Agreement test of statistical analysis results between online bayes estimation versus t-test in Mahidol Dental Journal.
Article Sidebar
Main Article Content
Abstract
Objective: This research aimed to assess agreement between t-test and Bayesian estimation using the results from Mahidol Dental Journal. In general, to reveal the difference of means between two sample groups, Inferential statistics using Null Hypothesis Significant Testing (NHST), particularly t-test, has long been accepted. However, statistical analysis revolutionized by the Journal of the Basic and Applied Social Psychology (BASP), almost dismissed papers with NHST. Later, American and Thai Statistical Association also published articles that explained limitation of P-value. Alternatively, Bayesian estimation which has been developed for more than 200 years, has been recommended as a substitution for t-test.
Materials and methods: Upon completion of ethical approval, data were pooled from the articles using t-test in Mahidol Dental Journal from 2007-2017. Then the mean, standard deviation and sample size published in these articles were used to calculate the t-value. Online Bayesian estimation program (http://pcl.missouri.edu/bayesfactor) was applied utilizing the aforementioned calculated t-values. Agreement percentage and Cohen Kappa Coefficient were also computed.
Results: From the overall 274 articles, 21 articles adopted independent sample t-test and 2 articles adopted one sample t-test statistical analyses. Eighty-seven percent of the articles published in Mahidol Dental Journal showed agreement of research results between t-test and Bayesian estimation. The Cohen Kappa Coefficient was 0.73 indicating substantial agreement between these two tests. Further, the tendency of disagreements occurred with P-values starting from 0.05 to 0.085.
Conclusion: Mahidol Dental Journal showed substantial agreement for both statistical analyses. Future study will suggest the detail investigation on how Bayes theorem clarifies the disagreement between these two statistical test results and the situation when Bayes may perform better.
Article Details
References
Altman D. Practical statistics for medical research. Chapman and Hall CRC: London, 1991.
Wasserstein RL, Lazar NA. The ASA’s Statement on p -Values: Context, Process, and Purpose. Am Stat 2016; 70: 129–133.
Wasserstein RL, Schirm AL, Lazar NA. Moving to a World Beyond “p < 0.05”. Am Stat 2019; 73: 1–19.
Scheutz F, Anderson B, Wulff HR. What Do Dentists Know About Statistics? Eur J Oral Sci 1988; 96: 281–287.
Lecoutre MP, Poitevineau J, Lecoutre B. Even Statisticians Are Not Immune to Misinterpretations of Null Hypothesis Significance Tests. Int J Psychol 2003; 38: 37–45.
Goodman S. A Dirty Dozen: Twelve P-Value Misconceptions. Semin Hematol 2008; 45: 135–140.
Trafimow D. Editorial. Basic Appl Soc Psych 2014; 36: 1–2.
Trafimow D, Marks M. Editorial. Basic Appl Soc Psych 2015; 37: 1–2.
Nguyen H. Why P-values are Banned? Thail Stat 2016; 14: 448.
O’Hagan A. Bayesian statistics: principles and benefits. In: van Boekel M, Stein A, van Bruggen A (eds). Bayesian Statistics and Quality Modelling in the Agro-Food Production Chain. Springer: Wageningen, The Netherlands, 2004, pp 31–45.
Van de Schoot R, Kaplan D, Denissen J, Asendorpf JB, Neyer FJ, van Aken MAG. A Gentle Introduction to Bayesian Analysis: Applications to Developmental Research. Child Dev 2014; 85: 842–860.
van de Schoot R, Depaoli S. Bayesian Analyses: Where to Start and What to Report. Eur Helath Psychol 2014; 16: 75–84.
Rouder JN, Speckman PL, Sun D, Morey RD, Iverson G. Bayesian T Tests for Accepting and Rejecting the Null Hypothesis. Psychon Bull Rev 2009; 16: 225–237.
Jeffreys H. Theory of probability. Third edition. The Clarendon Press, Oxford University Press: New York, 1983.
Kass RE, Raftery AE. Bayes Factors. J Am Stat Assoc 1995; 90: 773–795.
Kruschke JK. Bayesian Estimation Supersedes the T Test. J Exp Psychol Gen 2013; 142: 573–603.
Kruschke JK, Liddell TM. Bayesian Data Analysis for Newcomers. Psychon Bull Rev 2018; 25: 155–157.
Suebnukarn S, Rungcharoenporn N, Sangsuratham S. A Bayesian Decision Support Model for Assessment of Endodontic Treatment Outcome. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2008; 106: e48–e58.
Thanathornwong B, Suebnukarn S, Songpaisan Y, Ouivirach K. A System for Predicting and Preventing Work-Related Musculoskeletal Disorders among Dentists. Comput Methods Biomech Biomed Engin 2014; 17: 177–185.
Morey RD. Using the ‘BayesFactor’ Package, version 0.9.2+. 2015. Available from https://richarddmorey.github.io/BayesFactor/ (accessed 28 Mar2020).
Berger JO, Mortera J. Interpreting the Stars in Precise Hypothesis Testing. Int Stat Rev 1991; 59: 337–353.
Evett IW. Bayesian Inference and Forensic Science: Problems and Perspectives. Stat 1987; 36: 99–105.
Dienes Z. Using Bayes to Get the Most out of Non-Significant Results. Front Psychol 2014; 5: 1–17.
Wagenmakers EJ, Marsman M, Jamil T, Ly A, Verhagen J, Love J et al. Bayesian Inference for Psychology. Part I: Theoretical Advantages and Practical Ramifications. Psychon Bull Rev 2018; 25: 35–57.
Wagenmakers EJ, Love J, Marsman M, Jamil T, Ly A, Verhagen J et al. Bayesian Inference for Psychology. Part II: Example Applications with JASP. Psychon Bull Rev 2018; 25: 58–76.