Bayesian estimation with informative priors is indistinguishable from data falsification

  1. Miguel Ángel García-Pérez 1
  1. 1 Universidad Complutense de Madrid
    info

    Universidad Complutense de Madrid

    Madrid, España

    ROR 02p0gd045

Revista:
The Spanish Journal of Psychology

ISSN: 1138-7416

Año de publicación: 2019

Número: 22

Páginas: 1-13

Tipo: Artículo

DOI: 10.1017/SJP.2019.41 DIALNET GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: The Spanish Journal of Psychology

Resumen

Criticism of null hypothesis significance testing, confidence intervals, and frequentist statistics in general has evolved into advocacy of Bayesian analyses with informative priors for strong inference. This paper shows that Bayesian analysis with informative priors is formally equivalent to data falsification because the information carried by the prior can be expressed as the addition of fabricated observations whose statistical characteristics are determined by the parameters of the prior. This property of informative priors makes clear that only the use of non-informative, uniform priors in all types of Bayesian analyses is compatible with standards of research integrity. At the same time, though, Bayesian estimation with uniform priors yields point and interval estimates that are identical or nearly identical to those obtained with frequentist methods. At a qualitative level, frequentist and Bayesian outcomes have different interpretations but they are interchangeable when uniform priors are used. Yet, Bayesian interpretations require either the assumption that population parameters are random variables (which they are not) or an explicit acknowledgment that the posterior distribution (which is thus identical to the likelihood function except for a scale factor) only expresses the researcher’s beliefs and not any information about the parameter of concern.

Referencias bibliográficas

  • Alcalá-Quintana R., & García-Pérez M. A. (2004). The role of parametric assumptions in adaptive Bayesian estimation. Psychological Methods, 9, 250-271. https://doi.org/10.1037/1082-989X.9.2.250 10.1037/1082-989X.9.2.250
  • Arbuthnott J. (1710). An argument for Divine Providence, taken from the constant regularity observ'd in the births of both sexes. Philosophical Transactions of the Royal Society of London, 27, 186-190. https://doi.org/10.1098/rstl.1710.0011 10.1098/rstl.1710.0011
  • Bayes T. (1763). An essay towards solving a problem in the doctrine of chances. By the late Rev. Mr. Bayes, F. R. S. communicated by Mr. Price., in a letter to John Canton, A. M. F. R. S. Philosophical Transactions of the Royal Society of London, 53, 370-418. https://doi.org/10.1098/rstl.1763.0053 10.1098/rstl.1763.0053
  • Berger J. (2006). The case for objective Bayesian analysis. Bayesian Analysis, 1, 385-402. https://doi.org/10.1214/06-BA115 10.1214/06-BA115
  • Cassidy S. A., Dimova R., Giguère B., Spence J. R., & Stanley D. J. (2019). Failing grade: 89% of Introduction-to-Psychology textbooks that define or explain statistical significance do so incorrectly. Advances in Methods and Practices in Psychological Science, 2, 233-239. https://doi.org/10.1177/2515245919858072
  • Chang C.-H., Lin J.-J., Pal N., & Chiang M.-C. (2008). A note on improved approximation of the binomial distribution by the skew-normal distribution. The American Statistician, 62, 167-170. https://doi.org/10.1198/000313008X305359 10.1198/000313008X305359
  • Cumming G. (2014). The New Statistics: Why and how. Psychological Science, 25, 7-29. https://doi.org/10.1177/0956797613504966 10.1177/0956797613504966
  • Depaoli S. (2014). The impact of inaccurate "informative" priors for growth parameters in Bayesian growth mixture modeling. Structural Equation Modeling: A Multidisciplinary Journal, 21, 239-252. https://doi.org/10.1080/10705511.2014.882686 10.1080/10705511.2014.882686
  • Etz A., & Vandekerckhove J. (2018). Introduction to Bayesian inference for psychology. Psychonomic Bulletin & Review, 25, 5-34. https://doi.org/10.3758/s13423-017-1262-3 10.3758/s13423-017-1262-3
  • Finch W. H., & Miller J. E. (2019). The use of incorrect informative priors in the estimation of MIMIC model parameters with small sample sizes. Structural Equation Modeling: A Multidisciplinary Journal, 26, 497-508. https://doi.org/10.1080/10705511.2018.1553111 10.1080/10705511.2018.1553111
  • García-Pérez M. A. (2005). On the confidence interval for the binomial parameter. Quality and Quantity, 39, 467-481. https://doi.org/10.1007/s11135-005-0233-3 10.1007/s11135-005-0233-3
  • García-Pérez M. A. (2009). Letter to the editor. The American Statistician, 63, 102. https://doi.org/10.1198/tast.2009.0018 10.1198/tast.2009.0018
  • García-Pérez M. A. (2010). Confidence intervals for true scores using the skew-normal distribution. Journal of Educational and Behavioral Statistics, 35, 762-773. https://doi.org/10.3102/1076998609359786 10.3102/1076998609359786
  • García-Pérez M. A. (2012). Statistical conclusion validity: Some common threats and simple remedies. Frontiers in Psychology, 3, Article 325. https://doi.org/10.3389/fpsyg.2012.00325
  • García-Pérez M. A. (2017). Thou shalt not bear false witness against null hypothesis significance testing. Educational and Psychological Measurement, 77, 631-662. https://doi.org/10.1177/0013164416668232 10.1177/0013164416668232
  • García-Pérez M. A. (2018). Order-constrained estimation of nominal response model parameters to assess the empirical order of categories. Educational and Psychological Measurement, 78, 826-856. https://doi.org/10.1177/0013164417714296 10.1177/0013164417714296
  • García-Pérez M. A., & Alcalá-Quintana R. (2005). Sampling plans for fitting the psychometric function. The Spanish Journal of Psychology, 8, 256-289. https://doi.org/10.1017/S113874160000514X 10.1017/S113874160000514X
  • García-Pérez M. A., & Alcalá-Quintana R. (2016). The interpretation of scholars' interpretations of confidence intervals: Criticism, replication, and extension of Hoekstra et al. (2014). Frontiers in Psychology, 7, Article 1042. https://doi.org/10.3389/fpsyg.2016.01042
  • García-Pérez M. A., & Alcalá-Quintana R. (2017). The indecision model of psychophysical performance in dual-presentation tasks: Parameter estimation and comparative analysis of response formats. Frontiers in Psychology, 8, Article 1142. https://doi.org/10.3389/fpsyg.2017.01142
  • Gross C. (2016). Scientific misconduct. Annual Review of Psychology, 67, 693-711. https://doi.org/10.1146/annurev-psych-122414-033437 10.1146/annurev-psych-122414-033437
  • Haig B. D. (2017). Tests of statistical significance made sound. Educational and Psychological Measurement, 77, 489-506. https://doi.org/10.1177/0013164416667981 10.1177/0013164416667981
  • Hartgerink C. H. J., Wicherts J. M., & van Assen M. A. L. M. (2016). The value of statistical tools to detect data fabrication. Research Ideas and Outcomes, 2, e8860. https://doi.org/10.3897/rio.2.e8860
  • Ivarsson A., Andersen M. B., Stenling A., Johnson U., & Lindwall M. (2015). Things we still haven't learned (so far). Journal of Sport and Exercise Psychology, 37, 449-461. https://doi.org/10.1123/jsep.2015-0015 10.1123/jsep.2015-0015
  • Liu H., Zhang Z., & Grimm K. J. (2016). Comparison of inverse Wishart and separation-strategy priors for Bayesian estimation of covariance parameter matrix in growth curve analysis. Structural Equation Modeling: A Multidisciplinary Journal, 23, 354-367. https://doi.org/10.1080/10705511.2015.1057285 10.1080/10705511.2015.1057285
  • Kruschke J. K., & Liddell T. M. (2018 a). Bayesian data analysis for newcomers. Psychonomic Bulletin & Review, 25, 155-177. https://doi.org/10.3758/s13423-017-1272-1 10.3758/s13423-017-1272-1
  • Kruschke J. K., & Liddell T. M. (2018 b). The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin & Review, 25, 178-206. https://doi.org/10.3758/s13423-016-1221-4 10.3758/s13423-016-1221-4
  • Leek J. T., & Peng R. D. (2015). P values are just the tip of the iceberg. Nature, 520, 612. https://doi.org/10.1038/520612a 10.1038/520612a
  • Marcoulides K. M. (2018). Careful with those priors: A note on Bayesian estimation in two-parameter logistic item response theory models. Measurement:Interdisciplinary Research and Perspectives, 16, 92-99. https://doi.org/10.1080/15366367.2018.1437305
  • Miller J. (2017). Hypothesis testing in the real world. Educational and Psychological Measurement, 77, 663-672. https://doi.org/10.1177/0013164416667984 10.1177/0013164416667984
  • Nelson L. D., Simmons J., & Simonsohn U. (2018). Psychology's renaissance. Annual Review of Psychology, 69, 511-534. https://doi.org/10.1146/annurev-psych-122216-011836 10.1146/annurev-psych-122216-011836
  • Newcombe R. G. (1998). Two-sided confidence intervals for the single proportion: Comparison of seven methods. Statistics in Medicine, 17, 857-872. https://doi.org/10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E 10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E
  • Pickles A., & Croudace T. (2010). Latent mixture models for multivariate and longitudinal outcomes. Statistical Methods in Medical Research, 19, 271-289. https://doi.org/10.1177/0962280209105016 10.1177/0962280209105016
  • Raykov T., & Marcoulides G. A. (2019). A note on the presence of spurious pseudo-guessing parameters for three-parameter logistic models in heterogeneous populations. Educational and Psychological Measurement. Advance online publication. https://doi.org/10.1177/0013164419850882
  • Resnik D. B. (2014). Data fabrication and falsification and empiricist philosophy of science. Science and Engineering Ethics, 20, 423-431. https://doi.org/10.1007/s11948-013-9466-z 10.1007/s11948-013-9466-z
  • Ribeiro M. D., & Vasconcelos S. M. R. (2018). Retractions covered by Retraction Watch in the 2013-2015 period: Prevalence for the most productive countries. Scientometrics, 114, 719-734. https://doi.org/10.1007/s11192-017-2621-6 10.1007/s11192-017-2621-6
  • Rouder J. N. (2014). Optional stopping: No problem for Bayesians. Psychonomic Bulletin & Review, 21, 301-308. https://doi.org/10.3758/s13423-014-0595-4 10.3758/s13423-014-0595-4
  • Sakaluk J. K. (2016). Exploring small, confirming big: An alternative system to The New Statistics for advancing cumulative and replicable psychological research. Journal of Experimental Social Psychology, 66, 47-54. https://doi.org/10.1016/j.jesp.2015.09.013 10.1016/j.jesp.2015.09.013
  • Schneider J. W. (2015). Null hypothesis significance tests. A mix-up of two different theories: The basis for widespread confusion and numerous misinterpretations. Scientometrics, 102, 411-432. https://doi.org/10.1007/s11192-014-1251-5 10.1007/s11192-014-1251-5
  • Simmons J. P., Nelson L. D., & Simonsohn U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. https://doi.org/10.1177/0956797611417632 10.1177/0956797611417632
  • Simonsohn U. (2013). Just post it: The lesson from two cases of fabricated data detected by statistics alone. Psychological Science, 24, 1875-1888. https://doi.org/10.1177/0956797613480366 10.1177/0956797613480366
  • Smid S. C., McNeish D., Miočević M., & van de Schoot R. (2019). Bayesian versus frequentist estimation for structural equation models in small sample contexts: A systematic review. Structural Equation Modeling: A Multidisciplinary Journal. Advance online publication. https://doi.org/10.1080/10705511.2019.1577140
  • Stigler S. M. (1982). Thomas Bayes's Bayesian inference. Journal of the Royal Statistical Society, Series A, 145, 250-258. https://doi.org/10.2307/2981538 10.2307/2981538
  • Trafimow D. (2019). A frequentist alternative to significance testing, p-values, and confidence intervals. Econometrics, 7, 26. https://doi.org/10.3390/econometrics7020026 10.3390/econometrics7020026
  • Vanpaemel W. (2011). Constructing informative model priors using hierarchical methods. Journal of Mathematical Psychology, 55, 106-117. https://doi.org/10.1016/j.jmp.2010.08.005 10.1016/j.jmp.2010.08.005
  • Wang T., Xing Q.-R., Wang H., & Chen W. (2019). Retracted publications in the biomedical literature from Open Access journals. Science and Engineering Ethics, 25, 855-868. https://doi.org/10.1007/s11948-018-0040-6 10.1007/s11948-018-0040-6
Fuente de los datos: Dialnet