Methods to estimate the variance of some indices of the signal detection theoryA simulation study

  1. Manuel Suero
  2. Jesús Privado
  3. Juan Botella
Revue:
Psicológica: Revista de metodología y psicología experimental

ISSN: 1576-8597

Année de publication: 2017

Volumen: 38

Número: 1

Pages: 149-175

Type: Article

DIALNET GOOGLE SCHOLAR lock_openAccès ouvert editor

D'autres publications dans: Psicológica: Revista de metodología y psicología experimental

Résumé

A simulation study is presented to evaluate and compare three methods to estimate the variance of the estimates of the parameters δ and C of the signal detection theory (SDT). Several methods have been proposed to calculate the variance of their estimators, d' and c. Those methods have been mostly assessed by comparing the empirical means and variances in simulation studies with the calculations done with the parametric values of the probabilities of giving a yes response on a signal trial (hits) and on a noise trial (false alarms). In practical contexts the variance must be estimated from estimations of those probabilities (empirical rates of hits and false alarms). The three methods to estimate the variance compared in the present simulation study are based in the binomial distribution of Miller, the normal approach of Gourevitch and Galanter and the maximum likelihood method proposed by Dorfman and Alf. They are compared in terms of relative bias (accuracy) and the mean squared error (precision). The results show that the last two methods behave indistinguishably for practical purposes and provide severe over-estimation errors in a range of situations that while not the most common are perfectly credible in several practical contexts. By contrast, the method of Miller provides better results (or at least similar) in all conditions studied. It is the recommended method to obtain estimates of the variances of these statistics for practical purposes.

€ Voir financement

Information sur le financement

Project ?Meta-an?lisis con ?ndices de la Teor?a de la Detecci?n deSe?ales?. (Reference: PSI2013-45513; MINECO)

Financeurs

Références bibliographiques

  • Bolker, B. (2015). Package “bbmle”. http://cran.r-project.org/web/packages/bbmle/ bbmle.pdf
  • Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: John Wiley and sons.
  • Botella, J., & Sánchez-Meca, J. (2015). Meta-análisis en Ciencias Sociales y de la Salud. Madrid: Editorial Síntesis.
  • Brown, G. S., & White, K. G. (2005). The optimal correction for estimating extreme discriminability. Behavior Research Methods, 37(3), 436–449.
  • Burton, A., Altman, D. G., Royston, P., & Holder, R. L. (2006). The design of simulation studies in medical statistics. Statistics in Medicine, 25, 4279–4292.
  • Dorfman, D. D. (1982). RSCORE II. In J. A. Swets & R. M. Pickett (Eds.), Evaluation of diagnostic systems: Methods from signal detection theory (pp. 208-232). New York: Academic Press.
  • Dorfman, D. D., & Alf, E. (1968). Maximum likelihood estimation of parameters of signal detection theory—A direct solution. Psychometrika, 33, 117–124.
  • Gardiner, J. M., Ramponi, C., & Richardson-Klavehn, A. (2002). Recognition memory and decision processes: A meta-analysis of remember, know, and guess responses. Memory, 10(2), 83-98.
  • Gourevitch, V., & Galanter, E. (1967). A significance test for one parameter isosensitivit functions. Psychometrika, 32, 25–33.
  • Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York: Wiley.
  • Hautus, M. J. (1995). Corrections for extreme proportions and their biasing effects on estimated values of d’, Behavior Research Methods,Instruments, & Computers, 26, 46-51.
  • Hautus, M. J., & Lee, A. (2006). Estimating sensitivity and bias in a yes/no task. The British Journal of Mathematical and Statistical Psychology, 59 (2), 257–273.
  • Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
  • Heinrichs, R. W., & Zakzanis, K. K. (1998). Neurocognitive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology, 12(3), 426.
  • Jesteadt, W. (2005). The variance of d’ estimates obtained in yes—no and two-interval forced choice procedures. Perception & psychophysics, 67(1), 72-80.
  • Kadlec, H. (1999). Statistical properties of d' and β estimates of signal detection theory. Psychological Methods, 4(1), 22.
  • Kaplan, A. (2009). A Comparison of Three Methods for Calculating Confidence Intervals around D-Prime. Unpublished manuscript.
  • Logan, G. D. (2004). Cumulative Progress in Formal Theories of Attention. Annual Review of Psychology, 55, 207-234.
  • Macmillan N. A., & Creelman C. D. (2005). Detection theory: A user’s guide. (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
  • Macmillan, N. A., Rotello, C. M., & Miller, J. O. (2004). The sampling distributions of Gaussian ROC statistics. Perception & Psychophysics, 66(3), 406-421.
  • Metz, C. E. (1989). Some practical issues of experimental design and data analysis in radiological ROC studies. Investigative Radiology, 24, 234-245.
  • Miller, J. (1996). The sampling distribution of d’. Perception & Psychophysics, 58, 65-72.
  • Murdock, B. B. Jr., & Ogilvie, J. C. (1968). Binomial variability in short-term memory. Psychological Bulletin, 70, 256-260.
  • R Core Team (2015). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/.
  • Rhodes, M. G., & Jacoby, L. L. (2007). On the dynamic nature of response criterion in recognition memory: Effects of base rate, awareness, and feedback. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(2), 305.
  • Snodgrass, J. J., & Corwin, J. (1988). Pragmatics of measuring recognition memory: Applications to dementia and amnesia. Journal of Experimental Psychology: General, 117, 34–50.
  • Suero, M., Botella, J., & Privado, J. (in preparation). Estimating the sampling variance of SDT indexes with heterogeneous individuals.
  • Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Psychological Science can improve diagnostic decisions. Psychological Science in the Public Interest, 1(1), 1-26.
  • Verde, M. F., Macmillan, N. A., & Rotello, C. M. (2006). Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of d’, Az, and A’. Perception & psychophysics, 68(4), 643-654.
  • Wickens, T. D. (2001). Elementary signal detection theory. Nueva York: Oxford University Press.
La source de données: Dialnet