El Problema del Análisis de la Evaluación de la Satisfacción Estudiantil en el Ámbito UniversitarioUn estudio de simulación

  1. L. Catheryne Lancheros-Florián 1
  2. Eduar S. Ramírez 1
  3. Jesús M. Alvarado 1
  1. 1 Universidad Complutense de Madrid, Facultad de Psicología
Journal:
Revista iberoamericana de diagnóstico y evaluación psicológica

ISSN: 1135-3848

Year of publication: 2022

Issue Title: Avances en Medición en Psicología

Volume: 5

Issue: 66

Pages: 81-90

Type: Article

DOI: 10.21865/RIDEP66.5.06 DIALNET GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Revista iberoamericana de diagnóstico y evaluación psicológica

Abstract

Student´s satisfaction measured has raised great challenges since data processing involves the use of models capable of explaining the complexity of this measurement. In recent decades, multilevel models have been accepted as an alternative assessment for this phenomenon. However, its restrictions make its practical use difficult to achieve. An alternative is the application of bias corrections on controlling variables that will reduce the validity of the normative methods used. An antecedent study reveals that freeing the data from the bias provided by the differences between the inclinations of the teachers causes factorial structures to emerge different from those estimated, without using said procedures. In the present study, a simulation was carried out to check these first conclusions and observe how much it influences the difference in grades between teachers with respect to the estimates of the factorial models. When the difference between teachers’ qualification increased, it was noted a paradoxical increase in the quality of the fit indices.

Bibliographic References

  • Acevedo R., & Olivares, M. (2010). Fiabilidad y validez en la evaluación docente universitaria. Actualidades Investigativas en Educación, 10(1), 1- 38. https://doi.org/10.15517/aie.v10i1.10089
  • Alsarhan, A. A. M. (2017). Alternative methods of estimating the degree of uncertainty in student ratings of teaching (Doctoral dissertation, Brigham Young University). Theses and Dissertations. 6939.
  • Arámburo Vizcarra, V., & Luna Serrano, E. (2013). La influencia de las características del profesor y del curso en los puntajes de evaluación docente. Revista Mexicana de Investigación Educativa, 18(58), 949-968. https://doi.org/10.15366/riee2018.11.2.001
  • Bacci, S., &a Caviezel, V. (2011). Multilevel IRT models for the university teaching evaluation. Journal of Applied Statistics, 38(12), 27752791. https://doi.org/10.1080/02664763.2011.570316
  • Boring, A., Ottoboni, K., & Stark, P. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research, 0(0), 1-11. https://doi.org/10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1
  • Borsboom, D., Mellenbergh, G. J., & Van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061. https://doi.org/10.1037/0033-295X.111.4.1061
  • Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford publications.
  • Cohen, A. S., Kane, M. T., & Kim, S.-H. (2001). The precision of simulation study results. Applied Psychological Measurement, 25(2), 136-145. https://doi.org/10.1177/01466210122031966
  • DeFrain, E. (2016). An analysis of differences in non-instructional factors affecting teachercourse evaluations over time and across disciplines. (Doctoral dissertation, University of Arizona.)
  • De Leeuw, J., Meijer, E., & Goldstein, H. (2008). Handbook of multilevel analysis. Springer.
  • Feistauer, D., & Richter, T. (2018). Validity of students’ evaluations of teaching: Biasing effects of likability and prior subject interest. Studies in Educational Evaluation, 59, 168178. https://doi.org/10.1016/j.stueduc.2018.07.009
  • Eber, F. J., Holtmann, J., & Eid, M. (2021). A Monte Carlo simulation study on the influence of unequal group sizes on parameter estimation in multilevel confirmatory factor analysis. Structural Equation Modeling: A Multidisciplinary Journal, 28(6), 827-838. https://doi.org/10.1080/10705511.2021.1913594
  • Franklin, J. (2001). Interpreting the numbers: Using a narrative to help others read student evaluations of your teaching accurately. New Directions for Teaching and Learning, 87, 85-100. https://doi.org/10.1002/tl.10001
  • Garduño, J. M. G. (2000). ¿Qué factores extra clase o sesgos afectan la evaluación docente en la educación superior? Revista Mexicana de Investigación Educativa, 5(10), 303 - 325. https://www.redalyc.org/articulo.oa?id=14001006
  • Gruber, T., Lowrie, A., Brodowsky, G. H., Reppel, A. E., Voss, R., & Chowdhury, I. N. (2012). Investigating the influence of professor characteristics on student satisfaction and dissatisfaction: A comparative study. Journal of Marketing Education, 34(2), 165-178. https://doi.org/10.1177/0273475312450385
  • Hox, J. J., Moerbeek, M., & Van de Schoot, R. (2017). Multilevel analysis: Techniques and applications. Routledge.
  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. https://doi.org/10.1080/10705519909540118
  • Kelloway, E. K. (1998). Using LISREL for structural equation modeling: A researcher's guide. Sage.
  • Lancheros-Florián L., Ramírez E., & Alvarado, J. (2022). Satisfacción con la calidad docente en el ámbito universitario: Potenciales sesgos y propuestas de análisis para su evaluación. Revista Iberoamericana de Diagnóstico y Evaluación – e Avaliação Psicológica, 65(4), 69-83. https://doi.org/10.21865/RIDEP65.4.06
  • Lang, J. W., & Kersting, M. (2007). Regular feedback from student ratings of instruction: Do college teachers improve their ratings in the long run?. Instructional Science, 35(3), 187-205. https://doi.org/10.1007/s11251-006-9006-1
  • Lee, S. (2015). Implementing a Simulation Study Using Multiple Software Packages for Structural Equation Modeling. SAGE Open, 5(3). https://doi.org/10.1177/2158244015591823
  • Luna, E., Arámburo, V., & Cordero, G. (2010). Influence of the pedagogical context on students evaluation of teaching. International Journal of Teaching and Learning in Higher Education, 22(3), 337-345. http://www.isetl.org/ijtlhe/
  • Martínez, B. M. T., del Carmen Pérez-Fuentes, M., & Jurado. (2022). Investigación sobre el compromiso o engagement académico de los estudiantes: Una revisión sistemática sobre factores influyentes y instrumentos de evaluación. Revista Iberoamericana de Diagnóstico y Evaluación – e Avaliação Psicológica, 1(62), 101-111. https://doi.org/10.21865/RIDEP62.1.08
  • Marsh, H. W., & Hattie, J. (2002). The relation between research productivity and teaching effectiveness: Complementary, antagonistic, or independent constructs?. The Journal of Higher Education, 73(5), 603-641. https://doi.org/10.1080/00221546.2002.11777170
  • Messick, S. (1998). Test validity: A matter of consequence. Social Indicators Research, 45(1), 35-44. https://doi.org/10.1023/A:1006964925094
  • Muñoz, C. P., Nieto, B. B., Méndez, M. J. M., & Morillejo, E. A. (2011). Evaluación de la actividad docente en el Espacio Europeo de Educación Superior: Un estudio comparativo de indicadores de calidad en universidades europeas. Revista Española de Pedagogía, 248, 145-163. https://dialnet.unirioja.es/servlet/articulo?codigo=3365101
  • Oermann, M. H., Conklin, J. L., Rushton, S., & Bush, M. A. (2018). Student evaluations of teaching (SET): Guidelines for their use. Nursing Fórum, 53(3), 280-285. https://doi.org/10.1111/nuf.12249
  • Pardo, A., Ruiz, M. Á., & San Martín, R. (2007). Cómo ajustar e interpretar modelos multinivel con SPSS. Psicothema, 19(2), 308-321. https://www.redalyc.org/articulo.oa?id=72719220
  • Park, J., & Yu, H. T. (2016). The impact of ignoring the level of nesting structure in nonparametric multilevel latent class models. Educational and Psychological Measurement, 76(5), 824-847. https://doi.org/10.1177/0013164415618240
  • Pascual-Gómez, I. (2007). Análisis de la satisfacción del alumno con la docencia recibida: un estudio con modelos jerárquicos lineales. Revista Electrónica de Investigación y Evaluación Educativa, 13(1), 127-138 https://doi.org/10.7203/relieve.13.1.4216
  • Pekka R. (2013) The number of feedbacks needed for reliable evaluation. A multilevel analysis of the reliability, stability and generalisability of students’ evaluation of teaching. Assessment & Evaluation in Higher Education, 38(2), 224-239, https://doi.org/10.1080/02602938.2011.625471
  • Rampichini, C., Grilli, L. & Petrucci, A. (2004). Analysis of university course evaluations: From descriptive measures to multilevel models. Statistical Methods & Applications 13, 357-373. https://doi.org/10.1007/s10260-004-0087-1
  • Rosseel Y. (2012). “lavaan: An R Package for Structural Equation Modeling.” Journal of Statistical Software, 48(2), 1-36. https://doi.org/10.18637/jss.v048.i02.
  • Ruiz, M. A., Pardo, A., & San Martín, R. (2010). Modelos de ecuaciones estructurales. Papeles del psicólogo, 31(1), 34-45. https://dialnet.unirioja.es/servlet/articulo?codi go=3150815
  • Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598-642. https://doi.org/10.3102/0034654313496870
  • Stake, R. E., García, M. I. A., & Pérez, G. C. (2017). Evaluando la calidad de la Universidad–Particularmente su enseñanza. REDU: Revista de Docencia Universitaria, 15(2), 125-142. https://doi.org/10.4995/redu.2017.6371
  • Toland, M. D., & De Ayala, R. J. (2005). A multilevel factor analysis of students’ evaluations of teaching. Educational and Psychological Measurement, 65(2), 272-296. https://doi.org/10.1177/0013164404268667
  • Turull, M., & Buxarrais, M. R. (2018). La evaluación de la docencia en las universidades públicas catalanas: Análisis comparativo de los diferentes manuales de evaluación. Revista de Educación y Derecho, 17, 1-30. https://doi.org/10.1344/re&d.v0i17.21842
  • Wolbring, T., & Riordan, P. (2016). How beauty works. Theoretical mechanisms and two empirical applications on students’ evaluation of teaching. Social Science Research, 57, 253272. https://doi.org/10.1016/j.ssresearch.2015.12.009
Data source: Dialnet