Adaptation of the Teacher Effi cacy Scale to Measure Effective Teachers’ Educational Practices Through Students’ RatingsA Multilevel Approach

  1. María José Lera Rodríguez 1
  2. José M. León-Pérez 1
  3. Paula Ruiz Zorrilla Blanco 2
  1. 1 Universidad de Sevilla

    Universidad de Sevilla

    Sevilla, España


  2. 2 Universidad Complutense de Madrid

    Universidad Complutense de Madrid

    Madrid, España

    ROR 02p0gd045


ISSN: 0214-9915 1886-144X

Year of publication: 2021

Issue Title: Monográfico Mariano Yela

Volume: 33

Issue: 3

Pages: 509-517

Type: Article

DOI: 10.7334/PSICOTHEMA2020.262 DIALNET GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Psicothema


Cited by

  • Scopus Cited by: 6 (17-11-2023)
  • Dialnet Métricas Cited by: 3 (03-12-2023)
  • Web of Science Cited by: 6 (19-10-2023)
  • Dimensions Cited by: 4 (18-03-2023)

JCR (Journal Impact Factor)

  • Year 2021
  • Journal Impact Factor: 4.104
  • Journal Impact Factor without self cites: 3.657
  • Article influence score: 0.928
  • Best Quartile: Q1
  • Area: PSYCHOLOGY, MULTIDISCIPLINARY Quartile: Q1 Rank in area: 36/148 (Ranking edition: SSCI)

SCImago Journal Rank

  • Year 2021
  • SJR Journal Impact: 1.018
  • Best Quartile: Q1
  • Area: Psychology (miscellaneous) Quartile: Q1 Rank in area: 51/284

Índice Dialnet de Revistas

  • Year 2021
  • Journal Impact: 3.640
  • Field: PSICOLOGÍA Quartile: C1 Rank in field: 3/111


  • Social Sciences: A+

Scopus CiteScore

  • Year 2021
  • CiteScore of the Journal : 5.9
  • Area: Psychology (all) Percentile: 86

Journal Citation Indicator (JCI)

  • Year 2021
  • Journal Citation Indicator (JCI): 1.26
  • Best Quartile: Q1
  • Area: PSYCHOLOGY, MULTIDISCIPLINARY Quartile: Q1 Rank in area: 35/211


(Data updated as of 18-03-2023)
  • Total citations: 4
  • Recent citations: 4
  • Field Citation Ratio (FCR): 2.93


Background: There is an increasing evidence of the role that teachers’ educational practices have for students’ school achievement and their well-being. However, there is a lack of valid measures in Spanish to address effective educational practices based on students’ perceptions. In response, this study aims to provide a valid, reliable scale for measuring educational practices in school settings: the Students’ ratings of Teachers’ Educational Practices Scale (STEPS). Methods: We analyzed the scale’s internal consistency and reliability, factor solution and invariance, and criterion validity, by using a multilevel approach in a sample of 2,242 students nested in 104 classrooms from 22 Spanish schools. Results: indicated that the scale exhibited good reliability according to the omega coefficient (within =.86 and between level =.98). The multilevel confirmatory factor analysis (MCFA) revealed a hierarchical factor solution: classroom management, instructional strategies, and students’ engagement as first-order factors, and a general second-order factor labeled as effective educational practices. The scale demonstrated configural invariance by teaching level, sex, and region. Effective educational practices were associated with student self-esteem at the individual level. Conclusions: This study offers a reliable, valid instrument, STEPS, for measuring effective educational practices.

Funding information

This study was supported by research transfer contract FIUS 0956/0426. PRZ is a PhD student funded by the FPU program of the Spanish Ministry of Science, Innovation and Universities (FPU18/03536).

Bibliographic References

  • Ashdown, D. M., & Bernard, M. E. (2012). Can explicit instruction in social and emotional learning skills benefit the social-emotional development, well-being, and academic achievement of young children? Early Childhood Education Journal, 39(6), 397-405. s10643-011-0481-x
  • Bliese, P. D., Maltarich, M. A., & Hendricks, J. L. (2018). Back to basics with mixed-effects models: Nine take-away points. Journal of Business and Psychology, 33(1), 1-23. 9491-z
  • Cerny, C. A., & Kaiser, H. F. (1977). A study of a measure of sampling adequacy for factor-analytic correlation matrices. Multivariate Behavioral Research, 12(1), 43-47. s15327906mbr1201_3
  • Fernández-Alonso, R., & Muñiz, J. (2019). Calidad de los sistemas educativos: modelos de evaluación. Propósitos y Representaciones, 7, e347.
  • Fernández-Alonso, R., Suárez-Álvarez, J., & Muñiz, J. (2012). Imputación de datos perdidos en las evaluaciones diagnósticas educativas. Psicothema, 24(1), 167-175.
  • Fernández-Rodríguez, C., Soto-López, T., & Cuesta, M. (2019). Needs and demands for psychological care in university students. Psicothema, 31(4), 414-421.
  • Green, S. B., & Yang, Y. (2015). Evaluation of dimensionality in the assessment of internal consistency reliability: Coefficient alpha and omega coefficients. Educational Measurement: Issues and Practice, 34(4), 14-20.
  • Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.
  • Hoge, D. R., Smit, E. K., & Hanson, S. L. (1990). School experiences predicting changes in self-esteem of sixth-and seventh-grade students. Journal of Educational Psychology, 82(1), 117.
  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55.
  • James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67(2), 219-229.
  • Kane, T., Kerr, K., & Pianta, R. (2014). Designing teacher evaluation systems: New guidance from the measures of effective teaching project. John Wiley & Sons.
  • Klassen, R. M., & Tze, V. M. (2014). Teachers’ self-efficacy, personality, and teaching effectiveness: A meta-analysis. Educational Research Review, 12, 59-76.
  • Korpershoek, H., Harms, T., de Boer, H., van Kuijk, M., & Doolaard, S. (2016). A meta-analysis of the effects of classroom management strategies and classroom management programs on students’ academic, behavioral, emotional, and motivational outcomes. Review of Educational Research, 86(3), 643-680. https://doi. org/10.3102/0034654315626799
  • Kraft, M. A., Blazar, D., & Hogan, D. (2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547-588. https://doi. org/10.3102/0034654318759268
  • Kyriakides, L., Christoforou, C., & Charalambous, C. Y. (2013). What matters for student learning outcomes: A meta-analysis of studies exploring factors of effective teaching. Teaching and Teacher Education, 36, 143-152.
  • Kyriakides, L., Creemers, B. P., & Panayiotou, A. (2018). Using educational effectiveness research to promote quality of teaching: The contribution of the dynamic model. ZDM Mathematics Education, 50(3), 381-393.
  • Li, J., Han, X., Wang, W., Sun, G., & Cheng, Z. (2018). How social support influences university students’ academic achievement and emotional exhaustion: The mediating role of self-esteem. Learning and Individual Differences, 61, 120-126.
  • Martín-Albo, J., Núñez, J. L., Navarro, J. G., & Grijalvo, F. (2007). The Rosenberg Self-Esteem Scale: Translation and validation in university students. The Spanish Journal of Psychology, 10, 458-467. http://
  • Molina, E., Fatima, S. F., Ho, A., Hurtado, C. M., Wilichowksi, T., & Pushparatnam, A. (2018). Measuring teaching practices at scale: Results from the development and validation of the TEACH classroom observation tool. Research Working Paper No. 8653. World Bank, Washington, DC.
  • Muijs, D., Kyriakides, L., Van der Werf, G., Creemers, B., Timperley, H., & Earl, L. (2014). State of the art-teacher effectiveness and professional learning. School Effectiveness and School Improvement, 25(2), 231- 256.
  • Muthé n, B. O. (1994). Multilevel covariance structure analysis. Sociological Methods and Research, 22(3), 376-398.
  • Perera, H. N., Calkins, C., & Part, R. (2019). Teacher self-efficacy profiles: Determinants, outcomes, and generalizability across teaching level. Contemporary Educational Psychology, 58, 186-203. https://doi. org/10.1016/j.cedpsych.2019.02.00
  • Peters, G. J. (2014). The alpha and the omega of scale reliability and validity: Why and how to abandon Cronbach’s alpha and the route towards more comprehensive assessment of scale quality. European Health Psychologist, 16(2), 56-69. h47fv
  • Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom Assessment Scoring System. Paul H. Brookes Publishing.
  • Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: a critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879-903.
  • Rolland, R. G. (2012). Synthesizing the evidence on classroom goal structures in middle and secondary schools: A meta-analysis and narrative review. Review of Educational Research, 82(4), 396-435.
  • Rosenberg, M. (1965). Society and the adolescent self-image. Princeton University Press.
  • Stockard, J., Wood, T. W., Coughlin, C., & Rasplica Khoury, C. (2018). The effectiveness of direct instruction curricula: A meta-analysis of a half century of research. Review of Educational Research, 88(4), 479- 507.
  • Suldo, S. M., Friedrich, A. A., White, T., Farmer, J., Minch, D., & Michalowski, J. (2009). Teacher support and adolescents’ subjective well-being: A mixed-methods investigation. School Psychology Review, 38(1), 67-85.
  • Taylor, P. J. (2010). An introduction to intraclass correlation that resolves some common confusions. Unpublished manuscript, University of Massachusetts, Boston, USA. Retrieved from http://www.faculty.umb. edu/peter_taylor/09b.pdf
  • Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17(7), 783-805.
  • Van de Grift, W. (2007). Quality of teaching in four European countries: A review of the literature and an application of an assessment instrument. Educational Research, 49, 127-152. https://doi. org/10.1080/00131880701369651
  • van der Lans, R. M., van de Grift, W. J., & van Veen, K. (2019). Same, similar, or something completely different? Calibrating student surveys and classroom observations of teaching quality onto a common metric. Educational Measurement: Issues and Practice, 38(3), 55-64. https://
  • van der Scheer, E. A., Bijlsma, H. J., & Glas, C. A. (2019). Validity and reliability of student perceptions of teaching quality in primary education. School Effectiveness and School Improvement, 30(1), 30-50.
  • Vandenbroucke, L., Spilt, J., Verschueren, K., Piccinin, C., & Baeyens, D. (2018). The classroom as a developmental context for cognitive development: A meta-analysis on the importance of teacher-student interactions for children’s executive functions. Review of Educational Research, 88(1), 125-164.
  • Wagner, W., Göllner, R., Werth, S., Voss, T., Schmitz, B., & Trautwein, U. (2016). Student and teacher ratings of instructional quality: Consistency of ratings over time, agreement, and predictive power. Journal of Educational Psychology, 108(5), 705-721. edu0000075
  • Watkins, D. (2000). Learning and teaching: A cross-cultural perspective. School Leadership & Management, 20(2), 161-173. https://doi. org/10.1080/13632430050011407
  • Woitschach, P., Zumbo, B. D., & Fernández-Alonso, R. (2019). An ecological view of measurement: focus on multilevel model explanation of differential item functioning. Psicothema, 31(2), 194-203. https://