Content Validation of an Instrument Used to Assess the Educational Quality of Blended Learning Courses

Gutiérrez-Pérez Bárbara Mariana, Martín-García Antonio Víctor


This article describes the content validation process of an instrument used to assess the educational quality of training designs based on the Blended Learning model. The Expert Judgement technique was applied, involving the participation of 11 judges who have been duly trained in the design, implementation, and assessment of the Blended Learning model. The judges assessed the items presented based on criteria of relevance and clarity. The level of inter-judge agreement was determined using the Coefficient of Variation (CV) and Kendall's W test. The results show significant agreement amongst the participating judges regarding the relevance and clarity of the items that comprise the proposed instrument. The development of this research provides the design of an instrument able to objectify and define the educational quality of Blended Learning from a global and systemic perspective. This research project will help ensure quality in designing and implementing educational processes in hybrid learning environments and educational transformation and improvement in higher education.


Keywords: blended learning, content validity, educational quality, expert judgement, higher education.

Full Text:



MONTALVO-GARCÍA, A., LONGO, F., PEÑA-MOLINA, A., and TORREZ-MERUVIA, H. The Blended Quality Development Curve: A Comparative and Longitudinal Study on the Improvement Rate by Using Blended Methodologies in Management Masters. Universal Journal of Educational Research, 2020, 8(5): 1653–1664.

JEBRAEILY, M., PIRNEJAD, H., and FEIZI, A. Evaluation of blended medical education from lecturers’ and students’ viewpoint: a qualitative study in a developing country. BMC Medical Education, 2020, 20(482): 1–11.

ARMELLINI, A., TEIXEIRA ANTUNES, V., and HOWE, R. Student Perspectives on Learning Experiences in a Higher Education Active Blended Learning Context. TechTrends, 2021, 65: 433–443.

DZIUBAN, C., GRAHAM, C., MOSKAL, P., NORBERG, A., and SICILIA, N. Blended learning: The new normal and emerging technologies. International Journal Educational Technology in Higher Education, 2018, 15(3): 1–16.

SHUKLA, T., DOSAYA, D., NIRBAN, V.S., and VAVILALA, M.P. Factors extraction of effective teaching-learning in online and conventional classrooms. International Journal of Information and Education Technology, 2020, 10(6): 422–427.

PARK, Y., YU, J.H., and JO, I.H. Clustering blended learning courses by online behavior data case study in a Korean higher education institute. Internet and Higher Education, 2016, 29: 1–11.

MATOSAS-LÓPEZ, L., AGUADO-FRANCO, J.C., and GÓMEZ-GALÁN, J. Constructing an instrument with behavioral scales to assess teaching quality in blended learning modalities. Journal of New Approaches in Educational. Research, 2019, 8(2): 142–165.

MAZOHL, P., OSSIANNILSSON, E., and MAKL, H. Technical Innovation in Blended Learning – Concepts for the Creation of High-Quality Continuous Vocational Education Courses using Multiple Devices. Proceedings of the 10th International Conference on Computer Supported Education (CSEDU 2018), 2018: 548–554.

SALINAS IBÁÑEZ, J., DE BENITO CROSETTI, B., PÉREZ GARCÍAS, A., and GISBERT CERVERA, M. Blended learning, más allá de la clase presencial [Blended learning, beyond the classroom]. RIED. Revista Iberoamericana de Educación a Distancia, 2018, 21(1): 195–213.

HALVERSON, L.R., and GRAHAM, C.R. Learner engagement in blended learning environments: A conceptual framework. Online Learning, 2019, 23(2): 145–178.

BARTOLOMÉ-PINA, A., GARCÍA-RUIZ, R., and AGUADED, I. Blended learning: panorama y perspectivas [Blended learning: overview and expectations]. RIED. Revista Iberoamericana de Educación a Distancia, 2018, 21(1): 33–56.

BERGA, K.A., VADNAIS, E., and NELSON, J. Blended learning versus face-to-face learning in an undergraduate nursing health assessment course: A quasi-experimental study. Nurse Educational Today, 2021, 96: 104622.

ZHANG, Y.G., and DANG, M.Y. Understanding Essential Factors in Influencing Technology-Supported Learning: A Model toward Blended Learning Success. Journal of Information Technology Education: Research, 2020, 19: 489–510.

ALIZADEH, M., MEHRAN, P., and KOGUCHI, I. Evaluating a blended course for Japanese learners of English: why Quality Matters. International Journal Educational Technology in Higher Education, 2019, 16: 1–21.

KEAR, K., ROSEWEEL, J., and WILLIAMS, K. Quality Assessment for E-learning: a Benchmarking Approach. Maastricht: European A., 2016.

LIU, L., LIU, X., and XU, F. The study of quality evaluation model for the real-time interactive online teaching. 10th International Conference on Educational and Information Technology, ICEIT, 2021: 61–65.

VAGARINHO, P., and LLAMAS-NISTAL, M. Process-Oriented Quality in e-Learning: A Proposal for a Global Model. IEEE Access, 2020, 8: 13710–13734.

MUHAMMAD, A.H., SIDDIQUE, A., and YOUSSEF, A.E. A hierarchical model to evaluate the quality of web-based e-learning systems. Sustainability, 2020, 12(10): 4071.

ATANASOVA, I.A. University Knowledge Management Tool for the Evaluation of the Efficiency and Quality of Learning Resources in Distance e-Learning. International Journal of Knowledge Management, 2019, 15(4): 38–55.

PHAM, L., KIM, K., and WALKER, B. Development and validation of an instrument to measure student perceived e-learning service quality. International Journal of Enterprise Information Systems, 2019, 15(2): 15–42.

GRUBA, P., CÁRDENAS-CLAROS, M.S., SUVOROV, R., and RICK, K. Blended language program evaluation. London: Springer, 2016.

CASANOVA, D., and MOREIRA, A. A Model for Discussing the Quality of Technology-Enhanced Learning in Blended Learning Programmes. International Journal of Mobile and Blended Learning, 2017, 9(4): 1–20.

MOZELIUS, P., and HETTIARACHCHI, E. Critical Factors for Implementing Blended Learning in Higher Education. International Journal of Information and Communication Technologies in Education, 2017, 6(2): 37–51.

GUTIÉRREZ-PÉREZ, B.M., and MARTÍN-GARCÍA, A.V. Evaluation of Quality in Blended Learning Training. In: MARTÍN-GARCÍA, A.V. (ed.) Blended Learning: convergence between technology and pedagogy. Switzerland: Springer, 2020: 91–111.

ALMANASREH, E., MOLES, R., and CHEN, T.F. Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 2019, 15(2): 214–221.

MONTERO-MORA, J.G., and CANTÓN-CRODA, R.M. Validation of an instrument to measure user satisfaction in private sector education institutions: city of Xalapa-Enríquez (Mexico). Innovaciones Educativas, 2020, 22(32): 122–136.

GARCÍA-RUIZ, M.E., and LENA-ACEBO, F.J. Aplicación del metodo delphi en el diseño de una investigación cuantitativa sobre el fenómeno FABLAB [Application of the delphi method in the design of a quantitative investigation on the FABLABS]. EMPIRIA. Revista de Metodología de Ciencias Sociales, 2018, (40): 129–166.

KINTU, M.J., ZHU, C., and KAGAMBE, E. Blended Learning effectiveness: the relationship between student characteristics, design features, and outcomes. International Journal of Educational Technology in Higher Education, 2017, 14(7): 1–20.

MENDOZA-USTARIZ, E.F., and SEOANES-LEÓN, J.F. Indicadores básicos para la evaluación del b-learning y su incidencia en los procesos de aprendizaje de los estudiantes de la secundaria y media vocacional: Caso de estudio Institución Educativa Técnico Upar-Valledupar Colombia [Basic indicators for the evaluation of b-learning and its impact on the learning processes of secondary and vocational secondary school students: Case study of the educational institution Técnico Upar-Valledupar Colombia]. Seminario Modelos innovadores en las aulas: aprender en la Sociedad del Conocimiento Escuelas y Tecnologías, 2016: 1–16.

GALVIS, Á.H. Supporting decision-making processes on blended learning in higher education: literature and good practices review. International Journal of Educational Technology in Higher Education, 2018, 15(1): 1–38.

CASTAÑO, R., JENARO, C., and FLORES, N. Student Teachers’ perceptions towards the process and outcomes of Blended Learning. Revista de Educación a Distancia, 2017, 17(52): 1–19.

ALDANA VARGAS, M.F., and OSORIO, L.A. Pedagogical Guidelines for the Design of Blended Learning Environments. EDUTECH REVIEW. International Education Technologies Review, 2019, 6(1): 23–37.


  • There are currently no refbacks.