Design and validation of an instrument to measure digital Competencies in artificial intelligence among Higher education students
DOI:
https://doi.org/10.35997/unaciencia.v18i35.1254Keywords:
Artificial intelligence, Digital skills, Ethical perception, Knowledge assessment, StudentsAbstract
This study describes the design and validation of an instrument to assess digital competencies in artificial intelligence (AI) among higher education students. Given the increasing integration of AI in academic and professional settings, reliable tools are needed to measure students’ knowledge, attitudes, and ethical perceptions regarding this technology. The research was conducted with a sample of 200 students from the University of Montemorelos during the 2024–2025 academic year. The instrument demonstrated adequate internal consistency (0.880), with satisfactory reliability coefficients across its three dimensions: Knowledge (0.774), Attitudes (0.859), and Ethics (0.807). Exploratory factor analysis confirmed the three-factor structure, supported by sampling adequacy (0.882) and a significant Bartlett’s test of sphericity (1137.600, 0.001). The three factors accounted for 62.088 of the total variance: Attitudes (40.863), Ethics (12.410), and Knowledge (8.815). Communalities indicated a strong representation of key items within each dimension, highlighting perceptions related to AI’s potential to optimize workload (ACT3), the responsible and ethical use of AI (ETI6), and knowledge of AI-driven personalized learning (CON3). These findings support the internal validity of the instrument and its reliability in measuring digital competencies in AI. The results underscore the importance of enhancing technical knowledge, fostering trust in AI, and promoting its ethical integration within higher education curricula.
Downloads
References
Abdelaal, N. M. & Al Sawi, I. (2024). Perceptions, challenges, and prospects: University professors' use of artificial intelligence in education. Australian Journal of Applied Linguistics, 7(1), 1-24. https://doi.org/10.29140/ajal.v7n1.1309
Abou-Hashish, E. A. & Alnajjar, H. (2024). Digital proficiency: Assessing knowledge, attitudes, and skills in digital transformation, health literacy, and artificial intelligence among university nursing students. BMC Medical Education, 24(1), 508. https://doi.org/10.1186/s12909-024-05482-3
Acosta-Enríquez, B. G., Arbulú-Ballesteros, M. A., Arbulu-Pérez-Vargas, C. G., Orellana-Ulloa, M. N., Gutiérrez-Ulloa, C. R., Pizarro-Romero, J. M., Gutiérrez-Jaramillo, N. D., Cuenca-Orellana, H. U., Ayala-Anzoátegui, D. X., & López Roca, C. (2024). Knowledge, attitudes, and perceived ethics regarding the use of ChatGPT among generation Z university students. International Journal for Educational Integrity, 20(1), 10.https://doi.org/10.1007/s40979-024-00157-4
Ahmed, Z., Bhinder, K. K., Tariq, A., Tahir, M. J., Mehmood, Q., Tabassum, M. S., & Yousaf, Z. (2022). Knowledge, attitude, and practice of artificial intelligence among doctors and medical students in Pakistan: A cross-sectional online survey. Annals of Medicine and Surgery, 76, 103493. https://doi.org/10.1016/j.amsu.2022.103493
Akgun, S. & Greenhow, C. (2022). Artificial intelligence in education: Addressing ethical challenges in K-12 settings. AI and Ethics, 2(3), 431–440. https://doi.org/10.1007/s43681-021-00096-7
Al Saad, M. M., Shehadeh, A., Alanazi, S., Alenezi, M., Eid, H., Alfaouri, M. S., & Alenezi, R. (2022). Medical students’ knowledge and attitude towards artificial intelligence: An online survey. The Open Public Health Journal, 15(1), e001.
https://doi.org/10.2174/18749445-v15-e2203290
Alghamdi, S. A. & Alashban, Y. (2023). Knowledge, attitudes and practices towards artificial intelligence (AI) among radiologists in Saudi Arabia. Journal of Radiation Research and Applied Sciences, 16(2), 100569. https://doi.org/10.1016/j.jrras.2023.100569
Allam, A. H., Eltewacy, N. K., Alabdallat, Y. J., Owais, T. A., Salman, S. & Ebada, M. A. (2024). Knowledge, attitude, and perception of Arab medical students towards artificial intelligence in medicine and radiology: A multi-national cross-sectional study. European Radiology, 34(7), 1–14. https://doi.org/10.1007/s00330-023-10509-2
Al-Qerem, W., Eberhardt, J., Jarab, A., Al Bawab, A. Q., Hammad, A., Alasmari, F.,& Al-Beool, S. (2023). Exploring knowledge, attitudes, and practices towards artificial intelligence among health professions students in Jordan. BMC Medical Informatics and Decision Making, 23 (288), 1-7. https://doi.org/10.1186/s12911-023-02403-0
Atalla, A. D. G., El-Ashry, A. M., & Mohamed Sobhi Mohamed, S. (2024). The moderating role of ethical awareness in the relationship between nurses’ artificial intelligence perceptions, attitudes, and innovative work behavior: A cross-sectional study. BMC Nursing, 23(1), 488. https://doi.org/10.1186/s12912-024-02143-0
Awad, S. O., Mohamed, Y. & Shaheen, R. (2022). Applications of artificial intelligence in education. Al-Azkiyaa - Jurnal Antarabangsa Bahasa dan Pendidikan, 1(1), 71–81. https://doi.org/10.33102/alazkiyaa.v1i1.10
Bakhteev, D. V. (2023). Ethical-legal models of the society interactions with artificial intelligence technology. Journal of Digital Technologies and Law, 1(2), 520–539. https://doi.org/10.21202/jdtl.2023.22
Bergdahl, J., Latikka, R., Celuch, M., Savolainen, I., Soares Mantere, E., Savela, N. & Oksanen, A. (2023). Self-determination and attitudes toward artificial intelligence: Cross-national and longitudinal perspectives. Telematics and Informatics, 82, 102013. https://doi.org/10.1016/j.tele.2023.102013
Burton, E., Goldsmith, J., Koenig, S., Kuipers, B., Mattei, N. & Walsh, T. (2017). Ethical Considerations in Artificial Intelligence Courses. AI Magazine, 38(2), 22–34. https://doi.org/10.1609/aimag.v38i2.2731
Casas-Roma, J., Conesa, J. & Caballé, S. (2021). Education, ethical dilemmas and AI: From ethical design to artificial morality. En R. A. Sottilare & J. Schwarz (Eds.), Adaptive instructional systems. Design and evaluation (Vol. 12792, pp. 167–182). Springer International Publishing. https://doi.org/10.1007/978-3-030-77857-6_11
Chen, D. & Zhang, L. (2023). Monotonicity for AI ethics and society: An empirical study of the monotonic neural additive model in criminology, education, health care, and finance. ArXiv, abs/2301.07060. https://doi.org/10.48550/ARXIV.2301.07060
Chuang, S. (2020). An empirical study of displaceable job skills in the age of robots. European Journal of Training and Development. https://doi.org/10.1108/EJTD-10-2019-0183
Holmes, W., Bialik, M. & Fadel, C. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign.
Kizilcec, R. F., Huber, E., Papanastasiou, E. C., Cram, A., Makridis, C. A., Smolansky, A., & Raduescu, C. (2024). Perceived impact of generative AI on assessments: Comparing educator and student perspectives in Australia, Cyprus, and the United States. Computers and Education: Artificial Intelligence, 7, 100269.
https://doi.org/10.1016/j.caeai.2024.100269
Kanont, K., Pingmuang, P., Simasathien, T., Wisnuwong, S., Wiwatsiripong, B., Poonpirome, K., Songkram, N. & Khlaisang, J. (2024). Generative-AI, a Learning Assistant? Factors Influencing Higher-Ed Students’ Technology Acceptance. The Electronic Journal Of e-Learning, 22(6), 1833. https://doi.org/10.34190/ejel.22.6.3196
Lérias, E., Guerra, C. & Ferreira, P. (2024). Literacy in artificial intelligence as a challenge for teaching in higher education: A case study at Portalegre Polytechnic University. Information, 15(4), 205. https://doi.org/10.3390/info15040205
Liehner, G., Hick, A., Biermann, H., Brauner, P. & Ziefle, M. (2023). Perceptions, attitudes, and trust toward artificial intelligence: An assessment of public opinion. Artificial Intelligence and Social Computing, 72, 1–9. https://doi.org/10.54941/ahfe1003271
Luckin, R. (2018). Machine learning and human intelligence: The future of education for the 21st century. UCL IOE Press.
Luckin, R., Holmes, W., Griffiths, M. & Forcier, L. B. (2018). Intelligence unleashed: An argument for AI in education. Pearson.
Masters, K. (2023). Ethical use of artificial intelligence in health professions education: AMEE Guide No. 158. Medical Teacher, 45(6), 574–584. https://doi.org/10.1080/0142159X.2023.2186203
Muñoz-Vela, J. M. (2024). Inteligencia artificial generativa. Desafíos para la propiedad intelectual. Revista de Derecho de la UNED, 33, 17–75. https://doi.org/10.5944/rduned.33.2024.41924
Ofosu-Ampong, K. (2024). Beyond the hype: Exploring faculty perceptions and acceptability of AI in teaching practices. Discover Education, 3, 38. https://doi.org/10.1007/s44217-024-00128-4
Perkins, M., Furze, L., Roe, J. & MacVaugh, J. (2024). The artificial intelligence assessment scale (AIAS): A framework for ethical integration of generative AI in educational assessment. Journal of University Teaching and Learning Practice, 21(6). https://doi.org/10.53761/q3azde36
Popenici, S. A. D. & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12(1), 22. https://doi.org/10.1186/s41039-017-0062-8
Ranbhise, N., Rathod, S. & Talsandekar, A. (2023). A correlational study on knowledge and attitude regarding artificial intelligence in health care among nursing students of D. Y. Patil College of Nursing, Kolhapur, Maharashtra. International Journal for Multidisciplinary Research, 5(4), 12–23. https://doi.org/10.36948/ijfmr.2023.v05i04.5637
Rathakrishnan, T., Kumar, T. B., Tsen, M. K., Leong, M. K. & Yaacob, A. (2024). AI tools: Anxiety to achievement - Unveiling the psychological dynamics of technology adoption. In R. Kumar, E. Ong, S. Anggoro, T. Toh y M. Fukui (Eds.), Transdisciplinary teaching and technological integration for improved learning: Case studies and practical approaches (pp. 42–65). IGI Global. https://doi.org/10.4018/979-8-3693-8217-2.ch003
Rizvi, M. (2023). Exploring the landscape of artificial intelligence in education: Challenges and opportunities. En 2023 5th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA). IEEE.
https://doi.10.1109/HORA58378.2023.10156773
Saz-Pérez, F., Pizà-Mir, B. & Lizana-Carrió, A. (2024). Validación y estructura factorial de un cuestionario TPACK en el contexto de inteligencia artificial generativa (IAG). Hachetetepe. Revista científica de Educación y Comunicación, 28, 1–14. https://doi.org/10.25267/Hachetetepe.2024.i28.1101
Selwyn, N. (2019). Should robots replace teachers? AI and the future of education. Learning, Media and Technology, 44(2), 143–154. https://doi.org/10.1080/17439884.2019.1667891
Tyagi, M., Ranjan, S., Smiti & Gupta, A. (2022). Transforming education system through artificial intelligence and machine learning. En 2022 3rd International Conference on Intelligent Engineering and Management (ICIEM) (pp. 44–49). https://doi.org/10.1109/ICIEM54221.2022.9853195
UNESCO. (2021). Artificial intelligence in education: Challenges and opportunities. UNESCO Publishing. https://unesdoc.unesco.org/ark:/48223/pf0000376709
Ungerer, L. & Slade, S. (2022). Ethical considerations of artificial intelligence in learning analytics in distance education contexts. En P. Prinsloo, S. Slade y M. Khalil (Eds.), Learning analytics in open and distributed learning (pp. 105–120). Springer Nature Singapore. https://doi.org/10.1007/978-981-19-0786-9_8
Uunona, G. N. & Goosen, L. (2023). Leveraging ethical standards in artificial intelligence technologies: A guideline for responsible teaching and learning applications. En M. B. Garcia, M. V. Lopez Cabrera y R. P. P. De Almeida (Eds.), Advances in medical education, research, and ethics (pp. 310–330). IGI Global. https://doi.org/10.4018/978-1-6684-7164-7.ch014
Vázquez-Parra, J. C., Henao-Rodríguez, C., Lis-Gutiérrez, J. P. & Palomino-Gámez, S. (2024). Importance of university students’ perception of adoption and training in artificial intelligence tools. Societies, 14(8), 141. https://doi.org/10.3390/soc14080141
Vera, F. (2023). Integración de la inteligencia artificial en la educación superior: Desafíos y oportunidades. Transformar, 4(1), 17–34. https://www.revistatransformar.cl/index.php/transformar/article/view/84
Wadhwa, R., Rabby, F., Bansal, R. & Hundekari, S. (2024). Drivers and impact of artificial intelligence on student engagement. En AI algorithms and ChatGPT for student engagement in online learning (pp. 161–170). IGI Global.
https://doi.org/10.4018/979-8-3693-4268-8.ch011
Zawacki-Richter, O., Marín, V. I., Bond, M. & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–Where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0
Zhou, L., Li, F., Wu, S. & Zhou, M. (2020). Smart education in the era of AI: Opportunities and challenges. Education and Information Technologies, 25(3), 3439–3460. https://doi.org/10.1007/s10639-019-10063-8
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Unaciencia, Revista de Estudios e Investigaciones
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
