On October 21, 2026, MSc candidate Sara Kovačević successfully defended her thesis entitled “Generation and Quality Evaluation of Exam Questions Using Large Language Models and HPC Infrastructure” within the Artificial Intelligence Master’s programme at the Faculty of Information Systems and Technologies, University of Donja Gorica (UDG). The thesis was supervised by Prof. Dr. Tomo Popović. The research was partially supported by NCC Montenegro within the EuroCC3 project, including access to HPC infrastructure and support for experimental work. The study demonstrates the potential of combining generative AI and high-performance computing to support the preparation and evaluation of educational materials, with particular attention to the Montenegrin language.
ABSTRACT – This thesis presents the development and evaluation of an automated system for generating exam questions from video lectures using large language models (LLMs). The system integrates automatic transcription, extraction of key concepts, generation of multiple-choice and short-answer questions, and automated quality assessment. Nine LLM configurations were evaluated in English and Montenegrin, using local computing resources, the UDG HPC cluster, and API-based inference services.
The evaluation combined linguistic and structural metrics, Bloom’s taxonomy, and an LLM-as-Judge approach to assess the quality of generated questions. Experimental results highlighted challenges faced by smaller models when generating educational content in Montenegrin, while larger models achieved more consistent results. Experiments on the UDG HPC cluster demonstrated a 17.9× GPU speedup compared with CPU inference for the same model.
The research highlights the potential of AI and HPC technologies to assist educators in preparing assessment materials, while emphasizing the importance of human review and further pedagogical validation before practical deployment.