Assessing comprehension in students by processing cognitive data

Abstract

Conventional evaluation techniques frequently fail to capture the intricate cognitive processes like inference-making, metacognition, and information integration—that go into comprehension. This study advances comprehension assessment by developing and implementing a system that processes cognitive data from functional Magnetic Resonance Imaging (fMRI) scans to evaluate students’ reading comprehension levels in real time. There is a lack of comprehensive knowledge regarding how these patterns are closely associated with the wider spectrum of comprehension, despite the fact that eye-tracking research has revealed useful insights regarding visual attention patterns and readers' employed comprehension methods. Leveraging the Cross- Industry Standard Process for Data Mining (CRISP-DM) framework, we utilized "The Alice Dataset" to train two deep learning models—EEGNet and ResNet—to predict comprehension scores based on neural activity patterns. The implemented system integrates a web-based interface, a FastAPI backend, and cloud storage, enabling users to upload fMRI scans and receive comprehension scores ranging from 0 to 100, categorized into five levels (e.g., 90–100: Excellent). Testing revealed ResNet’s superior performance, with a Mean Absolute Error (MAE) reducing to 1.73, compared to EEGNet’s instability, highlighting the former’s suitability for neuroimaging- based assessments. While traditional methods like multiple-choice tests fail to capture underlying cognitive processes, this system offers objective, automated insights into comprehension, addressing limitations such as cost and scalability through affordable preprocessing techniques. Despite challenges like high computational demands and EEGNet’s overfitting, the findings enhance comprehension assessment practices, contributing to cognitive science and education by providing educators with precise tools for tailoring interventions. Future work aims to refine model stability and expand to multi-modal data integration.