Yuvaraj Rajamanickam

Objective This work aims to establish a framework in measuring the various attentional levels of the human operator in a real-time animated environment through a visual neuro-assisted approach.
Background With the increasing trend of automation and remote operations, understanding human-machine interaction in dynamic environments can greatly aid to improve performance, promote operational efficiency and safety.
Method Two independent 1-hour experiments were conducted on twenty participants where eye-tracking metrics and neuro activities from electroencephalogram (EEG) were recorded. The experiments required participants to exhibit attentive behaviour in one set and inattentive in the other. Two segments (“increasing flight numbers” and “relatively constant flight numbers”) were also extracted to study the participants’ visual behavioral differences in relation to aircraft numbers.
Results For the two experimental studies, those in the attentive behavioral study show incidences of higher fixation count, fixation duration, number of aircraft spotted, and landing fixations whereas those in inattentive behavior study reveal higher zero-fixation frame count. In experiments involving ‘increasing flight numbers’, a higher percentage of aircraft were spotted as compared to those with ‘constant flight numbers’ in both the groups. Three parameters (number of aircraft spotted, and landing fixations and zero-fixation frame count) are newly established. As radar monitoring is a brain engagement activity, positive EEG data were registered in all the participants. A newly Task Engagement Index (TEI) was also formulated to predict different attentional levels.
Conclusion Results provide a refined quantifiable tool to differentiate between attentive and inattentive monitoring behavior in a real-time dynamic environment, which can be applied across various sectors.
Recommendation With the quantitative TEI established, this paves the way for future studies into attentional levels by regions, time based, as well as eye signature studies in relation to visual task engagement and management and determining expertise levels to be explored. Factors relating to fatigue could also be investigated using the TEI approach proposed.

Measuring emotions in educational settings can provide important information in predicting and explaining student learning outcomes. Knowledge of student’s classroom emotions can also help teachers understand their students’ learning behaviors, improve their teaching methods, and optimize students’ learning and development. However, it can be highly challenging for teachers to monitor and accurately understand student emotions within classroom or group contexts, especially while they are actively teaching and attending to many students simultaneously. Video recording classroom activity can address that issue as high-definition cameras can be used to continuously record groups of students, enabling online or offline analyses of student emotions and supplementing teacher monitoring, retrieval, and interpretation of those processes. Teachers find it difficult to use existing emotion recognition methods to analyze student behaviors in videos due to a lack of user-friendly interfaces that facilitate automatic analysis. To address this challenge, we developed EmoRoom, an open-source tool designed to simplify the annotation and analysis of videos from an emotional perspective. EmoRoom tool offers a practical solution for quantifying and visualizing the frequency of emotions for individuals of interest. Furthermore, it can assist teachers in refining their teaching methods by offering a comprehensive overview of students’ emotional experiences during learning.

Affective computing is an emerging area of education research and has the potential to enhance educational outcomes. Despite the growing number of literature studies, there are still deficiencies and gaps in the domain of affective computing in education. In this study, we systematically review affective computing in the education domain. Methods: We queried four well-known research databases, namely the Web of Science Core Collection, IEEE Xplore, ACM Digital Library, and PubMed, using specific keywords for papers published between January 2010 and July 2023. Various relevant data items are extracted and classified based on a set of 15 extensive research questions. Following the PRISMA 2020 guidelines, a total of 175 studies were selected and reviewed in this work from among 3102 articles screened. The data show an increasing trend in publications within this domain. The most common research purpose involves designing emotion recognition/expression systems. Conventional textual questionnaires remain the most popular channels for affective measurement. Classrooms are identified as the primary research environments; the largest research sample group is university students. Learning domains are mainly associated with science, technology, engineering, and mathematics (STEM) courses. The bibliometric analysis reveals that most publications are affiliated with the USA. The studies are primarily published in journals, with the majority appearing in the Frontiers in Psychology journal. Research gaps, challenges, and potential directions for future research are explored. This review synthesizes current knowledge regarding the application of affective computing in the education sector. This knowledge is useful for future directions to help educational researchers, policymakers, and practitioners deploy affective computing technology to broaden educational practices.

Objective: Clinical diagnosis of epilepsy relies partially on identifying Interictal Epileptiform Discharges (IEDs) in scalp electroencephalograms (EEGs). This process is expert-biased, tedious, and can delay the diagnosis procedure. Beyond automatically detecting IEDs, there are far fewer studies on automated methods to differentiate epileptic EEGs (potentially without IEDs) from normal EEGs. In addition, the diagnosis of epilepsy based on a single EEG tends to be low. Consequently, there is a strong need for automated systems for EEG interpretation. Traditionally, epilepsy diagnosis relies heavily on IEDs. However, since not all epileptic EEGs exhibit IEDs, it is essential to explore IEDindependent EEG measures for epilepsy diagnosis. The main objective is to develop an automated system for detecting epileptic EEGs, both with or without IEDs. In order to detect epileptic EEGs without IEDs, it is crucial to include EEG features in the algorithm that are not directly related to IEDs.

Approach: In this study, we explore the background characteristics of interictal EEG for automated and more reliable diagnosis of epilepsy. Specifically, we investigate features based on univariate temporal measures (UTM), spectral, wavelet, Stockwell, connectivity, and graph metrics of EEGs, besides patient-related information (age and vigilance state). The evaluation is performed on a sizeable cohort of routine scalp EEGs (685 epileptic EEGs and 1229 normal EEGs) from five centers across Singapore, USA, and India. Main results: In comparison with the current literature, we obtained an improved Leave-One-Subject-Out (LOSO) cross-validation (CV) area under the curve (AUC) of 0.871 (Balanced Accuracy (BAC) of 80.9%) with a combination of 3 features (IED rate, and Daubechies and Morlet wavelets) for the classification of EEGs with IEDs vs. normal EEGs. The IED-independent feature UTM achieved a LOSO CV AUC of 0.809 (BAC of 74.4%). The inclusion of IED-independent features also helps to improve the EEG-level classification of epileptic EEGs with and without IEDs vs. normal EEGs, achieving an AUC of 0.822 (BAC of 77.6%) compared to 0.688 (BAC of 59.6%) for classification only based on the IED rate. Specifically, the addition of IED-independent features improved the BAC by 21% in detecting epileptic EEGs that do not contain IEDs.

Significance: These results pave the way towards automated detection of epilepsy. We are one of the first to analyse epileptic EEGs without IEDs, thereby opening up an underexplored option in epilepsy diagnosis.

Epilepsy is a chronic brain disorder characterized by recurrent unprovoked seizures. The treatment for epilepsy is influenced by the types of seizures. Therefore, developing a reliable, explainable, and automated system to identify seizure types is necessary. This study aims to automate the process of classification of five seizure types: focal non-specific, generalized, complex partial, absence, and tonic-clonic using electroencephalogram (EEG) signals and machine learning algorithms. The EEG signals of 2933 seizures from 327 patients were obtained from the publicly available Temple University Hospital dataset. Initially, the signals were preprocessed using a standard pipeline, and 110 features from the time, frequency, and time-frequency domain were computed from each seizure. Further, the features were ranked using the statistical test and extreme Gradient Boosting (XGBoost) algorithm to identify the significant features. We built binary and multiclass seizure-type classification systems using the identified features and machine learning algorithms. Our study revealed that the EEG band power between 11–13 Hz, 27–29 Hz, intrinsic mode function (IMF) band power 19–21 Hz, and delta band (1-4 Hz) played a crucial role in discriminating the seizures. We achieved an average accuracy of 88.21% and 69.43% for the binary and multiclass seizure-type classification, respectively, using the XGBoost classifier. We also found that the combination of features performed well compared to any single domain. This automated system has the potential to aid neurologists in making diagnosis of epileptic seizure types. The proposed methodology can be applied alongside the established clinical approach of visual evaluation for the classification of seizure-types.

In the process of diagnosing neurological disorders, neurologists often study the brain activity of the patient recorded in the form of an electroencephalogram (EEG). Identifying an abnormal EEG serves as a preliminary indicator before specialized testing to determine the neurological disorder. Traditional identification methods involve manual perusal of the EEG signals. This method is relatively slow and tedious, requires trained neurologists, and delays the treatment plan. Therefore, the development of an automated abnormal EEG detection system is essential. In this study, we propose a method based on short-time Fourier transform (STFT), which is a time-frequency (TF) representation, and deep convolutional neural network (CNN) to detect abnormal EEGs. First, the filtered time-series EEG signals are converted into TF images by applying STFT. Then, the images are fed to three popular configurable CNN structures, namely, DenseNet, SeizureNet, and Inception-ResNet-V2, to extract deep learned features. Finally, an extreme learning machine (ELM)-based classifier detects the input TF images. The proposed STFT-based CNN method is evaluated using the Temple University Hospital (TUH) abnormal EEG corpus, which is available under the public domain. The experiment showed that the combination of the SeizureNet-ELM model achieved an average (fivefold cross-validation) accuracy, specificity, sensitivity, and F1-score of 85.87%, 88.43%, 83.23%, and 0.858, respectively. The results demonstrate that the proposed framework may aid clinicians in abnormal EEG detection for the early treatment plan.

Biomedical signals provide unprecedented insight into abnormal or anomalous neurological conditions. The computer-aided diagnosis (CAD) system plays a key role in detecting neurological abnormalities and improving diagnosis and treatment consistency in medicine. This book covers different aspects of biomedical signals-based systems used in the automatic detection/identification of neurological disorders. Several biomedical signals are introduced and analyzed, including electroencephalogram (EEG), electrocardiogram (ECG), heart rate (HR), magnetoencephalogram (MEG), and electromyogram (EMG). It explains the role of the CAD system in processing biomedical signals and the application to neurological disorder diagnosis. The book provides the basics of biomedical signal processing, optimization methods, and machine learning/deep learning techniques used in designing CAD systems for neurological disorders.

Classroom EEG recordings classification has the capacity to significantly enhance comprehension and learning by revealing complex neural patterns linked to various cognitive processes. Electroencephalography (EEG) in academic settings allows researchers to study brain activity while students are in class, revealing learning preferences. The purpose of this study was to develop a machine learning framework to automatically classify different learning-style EEG patterns in real classroom environments. Method: In this study, a set of EEG features was investigated, including statistical features, fractal dimension, higher-order spectra, entropy, and a combination of all sets. Three different machine learning classifiers, random forest (RF), K-nearest neighbor (KNN), and multilayer perceptron (MLP), were used to evaluate the performance. The proposed framework was evaluated on the real classroom EEG dataset, involving EEG recordings featuring different teaching blocks: reading, discussion, lecture, and video. Results: The findings revealed that statistical features are the most sensitive feature metric in distinguishing learning patterns from EEG. The statistical features and RF classifier method tested in this study achieved an overall best average accuracy of 78.45% when estimated by fivefold cross-validation. Conclusions: Our results suggest that EEG time domain statistics have a substantial role and are more reliable for internal state classification. This study might be used to highlight the importance of using EEG signals in the education context, opening the path for educational automation research and development.

Teacher and student behavior during class is often observed by education professionals to evaluate and develop a teacher’s skill, adapt lesson plans, or monitor and regulate student learning and other activities. Traditional methods rely on accurate manual techniques involving in-person field observations, questionnaires, or the subjective annotation of video recordings. These techniques are time-consuming and typically demand observation and coding by a trained professional. Thus, developing automated tools for detecting classroom behaviors using artificial intelligence could greatly reduce the resources needed to monitor teacher and student behaviors for research, practice, or professional development purposes. This paper presents an automated framework using a deep learning approach to recognize classroom activities encompassing both student and teacher behaviors from classroom videos. The proposed method utilizes a long-term recurrent convolutional network (LRCN), which captures the spatiotemporal features from the video frames. For evaluation purposes, experiments were carried out on a subset of EduNet and an independent dataset composed of classroom videos collected from the internet. The proposed LRCN system achieved a maximum average accuracy (ACC) of 93.17%, precision (PRE) of 94.21%, recall (REC) of 91.76%, and F1-Score (F1-S) of 92.60% on the EduNet dataset when estimated by 5-fold cross-validation. The system provides ACC =83.33%, PRE =89.25%, REC =83.32%, and F1-S =82.14% when applied to independent testing, which ensures reliability. The study has significant methodological implications for the automated recognition of classroom activities and may assist in providing information about classroom behaviors that are worthy of inclusion in the evaluation of education quality.

Electroencephalogram (EEG) based emotion classification reflects the actual and intrinsic emotional state, resulting in more reliable, natural, and meaningful human-computer interaction with applications in entertainment consumption behavior, interactive brain-computer interface, and monitoring of psychological health of patients in the domain of e-healthcare. Challenges of EEG-based emotion recognition in real-world applications are variations among experimental settings and cognitive health conditions. Parkinson's Disease (PD) is the second most common neurodegenerative disorder, resulting in impaired recognition and expression of emotions. The deficit of emotional expression poses challenges for the healthcare services provided to PD patients. This study proposes 1D-CRNN-ELM architecture, which combines one-dimensional Convolutional Recurrent Neural Network (1D-CRNN) with an Extreme Learning Machine (ELM), robust for the emotion detection of PD patients, also available for cross dataset learning with various emotions and experimental settings. In the proposed framework, after EEG preprocessing, the trained CRNN can use as a feature extractor with ELM as the classifier, and again this trained CRNN can be used for learning of new emotions set with fine-tuning of other datasets. This paper also applied cross dataset learning of emotions by training with PD patients datasets and fine-tuning with publicly available datasets of AMIGOS and SEED-IV, and vice versa. Random splitting of train and test data with 80 − 20 ratio resulted in an accuracy of 97.75% for AMIGOS, 83.20% for PD, and 86.00% for HC with six basic emotion classes. Fine-tuning of trained architecture with four emotions of the SEED-IV dataset results in 92.5% accuracy. To validate the generalization of our results, leave one subject (patient) out cross-validation is also incorporated with mean accuracies of 95.84% for AMIGOS, 75.09% for PD, 77.85% for HC, and 84.97% for SEED-IV is achieved. Only a 1 − sec segment of EEG signal from 14 channels is enough to detect emotions with this performance. The proposed method outperforms state-of-the-art studies to classify EEG-based emotions with publicly available datasets, provide cross dataset learning, and validate the robustness of the deep learning framework for real-world application of psychological healthcare monitoring of Parkinson's disease patients.