Comparative evaluation of convolutional neural networks (CNNs) and ensemble models for focal versus nonfocal epilepsy detection using fluid-attenuated inversion recovery magnetic resonance imaging (FLAIR MRI) slices

Introduction

Background

Epilepsy is a neurological disorder characterized by recurrent seizures and heterogeneous clinical manifestations among people of all ages (1). Globally, the World Health Organization (WHO) reports more than 50 million people living with epilepsy (2). In the United States, approximately 150,000 new cases of epilepsy are diagnosed each year (3). Sudden Unexpected Death in Epilepsy (SUDEP) causes the unexplained death of more than one in every 1,000 people with epilepsy each year. To reduce SUDEP risk, patients are advised to follow strategies that support managing seizures in addition to taking medications and available treatments (4).

According to the Centers for Disease Control and Prevention (CDC), seizures can be generalized, focal, or unclassified, as a person with epilepsy may experience a variety of symptoms (5). While seizures arise from a particular lobe of the brain in focal epilepsy (6), generalized seizures begin in both hemispheres of the brain at the same time (7). Around 60% of patients experience focal seizures, which are classified and named based on the brain region of the seizure onset. For example, most patients with focal seizures are diagnosed with temporal lobe epilepsy (TLE), as repetitive temporal lobe seizures are frequently linked to shrinkage and scarring in the hippocampus region (3).

According to the Mayo Clinic, several evaluation methods support the diagnosis of epilepsy, including magnetic resonance imaging (MRI), which supports the detection of brain lesions related to epilepsy (8). However, interpreting medical imaging scans is a complex task that requires substantial time and expertise. In addition to the costs and resources required for specialized training for individuals, a considerable number of healthcare practitioners lack the necessary training to identify, diagnose, and manage this disease (2). In this context, deep learning (DL) techniques come into play, offering automated tools to analyze medical images by detecting subtle patterns that may be difficult even for experts to identify (9).

Among these techniques, convolutional neural networks (CNNs) are now widely utilized for medical image classification (10). Unlike traditional machine learning (ML) models that require manual feature extraction, CNNs are inherently designed to automatically recognize spatial patterns of structural abnormalities (11). Furthermore, CNN architectures seamlessly support transfer learning, which is a critical advantage when trying to achieve enhanced performance on limited clinical datasets. Accordingly, a baseline CNN trained from scratch was evaluated and compared with two pretrained CNN models, ResNet50 and Xception41, to assess whether transfer learning enhances the classification performance.

Rationale and knowledge gap

Many ML studies tackling epilepsy detection lean heavily on T1-weighted MRI sequences (12-14), or on multimodal approaches that combine T1-weighted imaging with other sequences (15,16). Fluid-attenuated inversion recovery (FLAIR) sequence has demonstrated clinical value in improving the visibility of abnormal brain tissues (17) and demonstrated greater sensitivity to certain lesions than conventional MRI sequences (18). However, it has rarely been used as primary input for epilepsy detection in DL research. Similarly, when it comes to classification tasks, studies tend to compare focal epilepsy against healthy controls (12,14), leaving the focal and nonfocal epilepsy differentiation relatively underexplored. In this study, nonfocal refers to patients who are clinically diagnosed with epilepsy but either exhibit generalized seizure manifestations or have MRI-negative findings.

Publicly available MRI datasets for epilepsy remain limited, as many are restricted due to privacy concerns. This gap motivated the use of real-world hospital data in this study, where subtype labels were assigned based on expert clinical judgment to ensure reliable annotations.

In focal epilepsy specifically, FLAIR imaging has shown an ability to reveal lesions and structural abnormalities that might be undetectable on conventional T1- and T2-weighted sequences, contributing to approximately 30% improvement in MRI diagnostic yield (19). This has been proven by the study (20), showing that ResNet50 achieved a higher mean accuracy in detecting focal cortical dysplasia (FCD) type II when trained on FLAIR images rather than on T1-weighted images. Accordingly, this study utilized FLAIR MRI as the primary imaging sequence for training the DL models. To the best of our knowledge, only a limited number of studies have investigated slice-based FLAIR MRI for distinguishing focal from nonfocal epilepsy using real-world clinical data.

Objective

The objectives of this study can be summarized in two main points:

• Collection of a real-world FLAIR MRI dataset from routine hospital clinical settings, categorized into focal or nonfocal cases based on expert radiological review.
• Systematic comparative analysis of three DL models followed by an ensemble learning approach.

Methods

In this study, the methodology begins with data collection and labelling, followed by data preprocessing, DL model implementation, and concludes with model evaluation.

Data collection and labelling

Study design and study population

This study included patients with a prior clinical diagnosis of epilepsy who were referred for epilepsy-protocol MRI examination accordingly, based on medical history/semiology and neurologists’ diagnosis. Patients’ records were distributed across two systems: picture archiving and communication system (PACS) and Sectra. Accordingly, available MRI scans and corresponding imaging reports were retrospectively retrieved from both systems. The radiological reports contained clinical indication/details provided for the MRI examination, patients’ history when available, along with the King Abdulaziz University Hospital (KAUH) radiologist’s imaging findings and impressions that provided supplementary information that served the MRI annotation process. Cases without an available FLAIR sequence were excluded, as it does not serve the purpose of the study.

This study collected 8,433 FLAIR brain MRI slices of 175 adolescent and adult participants aged ≥14 years, with heterogeneous radiological reports at a university hospital in Jeddah, Saudi Arabia. The researchers had no contact with participants.

Accurate data labeling is a crucial step to avoid the risk of training the model based on incorrect information. Accordingly, two expert radiologists from different hospitals were recruited to conduct the annotation process for greater reliability in data classification.

The first annotator is a Radiology Consultant at King Fahad Armed Forces Hospital, Jeddah, whereas the second annotator is the Director of Radiology at King Abdullah Medical City, Mecca. MRI scans were manually analyzed using a Digital Imaging and Communications in Medicine (DICOM) viewer, and both annotators considered the MRI images and radiological reports when classifying the patients. Furthermore, an Excel file containing three columns was provided to the annotators. The first column listed the patients’ file numbers. The second column was designated for patient-level labeling, where each case was classified as either Focal or Nonfocal. The third column was allocated for the annotators’ comments and clinical observations of each case. Both radiologists reached an agreement in 175 cases, while the remaining cases, for which the radiologists disagreed, were excluded from the study to avoid ambiguity and ensure data reliability.

For the focal category, the final dataset includes selected MRI slices from both preoperative and postoperative cases, where structural abnormalities related to focal lesions can be identified. It is worth mentioning that some focal cases showed both focal lesions and other abnormalities related to other brain diseases, such as Alzheimer disease (AD). Accordingly, MRI slices mainly showing abnormalities unrelated to epilepsy were excluded as well. On the other hand, the nonfocal category includes patients who are clinically diagnosed with epilepsy and exhibit symptoms of generalized epilepsy or have non-lesional MRI findings. Ultimately, a total of 8,433 MRI images were included in the final dataset, including 110 patients in the nonfocal class with 6,291 original MRI images, whereas 65 patients for Focal category with 2,142 original images.

Inclusion criteria

During the annotation process, the collected cases were categorized into three diagnostic groups based on the imaging findings.

Nonfocal group

This group included patients with no detectable structural abnormalities, no abnormal cortical or subcortical signal intensity, and symmetric hippocampi without abnormal FLAIR signal. These cases were classified as MRI-negative epilepsy, representing individuals without radiologically apparent epileptogenic substrates, with classification based primarily on MRI assessment and guided by available radiological reports.

For instance, Figure 1 presents a radiological report and the accompanying FLAIR MRI slices collected from a patient classified as nonfocal. Although the clinical information stated in the report suggested right TLE, no structural abnormality was identified on FLAIR MRI by either the original radiologist at KAUH or the two external radiology experts, as shown in the representative report. Therefore, within the image-based classification framework adopted in this study, the case was assigned to the nonfocal category. This multi-stage review process increased the robustness of the image-based classification and helped minimize labeling inconsistencies.

Figure 1 Example of a nonfocal epilepsy case demonstrating normal FLAIR MRI findings despite clinical details suggestive of right temporal lobe epilepsy. (A) Representative FLAIR MRI slices extracted from a nonfocal case after CLAHE application, showing no identifiable structural abnormality. (B) Corresponding radiological report indicating clinical information suggestive of right temporal lobe epilepsy. CLAHE, contrast limited adaptive histogram equalization; FLAIR, fluid-attenuated inversion recovery; MRI, magnetic resonance imaging.

Focal group

This group included patients demonstrating focal structural abnormalities potentially related to epileptogenesis. These abnormalities included space-occupying lesions, vascular malformations, chronic cerebrovascular changes, congenital structural abnormalities, mesial temporal sclerosis (MTS), encephalomalacia or gliotic changes, demyelinating lesions, significant cerebral or cerebellar atrophy, and postoperative or post-traumatic alterations.

Although the spectrum of abnormalities varied, all demonstrated focal FLAIR signal changes suggestive of a possible epileptogenic focus. For instance, Figure 2 presents several FLAIR MRI slices extracted from a postoperative case. The patient has undergone epilepsy surgery to remove a part of the right temporal lobe, but a portion of the damaged tissue responsible for the seizures—known as MTS—remains in the brain. The following slices were classified as focal to train the models to identify all structural brain abnormalities associated with epilepsy. In contrast, all normal slices for this focal case were excluded to avoid label noise.

Figure 2 FLAIR MRI slices extracted from a postoperative focal case (after CLAHE application). CLAHE, contrast limited adaptive histogram equalization; FLAIR, fluid-attenuated inversion recovery; MRI, magnetic resonance imaging.

Another example is represented in Figure 3, which demonstrates several FLAIR MRI slices extracted from a patient with FCD.

Figure 3 FLAIR MRI slices extracted from a focal case with FCD (after CLAHE application). CLAHE, contrast limited adaptive histogram equalization; FCD, focal cortical dysplasia; FLAIR, fluid-attenuated inversion recovery; MRI, magnetic resonance imaging.

Others (incidental findings)

This category included findings unrelated to epilepsy that were incidentally identified in some MRI slices, such as age-associated cerebral changes and microangiopathy, vascular dolichoectasia, benign congenital cysts, and middle ear or skull findings without neurological significance. These findings were not considered contributory to seizure activity.

MRI slice extraction

Since the dataset was developed primarily for research purposes to train DL models for binary classification and abnormality detection, the MRI images were manually extracted in PNG format by the radiologist. For the focal category, only slices with lesions related to epileptic seizures were included, whereas all MRI slices classified under the other categories were excluded. For the Nonfocal category, the entire series of each case was included. Slices that lacked sufficient anatomical content, such as those containing minimal brain coverage or displaying no visible signal, were excluded during preprocessing to maintain data quality.

Data organization and split

To prevent data leakage, MRI images were split on a patient level into two sets: 80% for training and 20% for testing, with 10-fold group cross-validation performed on the training set, using 10% of each fold for validation. Table 1 below provides better insights into the distribution of MRI slices across patients.

As shown in Table 1, the nonfocal class included 88 cases for training (5,119 slices) and 22 cases for testing (1,172 slices). On the other hand, the focal class included 52 cases (1,803 slices) for training and 13 cases for testing (339 slices). The final dataset demonstrates severe imbalance in both training and testing sets in terms of MRI slices. To address this issue, data augmentation was applied to the training set. In addition, performance metrics are reported per class for the testing set to ensure a more accurate representation of the model’s performance.

Data preprocessing

Several preprocessing steps were conducted on the whole dataset, starting with applying Contrast Limited Adaptive Histogram Equalization (CLAHE) technique on the whole dataset with a clip limit of 2.0 and a grid size of 8×8, as utilized in the study (21). Figure 4 presents samples of the original FLAIR MRI slices before and after applying the CLAHE technique.

Figure 4 Samples of FLAIR MRI slices from (A) a focal case with a focal lesion in left TLE and (B) a nonfocal case with no identified abnormalities, before and after applying CLAHE technique. CLAHE, contrast limited adaptive histogram equalization; FLAIR, fluid-attenuated inversion recovery; MRI, magnetic resonance imaging; TLE, temporal lobe epilepsy.

Figure 4A presents representative slices from a focal case with a focal lesion in the left temporal lobe, whereas Figure 4B presents a nonfocal case with no identified abnormalities in the MRI scan. Images on the left show original FLAIR MRI slices, whereas those on the right show the corresponding slices after CLAHE preprocessing.

To maintain consistency across all models, all input images were resized to 256×256 pixels, converted into tensors, and normalized using standard ImageNet mean and standard deviation values.

Image augmentation

Looking at the raw class distribution, nonfocal images represent approximately 75% of the entire dataset, whereas focal images represent only 25%, indicating a high imbalance between the two classes. To address this, online augmentation was applied during training to increase the number of focal slices, while maintaining the balance between both classes.

To investigate the most effective data augmentation techniques for medical imaging applications, a previous study (22) compared eight techniques on brain T1-weighted and FLAIR scans of low-grade glioma patients. The approaches conducted include flipping (vertical and horizontal), noise, rotations (90° and 180°), shearing, blurring, cropping and scaling. According to the findings, rotation outperformed other techniques in enhancing the model performance under limited data conditions. Accordingly, this study applied 90° and 180° rotations to minimize the risk of overfitting.

Furthermore, elastic deformation has been widely applied as an augmentation technique to address a limited amount of data in the training set and enhance the network’s ability to learn invariance and simulate realistic deformations (23). Accordingly, elastic deformation was also applied with α and σ of 30 and 5, respectively, where α controls the magnitude of the deformation and σ represents the degree of Gaussian smoothing applied to the deformation field (24).

DL models implementation

Three DL models were implemented for focal and nonfocal epilepsy classification: a baseline CNN, Xception41, and ResNet50. To improve generalization beyond what any single model could achieve, an ensemble approach was also applied.

Models’ architecture

CNN architecture

A simple CNN architecture was employed for MRI classification. The network takes RGB images of size 256×256 as input and consists of three convolutional blocks for feature extraction, each following a Conv(3×3)-BatchNorm-rectified linear unit (ReLU)-MaxPool structure, with filter sizes of 64, 128, and 256, respectively, across the three blocks. This progressive increase in channel depth allows the network to capture increasingly abstract features, while max pooling reduces spatial resolution from 256×256 down to 32×32. After feature extraction, the flattened feature maps pass through four fully connected (FC) layers that progressively reduce dimensionality from 256×32×32 to 256, 128, and 64, before a final layer maps the features to two output classes: Focal and Nonfocal. The first three FC layers are each followed by ReLU activation, with dropout (P=0.5) applied after the first FC layer to reduce overfitting. The final layer produces output scores for each class without activation. No pretrained weights were used, as the model was trained end-to-end from scratch.

ResNet50 architecture

ResNet50 was employed with weights pretrained on ImageNet (25,26). To adapt the model to the binary classification task, the original FC layer was replaced with a dropout layer (P=0.4) followed by a new FC layer mapping features to two output classes: Focal and Nonfocal. The remaining layers were fine-tuned during training. The entire network, including the pretrained backbone, was fine-tuned end-to-end during training, allowing the learned representations to adapt to the characteristics of the MRI data.

Xception41 architecture

An Xception41 model based on depthwise separable convolutions, with weights pretrained on ImageNet (27), was implemented and fine-tuned for binary classification. The architecture processes input images through a Stem section, followed by Entry Flow, Middle Flow, and Exit Flow stages that progressively extract hierarchical features at increasing depths. The original 1,000-class output layer was replaced with a fully connected layer mapping features to two output classes: Focal and Nonfocal.

Ensemble learning

In this study, the ensemble approach combines the predictions of the three trained classifiers: CNN, ResNet50, and Xception41. The ensemble applies soft voting by averaging the softmax probability outputs of all three models and selecting the class with the highest mean probabilities as the final prediction. This strategy enhances robustness and generalization by combining diverse model decisions without introducing additional trainable parameters.

Training procedures

The models were trained using the cross-entropy loss function. The Adam optimizer was employed with an initial learning rate of 0.001 for the CNN model and 0.0001 for the pre-trained ResNet50 and Xception41 models. The batch size was set to 32 for the CNN and ResNet50 models, and 16 for the Xception41 model. Training was conducted for 10 epochs with an early stopping criterion based on the macro recall on the validation set, using a patience of five epochs to prevent overfitting. Moreover, gradient clipping with a maximum threshold limit of one was also used during training to prevent dramatic weight updates that may affect training stability, as proposed by (28).

A weighted random sampling strategy using WeightedRandomSampler was applied to ensure data balance during training. A data transformation module (RandomChoice) was also implemented to randomly apply a single transformation to each input. Four transformations were included: rotation at 90°, rotation at 180°, elastic deformation, and a lambda transformation, in which the MRI slice is retained in its original form. As shown in Table 2 below, each transformation had an equal probability (25%) of being applied to the MRI slice.

Evaluation

Validation strategy

In this study, 10-fold group cross-validation was implemented to evaluate models’ performance, while splitting data at the patient level to prevent data leakage and ensuring that patients’ slices do not appear in both training and testing sets. For each fold, the model was trained independently, and the epoch achieving the highest macro-average recall on the validation set was selected as the optimal state for that fold. The corresponding model weights were saved for evaluation.

Confusion matrices

Based on the confusion matrix for binary classification, which includes true positives (TPs), true negatives (TNs), false positives (FPs), and false negatives (FNs), several metrics can be derived to evaluate model performance from different perspectives. This study incorporates the following metrics (29).

Accuracy

The accuracy metric evaluates the model in terms of overall correctness. In other words, the evaluation is conducted based on the number of correct predictions of instances in the testing set. The accuracy value can be calculated as below:

Accuracy=TP+TNTP+FP+TN+FN[1]

Precision (p)

A metric used for calculating the correct predictions of the positive instances out of the overall predictions of instances in the positive class. The precision value can be calculated as below:

Precision=TPTP+FP[2]

Recall (r)

A metric used for measuring the portion of positive instances that were rightly predicted. The recall value can be calculated as below:

Recall=TPTP+FN[3]

F-measure

It is a harmonic mean between the values of two evaluation metrics, which are precision and recall. The f-measure value can be calculated as below:

F−Measure=2×p×rp+r[4]

Receiver operating characteristic (ROC) curve

The ROC curve is constructed by tabulating the values of both sensitivity and specificity produced by continual test measures (30). While the ROC curve offers information about the model’s performance, the area under the curve (AUC) is used to summarize it.

AUC is a well-known metric that has been used in medical diagnosis (31). It is a universal metric of how well a test can distinguish between positive and negative cases of a specific condition with a specific condition (30). The AUC facilitates the comparison between the ROC curves, and its value can be calculated as below (29):

AreaUndertheROCCurve=Sp−nP(nn+1)/2nPnn[5]

Where Sp represents the total number of positive instances ranked, np represents the number of positive instances, and nn represents the number of negative instances.

Statistical analysis

The performance of the models was assessed using several evaluation metrics, including precision, recall, and F1-score. To explore the models’ performance in identifying each class and their overall performance, these metrics were calculated for individual classes (focal and nonfocal) and reported as macro-averages. The area under the receiver operating characteristic curve (ROC-AUC) was also computed as an overall measure of discrimination between the two classes. Since each patient has multiple MRI slices in the independent test set, we estimated 95% confidence intervals using patient-level bootstrap resampling to account for this clustered data. Model calibration was also assessed using the expected calibration error (ECE), which was visualized using reliability diagrams.

Threshold selection

In this study, we prioritize detecting focal cases rather than nonfocal ones. Accordingly, it is assumed that misclassifying an image containing a focal lesion is twice as costly as misclassifying a nonfocal slice. Therefore, a cost-sensitive thresholding approach was adopted, aiming to minimize the cost of misclassification rather than accuracy maximization. The standard threshold of 0.5 was lowered to 0.33 on both validation and testing sets, increasing the models’ sensitivity to the focal category. In other words, a 33% focal probability is sufficient to classify the image as focal due to subtle lesions of focal epilepsy.

Ethical consideration

The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by Institutional Review Board (IRB) of King Abdulaziz University (KAU), Jeddah, Saudi Arabia (reference No. 648-23, dated December 5, 2023). Participants recruitment involved a retrospective record review approach, whereby existing medical records from KAU Hospital are accessed and analyzed after obtaining consent from the hospital and necessary ethical approvals. Individual participants are not directly contacted or involved in the study. All patients’ images were extracted while excluding personal information out of the slices to ensure confidentiality.

Results

Models’ performance on validation set

For evaluating the models’ validation performance during the training process, the metrics of all epochs with the highest macro average recall per fold were averaged and summarized in Table 3.

According to Table 3, CNN showed the lowest validation performance, across all metrics, with 73.23% balanced accuracy, 73.23% recall, 67.89% precision, and the lowest ROC-AUC of 78.33%. On the other hand, Xception41 achieved higher performance with a balanced accuracy and recall of 81.96%. While ResNet50 achieved the highest discrimination ability, with an ROC-AUC of 88.37%, as well as the highest balanced accuracy and recall of 82.06%, Xception41 achieved the highest precision and F1-score, at 79.94% and 80.06%, respectively.

Models’ performance on testing set

After saving the models at the epoch with the highest macro average, they were evaluated on the unseen testing set at the slice level. As mentioned in Table 4, the testing set includes 22 nonfocal cases (1,172 slices) and 13 focal cases only (339 slices), reflecting real-life scenarios when identifying an imbalanced number of patients and healthy controls. To address this issue, some performance metrics, such as accuracy, were not considered as a standard for evaluation due to the imbalance of slices in both classes. In other words, this study focuses on presenting testing performance per class for a more accurate representation of models’ capabilities.

According to the reported results, CNN showed poor performance with the lowest macro average and nonfocal metrics. Despite the highest focal recall achieved by CNN, the model still showed the lowest focal precision and a poor discrimination ability with a ROC-AUC of 69.16%. Accordingly, CNN model showed overall poor classification performance.

On the other hand, ResNet50 and Xception41 showed comparable ROC-AUC values of 83.76% and 83.29%, respectively. However, their performance varied across the remaining metrics. ResNet50 achieved the highest macro average precision and F1-score of 80.96% and 79.19%, respectively, in addition to the highest overall performance for the nonfocal class with a 93.09% recall and 91.30% F1-score. Nevertheless, ResNet50 showed a lower macro average recall and focal recall of 77.81% and 62.54%, respectively, demonstrating a larger proportion of missing focal samples. In contrast, Xception41 achieved the highest macro average recall of 80.40% and overall good performance for both classes, achieving 74.63% focal recall and 86.18% nonfocal recall with a precision of 92.15% for the nonfocal class. Despite the lower focal precision achieved by Xception41 compared to ResNet50, the focal F1-score of the two pretrained models were nearly identical, with 67.11% for Xception41 and 67.09% for ResNet50. This suggests a trade-off between the models, where Xception41 showed more sensitivity to focal slices, whereas ResNet50 achieved a higher focal precision and a higher macro-average F1-score.

Meanwhile, the ensemble model achieved the highest ROC-AUC of 83.94% and the second-highest focal recall of 76.99% compared to other models. Despite the high focal sensitivity, the ensemble achieved a low focal precision of 51.79%, resulting in a focal F1-score of 61.92%. Similarly, the ensemble showed an 85.27% nonfocal F1-score and 79.27% nonfocal recall, despite achieving the highest nonfocal precision of 92.25%. Consequently, the ensemble model did not demonstrate the strongest overall performance for both classes. This study mainly focuses on detecting lesions related to focal epilepsy through minimizing missed focal cases. As a result, the Xception41 model can be considered the most appropriate model to serve the purpose of this study, due to its high sensitivity to focal cases, in addition to the highest macro average recall of 80.40%.

Figure 5 presents the ROC curves for all models evaluated on the testing set. As demonstrated, CNN model showed the lowest discrimination ability among all models, with a ROC-AUC of 69.16% (95% CI: 55.40–80.50%). In contrast, the ensemble model achieved the highest ROC-AUC of 83.94% (95% CI: 69.80–93.80%), followed by ResNet50 with 83.76% (95% CI: 71.30–93.70%) and Xception41 with 83.29% (95% CI: 68.90–92.90%). Overall, all models’ performance achieved ROC-AUC values above 0.5, indicating learned discrimination between focal and nonfocal slices.

Figure 5 Models’ ROC-curves. AUC, area under the curve; CI, confidence interval; ROC, receiver operating characteristic.

Figure 6 presents the confusion matrices of all models, presenting values of TP, TN, FP, and FN at the slice level. Figure 6 shows that each model handled focal and nonfocal slices differently. The CNN model showed the highest sensitivity to focal slices with a focal recall of 79.35%, but produced a high number of FP predictions, showing lower reliability for nonfocal slices. In contrast, ResNet50 showed stronger performance in detecting nonfocal slices, but showed poor sensitivity to focal slices. Xception41, on the other hand, showed the most balanced classification performance, as it achieved high focal sensitivity and a stable performance in identifying nonfocal slices. Although the ensemble model combined all the models’ predictions, the Xception41 model showed more consistent classification performance for both categories.

Figure 6 Comparison of confusion matrices across the evaluated architectures. (A) CNN, (B) ResNet50, (C) Xception41, and (D) Ensemble model. CNN, convolutional neural network.

To measure how reliably we can trust the models’ predicted probabilities, this paper presented calibration curves and ECE values in Figure 7. It is worth recalling that well-calibrated models follow the dashed diagonal line to show the alignment between the models’ confidence and the true observed frequencies.

Figure 7 Calibration curves of the evaluated models. CNN, convolutional neural network; ECE, expected calibration error.

As shown in Figure 7, ResNet50 showed the most reliable scores, achieving the lowest ECE with a value of 0.0754 and staying closest to the dashed line compared to other models. This was followed by the ensemble model, which achieved an ECE of 0.1206. Both models demonstrate greater stability along the perfect calibration line. On the other hand, the CNN model and Xception41 showed overconfidence by assigning high probability scores to nonfocal slices, resulting in the highest error with an ECE of 0.2340 for the CNN model, followed by Xception41 with an ECE of 0.1380.

Since calibration measures the reliability of the model’s predicted probabilities, differentiating between classification performance and calibration remains essential. While the ensemble and ResNet50 produced more reliable probability predictions, Xception41 provided a higher balance in detecting both focal and nonfocal slices, as shown in the confusion matrix. Therefore, this study considers Xception41 model as the most appropriate model due to its high sensitivity to focal slices, despite the slight overconfidence in its probability scores for some cases.

Discussion

Model performance and dataset characteristics

Deep CNNs are now widely used in medical imaging for classification, segmentation, and object localization (10). They have shown strong performance in feature extraction and image classification (11), and they have been increasingly applied to epilepsy diagnosis across several neuroimaging modalities, including electroencephalography (EEG) (32). Building on this progress, the present study applied CNN-based architectures to classify FLAIR MRI slices into focal and nonfocal epilepsy. Rather than replacing the diagnostic workflow, this study aims to evaluate the performance of DL models using FLAIR MRI alone, without incorporating other MRI sequences or additional clinical information, in an image-based classification setting. Although multimodal approaches may achieve better performance, understanding the contribution of FLAIR MRI alone remains an important research question.

In this study, three CNN architecture were trained to classify MRI slices into focal and nonfocal categories, and their predictions were combined using an ensemble approach. While the lightweight baseline CNN achieved the highest sensitivity for focal slices, it produced a high number of FPs, resulting in the weakest overall classification performance and the poorest probability calibration. This highlights the challenges of training a network from the scratch on a small single dataset without producing overconfident and unreliable predictions. In contrast, the pretrained models demonstrated a stronger classification performance. ResNet50 showed higher sensitivity to nonfocal slices and achieved more reliable probability scores. Meanwhile, the ensemble achieved the highest ROC-AUC, indicating a good overall classification performance, supported by strong confidence intervals. However, the Xception41 offered greater balance and high sensitivity to focal slices while maintaining stability in detecting the nonfocal ones. The detailed slice-level annotation strategy, together with the use of pretrained architectures, likely contributed to the overall promising performance. Nonetheless, the limited dataset size remains a major constraint and suggests that larger and more diverse training data could further improve both classification balance and calibration reliability.

DL algorithms generally require large amounts of data, as both dataset size and the number of healthy controls can substantially affect model performance. Limited data availability, in turn, increases the risk of overfitting or underfitting during training (22). However, acquiring large, well-annotated medical imaging datasets is challenging because of privacy regulations, heterogeneous imaging protocols, and restricted data sharing. In this context, the present study constructed a real-world FLAIR MRI dataset from KAUH, categorized into focal and nonfocal cases based on independent expert review by two radiology specialists. Although all patients were clinically diagnosed with epilepsy, they displayed distinct MRI findings in this dataset: the focal category comprised positive FLAIR MRIs with abnormalities related to focal epilepsy, whereas the nonfocal category comprised only negative FLAIR MRIs.

Comparison with existing literature

The proposed work shares several similarities with recent efforts in the literature. Lasagni et al. (33) evaluated 3D-CNNs (ResNet18, ResNet34, and ResNet50) for detecting FCD type II, adapting the original 2D-ResNet architecture into a full-brain 3D-ResNet classification model. To improve generalizability, they applied random rotation data augmentation within a range of ±15 degrees. Their models were trained from scratch and subsequently compared with models initialized using two transfer learning strategies based on Med3D weights provided by Chen et al. (34): one pretrained on a lung CT segmentation task and another on 23 segmentation tasks, including brain tumor segmentation, with fine-tuning restricted to the final block and layer in both cases.

Lasagni et al. (33) showed that applying transfer learning improved classification performance and increased accuracy, regardless of the specific ResNet variant and MRI modality, particularly when using weights pretrained on tasks that were closely aligned with the study objective. Compared with training models from scratch, transfer learning enhanced both optimization and final accuracy. When ResNet50 was trained on FLAIR images from scratch, it achieved a mean accuracy of 72.9%, which increased slightly to 73.5% when using weights pretrained on the single segmentation task. A more substantial improvement was observed when weights pretrained on 23 segmentation tasks were used, yielding a mean accuracy of 80.3% and a mean ROC-AUC of 0.817. This performance is notably close to the ROC-AUC of 0.816 achieved by the ensemble model in the present study.

In another related work, the study (35) evaluated six pretrained CNN architectures—VGG16, VGG19, Xception, Inception-ResNetV2, DenseNet201, and MobileNetV2—for FCD type II detection, with all the models initialized using ImageNet weights. Among these, Xception achieved the highest performance on 2D FLAIR images when combining axial, sagittal, and coronal planes, with 92.1% accuracy, 93.7% weighted precision, and 84.4% weighted F1-score. This finding is broadly consistent with our results, in which pretrained Xception41 showed the highest discriminative ability between focal and nonfocal cases, achieving a ROC-AUC of 0.843.

Despite these similarities, important differences exist between previous studies and the current work. Both Lasagni et al. (33) and the study in (35) used an open-access MRI dataset from the Life and Brain Center (Bonn), which includes T1-weighted and FLAIR scans from 85 patients with FCD type II and 85 age- and sex-matched healthy controls. The availability of balanced classes and matched controls in that dataset provides favorable conditions for model training and evaluation.

In contrast, the dataset constructed in the present study included 110 nonfocal and 65 focal patients, with a variable number of slices per patient. This class imbalance complicates model training and evaluation. To address this, performance metrics were reported separately for each class rather than relying on overall accuracy, which can be misleading in imbalanced settings. Despite these constraints, the performance achieved in this study is broadly comparable to that reported in the above-mentioned works, supporting the feasibility of using pretrained CNN architectures for MRI-based epilepsy classification even in smaller, imbalanced, single-center datasets.

Overall, our findings and those of related studies converge on the promising potential of CNN-based architectures for detecting focal epilepsy using FLAIR MRI slices. However, the relatively weak performance observed when training a lightweight network from scratch reinforces the importance of transfer learning and model capacity. Despite the limited size and single-source nature of our dataset, the use of pretrained models, combined with both original and augmented FLAIR images, appears to have enhanced feature extraction and improved classification performance.

Study limitations and future directions

Despite the promising results, the reported performance should be interpreted within the defined scope of the study. Moreover, no external validation on an independent cohort was performed, and the analysis relied on a relatively small, single-center dataset. Therefore, the generalizability of the models to data from external sites remains uncertain. Future work should include multi-center external validation using complete MRI examinations and patient-level evaluation to better reflect real-world diagnostic workflows and assess clinical applicability and model generalizability.

Moreover, lesions in focal epilepsy are non-uniformly distributed and vary considerably in location and size (36). As a result, larger and more heterogeneous training sets that capture this variability are likely to further improve lesion detection. Future work should therefore prioritize addressing the limitations of dataset size and diversity by exploring additional pretrained DL models instead of training shallow models from scratch and incorporating multi-center or multi-source datasets to improve model generalizability to different lesion patterns, in addition to applying more extensive augmentation strategies, such as generating synthetic MRI images to enhance focal lesion detection. As found by Kaur et al. (37), incorporating synthetic images improved the sensitivity of FCD detection in a low-data setting and demonstrated the potential to reduce the need for manual data labeling.

Future research could also investigate DL frameworks that utilize alternative modalities, such as T1-weighted imaging, in the absence of FLAIR, or integrate information from multiple MRI sequences to enhance both clinical applicability and diagnostic accuracy. Moreover, many existing studies (12,33,38) focus on a single epilepsy subtype, which restricts the generalizability of their findings across broader epilepsy syndromes. Extending the present approach to multiclass classification of different epilepsy subtypes, while examining the most discriminative imaging features learned by the models and incorporating interpretability methods, may provide deeper insights into the imaging biomarkers and structural abnormalities underlying epilepsy and support more transparent clinical decision-making.

Conclusions

In conclusion, the findings demonstrate that slice-level data labeling, augmentation strategies, 2D pretrained CNN models, a lowered classification threshold of 0.33, and training models on FLAIR MRI sequence collectively contribute to improving focal and nonfocal epilepsy detection, even with a limited single-source dataset. However, the conducted evaluation revealed different diagnostic trade-offs. While the ensemble achieved the highest overall discrimination with a ROC-AUC of 83.94% and ResNet50 provided the highest reliability in probability calibration (ECE =0.0754), Xception41 was the most practical model for the proposed study, as it showed a more balanced classification performance. It demonstrated high sensitivity to the minority focal class while preserving stable performance in detecting the nonfocal slices, reflecting the effectiveness of applying transfer learning in low-data medical imaging settings. Conversely, training a baseline CNN from scratch resulted in high focal sensitivity accompanied by excessive FPs and unreliable, overconfident predictions. Overall, these results suggest that strategic preprocessing, threshold tuning, and careful model selection are critical factors in developing a reliable, image-based epilepsy classification framework that could support clinical decision-making.

Acknowledgments

The authors would like to express their sincere gratitude to Abdulhaleem A. Almarghalani for his generous technical support throughout this work. He kindly provided access to the high-performance computing resources necessary for running the experiments, and offered his time and energy assisting with debugging and troubleshooting during the implementation phase. His support was invaluable in keeping the work moving forward. The authors also thank Majed T. Osaylan for his early coordination efforts in this study. Although he was unable to directly participate in the data collection phase, he was instrumental in identifying and preparing a qualified replacement, ensuring continuity in the data extraction process. The authors sincerely thank Sara Alreeshi for her valuable assistance with data collection. During the preparation of this manuscript, OpenAI ChatGPT has been used to improve the clarity, grammar, and readability of the written manuscript.

Footnote

Data Sharing Statement: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-2026-0098/dss

Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-2026-0098/prf

Funding: This research work was funded by Institutional Fund Projects (No. IFPRC-186-140-2020). Therefore, the authors gratefully acknowledge financial support from the Ministry of Education and King Abdulaziz University, Jeddah, Saudi Arabia.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-2026-0098/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by Institutional Review Board (IRB) of King Abdulaziz University (KAU), Jeddah, Saudi Arabia (Reference No. 648-23, dated December 5. 2023). Participants recruitment involved a retrospective record review approach, whereby existing medical records from KAU Hospital are accessed and analyzed after obtaining consent from the hospital and necessary ethical approvals. Individual participants are not directly contacted or involved in the study. All patients’ images were extracted while excluding personal information out of the slices to ensure confidentiality.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Mayo Clinic. [cited 2023 Jun 21]. Epilepsy - Symptoms and causes. Available online: https://www.mayoclinic.org/diseases-conditions/epilepsy/symptoms-causes/syc-20350093.
2. Epilepsy. [cited 2023 Jun 22]. Available online: https://www.who.int/health-topics/epilepsy.
3. Epilepsy and Seizures | National Institute of Neurological Disorders and Stroke. [cited 2026 Feb 28]. Available online: https://www.ninds.nih.gov/health-information/disorders/epilepsy-and-seizures.
4. Epilepsy Foundation. [cited 2023 Jun 22]. SUDEP. Available online: https://www.epilepsy.com/complications-risks/early-death-sudep.
5. Types of Seizures | Epilepsy | CDC. 2020 [cited 2023 Jun 21]. Available online: https://www.cdc.gov/epilepsy/about/types-of-seizures.htm.
6. Focal Epilepsy. 2021 [cited 2023 Jul 22]. Available online: https://www.hopkinsmedicine.org/health/conditions-and-diseases/epilepsy/focal-epilepsy.
7. Diagnosing Seizures and Epilepsy. 2021 [cited 2023 Jul 23]. Available online: https://www.hopkinsmedicine.org/health/conditions-and-diseases/epilepsy/diagnosing-seizures-and-epilepsy.
8. Epilepsy diagnosis. Mayo Clinic. 2021 [cited 2023 Jun 21]. Available online: https://www.mayoclinic.org/diseases-conditions/epilepsy/diagnosis-treatment/drc-20350098.
9. Mehta O, Liao Z, Jenkinson M, et al. Machine Learning in Medical Imaging – Clinical Applications and Challenges in Computer Vision. In: Raz M, Nguyen TC, Loh E. editors. Artificial Intelligence in Medicine: Applications, Limitations and Future Directions. Singapore: Springer Nature; 2022:79-99. [cited 2025 Sep 7].
10. Elyan E, Vuttipittayamongkol P, Johnston P, et al. Computer vision and machine learning for medical image analysis: recent advances, challenges, and way forward. Art Int Surg 2022;2:24-45.
11. Liu H, Lang B. Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey. Appl Sci 2019;9:4396.
12. Chang AJ, Roth R, Bougioukli E, et al. MRI-based deep learning can discriminate between temporal lobe epilepsy, Alzheimer's disease, and healthy controls. Commun Med (Lond) 2023;3:33. [Crossref] [PubMed]
13. Wang H, Ahmed SN, Mandal M. Automated detection of focal cortical dysplasia using a deep convolutional neural network. Comput Med Imaging Graph 2020;79:101662. [Crossref] [PubMed]
14. Kaestner E, Hassanzadeh R, Gleichgerrcht E, et al. Adding the third dimension: 3D convolutional neural network diagnosis of temporal lobe epilepsy. Brain Commun 2024;6:fcae346. [Crossref] [PubMed]
15. Gill RS, Lee HM, Caldairou B, et al. Multicenter Validation of a Deep Learning Detection Algorithm for Focal Cortical Dysplasia. Neurology 2021;97:e1571-82. Erratum in: Neurology 2022 May 24;98(21):907. [Crossref] [PubMed]
16. Kersting LN, Walger L, Bauer T, et al. Detection of focal cortical dysplasia: Development and multicentric evaluation of artificial intelligence models. Epilepsia 2025;66:1165-76. [Crossref] [PubMed]
17. Ashikaga R, Araki Y, Ishida O. MRI of head injury using FLAIR. Neuroradiology 1997;39:239-42. [Crossref] [PubMed]
18. De Coene B, Hajnal JV, Gatehouse P, et al. MR of the brain using fluid-attenuated inversion recovery (FLAIR) pulse sequences. AJNR Am J Neuroradiol 1992;13:1555-64. [PubMed]
19. Bergin PS, Fish DR, Shorvon SD, et al. Magnetic resonance imaging in partial epilepsy: additional abnormalities shown with the fluid attenuated inversion recovery (FLAIR) pulse sequence. J Neurol Neurosurg Psychiatry 1995;58:439-43. [Crossref] [PubMed]
20. Lasagni L, Ciccarone A, Guerrini R, et al. Focal Cortical Dysplasia Type II Detection Using Cross Modality Transfer Learning and Grad-CAM in 3D-CNNs for MRI Analysis. arXiv; 2025 [cited 2025 Oct 9]. Available online: http://arxiv.org/abs/2504.07775.
21. Rahman MA, Miah MBA, Hossain MA, et al. Enhanced Brain Tumor Classification Using MobileNetV2: A Comprehensive Preprocessing and Fine-Tuning Approach. BioMedInformatics 2025;5:30.
22. Safdar MF, Alkobaisi SS, Zahra FT. A Comparative Analysis of Data Augmentation Approaches for Magnetic Resonance Imaging (MRI) Scan Images of Brain Tumor. Acta Inform Med 2020;28:29-36. [Crossref] [PubMed]
23. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv; 2015 [cited 2026 May 9]. Available online: http://arxiv.org/abs/1505.04597.
24. Chlap P, Min H, Vandenberg N, et al. A review of medical image data augmentation techniques for deep learning applications. J Med Imaging Radiat Oncol 2021;65:545-63. [Crossref] [PubMed]
25. He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition. arXiv; 2015 [cited 2026 May 11]. Available online: http://arxiv.org/abs/1512.03385.
26. He K. KaimingHe/deep-residual-networks. 2026 [cited 2026 May 12]. Available online: https://github.com/KaimingHe/deep-residual-networks.
27. huggingface. GitHub. [cited 2026 May 12]. pytorch-image-models/timm/models/xception_aligned.py at main · huggingface/pytorch-image-models. Available online: https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/xception_aligned.py.
28. Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks. In: Proceedings of the 30th International Conference on Machine Learning. PMLR; 2013:1310-8. [cited 2026 May 16]. Available online: https://proceedings.mlr.press/v28/pascanu13.html.
29. Hossin M, Sulaiman MN. A Review on Evaluation Metrics for Data Classification Evaluations. Int J Data Min Knowl Manag Process 2015;5:1-11.
30. Hoo ZH, Candlish J, Teare D. What is an ROC curve? Emerg Med J. BMJ Publishing Group Ltd and the British Association for Accident ...; 2017.
31. Huang J, Ling CX. Using AUC and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng 2005;17:299-310.
32. Salehzehi S, Irankhah E, Sabet M. Comparing Machine Learning and Deep Learning Approaches to Diagnose Epilepsy Disease. In: Triwiyanto T, Rizal A, Caesarendra W. editors. Proceeding of the 3rd International Conference on Electronics, Biomedical Engineering, and Health Informatics. Singapore: Springer Nature; 2023:703-16.
33. Lasagni L, Ciccarone A, Guerrini R, et al. Focal Cortical Dysplasia Type II Detection Using Cross Modality Transfer Learning and Grad-CAM in 3D-CNNs for MRI Analysis. 2025 [cited 2026 Apr 11]. Available online: https://arxiv.org/abs/2504.07775v1.
34. Chen S, Ma K, Zheng Y. Med3D: Transfer Learning for 3D Medical Image Analysis. arXiv; 2019 [cited 2026 Apr 1]. Available online: http://arxiv.org/abs/1904.00625.
35. Shankar A, Saikia MJ, Dandapat S, et al. Focal cortical dysplasia (type II) detection with multi-modal MRI and a deep-learning framework. Npj Imaging 2024;2:31. [Crossref] [PubMed]
36. Wagstyl K, Whitaker K, Raznahan A, et al. Atlas of lesion locations and postsurgical seizure freedom in focal cortical dysplasia: A MELD study. Epilepsia 2022;63:61-74. Erratum in: Epilepsia 2022 Apr;63(4):1018. [Crossref] [PubMed]
37. Kaur P, Ouaalam H, Kandemirli S, et al. Impact of Synthetic Lesional MR Images in Automated Focal Cortical Dysplasia Detection in Low-Data Scenarios. J Neuroimaging 2026;36:e70137. [Crossref] [PubMed]
38. Kim KM, Hwang H, Sohn B, et al. Development and Validation of MRI-Based Radiomics Models for Diagnosing Juvenile Myoclonic Epilepsy. Korean J Radiol 2022;23:1281-9. [Crossref] [PubMed]