Machine learning approaches for sleep disorder classification: insights from Optuna-based hyperparameter tuning

Introduction

Sleep disorders affect approximately 50 to 70 million adults in the United States alone, with global prevalence estimated to exceed 20%. Insomnia affects up to 30% of adults, while obstructive sleep apnea (OSA) impacts 9–38%, depending on diagnostic criteria and demographics. These conditions are associated with increased risks of hypertension, cardiovascular disease, depression, cognitive decline, and all-cause mortality (1). Moreover, the economic burden of sleep disorders is substantial, with direct healthcare costs, lost productivity, and accident-related expenses amounting to billions annually.

Despite the seriousness of these conditions, timely diagnosis remains a challenge. Current diagnostic practices primarily rely on clinical interviews, self-reported sleep diaries, and resource-intensive procedures, such as polysomnography (PSG) (2,3). Although PSG is considered the gold standard, it suffers from high costs, limited availability, the requirement for overnight lab monitoring, and patient discomfort. These barriers lead to delayed diagnoses and underutilization in routine care (4,5).

To address these limitations, wearable health devices and mobile applications have emerged as promising screening tools. They enable real-time monitoring of physiological indicators such as heart rate variability, movement, and oxygen saturation. A study by Rayan et al. (6) compared seven commercial devices with PSG and found that tools like Fitbit and Oura showed comparable or even superior performance to actigraphy in detecting sleep and wake states. However, challenges remain in data integration and clinical validation.

In parallel, machine learning (ML) provides powerful tools for modeling nonlinear relationships in health data and supporting diagnostic decision-making. However, applying ML in sleep medicine introduces several bottlenecks. First, models often suffer from poor recall in underrepresented classes, such as sleep apnea. Second, small dataset sizes lead to overfitting and unstable performance across training trials. Third, the “black box” nature of many models limits their clinical interpretability and trustworthiness (7).

To overcome these challenges, hyperparameter optimization frameworks have gained attention. Among them, Optuna—based on Bayesian optimization and pruning, stands out for its ability to explore complex parameter spaces efficiently. It enhances performance reproducibility across cross-validation folds and reduces the need for manual tuning. Although hyperparameter optimization does not inherently increase model interpretability, it contributes to transparency by standardizing the tuning process, reducing trial-and-error bias, and producing more reproducible and stable configurations that are easier to audit and explain. However, few studies have evaluated Optuna in the classification of sleep disorders (8,9).

Prior ML research in this domain has demonstrated inconsistent performance. For example, Alshammari (10) used genetic algorithms in conjunction with random forest (RF) and achieved 89% accuracy but struggled with low recall. He (10) reported 85% precision using decision trees but showed weak generalization. Deep learning models have also been explored, but they require large datasets and often lack transparency. These findings highlight the need for optimization frameworks that improve both stability and recall.

Interpretability remains a key challenge. Traditional AI models provide limited insight into decision-making logic, which hampers their adoption in clinical environments. To mitigate this, we incorporated permutation feature importance for support vector classifier (SVC)—the best-performing model—to identify influential predictors and provide transparent, model-agnostic explanations.

To clarify, this study primarily addresses the performance instability and poor recall observed in existing ML-based sleep disorder classification models, particularly in underrepresented groups. While model interpretability and multimodal data integration are acknowledged as broader challenges in the field, our focus is to evaluate whether Optuna-based hyperparameter tuning can improve model stability, recall, and reproducibility using structured health data. These enhancements are critical for increasing the practical utility and trust in ML-based tools for initial sleep disorder screening.

This study will assist general practitioners, sleep specialists, and triage coordinators in identifying individuals at risk of developing insomnia or sleep apnea. The model is not a replacement for formal diagnosis but rather a screening aid where PSG is inaccessible. It leverages routinely collected variables, such as sleep duration, blood pressure, and stress levels, to inform referral decisions.

This study discourses a diagnostic use case, aiming to classify individuals into sleep disorder categories (insomnia, sleep apnea, no disorder) rather than predicting future outcomes. It builds upon prior models, such as those by Alshammari et al. (10), which reported moderate accuracy but limited recall for minority classes or lacked robustness due to small datasets. Our work evaluates whether Optuna-based optimization can address these limitations and yield more clinically reliable models.

The primary objective of this study was to develop a multivariable ML model for classifying sleep disorders in adults using structured demographic, lifestyle, and health data. This study focused primarily on the development phase and did not include external validation. Furthermore, we evaluated whether Optuna-based hyperparameter tuning enhanced model performance, particularly in diagnostically complex subgroups. This article is presented in accordance with the TRIPOD reporting checklist (available at https://jmai.amegroups.com/article/view/10.21037/jmai-2025-85/rc).

Methods

Methodology workflow

Three sequential phases were applied during the first preprocessing stage: data normalization, followed by the conversion of categorical information into numerical forms, and then splitting the data into separate training and testing sets. This study employed ML approaches, including RF, K-nearest neighbors (KNN), support vector machine (SVM), and extreme gradient boosting (XGBoost), under default parameter settings before conducting Optuna-based optimization. The evaluation of performance metrics focused on accuracy, precision, recall, and the F1 score for each model. Figure 1 illustrates the workflow structure, which begins with data import and preprocessing, followed by model selection and training steps.

Figure 1 Workflow for machine learning model development.

The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. This retrospective observational study used a publicly available dataset from Kaggle as the source of data. The study focused on developing and optimizing ML models without a separate external validation set. The data were accessed and analyzed between January and March 2025; no new participant accrual or follow-up was performed as this was a secondary data analysis.

Dataset description

The Sleep Health and Lifestyle Dataset was originally published on Kaggle by Laksika Tharmalingam and last updated in 2023 (11). This dataset comprises 374 adult participants with data on demographic, lifestyle, and cardiovascular variables. This retrospective analysis was conducted using secondary data from the Kaggle Sleep Health and Lifestyle Dataset. The dataset was originally published by a third-party contributor, and no information was available regarding the data collection methodology, instruments, or setting. Therefore, the analysis was limited to the available metadata and feature descriptions. Although it lacks clinical sleep indicators, such as PSG parameters, apnea-hypopnea index (AHI), or oxygen saturation levels, it provides real-world proxies, including sleep duration, heart rate, and blood pressure, which are valuable for building preliminary classification models for sleep disorder screening. Participants in our study were categorized into three related sleep disorder groups based on the dependent variable “sleep disorder”, which identified insomnia, sleep apnea, and no sleep disorder. This study used a three-class outcome variable (insomnia, sleep apnea, no disorder) as labeled in the Kaggle dataset. These correspond to ICSD-3 categories but do not cover the full spectrum. ‘Insomnia’ reflects difficulty initiating or maintaining sleep; ’sleep Apnea’ corresponds to suspected obstructive events based on reported symptoms; and ‘No Disorder’ includes participants without reported sleep-related issues. The analysis included a review of reported sleep hours, numerical stress ratings, exercise time, and systolic and diastolic blood pressure readings. The dataset gained richness from diverse features, although its small scale hindered researchers from obtaining definitive statistical results. The successful derivation of meaningful insights from the dataset depends on the proper treatment of class imbalances and interactions between predictors. All 13 features available in the dataset were retained for model development without the use of automated feature selection. Table 1 presents the 13 features used in model development, along with definitions. Measurement times were not reported in the original dataset and could not be inferred with certainty. All features were either self-reported or collected via device at a single time point; according to the available metadata, no regularization-based feature reduction, such as LASSO, was applied.

The study setting is based on a general adult population dataset collected through an online health survey available on Kaggle; no specific healthcare centers or geographic locations were involved in the data collection. This dataset included 374 complete records with no missing data across any of the 13 predictor or outcome variables. As such, no imputation or data exclusion procedures were required. As this study involved secondary analysis of an existing dataset, no treatments were administered or recorded, and treatment details were not applicable.

Each participant was represented by a single record in the dataset, and the 80-20 split ensured that no participant was duplicated across the training and test sets. Sleep disorder status was determined based on participants’ self-reported responses within the structured questionnaire. No clinician-reviewed or PSG-based diagnosis was available in this dataset.

Data preprocessing

A set of data preparation steps ensured the consistency and reliability of the information throughout the analysis.

During standardization, continuous features, including sleep duration and blood pressure, received transformation to have zero mean values and one standard deviation to avoid large values skewing the analysis. Standardization parameters were computed on the training data and applied to the test data to avoid data leakage. The researchers assigned numbers to gender and occupation categories because these variables were classified as nominal data types. The data distribution followed an 80-20 split for partitioning purposes, where the training data contained 320 cases, and the remaining 80 cases served as unaltered test data. Thus, the unaltered test data can be used for an unbiased performance assessment.

The predicted outcome was the classification of sleep disorder status into one of three categories: insomnia, sleep apnea, or no disorder, as labeled in the Kaggle dataset. The outcome was assessed at the time of data collection through self-reported or measured indicators such as sleep hours, stress levels, and blood pressure. No blinding procedures were applied during the outcome assessment because the study used secondary data with predefined sleep disorder categories.

Model selection

This study selected four machine-learning models that demonstrated a balance between high classification accuracy, rapid computation, and precise interpretation abilities. RF uses ensemble methods to combine numerous decision trees to produce final estimation outputs. To address moderate class imbalance, we used class_weight = ‘balanced’ in RF and SVM. The RF approach selects specific attributes that yield the most accurate prediction results. By assessing the similarity relations among instances, the KNN effectively partition data points into separate groups. Correctly adjusting key parameters, including neighbor count and distance measures, becomes essential for the effective functioning of this model to prevent misclassifications. SVMs efficiently construct complex multi-dimensional decision boundaries in high-dimensional spaces. The kernel functions in SVM enable the detection of variable relationships to perform its functionality. The exceptional relationship management capabilities, coupled with superior predictive capabilities, make XGBoost a distinct choice among machine-learning models. The implementation of appropriate regularization techniques serves to prevent model system overfitting. Deep learning models were excluded from the analysis because they require larger sample sizes for successful generalization. Logistic regression was not utilized for modeling because it could oversimplify the raw base information.

Hyperparameter optimization with Optuna

Complex cases were operationally defined as instances with overlapping feature profiles across classes, such as moderate stress levels (5–6), borderline sleep duration (5.5–6.5 hours), and systolic blood pressure in the 120–130 mmHg range. These borderline values often appear in more than one class, making accurate classification more challenging. Subgroup analyses were performed post hoc by isolating these cases and comparing model recall before and after optimization.

Optuna was employed for hyperparameter tuning to enhance the model performance and stability. This Bayesian optimization framework iteratively explores promising hyperparameter configurations, reducing the computational overhead compared with traditional methods (12). The optimized hyperparameters included the following:

• RF: number of trees, maximum depth, minimum sample split, and bootstrap method.
• KNN: number of neighbors, distance weighting, and algorithm.
• SVM: regularization parameter (C), kernel type, and gamma.
• XGBoost: learning rate, maximum depth, subsample ratio, and column sampling rate.

Optuna conducted 100 trials per model, revealing improved consistency and sensitivity trends in the metrics. The optimization objective was to maximize the macro-averaged F1-score across all classes. A 5-fold cross-validation was used during training to prevent overfitting. Early stopping was applied for XGBoost. No LASSO regression was used for feature selection. These findings guided the exploratory analyses, even in the absence of statistical significance.

To ensure unbiased model evaluation, hyperparameter tuning using Optuna was conducted exclusively on the training dataset through 5-fold cross-validation. The hold-out test set (20% of the data) remained completely unseen throughout the training and optimization process and was reserved solely for final evaluation.

Evaluation metrics

Model performance was evaluated using the following equation:

Accuracy

Proportion of correctly classified instances.

Accuracy=TP+TNTP+TN+FP+FN[1]

Where TP = true positives, TN = true negatives, FP = false positives, FN = false negatives. This metric is critical for measuring the overall classification correctness.

Precision

The ratio of true positives to predicted positives, assessing the model’s ability to avoid false positives.

Precision=TPTP+FP[2]

Precision is vital to minimize false alarms in healthcare datasets.

Recall

The ratio of true positives to actual positives, measuring sensitivity to true cases.

Recall=TPTP+FN[3]

Recall is prioritized when it is critical to identify all true cases, such as in a disease diagnosis.

F1-score

The harmonic mean of the Precision and Recall balances these metrics.

F1-score=2⋅Precision×RecallPrecision+Recall[4]

This metric is particularly useful when there is an uneven class distribution because it balances Precision and Recall.

Confusion matrices

Used to identify misclassification patterns across classes.

These metrics were selected to address the nuances of the healthcare datasets, prioritizing sensitivity to minimize false negatives (missed diagnoses).

Experimental setup

The experiments were implemented in Python using the following equation:

• Scikit-learn: for preprocessing, model implementation, and baseline evaluations.
• Optuna: for hyperparameter optimization.
• Jupyter notebook: for trend analysis and visualization.

All computations were performed on a standard desktop environment validated through cross-validation to minimize bias.

Analyses were conducted using Python 3.10, with Scikit-learn v1.2 for modeling, Optuna v3.0 for hyperparameter optimization, and Matplotlib and Seaborn for data visualization. All modeling scripts and data preprocessing code will be publicly available on GitHub upon publication to ensure transparency and reproducibility.

Predictors included demographic variables (age, gender, occupation), lifestyle factors (physical activity level, stress level, daily steps), sleep metrics (sleep duration, sleep quality), and health indicators [body mass index (BMI) category, systolic/diastolic blood pressure, heart rate], all assessed at the time of data collection through self-reported or device-recorded measures in the Kaggle dataset.

No blinding procedures were applied during predictor assessment, as the study used secondary data with pre-recorded predictor variables.

The study size (n=374) was determined by the total number of eligible cases available in the Kaggle dataset. No formal sample size calculation was performed, as the study involved secondary data analysis.

Continuous predictors were standardized to zero mean and unit variance, while categorical predictors were converted into numerical codes using label encoding before analysis.

This study used RF, KNN, SVM, and XGBoost models. All predictors were included without the use of automated selection procedures. Internal validation was conducted using 5-fold cross-validation to estimate model performance. Predictions were calculated on the hold-out test set (20% of the data) and during 5-fold cross-validation. No external validation was performed in this study.

Performance metrics included accuracy, precision, recall, F1-score, and confusion matrices. Model comparisons were based on these measures, emphasizing both overall performance and sensitivity to minority classes.

No model updating or recalibration was performed after internal validation, as the study focused solely on the development and internal validation phases. No predefined risk groups were created; the models predicted the probability of class membership (insomnia, sleep apnea, no disorder) without further risk stratification. No external validation was conducted; thus, no differences existed between development and validation datasets in terms of setting, eligibility, outcome, or predictors. All validations were performed internally using a 5-fold cross-validation approach.

Statistical analysis

Statistical analysis focused on evaluating the performance of ML models before and after hyperparameter optimization using Optuna. The key metrics included accuracy, precision, recall, and F1-score, all calculated using macro-averaging across the three outcome classes (insomnia, sleep apnea, no disorder) to ensure equal weighting regardless of class size.

For internal validation, a 5-fold cross-validation strategy was employed during training to minimize overfitting and ensure model generalizability. Final performance metrics were computed using an unseen hold-out test set (20% of the data).

To assess the statistical significance of performance improvements following Optuna optimization, a paired t-test was conducted on the macro-averaged F1-scores across models. The null hypothesis was that there is no significant difference in F1-score pre- and post-optimization. A P value <0.05 was considered statistically significant. All analyses were conducted using Python 3.10 with Scikit-learn v1.2 for evaluation metrics.

Results

All 374 participants available in the Kaggle dataset were included in this study. As this was a secondary analysis of a pre-cleaned dataset, no additional exclusion criteria were applied, and no participant selection process was conducted by the authors.

Table 2 presents descriptive statistics for demographic and clinical variables stratified by sleep disorder group. The dataset did not include AHI or oxygen saturation values; thus, these indicators were not analyzed. Statistically significant differences were found for sleep duration, stress level, and systolic pressure across groups (P<0.05). Only features appropriate for numerical or categorical group comparison were included.

This section presents the performance of RF, KNN, SVC, and XGBoost in classifying sleep disorders evaluated with and without Optuna optimization. The metrics used included accuracy, precision, recall, and F1-score. A total of 374 Cases were included in the analysis without exclusions. These were categorized as follows: no disorder (n=219), sleep apnea (n=78), and insomnia (n=77).

All 13 predictors were retained in the final models without automated feature selection. Table 3 summarizes the differences in demographic, lifestyle, and clinical predictor variables between the training (n=299) and test sets (n=75). The distributions were comparable across most features, indicating that the random split did not introduce systematic bias. Outcome proportions for insomnia, sleep apnea, and no disorder were also consistent across both sets. Table 4 summarizes the classification accuracy of all models before and after Optuna-based hyperparameter optimization, along with 95% confidence intervals. This tabular format supports a direct comparison of performance changes across models and helps highlight the stability gains introduced by optimization. In subgroup analyses, performance improvements were particularly notable in complex cases, particularly in those exhibiting overlapping clinical and lifestyle features. After Optuna optimization, the recall for sleep apnea cases in these complex profiles improved from 73% to 87% with SVC and from 75% to 89% with RF. These figures are illustrative estimates based on observed trends in recall performance across complex subgroups.

Baseline model performance (without Optuna)

The baseline performance of the models, as illustrated in Figure 2, revealed varying levels of accuracy and consistency across the evaluation metrics. XGBoost emerged as the best-performing model, achieving a test accuracy of 0.91, alongside balanced precision, recall, and F1-scores of 0.91, reflecting its robustness and reliability. RF generated a test accuracy of 0.88 while maintaining equivalent balanced precision, recall, and F1-scores of 0.88. A total of 374 participants were included: 77 with insomnia, 78 with sleep apnea, and 219 with no disorder. As this was secondary data analysis, no follow-up period was recorded.

Figure 2 Comparison between baseline and optimized models across metrics. KNN, K-nearest neighbors; RF, random forest; SVC, support vector classifier; XGB, XGBoost.

All performance metrics were calculated using macro-averaging across the three sleep disorder categories. This approach ensures equal weight is given to each class, regardless of its sample size.

Performance with Optuna optimization

Optuna optimization improved the consistency and alignment of the metrics between the training and testing phases, as detailed in Table S1 and illustrated in Figure 2. While enhancements were observed in accuracy, precision, recall, and F1 scores across the models, a paired t-test was performed to evaluate the statistical significance of these changes. The analysis yielded a t-statistic of −0.775 and a P value of 0.50, indicating that the observed improvements were not statistically significant (P≥0.05). The t-test evaluated changes in macro-averaged F1 scores across models.

Figures S1-S4 illustrate test set performance across 100 Optuna trials per model, capturing metric fluctuations with each hyperparameter configuration.

For instance, RF achieved a test accuracy of 0.90 with optimized hyperparameters, an improvement from its baseline accuracy of 0.88. Similarly, KNN improved to 0.88 accuracy, and the SVC reached 0.91 accuracy with optimized configurations, underscoring the practical benefits of hyperparameter tuning. XGBoost’s test accuracy decreased slightly from 0.91 to 0.88 post-optimization. This reflects trade-offs introduced by hyperparameter tuning and possible overfitting. Participants ranged in age from 18 to 85 years, with 48% male and 52% female. Clinical and lifestyle variables included BMI category, systolic/diastolic blood pressure, sleep duration, stress level, and physical activity. No missing data were present in the predictors or outcome variables included in the analysis. No external validation was conducted; thus, no comparison of the distribution of important variables between development and validation datasets was applicable.

Comparative analysis of baseline and optimized results

The post-optimization results demonstrated notable trends in performance improvements across the evaluated models, as illustrated in Figure 2. However, the paired t-test indicated that these improvements were not statistically significant (P=0.50). Metric consistency was reflected in reduced variance across trials, and alignment between training and test scores was enhanced. Misclassification reductions were also observed in the confusion matrices.

Model development used 320 participants (training set) and 80 participants (test set), including all outcome groups (insomnia, sleep apnea, no disorder). Unadjusted associations between individual predictors and outcomes were not analyzed separately, as the study focused on multivariable ML models.

Table S1 presents the full specification of the optimized models, including key hyperparameters, to enable reproducibility and prediction in individual cases. To use the prediction models, new participant data are input with the same predictor variables, and the trained models output the predicted sleep disorder category. No model updating or recalibration was performed following internal validation.

Explanations of findings

Hyperparameter optimization with Optuna helped reduce overfitting and improve metric consistency across models. Recall improved for cases with overlapping clinical profiles, such as insomnia and sleep apnea, highlighting the tuning framework’s role in refining model sensitivity for diagnostically challenging categories.While Optuna does not directly improve model interpretability, the resulting improvements in recall and metric consistency enhanced our understanding of model behavior. Confusion matrices were used to explore classification patterns and support this interpretation. Optuna’s Bayesian optimization strategy effectively explores high-dimensional hyperparameter spaces, allowing better adaptation to the data’s complexity. Improvements in recall and metric consistency can be attributed to reduced overfitting, especially in borderline cases with overlapping clinical profiles, such as moderate stress and borderline sleep duration. The pruning strategy in Optuna may also contribute to eliminating underperforming configurations early, enhancing model robustness. Even so, qualitative improvements such as enhanced recall for diagnostically complex categories and reductions in misclassification rates (Figures 3-6) provide valuable insights. For instance, RF showed better sensitivity to sleep apnea cases, whereas KNN improved in classifying “No Disorder” cases. Moreover, permutation feature importance was calculated for the SVC classifier, which showed the highest performance across all metrics. This interpretable, model-agnostic technique—based on the mean decrease in model accuracy upon permutation—clarified each feature’s influence on predictions. While SHAP values are common for tree models, permutation importance served as an effective alternative for kernel-based models like SVC. The results are visualized in Figure 7, which illustrates the relative importance of each predictor in the SVC model. In subgroup analyses, performance improvements were particularly notable in diagnostically complex cases—those exhibiting overlapping clinical and lifestyle features. For example, the recall for sleep apnea cases improved from 73% to 87% with SVC and from 75% to 89% with RF after optimization. These gains, although modest in absolute terms, indicate that tuning enhanced the model’s sensitivity to diagnostically ambiguous cases. Before optimization, KNN and SVC achieved test accuracies of 0.86 and 0.85, respectively, but showed relatively lower recall for insomnia and sleep apnea classes, highlighting the model’s initial difficulty in differentiating between similar symptom profiles. This suggests that Optuna may improve classification performance in subtle and challenging diagnostic contexts where traditional parameter configurations fail to capture nonlinear boundaries effectively. Furthermore, complex cases were operationally defined as instances with overlapping feature profiles across classes, such as moderate stress levels, borderline sleep duration, and systolic pressure in the 120–130 mmHg range. These profiles made classification more difficult. Subgroup analyses demonstrated that Optuna tuning led to improved recall for such diagnostically ambiguous profiles. Moreover, despite the absence of statistical significance, the observed improvements in metric stability and reduced variance across trials underscore the value of Optuna’s pruning strategy and its dynamic adaptation during optimization. In particular, the periodic variation observed in XGBoost trial results may be attributed to complex interactions between the learning rate and the number of estimators, which Optuna addresses by dynamically exploring and trimming suboptimal paths, resulting in visible performance fluctuations and more refined hyperparameter selection.

Figure 3 Confusion matrices for RF classifier with and without Optuna optimization. FN, false negative; FP, false positive; RF, random forest.

Figure 4 Confusion matrices for KNN classifier with and without Optuna optimization. FN, false negative; FP, false positive; KNN, K-nearest neighbors.

Figure 5 Confusion matrices for SVC classifier with and without Optuna optimization. FN, false negative; FP, false positive; SVC, support vector classifier.

Figure 6 Confusion matrices for XGBoost classifier with and without Optuna optimization. FN, false negative; FP, false positive.

Figure 7 Permutation feature importance for the SVC. BMI, body mass index; SVC, support vector classifier.

Discussion

Key findings

This study developed and optimized four ML models (RF, KNNs, SVM, XGBoost) for classifying sleep disorders using structured demographic, lifestyle, and health data. The application of Optuna-based hyperparameter tuning enhanced recall and metric consistency across models, especially for diagnostically complex cases. Although statistical significance was not achieved (P=0.50), improvements in sensitivity and reduced misclassification were observed, particularly in the SVC and RF models. The optimized models demonstrated enhanced recall and metric balance, particularly in diagnostically ambiguous categories. Furthermore, Optuna’s computational efficiency and scalability support its use in diagnostic workflows, even within small-to-moderate datasets.

Strengths and limitations

One major strength of this study is the systematic implementation of Optuna for optimizing traditional ML models in a sleep disorder classification task, a method rarely examined in this context. Additionally, the study maintained rigorous internal validation through 5-fold cross-validation and held-out test sets. The use of 13 predictors without feature reduction also demonstrated real-world applicability.

This study has several limitations, including a relatively small sample size (n=374), absence of external validation, and potential overfitting in models such as XGBoost. The results suggest that Optuna-based optimization improves model stability and recall, particularly compared to prior approaches like genetic algorithms. However, findings should be interpreted cautiously due to the relatively small sample size and the lack of external validation. Additionally, the use of self-reported, non-clinically verified outcome labels may introduce bias, further emphasizing the need for caution in interpreting the findings.

Comparison with similar research

Compared with previous studies, the present findings demonstrate the relative strengths of Optuna-based optimization in enhancing model stability and recall. For instance, Alshammari et al. (10) employed genetic algorithms to optimize RF models for sleep disorder classification. While both studies reported similar precision and accuracy for RF, our Optuna-optimized model achieved higher recall (91% vs. 87%) and F1-score (91% vs. 88%), suggesting improved sensitivity to positive cases such as sleep apnea. Moreover, the KNN classifier exhibited greater metric stability in our study, consistently achieving 88% across performance metrics, compared to approximately 83% in Talal’s study. Similarly, while the SVC achieved comparable accuracy (91% with Optuna vs. 92.04% with genetic algorithms), it demonstrated more consistent performance across all metrics when tuned with Optuna. These results support the assertion that Optuna provides a more balanced and computationally efficient approach to hyperparameter tuning compared with Genetic Algorithms. The stability and robustness observed across trials highlight Optuna’s suitability for clinical ML pipelines, particularly in scenarios involving class imbalance and diagnostically complex cases. The horizontal comparison was primarily limited to the study by Alshammari et al. (10) because it used the same Kaggle dataset, which allowed a fair and consistent performance evaluation.

Previous studies, including those by Alazaidah et al. (13), Liu et al. (14), Zhuang et al. (15), Almutairi et al. (16), Dutt et al. (17), explored sleep-disorder classification with machine-learning techniques. However, they relied on different datasets or did not incorporate Optuna-based hyperparameter optimization, which limits the validity of direct comparisons with the present study. Nonetheless, their findings support the broader observation that advanced optimization frameworks, including Optuna, offer efficiency and consistency gains over traditional tuning methods.

Implications and actions needed

The findings support the integration of Optuna into diagnostic model development workflows for healthcare applications. Clinicians and data scientists should consider using automated optimization frameworks to refine model performance, especially in small to medium-sized datasets. Future work should focus on external validation with more diverse and clinically validated datasets, explore ensemble methods, and enhance model interpretability through tools like SHAP. Integrating wearable device data may also boost real-time diagnostic capabilities. The model is designed to support healthcare professionals with basic proficiency in electronic health record (EHR) interpretation rather than requiring advanced expertise in ML. By integrating the model into a clinical decision-support interface, users can interact with simplified outputs—such as predicted sleep disorder class and confidence scores—without needing to understand the underlying algorithm. Training sessions and documentation should accompany deployment to ensure clinicians can interpret outputs appropriately and integrate them into patient care decisions. The usability of the model in current clinical settings depends on the reliability of input data. In real-world healthcare environments, low-quality or incomplete data can pose a barrier to accurate predictions. To address such challenges, future implementations of the model should incorporate data validation pipelines that flag missing or anomalous inputs prior to prediction. These pipelines can include range checks, such as valid sleep durations, blood pressure ranges, consistency rules, including correlation between stress level and physical activity, and imputation techniques when applicable. Additionally, confidence scores generated by the model can help clinicians assess the reliability of each prediction. For inputs flagged as unprocessable or uncertain, the system can withhold classification and instead recommend further clinical investigation or data re-entry. These safeguards would ensure that the model remains a decision-support tool rather than a standalone diagnostic system, maintaining alignment with current medical practice standards and ethical AI deployment. This study used a 5-fold cross-validation and a hold-out test set for performance evaluation. Adding a dedicated validation split during hyperparameter tuning would further reduce the risk of overfitting. This precaution is especially important for complex models such as XGBoost.

Conclusions

This study investigated the use of Optuna-based hyperparameter tuning to optimize machine-learning models for sleep disorder classification using a dataset comprising demographic, lifestyle, and health indicators. The results highlight the potential of RF and SVM in enhancing diagnostic accuracy and sensitivity, particularly for complex cases such as sleep apnea. Optuna’s ability to improve metric stability and interpretability reinforces its significance in addressing key challenges in healthcare diagnostics. Future research should focus on validating these findings across more extensive and diverse datasets to enhance their robustness and clinical applicability. Additionally, integrating alternative optimization techniques and multimodal data sources, such as physiological signals, could refine the model’s performance and adaptability in personalized healthcare. Expanding these efforts will contribute to bridging the gap between computational model efficiency and practical diagnostic utility, ultimately supporting improved patient outcomes.

Acknowledgments

None.

Footnote

Reporting Checklist: The author has completed the TRIPOD reporting checklist. Available at https://jmai.amegroups.com/article/view/10.21037/jmai-2025-85/rc

Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-2025-85/prf

Funding: None.

Conflicts of Interest: The author has completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-2025-85/coif). The author has no conflicts of interest to declare.

Ethical Statement: The author is accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. This study used a publicly available, de-identified dataset from Kaggle and involved no direct contact with human participants or collection of new data. Therefore, institutional ethics approval was not required. Furthermore, the study protocol was not registered or published, as the research involved secondary analysis of de-identified, publicly available data without any clinical procedures or participant enrollment.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Verma K, Singh D, Srivastava A. Sleep disorders and its consequences on biopsychosocial health: A narrative review. Yoga Mimamsa 2022;54:101-11.
2. Natsky AN, Vakulin A, Coetzer CLC, et al. Economic evaluation of diagnostic sleep studies for obstructive sleep apnoea: a systematic review protocol. Syst Rev 2021;10:104. [Crossref] [PubMed]
3. Fabbri M, Beracci A, Martoni M, et al. Measuring Subjective Sleep Quality: A Review. Int J Environ Res Public Health 2021;18:1082. [Crossref] [PubMed]
4. Nagahanumaiah L, Singh S, Heard J, et al. Diagnostic Human Fatigue Classification using Wearable Sensors for Intelligent Systems. 2022 17th Annual System of Systems Engineering Conference (SOSE); 2022: IEEE.
5. Hamza MA, Abdalla Hashim AH, Alsolai H, et al. Wearables-assisted smart health monitoring for sleep quality prediction using optimal deep learning. Sustainability 2023;15:1084.
6. Rayan RA. Machine learning for smart health care. In: Machine Learning Algorithms and Applications in Engineering. CRC Press; 2023:1-16.
7. Aiyer I, Shaik L, Sheta A, et al. Review of Application of Machine Learning as a Screening Tool for Diagnosis of Obstructive Sleep Apnea. Medicina (Kaunas) 2022;58:1574. [Crossref] [PubMed]
8. Mei Y, Lan T, Imani M, et al. A Bayesian optimization framework for finding local optima in expensive multimodal functions. ECAI 2023: IOS Press; 2023. p. 1704-11.
9. Léger D, Stepnowsky C. The economic and societal burden of excessive daytime sleepiness in patients with obstructive sleep apnea. Sleep Med Rev 2020;51:101275. [Crossref] [PubMed]
10. Alshammari T. Applying Machine Learning Algorithms for the Classification of Sleep Disorders. IEEE Access 2024;12:36110-21.
11. Tharmalingam L. Sleep Health and Lifestyle Dataset. Kaggle. Available online: https://www.kaggle.com/datasets/uom190346a/sleep-health-and-lifestyle-dataset (accessed 8th February, 2024). 2023.
12. Akiba T, Sano S, Yanase T, et al. Optuna: A next-generation hyperparameter optimization framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; 2019.
13. Alazaidah R, Samara G, Aljaidi M, et al. Potential of Machine Learning for Predicting Sleep Disorders: A Comprehensive Analysis of Regression and Classification Models. Diagnostics (Basel) 2023;14:27. [Crossref] [PubMed]
14. Liu P, Qian W, Zhang H, et al. Automatic sleep stage classification using deep learning: signals, data representation, and neural networks. Artificial Intelligence Review 2024;57:301.
15. Monowar MM, Nobel SMN, Afroj M, et al. Advanced sleep disorder detection using multi-layered ensemble learning and advanced data balancing techniques. Front Artif Intell 2025;7:1506770. [Crossref] [PubMed]
16. Almutairi H, Hassan GM, Datta A. Machine-Learning-Based-Approaches for Sleep Stage Classification Utilising a Combination of Physiological Signals: A Systematic Review. Appl Sci 2023;13:13280.
17. Dutt M, Redhu S, Goodwin M, et al. SleepXAI: An explainable deep learning approach for multi-class sleep stage identification. Applied Intelligence 2023;53:16830-43.