AI²-SXI algorithm enables predicting and reducing the risk of less than 30 days patient readmissions with 99% accuracy and precision

Introduction

The continuing efforts to lower hospital readmission rates in the United States of America (USA) from a profound awareness of the frequency of readmissions and their financial burden on the healthcare system. Hospital readmissions are prevalent among discharged patients in the USA, particularly among those with pre-existing health conditions (1) the elderly demographic (2). Furthermore, decreasing readmission rates is acknowledged as integral to national quality improvement strategies, incentivized by healthcare policies. Consequently, there is a high demand for predictive models to assess readmission risk, aiming to enhance patient care and diminish healthcare expenditures.

As healthcare data becomes increasingly accessible, machine learning (ML) techniques are becoming more popular for clinical applications. Commonly used algorithms in predicting hospital readmissions in the USA include tree-based methods, neural networks (NNs), regularized logistic regression (LR), and support vector machines (SVMs). Many studies have reported ML algorithms achieving an area under the curve (AUC) above 0.70, with variability in reported AUC values ranging from 0.50 to 0.90 and a median of 0.68 [interquartile range (IQR), 0.64–0.76].

Another study conducted a systematic review of risk prediction models for hospital readmission, which varied based on the timeframe of readmission (ranging from 15 days to 12 months after discharge), the population studied (including different age ranges, Medicare, and Medicaid beneficiaries), geographical scope (from nationwide to hospital-specific), and the data sources used (such as administrative claims, real-time data, and clinical data). Additionally, it was found that most predictive models for readmission performed poorly, with reported values of the AUC falling between 0.5 and 0.7 on the receiver operating characteristic (ROC) curve (3).

In another three different studies, predictive models developed for the Center For Medicare and Medicaid Services (CMS) exhibited relatively limited ability to discriminate, with corresponding AUC values of 0.61, 0.63, and 0.63 for heart failure (HF), acute myocardial infarction (AMI), and pneumonia (PN), respectively. These models utilized hierarchical regression techniques to estimate risk-standardized mortality rates (RSMRs) or risk-standardized readmission rates (RSRRs) for individual hospitals (4). In another study examining prediction of all-cause readmissions, the highest reported AUC of 0.8 was achieved using administrative claims data. This investigation employed a LR model with inpatient data sourced from the Veterans Healthcare Network in New York, validated through a two cross-fold method, yielding an AUC of 0.79 (5).

Fernández-Delgado et al. (6) conducted research on various classifiers and predictive algorithms, including LR and ML methods, with their findings indicating that random forest (RF) yielded the most favorable outcomes. Kulkarni et al. (7) compared decision trees, NNs, and LR models for predicting readmission risk using patient administrative data, revealing that ML algorithms can enhance LR’s AUC. Additionally, Au et al. (8) found that simpler models like LACE [which incorporates length of stay (LOS), admission acuity, comorbidities, and prior emergency department visits] outperformed CMS models. Yu et al. [2015] (9) compared time-to-event modeling (Cox model) with SVM and LACE model, demonstrating that SVM performed better than the other models for predicting readmissions related to AMI, PN, HF, and all causes. Vedomske et al. (10) crafted a predictive framework specifically for HF readmissions, employing the RF algorithm. Their model exhibited superior performance with an AUC of 0.84, leveraging diagnosis and procedural data as input variables.

In another study published by BMC Health Services Research [2022] evaluated 428,669 patients data finding that 24,974 (5.83%) were readmitted within 30 days of discharge for any reason. Patients were more likely to be readmitted if they utilized hospital care more, had more physician office visits, had more prescriptions, had a chronic condition, or were 65 years old or older. The LACE readmission prediction model had an AUC of 0.66±0.0064, while the ML model’s test set AUC was 0.83±0.0045, based on learning a gradient boosting machine on a combination of machine-learned and manually-derived features (11).

Furthermore, Xiao et al. (11) proposed a deep learning model for readmission prediction. Their model achieved an AUC of 0.6103±0.0130, compared to the second-best model’s AUC of 0.5998±0.0124. The derived patient representations were further utilized for patient phenotyping, providing a more precise understanding of readmission risks (12).

The above studies predominantly utilized ML methods to address the challenge of hospital readmission prediction, specifically in determining whether a patient would be readmitted within 30 days. However, they lacked efforts to explain and interpret the impact of various features on the prediction outputs, forecast short-, mid-, and long-term reductions in readmission within 30 days rates, or implement a comprehensive model for testing predictions without prior training. To address these gaps, this study concentrated on: (I) evaluating the performance of the Sriya Expert Index (SXI) model as a multivariate scoring system to predict readmissions as a binary classification problem, (II) enhancing the SXI scoring methodology for each patient using the Proprietary Deep Neural Network algorithm and correlating it with readmission 30 days, and (III) employing a targeted decision tree framework to interpret the most effective pathways leading to both positive and negative patient outcomes, thereby offering evidence-based recommendations to improve patient care and reduce readmission within 30 days.

Methods

Data description

The dataset represents a span of 10 years [1999–2008] and comprises clinical data from 130 hospitals and integrated delivery networks across the USA. It consists of over 50 features reflecting both patient and hospital outcomes where 46,902 patients with readmitted within 30 days and readmitted greater than 30 days. This data was derived from encounters that met specific criteria: they were inpatient admissions, focused on diabetic care (where any form of diabetes was diagnosed), had a duration of stay ranging from 1 to 14 days, involved laboratory tests and medication administration during the encounter.

• Dataset source: the dataset used is from Kaggle, a reputable platform for datasets commonly used in ML projects. Kaggle datasets are widely recognized and used for educational purposes, competitions, and baseline model development. While the dataset may be older, it provides a valuable foundation for developing and testing initial models.
• Historical data value: historical data, despite being older, offers significant value for initial model development. It allows us to establish baseline performance metrics and understand fundamental patterns and correlations in readmission within 30 days data. Models trained on historical data can be benchmarked and further refined with more recent data as it becomes available.
• Impact of Hospital Readmission Reduction Program (HRRP): the CMS HRRP, implemented in 2012, indeed marked a significant change in the healthcare landscape. While our dataset predates this, it still captures underlying patient behavior and readmission patterns that are valuable for model training. Future iterations of the model can incorporate post-2012 data to account for the impact of HRRP and enhance the model’s applicability to contemporary healthcare settings.
• Current use case: the current model serves as a foundational tool for understanding the mechanics of readmission prediction and provides a starting point for further development. It is intended for educational and preliminary research purposes, with the understanding that real-world deployment would require more recent data and continuous model updates. By addressing these points, we can ensure that the model remains relevant and improves over time as more data becomes available. We appreciate the feedback and are committed to refining our approach to deliver accurate and effective predictive models.

Attributes within the dataset include patient identifiers, demographic information such as race, gender, and age, admission details like type and duration, the medical specialty of admitting physicians, the number of laboratory tests performed, glycated haemoglobin (HbA1c) test results, diagnoses, medication details including diabetic medications, and the frequency of outpatient, inpatient, and emergency visits in the year preceding hospitalization.

The target variable in this dataset is ‘readmitted’, categorized into two groups: readmitted greater than 30 days and readmitted less than 30 days. Among the 46,902 individuals in the dataset, 35,545 were classified as readmitted greater than 30 days, while 11,357 were readmitted less than 30 days, representing 75.78% and 24.21% of the total. To prepare the data for analysis, preprocessing techniques are employed to address potential irregularities such as blanks, nulls, and outliers.

The original dataset contained a target variable that categorized patient readmission status into three distinct groups: “No”, “readmission more than 30 days” and “readmission within 30 days”. The “No” category represented patients who were not readmitted, “readmission more than 30 days” indicated patients who were readmitted more than 30 days after discharge, and “readmission within 30 days” denoted patients who were readmitted within 30 days. In terms of distribution, the dataset included 35,545 instances for the “readmission more than 30 days” category, 54,864 instances for the “No” category, and 11,357 instances for the “readmission within 30 days” category.

For the purpose of this experiment, the focus was narrowed down to only the “readmission more than 30 days” and “readmission within 30 days” categories, which directly pertain to the study’s objective of analyzing readmissions within and beyond 30 days. The “No” category, representing patients who were not readmitted, was excluded from the analysis to streamline the investigation and ensure that the study concentrated solely on patients who experienced a readmission. Consequently, a new dataset was filtered out, containing only the “readmission more than 30 days” and “readmission within 30 days” instances. This refined dataset was then used for all subsequent pre-processing steps and model training, enabling a more targeted and relevant analysis of patient readmission patterns.

The dataset contains several columns with missing values, which we have listed in Table 1. We developed a technique to handle these missing values based on their amount. If a particular feature column has more than 40% of its values missing, we remove that column to maintain data integrity. For numeric columns with missing values, we use mean imputation, filling in the gaps with the average value of the respective column. For categorical columns, we apply mode imputation, replacing missing values with the most common category. This approach ensures that our dataset is not only complete but preserving the overall quality and consistency of the data. Whenever necessary we have also used variational autoencoder (VAE) and generative adversal network (GAN) algorithms for missing values and synthetic data generation.

Additionally, feature encoding methods such as one-hot encoding and label encoding are utilized for variables with less than 10 unique values and more than 10 unique values, respectively.

SXI model framework

SXI is a dynamic score/index obtained from a proprietary formula consisting of weights from 5–10 ML algorithms. SXI is a super feature and is a true weighted representative of all important features which converts a multi-dimensional hard to solve problem into a simpler 2-dimensional solution. SXI with Proprietary Deep Neural Network algorithm involves an iterative approach where it dynamically adjusts algorithm weights based on the most significant weights provided by the 5 to 10 ML algorithms. This process aims to enhance the correlation between SXI scores and business outcomes, thereby improving accuracy and delineation.

The SXI framework (Figure 1) employs a variety of mathematical techniques ranging from arithmetic to statistical analyses and non-ML algorithms to process cleaned data effectively. By leveraging this diverse framework, the SXI utilizes multiple functions, including statistical measures like standard deviation and various types of correlations, alongside algebraic operations such as proportionality constants. Through these operations, the SXI Engine calculates a base SXI score by bringing together various statistical analysis and algebraic computations.

Figure 1 SXI framework. Score + Proprietary Deep Neural Network algorithm + Correlate = Improve. SXI, Sriya Expert Index; Min, minimum; Max, maximum; ML, machine learning; indv., individual; ACC, accuracy; MAE, mean absolute error; AUC, area under the curve; w.r.t., with respect to; DTs, decision trees.

The SXI Engine utilizes algebraic and statistical methods to derive weights for individual user input parameters within a user vector. These weights act as multipliers, enabling adjustments to the significance of each parameter. For example, bi-variate and multi-variate analyses may yield correlation coefficients, which are then employed as multipliers for the user vector components. This process contributes to the generation of a robust base SXI score, serving as the foundation for subsequent steps in the SXI System.

Once the base SXI score is established, the system incorporates weights obtained from 5 to 10 ML algorithms. These weights collectively contribute to form the final SXI scores, which serve as the initial SXI or the Benchmark SXI score in the SXI System. This integration of diverse mathematical functions, statistical analyses, and ML algorithms highlights the versatility of the SXI, providing a comprehensive approach to scoring and delineating business outcomes based on user input parameters.

Benchmark analysis of the baseline score involves extracting key metrics such as the average SXI Score (Benchmark Score), percentage of good and bad outcomes, and the distribution of outcomes above and below the average SXI. This analysis offers insights into the performance of the SXI system in distinguishing between positive and negative outcomes.

The Proprietary Deep Neural Network algorithm plays a crucial role in optimizing the SXI system. It adjusts weights dynamically based on the performance of the system, aiming to enhance the correlation between SXI scores and business outcomes. The Proprietary Deep Neural Network algorithm agent identifies the top features from ML weights and adjusts the weights accordingly. Additionally, it introduces a custom weight initialization strategy based on feature importance, aligning the weight initialization process with the significance of individual features.

The deep learning architecture consists of automated hyperparameter tuning, flexibility in activation functions and optimizers, dynamic architecture based on dataset size, and structured evaluation metrics. These characteristics ensure adaptability, efficiency, and thorough evaluation of the deep neural network model.

The workflow of the Proprietary Deep Neural Network algorithm involves steps such as train-test split, in our study, we divided the dataset into 70% for training, 20% for testing, and 10% for validation. The validation set, making up 10% of the data, plays a crucial role in the design of SXI model by aiding in hyperparameter tuning, overfitting detection, and appropriate model selection, ensuring that the model can generalize well to new, unseen data. Overfitting, where a model performs well on training data but poorly on real-world data, can be mitigated with the validation set. This thorough process ensures that the model is not only accurate on the training data but also reliable and generalizable. The validation dataset is thus a critical stage in the SXI workflow, confirming that the model remains highly accurate and generalizable when applied to new data. In this model, we adopted a 70-20-10 split for training, testing, and validation. This approach ensures that the model is thoroughly evaluated and fine-tuned before being tested on the 20% of data set aside for final performance assessment. The testing set, which comprises 20% of the data, is used to evaluate the model’s performance and ensure its accuracy on new, unseen data.

Bayesian optimization for hyperparameter tuning, model configuration, and model training and evaluation with the best hyperparameters. Iterative weight calibration is performed to refine the SXI system, where it adjusts weights, applies rewards and punishments based on performance improvements, until an optimal delineator is achieved.

In the reward and punishment mechanism, improvements in both SXI score and class delineation accuracy lead to a reward. This involves integrating the newly adjusted weights into the existing ones and recalculating SXI scores to evaluate the enhancements in delineation and accuracy. If these improvements occur without any weight adjustment, there is a positive weight adjustment, initially ranging from 0% to 100% percentage increase weights. Further iterations iteratively adjust weights until maximum accuracy is achieved. If positive adjustments fail to surpass the initial class delineation, negative weight adjustments from 0% to −100% percentage increase weights are implemented. If these adjustments yield improved delineation and accuracy, subsequent iterations continue adjusting weights negatively until no further improvement is possible. In cases where no improvement is observed for both positive and negative adjustments, the reinforced function penalizes by providing the next set of weights, along with additional weightage for the top 5 most important features in the hidden layers of the deep neural network.

The iterative process continues until the SXI scores cannot be further improved. The final new SXI scores are then updated to the system for further analysis and decision-making regarding business outcomes.

In the context of business outcomes, this score can be correlated with specific classes such as “good” and “bad” within the dataset. One common metric for evaluating the strength of this correlation is the coefficient of determination, often denoted as R-squared (R²).

If the SXI scores (x) are the independent variable and the target outcome (y) is the dependent variable, a polynomial regression equation can be expressed as:

y=b0+b1x+b2x2+…+bnxn+∈[1]

where y is the target outcome, x is the SXI scores, b₀, b₁, b₂, …, b_n are the coefficients representing the weights of the polynomial terms, n is the degree of the polynomial, ∈ is the error term.

The coefficient of determination, denoted as R², quantifies how well the SXI scores explain the variance in the business outcomes. By analyzing polynomial regression plots, distinct improvement phases can be identified: initial, mid-term, and long-term improvements. These phases delineate where changes in SXI scores most significantly impact business outcomes, aiding in the strategic interpretation of data trends and patterns.

For the initial target improvement for the business, when a positive correlation is identified between SXI scores and business outcomes, it suggests that higher SXI scores coincide with better performance. In such cases, positive weighted features, which contribute positively to the business outcome, are adjusted upwards by a specified percentage increase, while negative weighted features, which have a negative impact when their values rise, are adjusted downwards by a percentage decrease. Conversely, when a negative correlation is observed between SXI scores and business outcomes, positive weighted features are now negatively impact the outcome when their values rise, thus requiring a percentage decrease adjustment, while negative weighted features, which positively influence outcomes with higher values, receive a percentage increase adjustment. This adjustment strategy is applied to the dataset for each observation, simulating an initial improvement in the data.

Following the data adjustment process, a RF model is employed, utilizing the adjusted dataset as training data reflecting the initial improvement. Within the RF, a decision tree emerges as the target decision tree path. This decision tree encapsulates the intricate relationships between the adjusted features and the business outcome, offering insights into how specific changes in features may impact the overall business outcome. By using this scientific approach, businesses can understand how SXI scores are related to outcomes. They can then make strategic changes to features to gain a better understanding of how those changes affect overall performance. This method helps organizations gain valuable insights into how features and improvements affect their business, which can help them make better decisions and optimize their strategies.

Statistical analysis

Data statistics

The dataset utilized in this study spans 10 years [1999–2008] and comprises clinical data from 130 hospitals and integrated delivery networks across the USA. It includes a total of 46,902 patient records, categorized into two groups based on readmission status: patients readmitted more than 30 days after discharge (35,545 patients, 75.78%) and patients readmitted within 30 days (11,357 patients, 24.21%). The dataset contains over 50 features, encompassing demographic information, admission details, laboratory tests, medication details, and patient outcomes. Key attributes include patient identifiers, demographic data (such as race, gender, and age), admission type and duration, laboratory test results (e.g., HbA1c levels), and the frequency of healthcare visits prior to hospitalization.

Handling missing values

To ensure the integrity of the dataset, missing values were meticulously addressed. Any feature with more than 40% missing values was removed to eliminate potential bias or noise. For numeric columns with missing values below the 40% threshold, mean imputation was used, whereby the mean of the available values filled in the gaps, maintaining the overall data distribution. For categorical columns, mode imputation was applied, replacing missing values with the most frequently occurring category. This method preserved the commonality of categories, which is essential for accurate categorical data analysis. Whenever necessary we have also used VAE and GAN algorithms for missing values and synthetic data generation.

Data preprocessing steps

In preparing the data for ML algorithms, several preprocessing steps were undertaken. For feature encoding, one-hot encoding was applied to categorical features with fewer than 10 unique values, creating binary columns for each category and enabling effective processing of categorical data without assuming any ordinal relationship. For categorical variables with more than 10 unique values, OneHot Encoding was used, assigning a unique integer to each category while maintaining the order. To scale numerical variables, min-max scaling was applied, transforming each feature to a range between 0 and 1. This normalization facilitated better convergence during model training and improved overall model performance.

Outliers in numerical variables were managed using statistical techniques such as z-score analysis and the IQR method. Z-score analysis identified and removed data points that deviated significantly from the mean, while the IQR method removed outliers that fell below the first quartile or above the third quartile by a specified multiplier (typically 1.5 times the IQR). These methods ensured that extreme values did not skew the model’s performance.

SXI score calculation

To derive the base SXI score, the study employed a variety of statistical measures, including standard deviation and correlation coefficients. These measures were calculated based on patient data, capturing critical factors such as age, comorbidities, length of stay, and prior readmissions. The SXI algorithm applied algebraic operations, like proportionality constants, to weigh these factors and compute a robust base SXI score, reflecting the severity and risk of readmission within 30 days for each patient. Following the computation of the base SXI score, the study integrated weights obtained from 5 to 10 ML algorithms. These weights were dynamically applied to adjust the significance of various patient parameters, refining the accuracy of the Final SXI scores.

SXI score optimization using Proprietary Deep Neural Network algorithm

The study conducted a benchmark analysis of the SXI scores to identify key metrics such as the average SXI Score (Benchmark Score), percentage of good and bad outcomes, and the distribution of outcomes relative to the average SXI. This analysis provided insights into the effectiveness of the SXI algorithm in distinguishing between patients likely to be readmitted within 30 days and those with lower risk.

The Proprietary Deep Neural Network algorithm was employed to further optimize the SXI scores. The workflow of the Proprietary Deep Neural Network algorithm involves steps such as train-test split, Bayesian optimization for hyperparameter tuning, model configuration, K-fold cross-validation, and model training and evaluation with the best hyperparameters. Iterative weight calibration is performed to refine the SXI system, where it adjusts weights, applies rewards and punishments based on performance improvements, until an optimal delineator is achieved. It dynamically adjusted the weights based on system performance, enhancing the correlation between SXI scores and actual readmission outcomes.

Correlating SXI w.r.t. readmission within 30 days rate

The study utilized polynomial regression to explore the relationship between SXI scores and readmission within 30 days rates. By analysing the coefficient of determination (R²), the research team identified phases of improvement—initial, mid-term, and long-term—where changes in SXI scores significantly impacted readmission within 30 days rates.

The term initial improvement (short-term) in our study refers to the first stage of SXI scores, where a positive impact on the business outcome is observed. This phase typically occurs within the first few months after implementation of actionable insights.

Mid-term improvement represents the next stage of progress, identified as the period when SXI scores consistently show positive levels over a longer duration, generally ranging from 3 to 12 months, though this can vary.

Long-term improvement occurs when the business outcome either continues to improve gradually or reaches a stable plateau over an extended period, typically lasting a year or more in most cases.

Positive correlation between SXI scores and readmissions within 30 days led to an upward adjustment of positively weighted features, while negatively correlated features were adjusted downwards. Conversely, if a negative correlation was observed, adjustments were made accordingly. These refined weights were then applied to the dataset, simulating an initial improvement for attaining a target decision tree for scientific actionable insights or suggestions.

Model training and evaluation

The model training process starts with hyperparameter tuning, specifically focusing on the alpha parameter. This involves adjusting the SXI score by varying the alpha value between 0.5 and 1.5 in increments of 0.1. During each iteration, the SXI score is recalculated by multiplying the current SXI score with the alpha value, enabling fine-tuning to optimize the model’s performance.

SXInew=∝∗SXICurrent[2]

Where ∝ is the tuning parameter, ranging from 0.5 to 1.5 in increments of 0.1, SXI_Current is the Benchmark SXI score, SXI_new is the new SXI score after applying the alpha adjustment.

The dataset is divided into three parts: 70% for training, 20% for testing, and 10% for validation. This strategic split ensures the model is well-trained while also providing sufficient data for testing and validation, reducing the risk of overfitting and enabling an accurate assessment of model performance. During training, the optimized SXI scores are used as a “Super Feature”, playing a crucial role in enhancing the model’s predictive accuracy.

To thoroughly evaluate the model’s performance, various metrics are employed. Accuracy measures the overall correctness of the model’s predictions, while precision focuses on the proportion of true positive results among all positive predictions. The confusion matrix offers a detailed breakdown of true positive, true negative, false positive, and false negative outcomes, providing deeper insights into the model’s classification abilities. Additionally, the AUC is utilized to assess the model’s effectiveness in distinguishing between classes, serving as a robust indicator of its predictive power.

Accuracy=TP+TNTP+TN+FP+FN[3]

Precision=TPTP+FP[4]

where TP is the number of true positives, TN is the number of true negatives, FP is the number of false positives, FN is the number of false negatives.

The AUC for the ROC curve measures the classifier’s ability to distinguish between positive and negative classes. It is calculated based on the true positive rate (TPR) and false positive rate (FPR), with the ROC curve plotting TPR against FPR at different threshold settings. TPR and FPR are defined as follows:

TPR=TPTP+FN[5]

TPR=FPFP+TN[6]

AUC≈∑i=1n−1(TPRi+1+TPRi2)(FPRi+1−FPRi)[7]

where TPR_i and TPR_i+1 are the true positive rates at consecutive thresholds, FPR_i and FPR_i+1 are the false positive rates at consecutive thresholds.

Actionable insights for reducing readmission within 30 days.

In the decision tree model, features that positively influence readmission within 30 days are assigned positive weights, while those with a negative impact receive negative weights.

To initiate improvements, users specify the percentage increase for positive weighted features and the percentage decrease for negative weighted features. During the data transformation process, these adjustments are applied to each observation: positive weighted features are increased by the specified percentage, and negative weighted features are decreased. This transformation aims to simulate an initial enhancement in the dataset.

If the SXI scores are positively correlated with readmission within 30 days, the adjustment strategy for positive weighted features is as follows:

Adjusted Positive Feature Value = Orginal Feature Values∗(1+Percentage Increase)[8]

For negative weighted features, the strategy is:

Adjusted Negative Feature Value = Orginal Feature Values∗(1−Percentage Decrease)[9]

when a negative correlation exists between SXI scores and business outcomes, higher SXI scores are associated with poorer results. In this scenario, features that negatively impact business outcomes are given positive weights, while features that contribute positively are assigned negative weights.

Now, positive weighted features negatively affect business outcomes as their values increase, necessitating a strategy involving a percentage decrease. On the other hand, negative weighted features positively influence outcomes when their values rise, requiring a strategy involving a percentage increase. To drive improvements, users specify a percentage decrease for positive weighted features and a percentage increase for negative weighted features. During data transformation, positive weighted features are reduced by the specified percentage, while negative weighted features are increased. This process aims to simulate an initial improvement in the dataset, acknowledging the negative correlation.

In the case of a negative correlation, the adjustment strategy for positive weighted features that worsen outcomes when their values rise is:

Adjusted Positive Feature Value = Orginal Feature Values∗(1−Percentage Decrease)[10]

For negative weighted features, the strategy is:

Adjusted Negative Feature Value = Orginal Feature Values∗(1+Percentage Increase)[11]

Once the dataset has been adjusted, it is used as the training data for the RF model. The model learns the relationships between the features and outcomes by analyzing subsets of the data through multiple decision trees. Each decision tree within the RF captures different aspects of the feature-outcome relationship, and among these, a target decision tree path is identified. This path represents the sequence of feature splits that most accurately predict the outcome based on the adjusted data, encapsulating the complex relationships between the adjusted features and the business outcomes.

Feature importance

Table 2 highlights the top 5 most important features in the dataset, as identified by the top 5 algorithms. The feature num_lab_procedures stands out as the most significant, appearing 4 times across the algorithms. Following closely, num_medications, time_in_hospital, and level2_diag3 each appear 3 times, indicating their strong influence on the model outcomes. Finally, level2_diag2 is noted to appear twice, reflecting its relative importance. The table effectively illustrates the key features that consistently contribute to predictive performance across top 5 algorithms.

Classification metrics

Precision = (4,690/4,690 + 0) = 1. Recall = (4,690/4,690 + 0) = 1. F1-Score = (2 × 1 + 11 × 1) = 1. Based on the given calculations, the model demonstrates perfect performance across key evaluation metrics. Precision, which measures the proportion of correctly predicted positive cases out of all predicted positives, is calculated as 1, given that all 4,690 positive predictions are true positives (no false positives). Similarly, recall, which evaluates the model’s ability to identify all actual positive cases, is also 1, as there are no false negatives and the model successfully identifies all true positive cases. Consequently, the F1-Score, which represents the harmonic mean of precision and recall, is also 1, indicating an ideal balance between precision and recall. This suggests the model has made flawless predictions in this scenario.

Results

Discussions

The SXI delineation in (Figure 2) reveals that for readmissions within 30 days, 25.63% are categorized as above SXI, while for readmissions over 30 days, 76.78% fall below SXI. In terms of the overall class, 11.96% of readmissions within 30 days are above SXI, representing 24.22% of the total, whereas 38.74% of readmissions over 30 days are below SXI, constituting 75.78% of the total. Model evaluation indicates a training size of 32,831 and a test size of 9,380. Among the test cases, all 9,380 readmissions within and over 30 days were correctly predicted, resulting in an SXI accuracy of 100%.

Figure 2 SXI distribution graph. SXI, Sriya Expert Index.

The correlation coefficient of 0.95 (positively) in (Figure 3) between SXI and readmissions within 30 days indicates an exceptionally strong and positive correlation between these two variables. In practical terms, this implies that as the SXI score increases, there is a proportional and highly positive impact on readmissions within 30 days. So based on the correlation the SXI hypothesis would be higher the SXI, the better is the likelihood of patients with readmissions within 30 days and hence decreasing the SXI score should lead to a decrease in readmissions less than 30-day rates.

Figure 3 SXI correlation graph. SXI, Sriya Expert Index.

In more specific terms, with an initial 37.11% decrease in SXI scores, there is a corresponding and proportional 5% decrease in readmissions less than 30-day rates. Moving into the mid-term, a 62.26% decrease in SXI scores corresponds to a similar 10% decrease in readmissions less than 30-day rates. Looking towards the long-term, a 94.97% decrease in SXI scores is associated with a substantial 18.7% decrease in the likelihood of readmissions less than 30-day rates. This demonstrates the compounding effect of long-term improvements in reducing the negative outcomes, reinforcing the notion that a sustained focus on reducing the SXI can lead to significant and reducing the impact of patients with readmissions less than 30 days.

For the current decision (Figure 4) path for readmission more than 30 days, factors such as low service utilization (less than 6), reduced probability of insulin usage by 50%, and an infrequent occurrence of level 1 diagnosis1 (below 2 times) are indicative for those prone to readmission more than 30 days. Conversely, patients more likely to be readmitted less than 30 days tend to exhibit higher service utilization (greater than 5), frequent occurrence of level 1 diagnoses 2 and 3 (more than 7 times), and an increase in service utilization to exceed 6.

Figure 4 Current SXI decision tree. diag, diagnosis; num, number; SXI, Sriya Expert Index.

For the target decision (Figure 5) path for readmission more than 30 days, characteristics such as younger age (less than 30 years), lower service utilization (less than 4), and a lower number of medications taken (less than 26) are indicative factors for patients prone to readmission more than 30 days. Conversely, for those more likely to be readmitted less than 30 days, older age (greater than 30 years), longer hospital stays (more than 3.75 days), and a higher number of medications taken (more than 35) play significant roles.

• The study’s results in Table 3 clearly indicate that the SXI model outperforms traditional ML models in terms of accuracy, precision, and AUC. Notably, the SXI model shows a substantial improvement over extreme gradient boosting (XGBoost) and traditional ML from different studies, achieving a perfect accuracy rate of 100% representing a 30–40% increase in improvements. Additionally, the SXI model demonstrates significantly higher precision, reaching 100% compared to XGBoost and other traditional ML algorithms used in different studies. Moreover, with an AUC score of 1, the SXI model significantly outshines XGBoost AUC of 0.56, 0.84 of RF, [0.64–0.76] of NN and 0.8 of LR further highlighting the superiority of the SXI model. There are 4,690 (TN) cases in (Table 4) where the model correctly predicted “Readmitted within 30 days” when the actual value was “Readmitted within 30 days”.
• There are 4,690 (TP) cases in (Table 4) where the model correctly predicted “Readmitted more than 30 days” when the actual value was “Readmitted more than 30 days”.
• There are 0 (FP) cases in (Table 4) where the model incorrectly predicted “Readmitted more than 30 days” when the actual value was “Readmitted more than 30 days”.
• There are 0 (FN) cases in (Table 4) where the model incorrectly predicted “Readmitted within 30 days” when the actual value was “Readmitted within 30 days”.

Figure 5 Target SXI decision tree. num, number; SXI, Sriya Expert Index.

Sensitivity (recall or true positive rate)

Sensitivity=TPTP+FN[12]

Sensitivity = 4,690/(4,690 + 0) = 1 (or 100%).

Specificity (true negative rate)

Specificity=TNTN+FP[13]

Specificity =4,690/(4,690 + 0) = 1 (or 100%).

Conclusions

Traditional ML models are trained once for prediction, while SXI scoring is utilized for individual data points to determine their likelihood of patients with readmissions within 30 days. The scoring system correlates with readmissions within 30 days by identifying the strong relationship with r-squared value, which informs decision trees for achieving target reductions in readmissions within 30 days. These experimental suggestions inform necessary business decisions, including initial, mid-term, and long-term strategies. Future data points will receive scores to determine based on which we will determine whether they are likely to result in readmissions within 30 days or more.

This study highlights the predictive ability of SXI model, in analyzing historical patient data from 130 hospital in USA between 1999 to 2008 to predict and mitigate the impact of patients with experiencing readmissions within 30 days. By leveraging SXI, a dynamic score representing the likelihood of readmissions within 30 days, and employing the Proprietary Deep Neural Network model used in the study stands out for its advanced architecture and high performance in predicting patient readmissions within 30 days. Unlike traditional models, which often struggle with the complexity and high dimensionality of healthcare data, the Proprietary Deep Neural Network features a sophisticated structure designed to handle these challenges. The model iteratively adjusts algorithm weights based on significant feature contributions, optimizing predictive accuracy. This iterative adjustment is crucial for making precise predictions in clinical settings, where accurate forecasts can significantly impact patient outcomes and reduce healthcare costs.

One of the most significant advantages of the deep neural network model is its remarkable accuracy and precision. In this study, the model achieved a perfect accuracy of 100% and an AUC score of 1, outperforming other traditional models like XGBoost, which had an accuracy of 75.69% and an AUC of 0.56. This level of accuracy ensures that healthcare providers can rely on the model to make critical decisions about patient care, minimizing the likelihood of readmissions within 30 days and improving overall patient outcomes.

The study introduces a novel concept called AI², or two-layer artificial intelligence (AI), which enhances the model’s performance. The first layer of AI generates the SXI, a super feature that integrates insights from multiple ML algorithms. The second layer uses this SXI to refine predictions further, leading to better accuracy and performance. This two-layer approach allows the model to leverage the strengths of multiple algorithms, creating a robust and comprehensive predictive tool.

Key features of the Proprietary Deep Neural Network model include

• Automated hyperparameter tuning: by automatically locating the ideal hyperparameters using Bayesian optimization, the code allows the neural network to adjust and perform better without requiring human intervention.
• Flexibility in activation functions and optimizers: the model allows the optimization of activation functions and optimizers, providing flexibility to explore different configurations and adapt the neural network’s architecture to diverse datasets.
• Dynamic architecture based on dataset size: the architecture dynamically adjusts based on the size of the training dataset, demonstrating adaptability to different data scales and potentially improving generalization.
• The deep neural network model’s dynamic adjustment capability is another key feature that sets it apart. By continuously adjusting weights based on feature importance, the model remains highly responsive to the most relevant data. This dynamic nature is particularly important in healthcare, where patient data and conditions can vary widely, ensuring that the model’s predictions are always based on the most current and significant information.
• The SXI can be utilized in real-life healthcare scenarios as a predictive and decision-making tool, particularly in managing and reducing hospital readmission rates within 30 days.

Utilization in real-life scenarios

• Predicting readmission risks: SXI can predict the likelihood of patients being readmitted within 30 days after discharge. Healthcare providers can use this information to identify high-risk patients and intervene early to prevent readmissions, which can improve patient outcomes and reduce healthcare costs.
• Resource allocation: by identifying patients with high SXI scores (indicating a higher risk of readmission within 30 days), hospitals can allocate resources more effectively. For instance, more intensive follow-up care or post-discharge support can be provided to these patients to reduce their readmission risk.
• Personalized care plans: SXI can help tailor personalized care plans based on the patient’s risk profile. This could involve adjusting medications, scheduling more frequent check-ups, or providing additional education and support to ensure patients adhere to their treatment plans.
• Improving healthcare quality metrics: reducing readmission rates is a key metric in healthcare quality assessments. By utilizing SXI, healthcare organizations can improve their performance on this metric, which could enhance their reputation and financial incentives.

How SXI could help solve real-life healthcare problems

• Reducing readmission rates: SXI is directly correlated with readmission rates within 30 days, meaning that by monitoring and managing SXI, hospitals can effectively reduce these rates.
• Enhancing predictive accuracy: SXI has been shown to provide 100% accuracy in predicting readmission within 30 days rates, making it a highly reliable tool for healthcare providers.
• Cost efficiency: by reducing unnecessary readmissions, SXI can help healthcare providers save on costs associated with repeated hospital stays and treatments.

Actionable factors

• Patient monitoring: continuous monitoring of SXI for admitted patients to identify those at risk of readmission.
• Data-driven decisions: using SXI to guide clinical decisions, such as adjusting discharge plans or post-discharge follow-up schedules.
• AI-enhanced care: leveraging the AI-ML algorithms that underpin SXI to continually refine and improve patient care strategies.
• Cross-functional integration: integrating SXI into electronic health records (EHRs) systems to ensure that all healthcare providers have access to real-time risk assessments and can act accordingly.
• Correlation and decision support: by analyzing the correlation graph between SXI and readmission rates within 30 days over time (initial, mid-term, and long-term), healthcare providers can observe the effectiveness of interventions. The target decision tree provides actionable insights, enabling more precise decisions that can significantly reduce patient readmissions within 30 days. This combination of predictive analytics and decision support tools ensures that interventions are both timely and effective.
• Data driven decisions: with the actionable insights provided by the target decision tree, doctors can make more informed, data-driven decisions to reduce patient readmissions within 30 days. This tool offers a clear framework for identifying the most effective strategies based on patient data, improving care outcomes through precise interventions.

Acknowledgments

Funding: None.

Footnote

Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-162/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-162/coif). D.M., P.Y., A.T.J., S.K. are employed by Sriya.AI LLC. Sriya.AI LLC has filed US provisional patents on the underlying core technology and its applications in the healthcare industry. Full utility patents will be filed soon. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Since the research involved secondary analysis of anonymized data and did not include direct interaction with or intervention in human subjects, an ethics board review was not required. The study did not involve sensitive or personal data that would necessitate informed consent.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Dharmarajan K, Hsieh AF, Lin Z, et al. Diagnoses and timing of 30-day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA 2013;309:355-63. [Crossref] [PubMed]
2. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med 2009;360:1418-28. Erratum in: N Engl J Med 2011;364:1582. [Crossref] [PubMed]
3. Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA 2011;306:1688-98. [Crossref] [PubMed]
4. Khera R, Wang Y, Nasir K, et al. Evaluation of 30-Day Hospital Readmission and Mortality Rates Using Regression-Discontinuity Framework. J Am Coll Cardiol 2019;74:219-34. [Crossref] [PubMed]
5. Shulan M, Gao K, Moore CD. Predicting 30-day all-cause hospital readmissions. Health Care Manag Sci 2013;16:167-75. [Crossref] [PubMed]
6. Fernández-Delgado M, Cernadas E, Barro S. Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res 2014;15:3133-81.
7. Kulkarni P, Smith LD, Woeltje KF. Assessing risk of hospital readmissions for improving medical practice. Health Care Manag Sci 2016;19:291-9. [Crossref] [PubMed]
8. Au AG, McAlister FA, Bakal JA, et al. Predicting the risk of unplanned readmission or death within 30 days of discharge after a heart failure hospitalization. Am Heart J 2012;164:365-72. [Crossref] [PubMed]
9. Yu S, Farooq F, van Esbroeck A, et al. Predicting readmission risk with institution-specific prediction models. Artif Intell Med 2015;65:89-96. [Crossref] [PubMed]
10. Vedomske MA, Brown DE, Harrison JH. Random forests on ubiquitous data for heart failure 30-day readmissions prediction. 013 12th International Conference on Machine Learning and Applications. Miami, FL, USA: IEEE, 2013:415-21.
11. Davis S, Zhang J, Lee I, et al. Effective hospital readmission prediction models using machine-learned features. BMC Health Serv Res 2022;22:1415. [Crossref] [PubMed]
12. Xiao C, Ma T, Dieng AB, et al. Readmission prediction via deep contextual embedding of clinical concepts. PLoS One 2018;13:e0195024. [Crossref] [PubMed]