Enhancing emergency medical services (EMS) efficiency: design and validation of an SXI++ large numerical model (LNM) for response time prediction
Highlight box
Key findings
• Superior predictive accuracy: the SXI++ large numerical model (LNM) model achieved >94% accuracy categorizing emergency medical services (EMS) response times as “quick” or “slow”.
• Outperform traditional models: compared to traditional machine learning (ML) algorithms (75.69% accuracy), our model provides higher precision.
• Data: effectively handles challenges such as missing values and data imbalance.
• Operational impact: offers a scalable tool for optimizing resource allocation and efficiency, and potentially improving patient outcomes.
What is known and what is new?
• Traditional ML models show limited success in healthcare, with low accuracy and difficulty handling large datasets and imbalanced data, lacking granularity for actionable insights.
• Our model predicts EMS response times with high accuracy and strong correlation with outcomes. It addresses common data challenges such as missing values and imbalances, serving as a reliable tool to optimize EMS operations and potentially improve patient outcomes.
What is the implication, and what should change now?
• Optimized EMS operations and patient care: accurate prediction of “response time” enables better resource allocation, reducing delays and improving efficiency and outcomes in emergencies.
• Integration into EMS workflows: embedding the SXI++ LNM model into documentation systems will enable real-time application and decision-making.
• Scalability across systems: validating the model in diverse EMS settings and geographic regions will ensure adaptability and effectiveness.
• Education and training: targeted training for EMS personnel to leverage model insights can maximize practical impact.
• Data-driven decision making: incorporating the model into EMS workflows can transform decision-making processes by providing actionable insights based on real-time and historical data.
Introduction
Background
Ambulances form a critical part of pre-hospital emergency medical services (EMS). Studies have shown that in the case of adult and pediatric trauma, restricting ambulance response time results in better clinical outcomes. The response time of an ambulance is defined as the time difference between an ambulance being dispatched and its arrival on scene. A variety of factors influence EMS response time, such as the availability of ambulances, emergency medical dispatch (EMD) triage process, and call location (1-11). While there is no set standard for response time (as opposed to scene time), we use the same standard of 10 minutes.
EMS serves as the first-line response for many medical emergencies in the United States (12,13) and internationally (14). However, the stress on EMS agencies is rising with greater utilization coupled with greater financial fragility (15). Previous studies have focused on demographics of use, trauma outcomes when EMS is used (16-19), increased urbanization (20), financial realities (21,22), traffic (23,24), and provider skills (25). Some work has focused on the impacts of high response time (4,26). However, no work approaches this problem with clear machine-learning-driven modeling.
Traditional machine learning methods were used to analyze this dataset. Of the algorithms considered, the one that provided the best predictions about response time was the XGBoost algorithm. The XGBoost algorithm is a standard machine learning algorithm that uses gradient boosting on decision trees to improve performance. More details about the XGBoost can be found here (27,28) and in other standard machine learning references.
In this work, we have designed and implemented a model called the SXI++ large numerical model (LNM). The SXI++ framework includes generating a super feature using a weighted ensemble of machine learning algorithms, followed by a deep neural network (DNN) to adjust weights. The LNM and how it is implemented are discussed later in section “Methods” and the SXI++ framework is described in detail in (29,30).
Rationale and knowledge gap
Previous studies primarily utilized machine learning techniques to tackle the challenge of predicting EMS response times, particularly in identifying high-risk scenarios in emergency medical settings (23-26). However, these studies often faced limitations in interpreting the impact of various operational and clinical factors on response time predictions (23,24). Furthermore, traditional methods lacked the integration of advanced LNMs, which can leverage extensive datasets, including dispatch times, patient demographics, and incident characteristics, to predict EMS response times more precisely for specific scenarios. By incorporating SXI++ LNM model, which utilizes latent variable modelling to aggregate hidden patterns across features, the potential for accurate and actionable predictions is significantly enhanced, providing a deeper understanding of operational vulnerabilities.
Objective
In this paper, we aim to predict causes of increased response time and provide the factors responsible. We also want to avoid a black box approach and provide clear explainability. Furthermore, we want to use the explainability to make changes to reduce response times. To address these challenges, in this study, we (I) design and implement the SXI++ LNM; (II) evaluate the performance of the SXI++ LNM model as a binary classification system to predict EMS response times (quick ≤600 seconds, slow >600 seconds); (III) enhance the model using a proprietary DNN algorithm to improve predictions and correlate them with operational efficiency; (IV) explain the outcomes of the model using a simple interpretable decision tree framework, which can lead to evidence-based recommendations for improving EMS efficiency; and (V) compare the SXI++ LNM model’s results with traditional machine learning algorithms like XGBoost, highlighting its superior predictive accuracy and actionable insights for EMS resource optimization. We present this article in accordance with the TRIPOD reporting checklist (available at https://jmai.amegroups.com/article/view/10.21037/jmai-25-76/rc).
Methods
Methodology overview
The methodology focuses on processing and analysing the 2022 National Emergency Medical Services Information System (NEMSIS) data to evaluate trauma-related events and response times, integrating the SXI++ LNM approach for advanced analytics. The primary dataset used to be extracted and converted from its original format for preprocessing. Traumatic injury cases were filtered using specific codes and data was processed in chunks to handle their large size effectively. The initial dataset of 382,787 entries was refined to 208,550 by addressing null values in critical features.
SXI++ LNM significantly contributed to achieving high accuracy on unseen data by employing a combination of advanced latent variable modelling. The method constructs a latent network that identifies hidden relationships between features, capturing interdependencies not directly observable in the raw data. This process involves creating synthetic data points by extrapolating patterns from existing data, allowing the model to generalize effectively. Through iterative training, ensure robustness in capturing diverse scenarios within the dataset.
For unseen data, SXI++ LNM excels due to its ability to simulate edge cases and address data sparsity. By mapping feature relationships into a latent space, it reduces noise and emphasizes critical predictors of response categories (e.g., quick vs. slow responses). Additionally, the model incorporates feature-specific weighting schemes to balance the influence of rare variables, minimizing overfitting to the training data while enhancing generalization. This is especially effective when applied to processed datasets where features like age, dispatch reason, and patient condition were carefully encoded.
The high accuracy on unseen data demonstrates the effectiveness of SXI++ LNM in leveraging latent patterns and synthetic indexing to predict EMS response times accurately. This innovative approach ensures the model’s reliability across varying dataset sizes and feature combinations, providing actionable insights for improving EMS operations.
Dataset description
This study utilizes data from NEMSIS, a nationally recognized repository of EMS activation records funded by the National Highway Traffic Safety Administration (NHTSA). The dataset includes standardized data elements such as dispatch and scene time stamps patient demographics, and incident characteristics. Focusing on traumatic injury cases, the dataset was refined to exclude features with significant missing values, resulting in a subset optimized for predictive modeling. This structured and standardized dataset provides a robust foundation for analyzing EMS performance metrics like “response time”, enabling insights to improve response efficiency and patient outcomes.
Dataset source
The data utilized in this study originates from the 2022 NEMSIS dataset. The dataset comprises millions of EMS-related records voluntarily submitted by participating EMS agencies across the United States. The NEMSIS database is widely recognized as a reliable source for analyzing EMS performance metrics, with structured data elements standardized according to the NEMSIS Data Dictionary.
Since the research involved secondary analysis of anonymized data and did not include direct interaction with or intervention in human subjects, an ethics board review was not required. The study did not involve sensitive or personal data that would necessitate informed consent.
Impact of EMS
EMS systems are a critical component of healthcare infrastructure, providing prehospital emergency care and transportation during time-sensitive scenarios. Key metrics such as “response time” have a direct impact on patient survival rates, particularly for conditions like trauma, cardiac arrest, and stroke. Analyzing EMS data enables agencies to identify inefficiencies, optimize response times, and improve overall service delivery, ultimately saving lives and reducing healthcare costs.
Current use case
This study focuses on predicting the “response time”, a vital EMS metric that measures the duration between unit arrival at the scene and departure with the patient. The SXI++ LNM model was trained and validated using a subset of the NEMSIS v3.4.0 dataset, specifically records indicating traumatic injury scenarios This use case aims to address operational inefficiencies by categorizing response times into “quick” (≤600 seconds) and “slow” (>600 seconds) as shown in the (Table 1). Predictive insights from the model enable EMS agencies to allocate resources more effectively and proactively address delays in real-time.
Table 1
| Response time | Target variable | ||
|---|---|---|---|
| Class | Count | % | |
| >600 s or >10 min | Slow response | 156,982 | 75.16 |
| <600 s or <10 min | Quick response | 51,868 | 24.84 |
Table shows the number of records in each of the “quick” and “slow” categories. Note the unbalanced data.
Data preprocessing
The data preprocessing involved refining the NEMSIS v3.4.0 dataset to address missing values, feature redundancy, and class imbalance (29,30). Records with significant missing or irrelevant data were removed, while categorical features were encoded using one-hot encoding. We dropped the features that had more than 40% missing values and dropped all time-related features other than “eTimes_03” (unit notified by dispatch) and “eTimes_06” (unit arrived on scene), to calculate the response time. Key time-related features, such as “response time”, were derived by calculating the difference between these timestamps. Missing values in relevant features were imputed using mode or mean imputation, and low-frequency categories were grouped under “others” to enhance consistency. The final dataset consisted of 208,550 records and 97 features optimized for modelling. List of actionable features (those that can be changed and/or controlled by the EMS system that directly influence or predict the business outcome) and non-actionable (features that provide additional context but cannot be controlled by the EMS system) features and their descriptions are shown in Table 2. The response category was determined based on the target feature “response time”, categorizing cases into two classes: quick response (≤600 seconds) and slow response (>600 seconds). This classification served as the binary target variable for predictive modelling, enabling the assessment of factors influencing EMS response times. The processed dataset showed 24.84% of cases as quick responses (51,868 records) and 75.16% as slow responses (156,982 records). This categorization facilitated the application of machine learning models to predict response categories using the features in Table 2, iteratively refining the feature sets for optimal model performance.
Table 2
| Target variable | Response category |
|---|---|
| Non-actionable | PcrKey |
| Dispatch_Reason | |
| EMD_Performed | |
| Emergency_Department_Disposition | |
| Hospital_Disposition | |
| Gender | |
| Primary Method of Payment | |
| Type of Service Requested | |
| Level of Care of This Unit | |
| Response Mode to Scene | |
| First EMS Unit on Scene | |
| Possible Injury | |
| Transport Mode from Scene | |
| Age | |
| Actionable | Cardiac_Arrest |
| Cardiac_Arrest_Etiology | |
| AED_Use_Prior_to_EMS_Arrival | |
| monitored_arrest_rhythm | |
| Reason CPR | |
| End_of_Cardiac_Arrest_Event | |
| Number of Patients at Scene | |
| Mass Casualty Incident | |
| Triage Classification for MCI Patient | |
| Chief Complaint Anatomic Location | |
| Chief Complaint Organ System | |
| Patient condition on Arrival |
AED, automated external defibrillator; CPR, cardiopulmonary resuscitation; EMD, emergency medical dispatch; EMS, emergency medical services; MCI, mass casualty incident.
SXI++ LNM framework
The SXI++ LNM simplifies complex EMS performance metrics into actionable insights using a weighted composite score derived from 5–10 machine learning algorithms. By evaluating critical factors such as dispatch delays and transport modes, it predicts “response time” with high accuracy.
A proprietary DNN iteratively adjusts weights to enhance correlation with response times, ensuring robust and adaptable predictions. This dynamic scoring system empowers EMS agencies to optimize resource allocation, improve efficiency, and reduce delays, ultimately enhancing patient outcomes and healthcare system performance.
Preprocessing and normalization
The SXI++ LNM scoring mechanism begins with the preprocessing and normalization of the input dataset. This critical stage standardizes the data to ensure that all features are on a comparable scale, preventing any single feature from unduly influencing the final score. The normalization process takes into account the minimum and maximum values of each feature and adjusts them based on their correlation with the target variable. Features that are positively correlated are normalized by dividing the feature value by the maximum value, while negatively correlated features are normalized by subtracting the feature value from the maximum value and then dividing by that maximum value. This results in a normalized dataset that accurately reflects the relative importance and scale of each feature.
Statistical analysis
Data characteristics and preprocessing
The NEMSIS v3.4.0 dataset, containing over 10 million records, served as the foundation for this study. After rigorous filtering for relevance, particularly focusing on traumatic injury cases, the dataset was refined to include 208,550 records and 97 features. Key variables such as dispatch time, scene departure time, and patient demographics were prioritized for their direct influence on the target variable, “response time”. Data cleaning addressed missing values by removing features with over 40% missing data and imputing remaining gaps with mode or mean values. Time-related features were normalized into seconds to compute “response time”, while categorical features were encoded using one-hot encoding. Features with low-frequency categories were consolidated under an “others” label for consistency.
Model training and validation
The dataset was split into three parts: 70% for training, 20% for testing and 10% for Validation. The SXI++ LNM model was trained to categorize response times into “quick” (≤600 seconds) or “slow” (>600 seconds). Traditional machine learning algorithms, such as XGBoost, served as baselines for comparative performance analysis.
Insights from advanced metrics and feature importance
The SXI++ LNM model revealed valuable insights through its feature importance rankings. Factors such as dispatch delays, transport mode, and scene conditions were identified as the most critical determinants of response times. Sensitivity analyses confirmed the model’s robustness across different parameter settings, EMS scenarios, and geographic regions. By handling data imbalance and noise effectively during preprocessing, the model not only enhanced prediction accuracy but also ensured the generalizability of results to diverse EMS settings. These findings highlight the SXI++ LNM model’s ability to transform EMS operations through precise and actionable analytics.
Bivariate correlation analysis
After normalization, the data undergoes a bivariate correlation analysis in Figure 1, where pairwise correlations between all features are calculated. This step generates bi-variate correlation weights, which represent the average correlation of each feature with all others in the dataset. These weights are essential for understanding the interdependencies among features and their collective influence on the target variable. By using these correlation weights, the SXI++ LNM mechanism identifies the most impactful features and adjusts their contributions to the final score accordingly.
Base SXI score calculation
Following the correlation analysis in Figure 1, the SXI++ LNM scoring system calculates the individual base SXI scores. This involves computing a weighted sum of the normalized feature values, where the weights are derived from the bi-variate correlation analysis. The individual scores are then averaged to obtain the base SXI score, which serves as a benchmark for further analysis. Additionally, binary labels, known as Base SXI++ LNM Flags, are created by comparing each individual score with the base SXI score, categorizing data points into two groups based on their relative performance.
Least absolute shrinkage and selection operator (LASSO) regression adjustment
The SXI++ LNM mechanism in Figure 1 then utilizes a LASSO regression model to update the min-max mapping of each feature. In this step, the model is fitted to the normalized data and base SXI++ LNM flags, allowing for adjustments to feature importance based on the LASSO coefficients. Features with positive coefficients have their max values updated, while those with negative coefficients have their min values adjusted. This iterative process ensures that the feature mappings accurately represent their true influence on the target variable, leading to a more refined set of normalized data.
Composite weight calculation and final SXI score
Finally, the SXI++ LNM scoring in Figure 1 mechanism integrates multiple machine learning algorithms to derive composite weights for the features. These algorithms include Complement Naïve Bayes, XGBoost, Mutual Information, LASSO regression, and principal component analysis (PCA). The weights generated by these algorithms are combined to compute the final SXI score, ensuring robustness and reliability. Features with non-zero weights from each algorithm are retained, and their contributions are adjusted based on their significance. The final composite weights are then used to calculate individual SXI scores, which are averaged to produce the final benchmark SXI score.
Comparison with delineation
To achieve maximum delineation, iterative improvements are made by comparing the initial SXI++ LNM distribution with subsequent iterations. This process ensures that the delineation between different outcome groups is optimized over time.
Working of proprietary DNN
The proprietary DNN architecture integrates independent features and a target variable, along with updated base SXI scores and feature importance weights derived from common top features in the machine learning model. Consider a dataset with features sorted from highest frequency to lowest frequency; extra weightage is given to the top 5 common features, with importance based on their frequency of occurrence.
Initially, each feature is assigned a baseline weight of 1, and this weight is increased based on how frequently the feature appears in the top 5 common features. This approach ensures that more frequently occurring features are given higher importance, thereby enhancing their role in the analysis. The Custom Kernel Initializer function is utilized to modify the weight initialization process, giving extra weightage to the most important features based on their calculated importance.
The custom kernel initialization strategy involves calculating the effective input size using feature importance, followed by employing the Xavier/Glorot initialization method to generate random weights that account for custom feature weightage. The DNN’s architecture is then configured with an 80/20 train-test split and fine-tuned using Bayesian optimization to determine the best hyperparameters, including neurons, activation functions, optimizers, learning rates, batch sizes, and epochs.
This optimization process is supported by stratified k-fold cross-validation to ensure the model’s performance and generalization are robust, particularly when data is limited. The final model is compiled with a chosen optimizer and loss function, such as binary cross-entropy, and trained using the best hyperparameters identified through Bayesian optimization. The weights of the first five layers are then used to generate new SXI scores, ensuring that the model’s predictions are based on the most refined feature importance.
Iterative weight calibration system
The iterative weight calibration system aims to improve the SXI score and accuracy through a series of weight adjustments. The process begins with an initial assessment, where the current weights are used to calculate the SXI score and class delineation accuracy, serving as a benchmark for future comparisons. The system first evaluates whether any improvements can be made without adjusting the weights. If no improvement is found, it proceeds with positive weight adjustments, ranging from 0% to 100%, calculating the new SXI score and accuracy at each step. If positive adjustments show improvements, further adjustments are made beyond 100% until no additional gains are observed.
If positive adjustments do not yield improvements, the system then explores negative weight adjustments from 0% to −100%, following a similar process to identify the maximum delineation and accuracy gains within this range. Further negative adjustments are made if necessary, continuing beyond −100% until no further gains are detected. If neither positive nor negative adjustments improve the SXI score and accuracy, a new set of weights is generated, positively weighting the top 5 most important features (based on importance) as identified in the hidden layers of the neural network. These weights are then adopted as the new benchmark for future comparisons. This iterative process is repeated multiple times, refining the system by adjusting weights, applying penalties, and rewarding positive outcomes until the optimal delineator is achieved, ensuring that the SXI scores are continually improved.
Correlation of SXI++ LNM w.r.t response time
The SXI++ LNM can be used to find the correlation between SXI scores and the response time. This mechanism is similar to the one used in (27,28). The same mechanism can be used to develop initial short-term improvement, mid-term improvement, and long-term improvement.
Model training and evaluation
The entire pipeline of model training and evaluation, consisting of hyperparameter tuning, the use of the SXI score as a “super feature”, and several methods for accuracy measurement, are also implemented using the methods as described in (27,28). These steps allow us to get accurate and reliable results that can be used to provide actionable insights, as we describe next.
Actionable insights for early response time detection
In the decision tree model for early response time detection, clinical features that contribute positively to the likelihood of response time are assigned positive weights, while those that indicate a lower likelihood of Response Time are assigned negative weights. During the data transformation process, positive weighted features are increased by the specified percentage, while negative weighted features are decreased, enhancing the dataset’s overall accuracy for detecting response time. The implementation of this methodology is described in detail in (27,28).
Once the dataset has been adjusted using the specified transformations, it is employed to train a decision tree. The tree learns the relationships between various clinical features and the likelihood of response time by analysing the adjusted dataset through multiple decision trees. The tree captures distinct aspects of the feature-response time relationship, and a target decision tree path is identified. This path represents the sequence of feature splits that most accurately predicts the risk of response time based on the adjusted data. By utilizing this approach, the model provides a detailed understanding of the factors contributing to response time risk and offers valuable insights for effective intervention strategies.
Ethical statement
The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. Since the research involved secondary analysis of anonymized data and did not include direct interaction with or intervention in human subjects, an ethics board review was not required. The study did not involve sensitive or personal data that would necessitate informed consent.
Results
From the model development described above, subsets of features were selected for further analysis. We used only actionable features, where the EMS system can make realistic changes, and another subset of features with all the actionable features identified previously plus three non-actionable features (which the EMS system cannot change) to provide insight into the system operations. These results are described below.
From the decision tree in (Figure 2), we observe that the classification of responses into “quick” or “slow” is influenced by key factors such as the probability of EMD_Performed, Response Mode to Scene, and Transport Mode from Scene. A quick response is more likely when the probability of EMD being “Yes, unknown if pre-arrival instructions given” is 40% or higher. Additionally, if the probability of the Response Mode to Scene being classified as “Non-emergent” is 60% or higher, or if the Transport Mode from Scene is “Emergent Downgraded to Non-Emergent” with a probability of 60% or more, the response is classified as quick. These conditions indicate that pre-arrival procedures and downgraded transport urgency contribute to faster response times.
On the other hand, a slow response is expected when the probability of EMD being “Yes, unknown if pre-arrival instructions given” is 40% or lower, or when the probability of Emergency Department Disposition being “Discharged/Transferred to another hospital” is 60% or lower. Furthermore, if the Response Mode to Scene is “Non-Emergent Upgraded to Emergent” with a probability of 60% or higher, the response is categorized as slow. These conditions suggest that a lack of pre-arrival guidance, limited hospital transfers, and the need for escalation in response contribute to slower response times.
From the decision tree in (Figure 3), we observe that the classification of responses into “quick” or “slow” is influenced by key factors such as the Level of Care of this Unit, Response Mode to Scene, and Emergency Department Disposition. A quick response is more likely when the probability of the Level of Care of this Unit being classified as “Non-Emergent Upgraded to Emergent” is 60% or higher. Additionally, if the probability of the Response Mode to Scene being “Emergent” is 60% or lower, or if the Emergency Department Disposition is classified as “Routine discharge” with a probability of 60% or more, the response is categorized as quick. These conditions suggest that upgrades in care level, lower emergency response mode, and routine hospital discharges contribute to faster response times.
Conversely, a slow response is expected when the probability of the Level of Care of this Unit being “Non-Emergent Upgraded to Emergent” is 60% or higher, or when the Response Mode to Scene is classified as “Emergent” with a probability of 60% or more. Additionally, if the probability of EMD_Performed being “Yes, Unknown If Pre-Arrival Instructions Given” is 40% or lower, or if the Type of Service Requested is “Hospital to Hospital Transfer” with a probability of 60% or more, the response is categorized as slow. These conditions indicate that escalated response needs, lack of pre-arrival guidance, and inter-hospital transfers contribute to slower response times.
Use case overview (Tables 3-9)
Table 3
| Metric | Data | LNM (unseen data) |
|---|---|---|
| Number of rows | 208,550 | 50,000 |
| Training | 145,985 | Only testing |
| Testing | 41,710 | – |
| Validation | 20,855 | – |
| Target variable distribution | ||
| Slow response | 75.16% | 73.4% |
| Quick response | 24.84% | 26.6% |
LNM, large numerical model.
Table 4
| Metric | SXI (data) | SXI (LNM) |
|---|---|---|
| Accuracy (95% CI) | 98.13% (0.9795–0.9830) | 99.13% (0.9900–0.9925) |
| Precision (95% CI) | 93.36% (0.9307–0.9364) | 97.43% (0.9723–0.9762) |
| Recall (95% CI) | 99.71% (0.9963–0.9978) | 99.57% (0.9948–0.9965) |
| AUC | 1.000 | 0.999 |
| Sensitivity (95% CI) | 99.71% (0.9963–0.9978) | 99.57% (0.9948–0.9965) |
| Specificity (95% CI) | 99.69% (0.9961–0.9976) | 99.58% (0.9949–0.9966) |
| Features | 95 (all) | 95 |
AUC, area under the curve; CI, confidence interval; LNM, large numerical model.
Table 5
| Metric | SXI (data) | SXI (LNM) |
|---|---|---|
| Accuracy (95% CI) | 87.26% (0.8691–0.8760) | 80.6% (0.8023–0.8096) |
| Precision (95% CI) | 53.19% (0.5262–0.5376) | 58.85% (0.5837–0.5933) |
| Recall (95% CI) | 99.58% (0.9949–0.9966) | 88.5% (0.8818–0.8881) |
| AUC | 0.900 | 0.860 |
| Sensitivity (95% CI) | 99.58% (0.9949–0.9966) | 88.5% (0.8818–0.8881) |
| Specificity (95% CI) | 99.58% (0.9949–0.9966) | 88.49% (0.8817–0.8880) |
| Features | 35 | 35 |
AUC, area under the curve; CI, confidence interval; LNM, large numerical model.
Table 6
| Metric | SXI (data) | SXI (LNM) |
|---|---|---|
| Accuracy (95% CI) | 98.56% (0.9840–0.9871) | 99.35% (0.9923–0.9946) |
| Precision (95% CI) | 94.55% (0.9428–0.9481) | 98.00% (0.9783–0.9817) |
| Recall (95% CI) | 99.71% (0.9963–0.9978) | 99.84% (0.9978–0.9989) |
| AUC | 0.996 | 1.000 |
| Sensitivity (95% CI) | 99.71% (0.9963–0.9978) | 99.84% (0.9978–0.9989) |
| Specificity (95% CI) | 99.86% (0.9980–0.9991) | 99.84% (0.9978–0.9989) |
| Features | 40 | 40 |
AUC, area under the curve; CI, confidence interval; LNM, large numerical model.
Table 7
| Metric | SXI (data) | SXI (LNM) |
|---|---|---|
| Target variable distribution | ||
| Slow response | 75.16% | 15,042 (75.21%) |
| Quick response | 24.84% | 4,958 (24.79%) |
| Accuracy (95% CI) | 98.05% (0.9782–0.9827) | 98.27% (0.9805–0.9848) |
| Precision (95% CI) | 94.66% (0.9427–0.9503) | 95.52% (0.9517–0.9586) |
| Recall (95% CI) | 99.17% (0.9900–0.9932) | 99.30% (0.9914–0.9943) |
| AUC | 0.998 | 0.998 |
| Sensitivity (95% CI) | 99.17% (0.9900–0.9932) | 99.30% (0.9914–0.9943) |
| Specificity (95% CI) | 99.14% (0.9897–0.9929) | 99.27% (0.9911–0.9941) |
| Features | 40 | 40 |
Training and validation were performed on a subset of unseen data with 20k records. AUC, area under the curve; CI, confidence interval; LNM, large numerical model.
Table 8
| Metric | SXI (data) | SXI (LNM) |
|---|---|---|
| Target variable distribution | ||
| Slow response | 75.16% | 3,814 (76.28%) |
| Quick response | 24.84% | 1,186 (23.72%) |
| Accuracy (95% CI) | 96.95% (0.9645–0.9739) | 96.55% (0.9601–0.9703) |
| Precision (95% CI) | 90.00% (0.8914–0.9081) | 91.02% (0.9020–0.9178) |
| Recall (95% CI) | 99.10% (0.9873–0.9938) | 98.50% (0.9804–0.9887) |
| AUC | 0.985 | 0.987 |
| Sensitivity (95% CI) | 99.10% (0.9873–0.9938) | 98.50% (0.9804–0.9887) |
| Specificity (95% CI) | 99.03% (0.9865–0.9932) | 98.55% (0.9809–0.9892) |
| Features | 40 | 40 |
Training and validation were performed on a subset of unseen data with 5k records. The model still performs accurately despite the small size of the dataset. AUC, area under the curve; CI, confidence interval; LNM, large numerical model.
Table 9
| Metric | SXI (data) | SXI (LNM) |
|---|---|---|
| Target variable distribution | ||
| Slow response | 75.16% | 770 (77.0%) |
| Quick response | 24.84% | 230 (23.0%) |
| Accuracy (95% CI) | 83.00% (0.8057–0.8523) | 79.07% (0.7645–0.8147) |
| Precision (95% CI) | 62.00% (0.5860–0.6529) | 59.59% (0.5613–0.6298) |
| Recall (95% CI) | 90.00% (0.8764–0.9201) | 86.20% (0.8357–0.8852) |
| AUC | 0.859 | 0.810 |
| Sensitivity (95% CI) | 90.00% (0.8764–0.9201) | 86.20% (0.8357–0.8852) |
| Specificity (95% CI) | 90.00% (0.8764–0.9201) | 86.11% (0.8347–0.8844) |
| Features | 40 | 40 |
Validation was performed on a subset of unseen data with 1,000 records. Despite the significantly smaller dataset, performance deteriorates only slightly, demonstrating the capability of the LNM. AUC, area under the curve; CI, confidence interval; LNM, large numerical model.
The dataset for this analysis consists of two components (see Table 3): the LNM dataset, which is unseen data containing 50,000 rows used exclusively for testing, and the main dataset comprising 208,550 rows. The main dataset is divided into training, testing, and validation subsets, with 145,985 rows for training, 41,710 rows for testing, and 20,855 rows for validation. The target variable in the main dataset is distributed as 75.16% “slow response” and 24.84% “quick response”. Similarly, the LNM dataset has a distribution of 73.4% “slow response” and 26.6% “quick response”.
In Case I (baseline model—all features), the model was trained using all 95 features without any feature reduction (see Table 4). For the main dataset, the model achieved an accuracy of 98.13%, a precision of 93.36%, and an AUC of 1.0. On the LNM dataset, the performance was slightly better, with an accuracy of 99.13%, a precision of 97.43%, and an AUC of 0.999.
In Case II (actionable features only), the model was trained using 35 actionable features (see Table 5). On the main dataset, the accuracy dropped to 87.26%, with a precision of 53.19% and an AUC of 0.9. The LNM dataset showed a similar trend, with an accuracy of 80.6%, a precision of 58.85%, and an AUC of 0.86.
Case III (actionable + 3 non-actionable features with age as numerical) introduced three non-actionable variables, including treating “age” as a numerical feature (see Table 6). This improved performance significantly. For the main dataset, the accuracy rose to 98.56%, the precision to 94.55%, and the AUC to 0.996. The LNM validation dataset also showed strong results, with an accuracy of 99.35%, a precision of 98.0%, and an AUC of 1.0.
In Case IV (subset with 20k records), a subset of 20,000 records was evaluated (see Table 7) using 40 features (35 actionable and 3 non-actionable). The subset had a target variable distribution of 75.21% “slow response” and 24.79% “quick response”. On the main dataset, the model achieved an accuracy of 98.05%, a precision of 94.66%, and an AUC of 0.998. On the LNM dataset, the accuracy was 98.27%, the precision was 95.52%, and the AUC was 0.998.
In Case V (subset with 5k records), a smaller subset of 5,000 records was used (see Table 8), maintaining the 40 features. The target variable distribution was 76.28% “slow response” and 23.72% “quick response”. The main dataset’s performance showed an accuracy of 96.95%, a precision of 90.0%, and an AUC of 0.985. The LNM dataset achieved an accuracy of 96.55%, a precision of 91.02%, and an AUC of 0.987.
Finally, in Case VI (subset with 1k records), a subset of 1,000 records was evaluated (see Table 9). This subset had a target variable distribution of 77.0% “slow response” and 23.0% “quick response”. The performance on the main dataset dropped significantly, with an accuracy of 83.0%, a precision of 62.0%, and an AUC of 0.859. Similarly, the LNM dataset had an accuracy of 79.07%, a precision of 59.59%, and an AUC of 0.81.
These cases illustrate the trade-offs between model complexity, feature selection, and dataset size in achieving optimal performance.
Discussion
Key findings
The study showcases the remarkable predictive accuracy of the SXI++ LNM model, achieving >94% accuracy in categorizing EMS response times as “quick” or “slow”. Specifically, we have an accuracy of 99.13% with 95% CI between 99% and 99.25%, precision of 97.43% and 95% CI between 97.23% and 97.62%, and recall of 99.57% with 95% CI between 99.48% and 99.65%. This accuracy outperformed traditional models such as XGBoost, which achieved a maximum accuracy of 68.58%. The model’s high correlation with slow response rates and effective handling of data imbalances underlines its robustness and utility for EMS.
Strengths and limitations
The primary strength lies in the SXI++ LNM model’s ability to deliver superior accuracy and insights using advanced latent variable modelling and pattern-based data comparison. Its reliance on detailed preprocessing and tailored weighting mechanisms ensures robust predictions. However, limitations include the need for significant computational resources and questions about scalability across diverse healthcare environments.
Comparison with similar research
Large language models (LLMs), large quantitative models (LQMs), and the SXI++ LNM represent three distinct yet complementary AI paradigms, each excelling in different domains. LLMs specialize in processing and generating human-like text, making them ideal for tasks involving natural language understanding, content creation, and communication-based applications across industries like customer service and healthcare. However, they struggle with numerical precision and complex quantitative analysis. LQMs, on the other hand, focus on handling large-scale numerical datasets, performing statistical modelling, optimization, and predictive analysis, making them suitable for fields such as finance and scientific research. Despite their potential, LQMs are still largely in the experimental phase and face challenges related to practical implementation and scalability. In contrast, the SXI++ LNM stands out by offering a robust and field-tested solution for structured numerical data, providing high-precision predictions and actionable insights through advanced modelling techniques like latent variable analysis and feature-specific weighting. Unlike LLMs and LQMs, the SXI++ LNM has already demonstrated real-world effectiveness, achieving exceptional accuracy in critical applications such as emergency response prediction, making it a superior choice for industries that demand reliable, data-driven decision-making.
Explanations of findings
The model’s success can be attributed to its unique preprocessing techniques, such as feature-specific normalization, and iterative weight calibration, which ensured the optimal importance of critical features. The strong correlation between SXI scores and response Time efficiency underscores the effectiveness of data-driven strategies in identifying slow response risks.
Implications, recommendations, and user interaction
Integrating the SXI++ LNM model into EMS workflows has the potential to revolutionize emergency response operations by enabling real-time decision-making, optimizing resource allocation, and minimizing delays, which can significantly enhance patient care and survival rates in time-sensitive emergencies. To maximize its impact, EMS agencies should adopt the model within electronic patient care reporting systems, coupled with comprehensive training programs to ensure personnel can effectively utilize its insights. Furthermore, validating the model across diverse healthcare settings will enhance its scalability and adaptability. To facilitate seamless adoption, user-friendly interfaces with real-time visualizations and alerts based on SXI scores should be developed, empowering EMS staff to make swift, data-driven decisions during critical scenarios.
Future investigation
While this study focuses on EMS response times, future investigations should explore broader applications, such as predicting patient outcomes or optimizing dispatch strategies. Additional research is needed to evaluate the model’s performance across varied geographic and operational contexts. Expanding the feature set and exploring advanced architectures could further enhance predictive capabilities.
Conclusions
The SXI++ LNM model represents a ground-breaking advancement in EMS analytics by delivering remarkable accuracy in predicting “response time” metrics, achieving >94% accuracy in classifying responses as “quick” or “slow”. This surpasses traditional models like XGBoost, which reached a maximum accuracy of 68.58 %, and demonstrates the transformative potential of leveraging advanced latent variable modelling and pattern-based data comparison. The model’s ability to effectively handle data imbalances, capture latent patterns, and provide actionable insights makes it a powerful tool for optimizing EMS workflows and improving patient outcomes. The model’s utility extends to unseen datasets, where its ability to generalize and simulate edge cases ensures robust predictions even in unfamiliar scenarios. By mapping complex relationships into a latent space, SXI++ LNM reduces noise and emphasizes critical predictors, enabling EMS agencies to allocate resources proactively and minimize delays. Its performance on unseen data, maintaining high accuracy and reliability, underscores its adaptability and essential role in addressing real-world EMS challenges. This research highlights the importance of integrating SXI++ LNM into EMS operations for real-time decision-making, resource optimization, and enhancing patient care, particularly in diverse and evolving emergency scenarios. Further validation, especially with external data, and adaptation in various healthcare settings can amplify its impact, ensuring its scalability and long-term relevance.
Acknowledgments
None.
Footnote
Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://jmai.amegroups.com/article/view/10.21037/jmai-25-76/rc
Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-25-76/prf
Funding: None.
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-25-76/coif). P.Y., R.K. and S.K. are employed by Sriya.AI LLC. M.B. is employed by Clarkson University and is a consultant with Sriya.AI LLC. The authors have no other conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. Since the research involved secondary analysis of anonymized data and did not include direct interaction with or intervention in human subjects, an ethics board review was not required. The study did not involve sensitive or personal data that would necessitate informed consent.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Gibson G. Evaluative criteria for emergency ambulance systems. Soc Sci Med 1967;1973:425-54. [Crossref] [PubMed]
- ReVelle C. Facility location: a review of context-free and EMS models. Health Serv Res 1977;12:129-46.
- Pepe PE. Emergency medical services. Acad Emerg Med 1994;1:131-3. [Crossref] [PubMed]
- Cabral ELDS, Castro WRS, Florentino DRM, et al. Response time in the emergency services. Systematic review. Acta Cir Bras 2018;33:1110-21. [Crossref] [PubMed]
- Do YK, Foo K, Ng YY, et al. A quantile regression analysis of ambulance response time. Prehosp Emerg Care 2013;17:170-6. [Crossref] [PubMed]
- Abdulla B, Birgisson B. Characterization of resilience of networks to uncertain disruptions: A case study of Houston road network during Hurricane Harvey. In: ASCE International Conference on Transportation and Development (ASCE ICTD). Reston, VA: ASCE; 2020a. doi:
10.1061/9780784483169.004 . - Abdulla B, Birgisson B. Predicting road network vulnerability to fluvial flooding using machine learning classifiers: Case study of Houston during Hurricane Harvey. In: Proc., Construction Research Congress (CRC) 2020. Reston, VA: ASCE; 2020b.
- Abdulla B, Kiaghadi A, Rifai HS, et al. Characterization of vulnerability of road networks to fluvial flooding using SIS network diffusion model. J Infrastruct Preserv Resil 2020; [Crossref]
- Christensen EF, Bendtsen MD, Larsen TM, et al. Trends in diagnostic patterns and mortality in emergency ambulance service patients in 2007-2014: a population-based cohort study from the North Denmark Region. BMJ Open 2017;7:e014508. [Crossref] [PubMed]
- Lowthian JA, Jolley DJ, Curtis AJ, et al. The challenges of population ageing: accelerating demand for emergency ambulance services by older patients, 1995-2015. Med J Aust 2011;194:574-8. [Crossref] [PubMed]
- Hagihara A, Hasegawa M, Hinohara Y, et al. The aging population and future demand for emergency ambulances in Japan. Intern Emerg Med 2013;8:431-7. [Crossref] [PubMed]
- McLay LA, Boone EL, Brooks JP. Analyzing the volume and nature of emergency medical calls during severe weather events using regression methodologies. Socio Econ Plan Sci 2012;46:55-66.
- Liu A, Soneja SI, Jiang C, et al. Frequency of extreme weather events and increased risk of motor vehicle collision in Maryland. Sci Total Environ 2017;580:550-5. [Crossref] [PubMed]
- Wang HE, Mann NC, Jacobson KE, et al. National characteristics of emergency medical services responses in the United States. Prehosp Emerg Care 2013;17:8-14. [Crossref] [PubMed]
- Lowthian JA, Cameron PA, Stoelwinder JU, et al. Increasing utilisation of emergency ambulances. Aust Health Rev 2011;35:63-9. [Crossref] [PubMed]
- Haagsma JA, Graetz N, Bolliger I, et al. The global burden of injury: incidence, mortality, disability-adjusted life years and time trends from the Global Burden of Disease study 2013. Inj Prev 2016;22:3-18. [Crossref] [PubMed]
- Benjet C, Bromet E, Karam EG, et al. The epidemiology of traumatic event exposure worldwide: results from the World Mental Health Survey Consortium. Psychol Med 2016;46:327-43. [Crossref] [PubMed]
- Williamson K, Ramesh R, Grabinsky A. Advances in prehospital trauma care. Int J Crit Illn Inj Sci 2011;1:44-50. [Crossref] [PubMed]
- Hedges JR, Feero S, Moore B, et al. Factors contributing to paramedic onscene time during evaluation and management of blunt trauma. Am J Emerg Med 1988;6:443-8. [Crossref] [PubMed]
- McDonald RI. The effectiveness of conservation interventions to overcome the urban-environmental paradox. Ann N Y Acad Sci 2015;1355:1-14. [Crossref] [PubMed]
- Beninde J, Veith M, Hochkirch A. Biodiversity in cities needs space: a meta-analysis of factors determining intra-urban biodiversity variation. Ecol Lett 2015;18:581-92. [Crossref] [PubMed]
- Kim SH, Lee YH. Iterative optimization algorithm with parameter estimation for the ambulance location problem. Health Care Manag Sci 2016;19:362-82. [Crossref] [PubMed]
- Peyravi M, Khodakarim S, Örtenwall P, et al. Does temporary location of ambulances ("fluid deployment") affect response times and patient outcome? Int J Emerg Med 2015;8:37. [Crossref] [PubMed]
- Nogueira LC Jr, Pinto LR, Silva PM. Reducing Emergency Medical Service response time via the reallocation of ambulance bases. Health Care Manag Sci 2016;19:31-42. [Crossref] [PubMed]
- Aringhieri R, Carello G, Morale D. Supporting decision making to improve the performance of an Italian Emergency Medical Service. Ann Oper Res 2016;236:131-48.
- Lawner BJ, Hirshon JM, Comer AC, et al. The impact of a freestanding ED on a regional emergency medical services system. Am J Emerg Med 2016;34:1342-6. [Crossref] [PubMed]
- Géron A. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. 3rd ed. Sebastopol, CA: O'Reilly Media; 2022.
- Howard J, Gugger S. Deep Learning for Coders with FastAI and PyTorch: AI Applications Without a PhD. Sebastopol, CA: O'Reilly Media; 2020.
- Mahto D, Yadav P, Joseph AT, et al. AI2-SXI algorithm enables predicting and reducing the risk of less than 30 days patient readmissions with 99% accuracy and precision. J Med Artif Intell 2025;8:10.
- Kumar R, Yadav P, Kilambi S, et al. Development and validation of the Sriya Expert Index Plus (SXI++) framework for heart disease prediction and risk reduction through clinical decision support. J Med Artif Intell 2025; [Crossref]
Cite this article as: Yadav P, Kumar R, Banavar M, Kilambi S. Enhancing emergency medical services (EMS) efficiency: design and validation of an SXI++ large numerical model (LNM) for response time prediction. J Med Artif Intell 2026;9:14.


