Developing an explainable artificial intelligence framework for patient-reported outcomes: an interpretable approach for understanding patient experiences in vascular care
Original Article

Developing an explainable artificial intelligence framework for patient-reported outcomes: an interpretable approach for understanding patient experiences in vascular care

Arash Harzand1,2, Sam Taimourzadeh3, Qianhui Lu3, Julie B. Prillinger3, Jennifer T. McCaney4, David Rhew5

1Division of Cardiology, Department of Medicine, Emory University School of Medicine, Atlanta, GA, USA; 2Atlanta VA Health Care System, Decatur, GA, USA; 3Abbott Laboratories, Santa Clara, CA, USA; 4Department of Medicine, University of California, Los Angeles, CA, USA; 5Microsoft, New York, NY, USA

Contributions: (I) Conception and design: All authors; (II) Administrative support: A Harzand; (III) Provision of study materials or patients: S Taimourzadeh, Q Lu, JB Prillinger; (IV) Collection and assembly of data: S Taimourzadeh, Q Lu; (V) Data analysis and interpretation: A Harzand, S Taimourzadeh, Q Lu, JB Prillinger; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Correspondence to: Arash Harzand, MD, MBA. Division of Cardiology, Department of Medicine, Emory University School of Medicine, 1670 Clairmont Road, CVNR (151R), Decatur, GA 30033, USA; Atlanta VA Health Care System, Decatur, GA, USA. Email: arash.harzand@emory.edu.

Background: Agentic artificial intelligence (AI) systems can autonomously generate insights from healthcare data, yet there remains a need for transparent, reproducible, and clinically interpretable approaches for translating patient-reported outcomes into actionable improvements. This study evaluated an explainable machine learning (ML) framework applied to structured survey data to identify drivers of perceived care quality following coronary and peripheral treatment.

Methods: A four-step analytic pipeline was applied to the United States (U.S.) cohort (n=131) of the Beyond Intervention (BI) Year 3 survey. Natural language processing aligned BI survey items with validated Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) domains. Patient-level features (n=31) were then used to train a stacked ensemble ML model (Support Vector Machine + Random Forest + Gradient Boosting → logistic regression meta-learner) for predicting survey responses. SHapley Additive exPlanations (SHAP) analysis quantified feature impact, and subgroup likelihood analyses evaluated demographic and clinical variation.

Results: Three BI survey items were most semantically aligned with HCAHPS quality domains. The stacked model achieved test set area under the curve values of 0.774 [95% confidence interval (CI): 0.680–0.867] for Question 1, 0.761 (95% CI: 0.634–0.887) for Question 2, and 0.717 (95% CI: 0.594–0.840) for Question 3. Age and digital health familiarity were consistent associative factors across outcomes. Younger patients (aged 35–44 years) were 4.7 (95% CI: 2.1–8.3) times more likely to report negative perceptions, while patients with greater digital health experience more often expressed uncertainty or feelings of burden. Socioeconomic status (SES) and prior cardiovascular events also emerged as important modifiers.

Conclusions: In the era of autonomous AI, interpretable ML frameworks are essential for clarifying why patient groups perceive care differently. Digital health literacy, age, and SES were consistently associated with patient experience, underscoring the need for explainable approaches that support targeted interventions and trustworthy feedback loops. Brier scores (0.197–0.224) confirm well-calibrated predictions.

Keywords: Patient-reported outcomes; cardiovascular care; machine learning (ML); digital health literacy


Received: 23 February 2026; Accepted: 10 August 2026; Published online: 20 September 2026.

doi: 10.21037/jmai-2026-0063


Highlight box

Key findings

• An interpretable stacked machine learning framework (Random Forest + Support Vector Machine + Gradient Boosting feeding a logistic meta-classifier) predicted three Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS)-aligned patient-experience items with area under the curve 0.717–0.774.

• Age and digital health familiarity consistently ranked among the top associative factors.

What is known and what is new?

• Patient-reported experience is increasingly used in vascular care.

• Our study adds an explainable pipeline that links survey items to validated HCAHPS domains and quantifies subgroup differences with SHapley Additive exPlanations and likelihood ratios.

What is the implication, and what should change now?

• Programs that bolster digital literacy and expectation-setting for younger, tech-engaged patients may improve post-discharge experience; explainable artificial intelligence helps target those interventions transparently.


Introduction

Cardiovascular disease (CVD) remains a leading cause of morbidity and mortality worldwide (1). While biomedical data and clinical outcomes have historically guided management, patient-reported outcomes and peri-hospitalization perceptions of care are increasingly recognized as essential for comprehensive, patient-centered care (2,3). Since the introduction of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey, use of patient experience surveys has grown significantly, and health systems now systematically collect large volumes of patient survey data. These data can potentially affect which treatments are prioritized or how treatments are administered, but analysis has not kept pace with the evolving landscape of artificial intelligence (AI) (4,5).

The rapid emergence of agentic AI systems—large language models capable of autonomous reasoning, task execution, and narrative synthesis—has transformed data analysis. These systems can ingest and analyze survey data end-to-end, raising the question of whether bespoke analytic frameworks remain necessary. However, agentic AI often functions as a ‘black box’, with limited transparency into how conclusions are reached, creating challenges for clinical trust, reproducibility, and governance.

This study responds to that gap by presenting a transparent and reproducible analytic pipeline that integrates natural language processing (NLP), supervised machine learning (ML), and explainable AI methods to identify factors associated with perceived vascular care quality (6-11). Using the BI Year 3 U.S. survey cohort (Abbott Laboratories, Santa Clara, CA, USA), a global dataset capturing patient experiences after coronary and peripheral interventions, the framework highlights how patient characteristics—including age, socioeconomic status (SES), and digital health literacy—are associated with perceptions of care (3,12). By contrasting this interpretable pipeline with the growing ubiquity of agentic AI, this work demonstrates the continued importance of explainability for translating patient-reported data into actionable insights. We present this article in accordance with the TRIPOD+AI reporting checklist (available at https://jmai.amegroups.com/article/view/10.21037/jmai-2026-0063/rc).


Methods

Design, population and dataset

Building on prior work (12), a new four-step analytic pipeline was applied to the Year 3 BI dataset: (I) an initial NLP model was used to identify questions the most relevant to patient perceived quality of care (QOC). (II) The available baseline demographic data were then used to train a supervised ML model to predict patients’ responses to the NLP identified questions. (III) Feature importance analysis then ranked patient features according to their predictive ability after which (IV) patient subgroups were compared to estimate the likelihood of receiving positive or negative responses for each question according to the most predictive features.

The global BI dataset includes responses from 1,350 patients with coronary artery disease (CAD) and peripheral artery disease (PAD) who were asked closed-ended questions on perceived care quality in the post-acute phase (e.g., post-discharge, post-intervention). The primary survey was conducted between March and May 2022 with participants from the U.S., Mexico, Brazil, Chile, Canada, United Kingdom, Italy, Spain, Germany, Russia, Saudi Arabia, South Africa, Australia/New Zealand, Japan, China, India, and France. The analysis was restricted to the U.S. patient cohort to minimize potential confounding from cross-national variations in cultural attitudes toward care, socioeconomic conditions, and structural factors within healthcare systems that may influence patient perceptions. The HCAHPS benchmark used is also U.S.-specific, further motivating this restriction. After merging survey responses with patient features and removing incomplete cases, 131 patients remained for modeling.

The 31 patient-level variables were derived directly from the BI Year 3 survey. Categorical variables (age group, income bracket, population density) were encoded as ordinal integers reflecting their natural ordering. Binary indicators (prior CAD diagnosis, prior PAD procedure) were coded as 0/1. Digital health experience variables (telemedicine, wearable monitors, implantable monitors, mobile apps, medical alert devices, remote monitoring) were scored on a 0–4 ordinal scale (none to very high). No imputation was required; respondents with incomplete feature data were excluded during the original survey administration, yielding complete cases for the modeling pipeline.

NLP pipeline

An NLP pipeline leveraging RoBERTa (A Robustly Optimized BERT Pretraining Approach), a transformer-based language model, was applied to semantically analyze the BI survey against a validated benchmark in order to identify the questions most relevant to patient-perceived care quality (12). The HCAHPS survey was selected as the benchmark because of its widespread adoption as a standardized measure of hospital care quality and patient perceptions.

Both surveys were manually preprocessed to exclude items unrelated to QOC. Composite questions containing multiple concepts or response options were divided into discrete, single-concept items to improve semantic clarity. For analysis, the processed items were tokenized and converted into contextual embeddings using RoBERTa, preserving the semantic and contextual meaning of the original text. Cosine similarity scores were then calculated between each BI question and the HCAHPS items to quantify semantic overlap, enabling identification and retention of BI questions most closely aligned with the HCAHPS benchmark and relevant to patient-perceived care quality.

Final item selection incorporated three criteria beyond cosine similarity: (I) sufficient response variation for binary classification—items required an adequate positive/negative split to support stable model training with n=131; (II) sample completeness—items with high missingness after data cleaning were deprioritized; and (III) clinical interpretability—expert review by co-authors with vascular experience confirmed that the selected items captured distinct, actionable constructs (lifestyle changes, recovery uncertainty, caregiver burden) rather than overlapping themes. This multi-criteria approach balanced semantic alignment with the HCAHPS benchmark, analytic feasibility, and clinical distinctiveness. Full NLP rankings for all 14 post-intervention items are provided in Table S1.

Critically, the NLP pipeline operates on question text and HCAHPS domain descriptions alone—no patient response data are used. Sentence embeddings were generated from the question wording alone, ensuring zero risk of data leakage between the NLP selection step and downstream predictive modeling. To verify robustness across architectures, the analysis was repeated using five diverse sentence-transformer models (all-MiniLM-L6-v2, all-mpnet-base-v2, paraphrase-MiniLM-L6-v2, stsb-roberta-large, nli-roberta-base) which all produced concordant top 3 rankings.

ML predictive modeling for survey responses

Consistent with prior work (12), a stacked ensemble of binary classifiers was trained to predict patient responses to each NLP-identified survey question (Figure 1). A classification framework was selected for its interpretability and alignment with the labeled structure of survey data. Models were trained on patient-level features, with individual responses serving as target variables. To address class imbalance, the minority class for each question was up sampled using linear interpolation via the Synthetic Minority Oversampling Technique (SMOTE) library in Python (13) which was applied exclusively within training folds to prevent data leakage.

Figure 1 Analysis pipeline for generating survey insights. (I) Natural language processing is used to identify relevant questions from the BI survey. (II) Supervised machine learning (stacked ensemble: RF + SVM + GBM → LR meta-learner with Kneedle-based feature selection) is trained to predict favorable responses based on patient characteristics. (III) Feature importance scores (via SHAP) are computed to identify the most influential characteristics. (IV) Likelihood estimates are calculated for each value of each impactful feature, that meets the significant criteria, to quantify the probability of a favor response, revealing key drivers of patient sentiment. BI, Beyond Intervention; GBM, Gradient Boosting Machine; HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; LR, likelihood ratio; ML, machine learning; NLP, natural language processing; RF, Random Forest; SHAP, SHapley Additive exPlanations; SVM, Support Vector Machine.

The supervised ML ensemble comprised three base classifiers—Random Forest (RF), Support Vector Machine (SVM) with a radial basis function kernel, and Gradient Boosting Machine (GBM; 100 estimators)—all implemented using the scikit-learn Python library. Their outputs were combined in a logistic regression meta-classifier that performed binary classification for each question. Model training used 5-fold stratified cross-validation (CV) (approximately 105 train/26 test per fold from n=131), with hyperparameters tuned to optimize performance while reducing the risk of overfitting. For survey items with multi-class responses (e.g., five-point Likert scales), responses were recoded into binary classes by collapsing to the nearest pole, producing dichotomous targets. Mean CV metrics were reported to summarize model accuracy.

To further mitigate overfitting with n=131, we applied the Kneedle algorithm (14) to SHapley Additive exPlanations (SHAP)-based feature importance curves to identify the point of maximum curvature (i.e., the “elbow”) in a monotonically decreasing importance curve. Only the features above the elbow were retained, yielding a feature set that could be identified as most impactful. To reduce overfitting, all but one of our final models were retrained using only these selected features.

Feature impact and subgroup analysis

To evaluate the influence of patient characteristics on post-discharge care perceptions, the SHAP Python library was used to calculate SHAP values quantifying each feature’s contribution to model predictions. Feature impact scores were ranked and analyzed using the Kneedle algorithm to determine a natural cutoff for model impact. The Kneedle point—defined as the point of maximum curvature in the importance curve—was used to distinguish features with meaningful impact from those with minimal influence. Features below this threshold were excluded, while those at or above the cutoff were retained for further analysis.

A likelihood analysis was then performed to interpret how retained features shaped care perceptions. For each impactful feature, the probability that a randomly selected patient with a given feature value (e.g., age group 35–44 years) would respond unfavorably was estimated. This was calculated using a distributional comparison method (12), in which the likelihood was defined as the feature’s ratio of density distribution among “No” responders and its density distribution among “Yes” responders, at that feature value. Conditional probabilities were estimated using kernel density estimation with Gaussian kernels (Scott’s bandwidth), and 95% confidence intervals (CIs) were computed using the bootstrap percentile method (B =1,000 resamples). Feature-value combinations where the CI excluded 1.0 are considered statistically significant. This enabled subgroup-level comparisons of response risk. Only statistically significant feature values were reported, with non-significant comparisons excluded. The complete list of additional patient features considered—including those later excluded from the likelihood analysis due to low impact—is provided in Table S2.

Statistical analysis

All analyses were performed in Python 3.11 using scikit-learn (v1.3), SHAP (v0.42), and SciPy (v1.11). Model performance was evaluated using 5-fold stratified CV, with SMOTE applied exclusively within training folds to prevent data leakage. Performance metrics—area under the curve (AUC), accuracy, F1 score, and Brier score—were averaged across folds, with 95% CIs computed as the mean ± 1.96 × standard deviation across folds. Feature selection was performed using the Kneedle algorithm applied to ranked SHAP importance curves; features above the point of maximum curvature were retained. For the likelihood analysis, 95% CIs were estimated via the bootstrap percentile method (B =1,000 resamples), and feature-value associations were considered statistically significant when the CI excluded 1.0. An ordinal regression sensitivity analysis was conducted to confirm concordance of findings under non-dichotomized response scales (Appendix 1). Cross-national generalizability was assessed descriptively using mean Spearman rank correlation of SHAP feature rankings across 16 markets. No adjustments for multiple comparisons were applied, consistent with the exploratory nature of the subgroup analyses.

Ethical consideration

The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. Under U.S. federal definitions [Common Rule, 45 CFR 46.102(e)(1)], the BI survey and this analysis do not constitute human subjects research. IRB review and consent to participate for this secondary analysis were therefore not required. The original BI survey respondents provided consent through market‑research panel procedures.


Results

Patient population and baseline survey results

A total of 31 patient characteristics from the US-based respondents (n=157) were included (Tables 1,2). These comprised 7 demographic/socioeconomic variables, 15 health conditions, 7 measures of health technology experience, and 2 related to insurance status and care provider. Most participants were male (57%), white (70%), and resided in suburban areas (37%). CAD was present in 101 patients (64%) and PAD in 84 patients (54%), with 46 patients (29%) having both. Most had a prior CAD-related diagnosis (89%), CAD-related symptoms (63%), or a prior CAD procedure (61%). Similarly, 81% reported a prior PAD-related diagnosis, 73% had PAD-related symptoms, and 47% had undergone a PAD-related procedure.

Table 1

Demographic and clinical characteristics of the patients (n=131)

Characteristic Values
Age (years) 60 (40–70)
   35–44 37 [28.2]
   45–54 20 [15.3]
   55–64 23 [17.6]
   65–74 33 [25.2]
   75+ 18 [13.7]
Male 76 [58]
Race or ethnicity
   White 91 [69.5]
   Black 14 [10.7]
   Hispanic 14 [10.7]
   Asian 7 [5.3]
   American Indian or Alaskan Native 1 [0.8]
   Other/multiracial 4 [3.1]
Local population density
   Rural 32 [24]
   Suburban 51 [39]
   Urban 48 [37]
Health conditions
   Hypertension 77 [59]
   Hypercholesterolemia 82 [63]
   Diabetes mellitus 70 [53]
   CAD 97 [74]
   PAD 80 [61]
Medical history
   Prior CAD procedure 96 [73]
   Recent CAD procedure 84 [64]
   Prior CAD related diagnosis 114 [87]
   Prior CAD related medical event 82 [63]
   Prior CAD related symptoms 83 [63]
   Prior PAD procedure 73 [56]
   Recent PAD procedure 47 [36]
   Prior PAD related diagnosis 106 [81]
   Prior PAD related medical event 49 [37]
   Prior PAD related symptoms 99 [76]

Values are median (interquartile range) or n [%]. All data was reported by the patients. “Recent” means that the procedure in question was the most recent procedure the patient has had out of a list of possible procedures. “Prior” means in the past without relevance to the order of events. CAD related diagnoses include ischemia and no obstructive coronary artery disease and coronary microvascular dysfunction. PAD include diabetes (type 2), high blood pressure (hypertension), high cholesterol (hypercholesterolemia), and critical limb ischemia. CAD, coronary artery disease; PAD, peripheral artery disease.

Table 2

Reported personal, social, and health technology experiences (n=131)

Characteristic Values
Annual household income
   <$25,000 21 [16]
   $25,000–$99,999 65 [50]
   $100,000–$149,999 28 [21]
   ≥$150,000 17 [13]
Financial hardship
   None 84 [64]
   Food, water, medicine, and/or healthcare 29 [22]
   Transportation 8 [6]
   Other 10 [8]
Type of doctor for CAD or PAD
   General doctor 13 [10]
   Cardiologist 73 [56]
   Vascular proceduralist 10 [8]
   Cardiologist and vascular 16 [12]
   Other 19 [15]
Comfort with health technology
   Very uncomfortable 11 [8]
   Uncomfortable 8 [6]
   Somewhat uncomfortable 20 [15]
   Somewhat comfortable 38 [29]
   Comfortable 28 [21]
   Very comfortable 26 [20]
Experience with telemedicine
   None 8 [6]
   Low 42 [32]
   Moderate 17 [13]
   High 32 [24]
   Very high 32 [24]
Experience with wearable monitors
   None 12 [9]
   Low 54 [41]
   Moderate 39 [30]
   High 7 [5]
   Very high 19 [15]
Experience with implantable monitors
   None 44 [34]
   Low 52 [40]
   Moderate 13 [10]
   High 11 [8]
   Very high 11 [8]
Experience with mobile health apps
   None 32 [24]
   Low 38 [29]
   Moderate 31 [24]
   High 17 [13]
   Very high 13 [10]
Experience with medical alert devices
   None 4 [3]
   Low 93 [71]
   Moderate 6 [5]
   High 14 [11]
   Very high 14 [11]
Experience with remote monitoring devices
   None 20 [15]
   Low 41 [31]
   Moderate 27 [21]
   High 24 [18]
   Very high 19 [15]

Values are n [%]. All data was self-reported by patients. Vascular proceduralist refers to vascular surgeons or interventional radiologists. Digital health experience levels represent frequency of use (none = never used). CAD, coronary artery disease; PAD, peripheral artery disease.

Over half of respondents (56%) received care from a cardiologist, while 17% saw both a cardiologist and vascular specialist. Comfort with health technology was mixed: 84% reported being at least somewhat comfortable, 16% described themselves as uncomfortable or very uncomfortable. Experience with digital health tools varied by category. Most participants reported at least some experience with telemedicine (95%), wearable monitors (90%), and mobile health apps (75%). Over half of survey respondents (64%) had a history of implantable monitors. The use of remote monitoring devices (85%) was also common, though experience levels were generally lower. Medical alert devices had the lowest reported use, with 72% indicating only minimal experience.

NLP-identified questions

Based on the similarity scores and selection criteria from the NLP-based analysis, three BI survey questions were identified as having the most relevant indicators of patient-perceived care:

  • Question 1: “I struggled with the lifestyle changes my physician talked about”.
  • Question 2: “I wasn’t sure what actions would help most with my recovery”.
  • Question 3: “I felt like a burden to my loved ones because I had to rely on them to help me navigate medical care and decisions”.

Model performance and key predictive features

Model performance metrics are summarized in Table 3, reporting mean accuracy, F1, and AUC values averaged across five-fold CV. The stacked models achieved AUC values of 0.774 (95% CI: 0.680–0.867) for Question 1, 0.761 (95% CI: 0.634–0.887) for Question 2, and 0.717 (95% CI: 0.594–0.840) for Question 3. Brier scores (0.198, 0.197, 0.224) confirm well-calibrated predictions. Feature impact values on model predictability, per question, are shown in Figure 2. To emphasize the most relevant findings, features are ranked by SHAP importance with the Kneedle elbow annotated. All three models found “age” within the top three most impactful features. The models for Questions 1 and 2 found CAD or PAD-related features in the top three, and the models for Questions 2 and 3 found digital health related features in the top three. The moderate accuracy (0.64–0.71) and variable F1 scores reflect the inherent difficulty of predicting subjective patient perceptions from baseline demographics alone; however, the framework’s primary contribution is explanatory rather than purely predictive. For Question 2, the Kneedle algorithm identified no natural elbow in the SHAP importance curve, resulting in retention of all 31 features. Coupled with the lower F1 score (0.520; 95% CI: 0.256–0.783) and wide CI, this suggests limited class separability. Retaining all features reflects the optimizer’s ability to identify a compact discriminative subset rather than evidence of a broadly distributed predictive signal. In contrast, Questions 1 and 3 yielded compact feature subsets of 8 and 5 features, respectively, with narrower CIs, supporting more reliable discrimination within this cohort.

Table 3

Model performance metrics (5-fold stratified cross-validation)

Question AUC [95% CI] Accuracy [95% CI] F1 [95% CI] Brier score [95% CI]
Q1: “I struggled with the lifestyle changes my physician talked about” 0.774 [0.680, 0.867] 0.711 [0.605, 0.817] 0.734 [0.625, 0.843] 0.198 [0.172, 0.225]
Q2: “I wasn’t sure what actions would help most with my recovery” 0.761 [0.634, 0.887] 0.696 [0.586, 0.806] 0.520 [0.256, 0.783] 0.197 [0.159, 0.236]
Q3: “I felt like a burden to my loved ones because I had to rely on them …” 0.717 [0.594, 0.840] 0.641 [0.512, 0.769] 0.649 [0.489, 0.809] 0.224 [0.193, 0.254]

Values are from 5-fold stratified cross-validation with SMOTE applied within training folds. Brier scores range from 0 (perfect) to 1 (worst). AUC, area under the curve; CI, confidence interval.

Figure 2 Feature impact analysis identifying patient characteristics associated with response predictions. (A-C) Ranked SHAP values for patient features, ordered by impact in predicting responses to each BI survey question: (A) “I struggled with lifestyle changes my physician talked about”, (B) “I wasn’t sure what actions would help most with my recovery”, and (C) “I felt like a burden to my loved ones because I had to rely on them to help me navigate medical care and decisions”. Higher SHAP values indicate stronger influence on the response predictions. The red dashed line indicates the Kneedle elbow (i.e., feature selection boundary). Question 1 retains 8 features, Question 2 all 31, Question 3 keeps 5 features. All features are kept for Question 2 due to a lower target variable positive rate. Hence, the amount of signal per feature is lower, and dropping features leads to a loss of predictability. BI, Beyond Intervention; CAD, coronary artery disease; PAD, peripheral artery disease; SHAP, SHapley Additive exPlanations.

Subgroup analysis

Likelihood ratios for positive and negative responses are shown in Figure 3. Age demonstrated a consistent gradient across all three questions, with younger patients more likely to report negative care experiences following treatment. 95% bootstrap CIs (B =1,000) are shown; feature values where the CI excludes 1.0 are statistically significant. A univariate trend of age versus the likelihood of a favorable response is visualized in Figure S1. Patients aged 55–64 and 65–74 years were 1.6 and 1.5 times, respectively, less likely to report struggling (Question 1), whereas younger patients aged 45–54 and 35–44 years were 1.4 and 4.7 (95% CI: 2.1–8.3) times more likely, respectively, to report struggling. Similarly, younger patients (aged 35–44 and 45–54 years) were 2.0 and 2.6 times, respectively, more likely to express uncertainty (Question 2), while patients aged 55–64 and 65–74 years were 2.7 and 1.5 times, respectively, less likely to feel uncertainty. On burden (Question 3), patients aged 35–44 and 45–54 years were 2.8 and 1.4 times, respectively, more likely to feel like a burden, whereas patients aged 65–74 and 75+ years were 1.8 and 3.4 times, respectively, less likely to feel like a burden.

Figure 3 LRs of patient characteristics driving positive or negative response predictions. (A-C) The LR that a randomly sampled patient with a particular value of the ML model’s driving feature is more likely to respond “No” (a “No” response corresponds to a positive patient reported outcome) for each NLP-identified survey question. The vertical line in each panel indicates an LR value of 1.0. Features are grouped into similar categories by color (e.g., age in red, no prior CAD or PAD experience in blue). Error bars indicate 95% bootstrap confidence intervals (B =1,000). Points are color-coded by feature category. CAD, coronary artery disease; LR, likelihood ratio; ML, machine learning; NLP, natural language processing; PAD, peripheral artery disease; SES, socioeconomic status.

Additional subgroup effects were observed by comorbidity, SES, and digital health experience. Patients with a prior history of CAD- or PAD-related events (procedures or symptoms) were 1.5 to 2.6 times more likely to report struggling with lifestyle changes (Question 1), compared to healthier patients who were nearly twice as likely to respond that they did not struggle with lifestyle changes. For Question 2, lower household income was associated with greater uncertainty; those earning <$25,000 annually were 2.8 times more likely to be unsure about which actions would aid their recovery, whereas patients with income between $100,000–$149,999 were 1.7 times less likely to be unsure. Digital health experience showed a strong scaling effect for Question 3. Patients with limited or no experience with digital health tools were less likely to feel like a burden to family. Conversely, patients with greater digital health experience (reported as “Moderate” to “Very high”) were 1.1 to 2.9 times more likely to feel like a burden Notably, four of the five most impactful predictors for Question 3 related to technology use, underscoring the central role of digital literacy in shaping patients’ perceived independence post-discharge. Correlation analysis confirmed that digital health features are not proxies for disease severity (mean |r|=0.23 vs. comorbidity burden; Appendix 2).


Discussion

This study integrates ML and NLP into cardiovascular care by linking computational analyses with patient-reported outcomes. Although autonomous AI systems can rapidly analyze survey data and generate insights, the present work demonstrates the continued importance of transparent, reproducible, and interpretable pipelines that link patient sentiment to validated QOC measures and provides interpretable outputs suitable for clinical use. By aligning BI survey items with validated HCAHPS domains and applying explainable ML methods, the framework provides clinically relevant insights into how patients interpret their vascular care experiences.

Recent related work has applied supervised ML to healthcare survey data, showing that patient characteristics can predict future responses and identify key predictive features (12). Other studies have quantified the extent to which demographics influence demand for care (15). These efforts align with the present study in demonstrating the feasibility of AI-enhanced survey analysis but differ in interpretability. By emphasizing explainability, this work demonstrates how interpretable pipelines support clinical trust and governance.

This work shows that patient perceptions are shaped not only by clinical outcomes but also by expectations, life context, and technological readiness. Younger and digitally engaged patients reported less favorable experiences, likely reflecting higher baseline expectations and greater demand for personalized guidance. Patients with prior CAD- or PAD-related events and those with lower SES were also more likely to express uncertainty or difficulty. Surprisingly, extensive exposure to digital health tools such as wearables and implantable monitors was associated with greater feelings of burden and reduced autonomy, underscoring a mismatch between patient expectations for precision, personalization, and support from these tools and clinical realities.

These results highlight an associative finding regarding patient experience: the misalignment between patient expectations and clinical realities. Younger, tech-savvy patients and those heavily engaged with digital tools report needing more granular information, rapid feedback, and better outcomes (16). Without structured interventions—such as tailored patient education, guided onboarding to health technologies, and responsive digital support—this mismatch can worsen post-discharge perceptions (17).

While aligning the BI survey with HCAHPS domains provides a valuable framework for translating specialized patient insights into institutional quality benchmarks, this mapping inherently compromises some disease-specific nuance. Vascular care involves highly distinct recovery trajectories, localized pain profiles, and specialized anatomical anxieties that standardized HCAHPS domains do not fully capture. The moderate (rather than maximal) cosine similarity scores between our selected items and HCAHPS domains reflect this intentional balance—items were selected not merely for HCAHPS replication, but for their ability to bridge standardized quality metrics with disease-specific patient experiences. Future work should explore hybrid frameworks that preserve these granular, vascular-specific sentiments alongside standardized metrics.

The identification of SES and digital literacy as predictive features must be approached with caution. Using such models to allocate clinical resources risks reinforcing existing disparities (i.e., digital redlining) or adopting paternalistic assumptions regarding patient needs. We emphasize that these features identify associations and not modifiable targets for intervention without broader contextual assessment.

The U.S. BI cohort is characterized by high digital health engagement (95% telemedicine experience, 90% wearable monitor use), reflecting the post-interventional population from which participants were recruited. While this limits generalizability to populations with lower digital adoption, it provides a unique window into how extensive digital monitoring experience shapes perceptions of care burden.

Limitations include the restriction to a U.S.-based cohort, which may reduce generalizability across health systems with different structures, cultural norms, or digital access. An exploratory cross-national comparison (Appendix 2; Figure S2) suggests moderate concordance of feature rankings across 16 markets (mean Spearman ρ =0.48 vs. U.S.). Binary dichotomization of Likert-scale outcomes sacrifices granularity, though ordinal regression sensitivity analysis (Appendix 1) confirmed concordant findings (Appendix 2 and Figure S3). No external validation dataset exists combining the same survey instrument, patient features, and post-interventional population. Potential response bias and the cross-sectional design may also limit insights into longitudinal changes in patient perceptions. Although the classification metrics for Questions 1 and 3 demonstrate a reliable signal within this cohort, as supported by limited feature selection and relatively narrow CIs, the analytic framework remains complex given the modest sample size (n=131). Moreover, the single-cohort, single-survey design limits the ability to determine whether these within-cohort associations extend to the broader population. Establishing generalizability will require replication in substantially larger cohorts, ideally an order of magnitude greater in size, to evaluate the stability of these findings and the clinical significance of identified subgroups with sufficient statistical power.

A key methodological contribution from this work is the deployment of a four-step analytic pipeline spanning (I) NLP-guided question selection, (II) supervised ML for predictive modeling, (III) SHAP-based feature attribution, and (IV) subgroup-level likelihood analysis. Anchoring the analysis to HCAHPS domains provided external validation and interpretability. The use of ensemble ML improved predictive accuracy—individual base classifiers produced discordant feature importance rankings (Figure S2), and the stacked architecture resolves these disagreements through meta-learning—while SHAP values and likelihood ratios preserved transparency, an essential advantage over autonomous systems that may achieve comparable accuracy without offering insight into why predictions are made. This work also demonstrates the feasibility of deploying AI to generate real-time, interpretable insights from patient-reported data—supporting both clinical practice and survey design (18).

By combining explainable AI with validated survey measures, this framework enables health systems to identify subgroups at risk of dissatisfaction and design targeted interventions, such as digital literacy support, tailored education, and proactive follow-up. While agentic AI offers scalability, frameworks like this remain essential for ensuring transparency, equity, and clinician trust. Future work should extend this approach to larger and more diverse populations, evaluate its performance in prospective longitudinal studies, and explore hybrid models that integrate agentic AI while retaining interpretable decision-making. Importantly, clinical implementation of subgroup-targeted interventions should remain contingent on validation at substantially larger scale. Although the present study demonstrates that the framework can identify meaningful discriminative patterns, confirmation of these associations across broader populations is necessary before informing resource allocation or care pathway modifications.


Conclusions

By aligning patient-reported survey items with validated HCAHPS domains and using an interpretable ensemble with SHAP-based attribution and subgroup likelihoods, we identified age, digital health familiarity, and socioeconomic factors as consistent associates of vascular care experience. Findings suggest targeted digital literacy support and expectation-setting—particularly for younger, tech-engaged patients—may improve perceived post-discharge care. Explainable AI enables transparent prioritization of such interventions and supports reproducible survey analytics across populations. With validation in larger, independent cohorts, this framework could further refine targeted care strategies.


Acknowledgments

The authors thank Abbott Laboratories for supporting the Beyond Intervention global survey program.


Footnote

Reporting Checklist: The authors have completed the TRIPOD+AI reporting checklist. Available at https://jmai.amegroups.com/article/view/10.21037/jmai-2026-0063/rc

Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-2026-0063/prf

Funding: This work was supported by Abbott Laboratories who also funded the Beyond Intervention survey. The study design, data analysis, interpretation of results, and manuscript preparation were conducted independently by the research team. The funder had no role in the decision to submit for publication.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-2026-0063/coif). A.H. reports that Abbott Laboratories supported the analysis and writing of this manuscript, including article processing charges and reports a minority interest in Switchboard MD, Inc. in the form of stock options and convertible debt. S.T., Q.L., and J.B.P. are current employees of Abbott Laboratories. D.R. is a full-time employee of Microsoft. The other author has no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. Under U.S. federal definitions [Common Rule, 45 CFR 46.102(e)(1)], the Beyond Intervention survey and this analysis do not constitute human subjects research. IRB review and consent to participate for this secondary analysis were therefore not required. The original BI survey respondents provided consent through market‑research panel procedures.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Martin SS, Aday AW, Allen NB, et al. 2025 Heart Disease and Stroke Statistics: A Report of US and Global Data From the American Heart Association. Circulation 2025;151:e41-e660. [Crossref] [PubMed]
  2. Gualandi R, Masella C, Viglione D, et al. Exploring the hospital patient journey: What does the patient experience?. PLoS One 2019;14:e0224899. [Crossref] [PubMed]
  3. West NEJ, Juneja M, Pinilla N, et al. Personalized vascular healthcare: insights from a large international survey. Eur Heart J Suppl 2022;24:H8-H17. [Crossref] [PubMed]
  4. Sisodia RC, Dankers C, Orav J, et al. Factors Associated With Increased Collection of Patient-Reported Outcomes Within a Large Health Care System. JAMA Netw Open 2020;3:e202764. [Crossref] [PubMed]
  5. Nakano A, Mainz J, Lomborg K. Patient perception and assessment of admission to acute cardiac care unit. Eur J Cardiovasc Nurs 2008;7:10-5. [Crossref] [PubMed]
  6. Khanbhai M, Warren L, Symons J, et al. Using natural language processing to understand, facilitate and maintain continuity in patient experience across transitions of care. Int J Med Inform 2022;157:104642. [Crossref] [PubMed]
  7. Nawab K, Ramsey G, Schreiber R. Natural Language Processing to Extract Meaningful Information from Patient Experience Feedback. Appl Clin Inform 2020;11:242-52. [Crossref] [PubMed]
  8. Maramba ID, Davey A, Elliott MN, et al. Web-based textual analysis of free-text patient experience comments from a survey in primary care. JMIR Med Inform 2015;3:e20. [Crossref] [PubMed]
  9. van Buchem MM, Neve OM, Kant IMJ, et al. Analyzing patient experiences using natural language processing: development and validation of the artificial intelligence patient reported experience measure (AI-PREM). BMC Med Inform Decis Mak 2022;22:183. [Crossref] [PubMed]
  10. Johnson KB, Wei WQ, Weeraratne D, et al. Precision Medicine, AI, and the Future of Personalized Health Care. Clin Transl Sci 2021;14:86-93. [Crossref] [PubMed]
  11. Guardado S, Karampela M, Isomursu M, et al. Use of Patient-Generated Health Data From Consumer-Grade Devices by Health Care Professionals in the Clinic: Systematic Review. J Med Internet Res 2024;26:e49320. [Crossref] [PubMed]
  12. Lu Q, Taimourzadeh S, Rhew D, et al. Machine learning methods application: generating clinically meaningful insights from healthcare survey data. J Med Artif Intell 2025;8:4.
  13. Chawla NV, Bowyer KW, Hall LO, et al. SMOTE: Synthetic Minority Over-sampling Technique. J Artif Intell Res 2002;16:321-57.
  14. Satopaa V, Albrecht J, Irwin D, et al. Finding a "Kneedle" in a Haystack: Detecting Knee Points in System Behavior. 2011 31st International Conference on Distributed Computing Systems Workshops. Minneapolis, MN, USA: IEEE; 2011:166-71.
  15. Orhan F, Kurutkan MN. Predicting total healthcare demand using machine learning: separate and combined analysis of predisposing, enabling, and need factors. BMC Health Serv Res 2025;25:366. [Crossref] [PubMed]
  16. Jaipaul CK, Rosenthal GE. Are older patients more satisfied with hospital care than younger patients?. J Gen Intern Med 2003;18:23-30. [Crossref] [PubMed]
  17. McLoughlin M, Imran A, Hannigan A, et al. Does Age Impact on Expectations and Concerns of Patients Attending a Chronic Pain Clinic?. Pain Pract 2018;18:23-8. [Crossref] [PubMed]
  18. Alemi F, Torii M, Clementz L, et al. Feasibility of real-time satisfaction surveys through automated analysis of patients' unstructured comments and sentiments. Qual Manag Health Care 2012;21:9-19. [Crossref] [PubMed]
doi: 10.21037/jmai-2026-0063
Cite this article as: Harzand A, Taimourzadeh S, Lu Q, Prillinger JB, McCaney JT, Rhew D. Developing an explainable artificial intelligence framework for patient-reported outcomes: an interpretable approach for understanding patient experiences in vascular care. J Med Artif Intell 2026;9:77.

Download Citation