Application of machine learning in predicting medication adherence of patients with cardiovascular diseases: a systematic review of the literature
Cardiovascular disease (CVD) is a category of diseases related to the heart and blood vessels (1). The most common CVD in the US that results in severe heart damage are coronary artery disease (CAD) and heart failure (HF) (1). Several risk factors have been identified for CVD including hypertension (HTN) and hypercholesterolemia (HCL) (2). Having HTN and HCL under control with medications, can have a significant impact on long-term outcomes of patients with CVD (2,3).
American Heart Associate reported that CAD leading to acute coronary syndrome (ACS), was responsible for 365,914 deaths in the US in 2017 (4). A metanalysis carried out in 2017 indicated that good adherence to evidence-based medications for CAD, was associated with a significantly lower probability of all-cause mortality (risk ratio =0.56; 95% CI: 0.45–0.69), mortality due to CVD (risk ratio =0.66; 95% CI: 0.51–0.87), and cardiovascular hospitalization (risk ratio =0.61; 95% CI: 0.45–0.82) (5).
HF is a growing health concern affecting more than 20 million people globally (6). The results of a study on 219 HF patients indicated that HF patients with poor medication adherence (MA) had higher risk of reporting dyspnea or ankle swelling (P=0.05) (7). Using a cox model, Wu et al. (7) reported a worse cardiac event-free survival associated with poor MA (P=0.006).
HTN is a major risk factor for CVD (8). About half of adults in the US (108 million) have HTN or are using medication for HTN while only one out of four HTN patients have their blood pressure in the acceptable range (9). A recent metanalysis (54,349 patients) assessed the association of antihypertensive MA to the recurrence of CVD events (10). They found a 9% reduction in the risk of CVD events for each 20% increase in adherence to antihypertensive medications (10).
HCL is a very common condition in the US. According to the Centers for Disease Control and Prevention (CDC), 73.5 million (31.7%) of US adults have LDL on higher-than-normal range (11). A metanalysis that included 710,504 patients assessed the incidence of stroke among patients using statins (12). Xu et al. (12) did a dose-response analysis that indicated an 8% reduction in the total risk of stroke for each 20% improvement in statin adherence.
Medical conditions explained above are modifiable with medications. However, maximum benefit is not possible unless patients have enough adherence to medications. According to the World Health Organization, MA is “the extent to which a person’s behavior—taking medication, following a diet, and/or executing lifestyle changes, corresponds with agreed recommendations from a health care provider” (13). Based on a meta-analysis of 376,162 patients, the estimated MA among patients using CVD preventive medications is about 57% (95% CI: 50–64%) with reported adherence in the included studies ranging from 32% to 68% (3). Naderi et al. (3) suggested that medication type and side effects of medications may not be the main causes of poor adherence to CVD preventive medications. Instead, it seems that the role of factors not related to medications are more prominent (14).
Many studies have used different regression methods to identify MA associated factors (15,16). Despite the improvements in health care claims databases and the increase in patient data, prediction of future MA has not witnessed a noticeable change. Traditional approaches were mostly focused on a small number of clinical characteristics specified by investigators and demographic factors available at the initiation of treatment. Many characteristics have been identified as predictors of MA including, age, race, socioeconomic status, psychological issues, cognitive problems, and the complexity of the treatment protocol (17-19). Even though these variables are in correlation with adherence, they provide very limited discrimination between adherent and non-adherent patients (20,21). As a result, the best adherence-prediction models based on traditional statistic methods provide at best, a moderate accuracy of 0.65 (22).
Improvements in processing power, memory, and storage capacity of computers have made it possible to conduct increasingly convoluted learning tasks through machine learning (ML) and deep learning techniques. The new techniques have improved prediction through variable selection algorithms and through evaluation of nonlinear associations between predictors and outcome or deep interactions among predictors. For instance, the selection of important variables does not require specific criteria or linearity check which may be useful for modeling a behavior as complex as MA (23).
In this review, our objective was evaluating the literature for the application of ML methods to predict MA among patients with CVD namely CAD and HF. In addition, due to the importance of MA for the management of HTN and HCL as the main risk factors of CVD, studies that used ML to predict medication adherence among patients with HTN or HCL were also included. We present the following article in accordance with the PRISMA reporting checklist (available at https://jmai.amegroups.com/article/view/10.21037/jmai-21-26/rc).
Data sources and systematic searches
This systematic review used the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) to guide conducting and reporting (24). Further, Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was used to extract information from the studies (25,26). We performed a review of original English research articles in PubMed, and Google Scholar database from January 1, 2000 until August 9, 2021. PubMed is the most used database that covers millions of publications in the field of medicine. Google Scholar provides studies across diverse disciplines including computer science where ML was originated. We selected these two search engines as our data source to retrieve publications from both fields of research, namely medicinal science and computer science. Cross-references were also used to identify more publications.
Eligibility criteria and study selection
Eligibility criteria for inclusion were: (I) utilization of at least one ML technique for medication adherence prediction; (II) must report details of the performance of the predictive model in terms of c-statistic or AUC; (III) being focused on patients with chronic CVD (CAD and HF) or their main risk factors (HTN and HCL) (IV) being an original research paper; and (V) availability of full texts in the English language. All letters to editor, opinions, and abstracts were excluded from this review. In addition, interventional studies that utilized ML to enhance MA were also excluded (27,28).
Search strategy and selection process
To ensure that all eligible publications are included, first we conducted a query using two brackets. The first bracket included various keywords for ML and the second bracket included various keywords for MA. Alternative search terms used for ML were obtained from previous literature (29). A list of all search terms used for ML and MA is provided in Appendix 1. The initial publications found were imported into an excel sheet. Then, all duplicates were identified and removed. The remaining titles and abstracts were independently screened by two authors (MZ and SMA) to identify articles that included the concepts of ML-based MA predictive models. Any disagreement between the two authors was resolved by the third author (SSS). The full texts of articles identified as potentially relevant based on their title and abstract were screened (by MZ and SMA) regarding the inclusion and exclusion criteria (Figure 1). Inclusion and exclusion criteria were made in advance to the literature search and were in accordance with the search strategy utilized in the identification process. Any discrepancies between the two reviewers (MZ and SMA) were resolved by the third reviewer (SSS).
This review focused on ML techniques utilized for prediction of MA among patients with CVD (CAD and HF) or their main risk factors (HTN and HCL). The list of extracted items was based on previous literature using predictive ML models, revised based on discussions among the authors. One author (MZ) extracted the following information and entered all the information in an excel sheet, and another reviewer (SMA) validated all the information. A third reviewer (SSS) resolved any discrepancies between the two reviewers. The extracted items were study design (retrospective/cross-sectional), location (the country where the study was carried out), data source [claims data/electronic health records (EHR)/survey], data collection period, clinical diagnosis and inclusion criteria, average patients’ age, conventional MA measurement tool (PDC/self-report/staff-report), and MA rate. Further, the initial factors in the model (sociodemographic/comorbidities/lab results/previous adherence), factor selection and grouping, data preparation and model evaluation (leave one out cross-validation/simple cross-validation/k-fold cross-validation), factors in the final model, and ML techniques used were evaluated and summarized. The outcomes of studies were evaluated based on the model accuracy (c-statistic) in predicting MA.
Model validation and risk of bias
To assess the validation in each study, validation techniques and alternative statistical methods used in each study were evaluated. To estimate the risk of bias in each study, we evaluated sample sizes, inclusion criteria, length of follow-up, MA measurement methods, and the number and types of variables used in each study.
Overall, 2,708 hits were identified from all sources (PubMed n=120, Google Scholar n=2,570, and Cross-references n=18). After removing duplicates, screening titles and abstracts for eligibility criteria and disease states was conducted. The full text of the 32 screened articles were evaluated and a total of 12 studies met the eligibility criteria (30-41). There were some studies that met some of the inclusion criteria but not all of them. For example, Li et al. (42) used ML for evaluation of MA among HTN patients. However, they did not provide an accuracy for their model. A summary of the selection process is available in Figure 1.
Characteristics of the selected studies
Summarized characteristics of the selected studies are provided in Table 1. Included characteristics were the study design, the country which the study was carried out, the clinical diagnosis and details of the inclusion criteria, the conventional method used for MA measurement, and the MA rate of the sample. Among the selected studies, two of them evaluated MA among patients with HF (40,41). Three of them evaluated MA among patients recently hospitalized due to ACS or myocardial infarction (MI) (32-34). Five studies measured MA among HTN patients (35-39). Two studies evaluated MA among patients using statins due to HCL (30,31). Most of the studies used retrospective designs (n=8) (30,31,33,34,36-38,40), while four studies used cross-sectional designs (32,35,39,41).
|Reference, year||Study design, location||Data source||Data collection period||Clinical diagnosis, inclusion criteria||Sample size||Mean age (years)||Conventional adherence measurement||Medication adherence rate (%)|
|Karanasiou et al. (40) 2016||Retrospective, Greece||Second Department of Cardiology, University Hospital of Ioannina||NP||HF diagnosed with (I) continuous symptoms and frequent recurrence, (II) functional NYHA II–IV classes with optimal treatment, (III) recent hospitalization, emergency admission or specialized consultation for decompensated HF, (IV) at least, have undergone in the last 12 months one ECG indicating HF compatible symptoms||90||NP||Clinician’s estimate Dichotomous||GA 82.2|
|Medication adherence 67.8|
|Son et al. (41) 2010||Cross-sectional, Korea||Self-report survey of outpatient clinics||Jan–Apr 2010||HF based on EHR||76||74.8±5.9||Single self-reported item Dichotomous||NP|
|Bourdès et al. (32) 2011||Cross-sectional, France||Patients enrolled by general practitioners||Oct 2006 to Apr 2007||Patients with recent first ACS under treatment with EBCM||2,132||64±12||Continuous usage of all the EBCM after 24 months Dichotomous||86.7|
|Zullig et al. (33) 2019||Retrospective, USA||Medicare dataset: part A (inpatients), part B (outpatient), part D (prescription medication)||2007–2013||Acute MI related admissions with a statin prescription at, or shortly after, discharge||11,969||76 (median)||PDC <80% Dichotomous||64|
|Hu et al. (34) 2020||Retrospective, New Zealand||PREDICT (a web-based CVD risk assessment and management decision support system for primary care)||Patient initial assessments, 2011–2014||MI, unstable angina, ischemic stroke, transient ischemic attack, and/or peripheral vascular disease based on ICD-10||2,322 for one year 1,955 for two years||30–85 (range)||A refill gap >120 days for one year, or two years Dichotomous||One year 71.9 Two years 67.8|
|Lee et al. (35) 2013||Cross-sectional Korea||Outpatient clinics at a teaching hospital||January to May 2011||Asthma, HTN, DM, COPD, liver cirrhosis, stroke, and CVD patients with normal cognitive function under treatment for >6 months||293||74.4±6.3||Four items self-reported Morisky scale Mean scores ≤2 Dichotomous||41|
|Aziz et al. (36) 2020||Retrospective survey-based, Malaysia||Outpatient Clinics at the University Kebangsaan Hospital||2011||Diagnosed with essential HTN, and being on at least one anti-HTN medication for >1 year||160||65±9||Self-reported Malaysian medication adherence scale based on BMQ (0–8 score) Dichotomous||NP|
|Gao et al. (37) 2020||Retrospective, China||Fangzhuang Community Health Service Center, Intelligent Chronic Disease Management System||June 2015 to June 2018||Essential HTN based on ICD-10, and having records of annual assessment of chronic diseases||7,638||67.6±11||PDC <80% Dichotomous||17.7|
|Galozy et al. (38) 2020||Retrospective, Sweden||Region Halland data warehouse containing detailed patient records from public primary, secondary, and specialist care facilities||2018||Essential HTN diagnosis based on ICD-10 at one point between the 1st November 2014 and 31st July 2018||18,581||67±12.3||PDC >80% Dichotomous||78.6|
|Taranik et al. (39) 2019||Cross-sectional, Russia||Scientific Research Institute of Cardiology||NP||Recently diagnosed HTN||62||23–85 (range)||Based on staff report Dichotomous||NP|
|Franklin et al. (30) 2016||Retrospective, USA||Medicare beneficiaries using CVS Caremark Census data||Jan 2006 to Dec 2008||Newly initiated statin, or statin combination||77,703||74||PDC <80% Dichotomous One year adherence from the index day||≤ 30-day supply 39.09|
|> 30-day supply 48.07|
|Delayed adherence: (91–365 days after index day)||≤ 30-day supply 37.82|
|> 30-day supply 41.70|
|Lucas et al. (31) 2017||Retrospective, USA||Military Health System for TRICARE beneficiaries||2005–2006||Newly initiated statin||138,731||49.5||PDC <80% Dichotomous||10.3|
ML, machine learning; CVD, cardiovascular disease; HTN, hypertension; HCL, hypercholesterolemia; NP, not provided; HF, heart failure; ACS, acute coronary syndrome; EBCM, evidence-based cardiovascular medications including antithrombotic + beta-blocker + statin + angiotensin converting enzyme inhibitor-ACEI and/or angiotensin-II receptor blocker-ARB; MI, myocardial infarction; PDC, proportion of days covered; DM, diabetes mellitus; COPD, chronic obstructive pulmonary disease; ICD-10, International Classification of Diseases, 10th Revision; TRICARE beneficiaries, active duty, retirees, and dependents of the US armed forces military.
Medication adherence measurement tools
The most used (n=5) MA measurement tool was the proportion of days covered (PDC) which is the ratio of the number of the days’ covered in a particular period divided by the number of days in the same period (43). Other studies used various methods for MA measurement including self-reported surveys (n=4), physician estimate (n=1), and hospital staff reports (n=1). All studies had dichotomous measures (yes/no) for identifying MA and none of them used continuous measures for MA. The percentage of patients identified as medication adherent in each study, had a range from 10.3% among patients with HCL to 86.7% among patients with recent ACS (31,32).
Table 2 provides a summary of the initial factors included in each model, methods used for factor selection and grouping, data preparation and model evaluation methods, factors included in the final models, the specific ML model utilized, and the accuracy of each model. The most used ML algorithms were random forest (RF; n=5), support vector machine (SVM; n=4), and neural network (NN; n=4). Many different techniques were used for factor selection and grouping including forward and backward stepwise feature selection, genetic input selection algorithm, and collapsing similar variables (32,33). For evaluation of the model performance, five studies performed standard logistic regression (LR) as an alternative method for accuracy measurement, while the other studies only relied on internal validation using different cross-validation methods (30,32-35). Worth noting, one study used an alternative method to check the accuracy of their model, while not performing any internal validity tests (35). The accuracy of the models based on AUC had a range from 0.53 to 0.97.
|Reference, year||Initial factors in the model||Factor selection and grouping||Data preparation and model evaluation||Factors in the final model||Modeling method details||Accuracy (AUC)|
|Karanasiou et al. (40) 2016||100 features: (I) general information: age, gender, caregiver, (II) allergies, (III) medical condition: KILLIP classification, NYHA class, smoking habit, alcoholism habit, comorbidities, (IV) drugs: the active substances, dosage, frequency, (V) biological data related with HF disease, (VI) clinical examinations: left bundle branch block, intraventricular delay, left ventricular ejection fraction||Features with >60% missing were removed 80 features remained A filter approach for FS measures was carried out||10-fold cross validation Imbalanced classes handled with SMOTE||Medical condition and drugs categories of features||RF, filter: symmetrical uncertainty||GA (0.76)|
|RF, filter: info-gain||MA (0.78)|
|Medical condition and drugs categories of features (MA included)||LMT, filter: relief-F||GA (0.82)|
|Medical condition and drugs categories of features (GA included)||SVM, filter: relief-F||MA (0.91)|
|Son et al. (41) 2010||Gender, age, spouse, education, monthly income, and duration of HF diagnosis, daily frequency of medication, MMSE-K, EF, medication knowledge, NYHA functional class||All possible combinations of features tested for the highest accuracy||LOOCV 4 kernel functions tested for the best accuracy (linear, sigmoid, polynomial, RBF)||5 features: gender, daily frequency of medication, medication knowledge, NYHA functional class, spouse||SVM with RBF||0.77|
|7 features: age, education, monthly income, EF, MMSE-K, medication knowledge, NYHA functional class||0.77|
|Bourdès et al. (32) 2011||Patients’ and ACS characteristics, medications at hospital discharge and at inclusion visit, cardiovascular risk factors, last biological measurements, concomitant cardiovascular and other diseases, concomitant treatments||Forward and backward stepwise feature selections and a genetic input selection algorithm||Simple cross-validation LR used as an alternative method to test the accuracy||Only NN model: FBS, treated DM, coronarography results, treated HTN, TG, oral anti-DM at inclusion visit, gender, severe RF, insulin, hypnotics at inclusion visit, stroke with sequelae||NN, (MLP) Variables selected by LR||0.78|
|NN, (MLP) Variables selected by NN||0.80|
|Both NN and LR models: interventional procedures, number of medications per day at inclusion visit, ACS oldness, education, BMI, smoking, treated HCL, peripheral vasodilators, COPD, asthma, accommodation, respiratory failure, HRT at inclusion visit||Std. LR||0.68|
|Zullig et al. (33) 2019||Statin use in the prior 6 months, race, number of CVD meds at discharge, region, CABG during index admission, Medicaid dual eligibility, hospitalization in prior 6 months, DM, CKD, nonacute institutional care in prior 6 months (only in the LR model), cerebrovascular disease||LASSO||80% training 20% test 10-fold cross validation LR used as an alternative method to test the accuracy||LASSO reg and RF: P2Y12 inhibitor at discharge, length of stay during index admission, PCI during index admission, rural location, peripheral arterial disease, age (10-year increase), CHF, prior MI, SNF, IRF, or LTC stay in prior 6 months, P2Y12 inhibitor in prior 6 months, female, HTN, cardiogenic shock index admission, drug-eluting stent placed during index admission, prior CABG||Std. LR||0.67|
|Hu et al. (34) 2020||Gender, ethnicity, deprivation, diabetes history, smoking status, BMI, age, days stayed in the hospital (within 3 days or over), diagnosis type (principal diagnosis or other relevant diagnosis), procedure of PCI or CABG, maximum CCL, new/continuous users||Grouped into new/continuous users||10-fold cross-validation LR used as an alternative method to test the accuracy||Variables added to the final model: CVD type (MI, UA, IS, TIA, or PVD), days’ supply of each of the lipid-lowering agents on the day of discharge, PDC score during the last 720 days, 360 days and the past 8 quarters (each 90 days) Days between the latest prehospitalization dispensing date and the hospital admission date||One year||Std. LR||0.73|
|Two years||Std. LR||0.72|
|Lee et al. (35) 2013||Age, gender, job, monthly income, spouse, educational level, activities of daily living, perceived health status, duration after diagnosis, number of medication types, daily pill counts, side effects of medication, self-efficacy, depression, health literacy, and medication knowledge||PCA was used to transform original features into salient features that are not correlated with each other, SFS was used for variable selection||Testing carried out on the same data used for trainingLR used as an alternative method to test the accuracy||SVM model (9 variables): self-efficacy, age, depression, health literacy, medication knowledge, number of medication type, daily pill counts, duration after diagnosis, and education level Std. LR model (all 16 variables)||SVM with RBF||0.97|
|Aziz et al. (36) 2020||Specific concerns scale, monthly income, general overuse, marital status, gender, educational level, general harm, age, occupational field, ethnicity, specific necessity scale, duration of anti-HTN medications intake, religion, counseling for anti-HTN medications, aids in anti-HTN medications intake, total number of daily anti-HTN medications, other concomitant diseases||RF variable importance method was used for feature selection||Data normalization; 10-fold cross- validation The final model was compared to the initial model with all variables||SF: educational level, marital status, general overuse, monthly income, and specific concerns||RF (all features)||0.79|
|ANN (all features)||0.81|
|SVR (all features)||0.53|
|Gao et al. (37) 2020||456 initial factors including age, gender, marital status, occupation, education, level, type of residence, dietary habits, physical exercise, smoking, BP, height, weight, abdominal girth, family history, patients’ course, severity, management of disease, comorbidities, types and frequency of medications used, diagnosis, and laboratory test results.||RP for groupingDT for subgrouping||Simple cross-validation||The number of instances of nonadherence in the year before the first prescription; types of drugs used in the year before the first prescription; weight; smoking; total number of hospital visits in the year before enrollment; total number of days of medication use in the year before enrollment; age; diabetes; follow-up rates||DT||0.81|
|Galozy et al. (38) 2020||Baseline predictors: age/gender at the time of prescription, prescriber age, aggregate predictors of current and past prescriptions, visit frequency and visit types (outpatient, inpatient and emergency care visits)||Stratified random split, splitting by patient, temporal split||Training (70%) Validation (15%) Test (15%)||Data regarding the five most recent prescriptions were also added to the models including the previous refill data||RF, LR, GB, k-nearest neighbor||No previous refill data: 0.56–0.65|
|Previous refill data included: 0.78–0.91|
|Taranik et al. (39) 2019||Gender, age, medical staff data: measurements, medications, procedures, and activitiesPatient interview data: activities and sleeping durationsMeasurement-based data: body temperature and blood pressure||Fuzzy logic algorithm||Simple cross- validation||All the initial factors during 14 days of treatment in the cardiovascular department after HTN diagnosis||NN (double layer with forward propagation)||0.79|
|Franklin et al. (30) 2016||Demographics : age, sex, raceBaseline predictors during the 180 days prior the index day: (I) IS variables : prior medication use, and comorbidities (II) census based variables  (IV) Hd-PS variables : based on 180 days prior to statin initiation Postbaseline variables : 30-, 60-, and 90-day adherence after the index day||Hd-PS variable selection algorithm used for the claims data for inclusion in a propensity score modelBoosting to improve predictive accuracy||10-fold cross validation LR used as an alternative method to test the accuracy||All baseline||LR||0.59|
|LR + boosting||0.59|
|Postbaseline + IS||LR||0.74|
|LR + boosting||0.75|
|All predictors (for all groups, all patients were included and one year adherence was evaluated)||LR||0.73|
|LR + boosting||0.75|
|Lucas et al. (31) 2017||850 counts (lab, pharmacy, coding) 106 lab values 874 prior PDC values for non-statin medications 10 patient and statin prescription characteristics||Dimension reduction approach to cluster the pharmacy, lab, and billing data into sets of correlated factors||30-fold cross validation||Age, gender, race, smoking, follow-up days, days’ supply, statin strength (mg), number of labs, number of concomitant drugs, number of codes, 30 individual factors based on EHR dataset||RF (without the first statin refill data)||0.73|
|RF (with the first statin refill data. PDC calculated after the first refill)||0.81|
Accuracy was based on c-statistic. Genetic algorithm selection is a heuristic seeking the optimal set of input variables. This heuristic builds a model by a succession of artificial transformations (mutation, crossover, and selection) from an initial population of variables sets. Specific concern scale, evaluates the possibility of adverse reactions resulting from consuming the prescribed medication. Specific Necessity scale, looks into the patient’s belief about their individual requirements in adhering to their prescribed medicine. To characterize the extent to which a particular code was relevant for a given individual for each of the data types (pharmacy, laboratory, and coding), they counted the number of times that code appeared in the medical record prior to statin initiation. ML, machine learning; CVD, cardiovascular disease; HTN, hypertension; HCL, hypercholesterolemia; KILLIP classification, is used in patients presenting with acute MI for the purpose of risk stratification; NYHA, New York Heart Association functional class; FS, factor selection; SMOTE, synthetic minority over-sampling technique; GA, global adherence; MA, medication adherence; RF, random forest; SF, specific features; LMT, logistic model trees; SVM, support vector machine; HF, heart failure; EF, ejection fraction; MMSE-K, Mini Mental Status Examination-Korean version; LOOCV, leave-one-out cross-validation; RBF, radial basis function; ACS, acute coronary syndrome; NN, neural network; MLP, multiple layer perceptron; GB, gradient boosting trees; LR, logistic regression; CABG, coronary artery bypass graft; CKD, chronic kidney disease; EHR, electronic health records; Std. LR, standard logistic regression; LASSO, least absolute shrinkage and selection operator, LASSO addresses the overfitting problem of LR with backward selection by minimizing Akaike’s information; PCI, percutaneous coronary intervention; CHF, congestive heart failure; MI, myocardial infarction; SNF, skilled nursing facility; IRF, inpatient rehabilitation facility; LTC, long term care; PCA, principal component analysis; SFS, sequential feature selection; ANN, artificial neural network; SVR, support vector regression; SF, selected features; RP, recursive partitioning; DT, decision tree; Hd-PS, high dimensional propensity score; IS, investigator specified; CCL, complication and comorbidity level; BRT, booster regression trees.
Selection of predictors
Candidate predictors across health conditions and across studies were selected based on literature review and several factor selection techniques. The final models included various predictors based on availability and applicability which might have resulted in heterogeneity in outcomes. Common predictors in the models were sociodemographic characteristics, past medical history, severity of the disease, lab values, current medications, and previous refill data. Several studies indicated that previous adherence to medication and other health related recommendations may increase the accuracy of the model’s performance for prediction of MA (30,31,33,34,37-40). One of them, used patient adherence to health professional recommendations (regarding nutrition, lifestyle, activities, and sleeping) as a predictor of patients’ MA (40). Six studies used patients’ MA history as an MA predictor (30,31,33,34,37,38).
Risk of bias
The main source of bias in the studies was the MA measurement methods. Several methods with different levels of accuracy were used for MA calculation in studies. Five studies used PDC which is among the most common measured parameters in pharmacy claim databases for MA calculation (30,31,33,37,38). Four studies used self-reported surveys (32,35,36,41) that generally tend to overestimate adherence due to recall or reporting bias (44). One study used healthcare provider’s opinion (40), and one study used healthcare staff report to estimate the MA (39). Lastly, one study used medication refill gap of >120 days as a definition for non-adherence (34).
ML models predicting MA compared to conventional methods
Based on our review, it seems that ML methods do not have better predictive values compared to conventional statistical techniques. In studies that used similar predictive variables, there was no noticeable difference between the accuracy of the models (30,33,34). In one of the studies that the accuracy of the ML model was significantly higher than the conventional methods (0.97 vs. 0.71), there was an internal validation issue (35). In Lee et al. (35) study, they tested the ML model with the same dataset previously used for model training and it resulted in a biased high accuracy. In two other studies that the ML model had higher accuracies, several variables related to patient’s past medical history were added to ML models that were not included in the LR models (32,34). At first glance it seems that using ML models provided the opportunity of adding several predictive factors to the model. However, evaluation of previous literature indicates that utilization of patients’ MA history as a predictor of MA had previously been used in LR models. One of the studies in this review that used LR with and without MA history also reported that inclusion of MA history will enhance the predictive accuracy of the model even without utilization of ML methods (30).
ML models predicting MA based on MA measurement tools
Several methods with different levels of accuracy were used for MA measurement in studies. Five studies used PDC which is among the most common measured parameters in pharmacy claim databases for MA calculation (30,31,33,37,38). Among these five studies, only two of them compared their ML model accuracy with the conventional methods and did not report any significant differences (30,33). One study used medication refill gap of >120 days as a definition for non-adherence (34) and reported higher accuracies in their ML models. First, studies have shown that refill gap as an MA measurement tool is less strongly correlated with patient outcomes in comparison with PDC (45). On the other hand, a more careful comparison of the variables included in their models indicated that several variables based on patients’ MA history were added to the ML model but not to the standard LR model (34). Four studies used self-reported methods (32,35,36,41) that generally tend to overestimate adherence due to recall or reporting bias (44). Among these four studies, only one of them compared their ML model accuracy to an LR model (35). While they reported a dramatically higher accuracy for their ML model in comparison to their LR model, their finding was affected by an internal validation issue. Lee et al. (35) tested the accuracy of their ML model using the same dataset previously used for model training. Two studies used healthcare provider’s opinion and report to estimate MA among patients, but none of them compared their model accuracy to other statistical methods (39,40).
ML models predicting MA based on health condition
Because of the cooccurrence of the clinical conditions included in this review, and the diversity of the variables used in each study, it is exceedingly difficult to directly compare predictive models for the same clinical condition across studies. However, because of the differences in the medications used for each health condition and possible differences in MA in each health condition, we also compared the performance of ML models from disease perspective.
ML models predicting MA among patients after ACS or MI
A total of three studies were focused on MA among patients after an ACS or an MI, with several noticeable strengths (32-34). All three studies were carried out on large populations and tested the accuracy of their models by LR as an alternative statistical method (32-34). Bourdès et al. (32) evaluated MA among patients during the first two years after hospitalization for ACS. They included several variables regarding patient’s characteristics, disease severity, lab values, and medications at discharge from the hospital. To have the most predictive variables in the model, they carried out forward and backward stepwise feature selection in addition to genetic input selection algorithm. Genetic algorithm selection is a heuristic method for selection of the optimal set of input factors. Using genetic algorithm, they built a model by several artificial transformations from an initial group of variable set (32). In addition, they internally validated their model by simple cross-validation which could have strengthen the study even more if replaced with a k-fold cross-validation. Despite having several strengths, Bourdès et al. (32) study was faced with some limitations. In their study, they used medication persistence instead of MA with a strict dichotomous definition. Patients who were still using all their prescribed medications two years after being discharged from the hospital, were considered to have medication persistence. Based on their definition, discontinuation of any of the medications, indicated non-persistence even if a medication contraindication or a side-effect was the reason for discontinuation. Considering all strengths and limitations, Bourdès et al. (32) fitted NN model showed about 0.09 higher prediction accuracy compared to the standard LR. However, the higher accuracy observed should be interpreted considering that several variables in the NN were missing from the LR model. The second study evaluated statin adherence after hospitalization due to MI (33). Zullig et al. (33) used a large sample of Medicare beneficiaries to predict patients’ MA based on PDC which is the preferred method to measure medication adherence (33,43). They used least absolute shrinkage and selection operator (LASSO), a regression analysis method that performs both variable selection and regularization (33). LASSO is used for the purpose of enhancing the prediction performance and interpretability of the fitted model (33). After including patients’ MA during the six months prior to the study, they internally validated their model by 10-fold cross-validation. Comparison between their models did not emerge any clear superiority due to ML utilization and none of the models had an acceptable discrimination with a c-statistic of 0.7–0.8 (33). In the third study by Hu et al. (34), they grouped patients into two cohorts of initial and continuous statin users (34). In their models, instead of MA, they fitted a model to predict medication persistence with a specific definition. Patients who had a gap of 120 days or longer during the study period, were considered as non-persistent to medication. They tried to check the accuracy of their model by conducting a standard LR model and found higher accuracy in their ML models. However, the inconsistency between the variables used in LR model and the ML models may explain the higher accuracy observed in the ML models. They used several PDC scores from the first two years after the initiation of the study as predictors in their ML models that were missing from their LR model. It is a well-known fact that including measures of previous medication adherence significantly enhances the accuracy of future medication adherence prediction compared to usual baseline measures typically obtained from claims data (46). Not having included previous PDC scores in their ML models, the accuracy of their models would have been lower than the values they reported.
ML models predicting MA among HF patients
MA among patients with HF was evaluated in two studies (40,41). Besides the small number of studies, each study had several limitations. Both studies had extremely small sample sizes (n=90 and n=76) which is one of the most important limitations for ML techniques (47). In addition, none of the fitted models were externally validated on other datasets. The study by Karanasiou et al. (40) relied on clinician’s judgement for patients’ adherence to medication which does not seem to be a reliable measure. It has been reported that physicians tend to overestimate MA of their non-psychiatric patients (48). Hence, there is a chance of potential bias for patients’ MA evaluated by physicians in their study. In addition, Karanasiou et al. (40) increased the accuracy of their model for MA prediction from 0.78 to 0.91 by adding patients’ global adherence (GA) as a predictive variable to the model. GA in these modes was based on physician’s judgement about patient’s adherence to recommendations regarding healthy nutrition and lifestyle. The extremely high accuracy observed, may be explained by considering that patients with high adherence to nutrition and lifestyle recommendations are more likely to have a higher MA (49). Even though they observed a high accuracy in their model, it does not seem to be practical to evaluate patients’ MA by subjectively evaluating another measure of adherence with possible bias. Despite the limitations in Karanasiou et al. (40) study, they had strengths including 10-fold cross-validation and handling the imbalanced classes with synthetic minority over-sampling technique (SMOTE). The second study on HF was carried out by Son et al. (41) and had several limitations. First, they evaluated MA by utilization of a single item on a self-reported survey. Studies have shown that self-reported adherence generally have good specificity but limited sensitivity for detecting poor adherence. In other words, self-reported nonadherence can be trusted while self-reported adherence is less reliable (50). Since they have not provided the MA rate in their study, further judgement in this regard is not possible. In addition, they used leave-one-out cross-validation (LOOCV) method with certain limitations (51). LOOCV which is an asymptotically equivalent of k-fold cross-validation, is subject to high variance and overfitting due to feeding the model with almost all the training data to learn and just a single observation to evaluate (51). Overfitting due to LOOCV utilization, might explain the moderate accuracy (0.77) observed in Son et al. (41) study with including only five or seven variables in each model. Had the model tested in a large external dataset with different baseline patient characteristics, the model’s external performance would have been lower than 0.77 and far from an optimal accuracy.
ML models predicting MA among patients with HTN
Five studies used ML for prediction of MA among HTN patients (35-39). One of the studies in this group had a wide inclusion criterion and covered several chronic conditions in addition to HTN (35). Among the studies, only Gao et al. and Galozy et al. (37,38) used large sample sizes (n=7,638 and n=18,581). The conventional method for MA measurement was PDC in two studies. Taranik et al. (39) used staff report for identifying MA while all other studies used different self-reported surveys. The highest ML accuracy was observed in Lee et al. (35) study (0.97) which was noticeably higher than the accuracy of their LR model (0.71). The reason for the exceptionally high accuracy observed in their ML model is the fact that the same data was used for training and testing the model, leading to a biased accuracy (35). Having used a k-fold cross-validation, the accuracy would have been far lower than the observed value in their study. The next study by Galozy et al. (38) had several strength points for evaluating MA among HTN patients due to utilization of a large sample size, using PDC for MA measurement, and internal validation of the model by simple cross-validation. using a limited number of variables in several ML models, they observed accuracies ranging from 0.56 to 0.65 which is close to the accuracy of conventional methods in a previous meta-analysis of 20 studies ranging from 0.32 to 0.68 (3). Adding patient’s MA history, they observed higher accuracies in a range from 0.78 to 0.91 indicating the importance of MA history as a predictor of future MA (38). Aziz et al. (36) performed an RF variable importance method for feature selection. By incorporating five variables with the highest predictive values in three ML models, they found similar accuracies. In addition, using Wilcoxon signed-rank test, they did not notice any significant difference between the actual MA and the MA predicted by ML models. Having their models validated on a different dataset would have enhanced their model external validity. The other study with a large sample size was carried out by Gao et al. (37) in China. They used recursive partitioning (RP) for grouping the patients. RP creates a decision tree (DT) that tries to make the best classification of the population by splitting it into sub-groups based on several binary independent factors (37). By including the number of instances of nonadherence during the year before the current prescription, they fitted a DT model with a 0.81 accuracy which was similar to Galozy et al. (38) study after addition of MA history (0.78–0.91). The last study was carried out by Taranik et al. (39) with critical weaknesses. The study used a small sample (n=62) during an extremely brief period (2 weeks) that patients were under observation in cardiovascular department. Incorporating patient’s adherence to different components of treatment plan including activities and sleeping duration was a strength for the study. Considering all strengths and weaknesses, they reported a 0.79 accuracy which was not externally validated.
ML models predicting MA among patients with HCL
Lastly, two studies used ML to evaluate MA among patients with HCL (30,31). Despite the small number of studies in this group, they had noticeable strengths including large sample sizes (n=77,703 and n=138,731) from huge data sources (Medicare and Military Health System), utilization of PDC as the conventional method for MA measurement, and great internal validation tests (10- and 30-fold cross-validation). Franklin et al. (30) used boosting which is an ML method to improve the predictive accuracy of regression models. They did not observe any improvement in the model performance. However, they found that adding patient’s initial refill data, increases the model performance from 0.59 to 0.73. The other study by Lucas et al. (31) used dimension reduction approach to cluster the data into sets of correlated factors. Using several values from the EHR and previous PDC values for adherence to statin and non-statin medications, they fitted two RF models. They observed a 0.73 accuracy for their initial model which was enhanced to 0.81 after adding the first statin refill data, and PDC recalculation for the period after the first refill. Their study incorporated an excellent internal validity test (30-fold cross validation) and could have been improved by external validation.
This study has limitations that needs to be addressed. First, this review was only focused on publications in English. Second, due to the small number of studies with several differences in variables and techniques in each health state, a limited comparison between different ML models in each health state was carried out. This review was based on the accuracy of the ML models used in each study. Hence, the evaluation of studies was limited to sample size of the studies, validation techniques used for internal validation, reliability of methods used for MA measurement, and above all, presence of external validation.
Medication adherence is an extremely complex behavior that is not easily predictable. We found a limited number of studies that developed predictive models using ML to predict medication adherence among patients with CVD and its risk factors namely HTN and HCL. Inclusion criteria of each health condition, variables used in each study, and the definitions of medication adherence varied between studies. Hence, the accuracy of each model needs to be interpreted with careful consideration of study characteristics. Compared to conventional statistical methods, the fewer restrictions for inclusion of variables in ML models, may enhance the model performance. More research is required to predict medication adherence with higher accuracy to facilitate the identification of patients that may benefit the most from different interventional techniques to increase MA.
Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at https://jmai.amegroups.com/article/view/10.21037/jmai-21-26/rc
Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-21-26/prf
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-21-26/coif). SMA reports grants from Regeneron, grants from Valeant, grants from Sanofi, and grants from Pfizer/BMS, outside the submitted work. The other authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
- WHO. Heart disease facts. National Center for Chronic Disease Prevention and Health Promotion, Division for Heart Disease and Stroke Prevention, 2020. Available online: https://www.cdc.gov/chronicdisease/resources/publications/aag/heart-disease-stroke.htm
- Francula-Zaninovic S, Nola IA. Management of Measurable Variable Cardiovascular Disease' Risk Factors. Curr Cardiol Rev 2018;14:153-63. [Crossref] [PubMed]
- Naderi SH, Bestwick JP, Wald DS. Adherence to drugs that prevent cardiovascular disease: meta-analysis on 376,162 patients. Am J Med 2012;125:882-7.e1. [Crossref] [PubMed]
- Benjamin EJ, Muntner P, Alonso A, et al. Heart Disease and Stroke Statistics-2019 Update: A Report From the American Heart Association. Circulation 2019;139:e56-e528. [Crossref] [PubMed]
- Du L, Cheng Z, Zhang Y, et al. The impact of medication adherence on clinical outcomes of coronary artery disease: A meta-analysis. Eur J Prev Cardiol 2017;24:962-70. [Crossref] [PubMed]
- Mazurek JA, Jessup M. Understanding Heart Failure. Heart Fail Clin 2017;13:1-19. [Crossref] [PubMed]
- Wu JR, Moser DK. Medication Adherence Mediates the Relationship Between Heart Failure Symptoms and Cardiac Event-Free Survival in Patients With Heart Failure. J Cardiovasc Nurs 2018;33:40-6. [Crossref] [PubMed]
- GBD 2017 Risk Factor Collaborators. Global, regional, and national comparative risk assessment of 84 behavioural, environmental and occupational, and metabolic risks or clusters of risks for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018;392:1923-94. [Crossref] [PubMed]
- CDC. Facts About Hypertension. Centers for Disease Control and Prevention, 2020. Available online: https://www.cdc.gov/bloodpressure/facts.htm
- Feng Y, Han M, Qie R, et al. Adherence to antihypertensive medications for secondary prevention of cardiovascular disease events: a dose-response meta-analysis. Public Health 2021;196:179-85. [Crossref] [PubMed]
- Ibrahim MA, Asuka E, Jialal I. Hypercholesterolemia. Treasure Island, FL, USA: StatPearls Publishing, 2021.
- Xu T, Yu X, Ou S, et al. Statin Adherence and the Risk of Stroke: A Dose-Response Meta-Analysis. CNS Drugs 2017;31:263-71. [Crossref] [PubMed]
- Yach D. Adherence to long-term therapies evidence for action, 2003. Available online: https://apps.who.int/iris/bitstream/handle/10665/42682/9241545992.pdf
- Leslie KH, McCowan C, Pell JP. Adherence to cardiovascular medication: a review of systematic reviews. J Public Health (Oxf) 2019;41:e84-94. [Crossref] [PubMed]
- Fernandez-Lazaro CI, Adams DP, Fernandez-Lazaro D, et al. Medication adherence and barriers among low-income, uninsured patients with multiple chronic conditions. Res Social Adm Pharm 2019;15:744-53. [Crossref] [PubMed]
- Al-Ganmi AHA, Alotaibi A, Gholizadeh L, et al. Medication adherence and predictive factors in patients with cardiovascular disease: A cross-sectional study. Nurs Health Sci 2020;22:454-63. [Crossref] [PubMed]
- Mann DM, Woodward M, Muntner P, et al. Predictors of nonadherence to statins: a systematic review and meta-analysis. Ann Pharmacother 2010;44:1410-21. [Crossref] [PubMed]
- Choudhry NK, Fischer MA, Avorn J, et al. The implications of therapeutic complexity on adherence to cardiovascular medications. Arch Intern Med 2011;171:814-22. [Crossref] [PubMed]
- Gellad WF, Grenard JL, Marcum ZA. A systematic review of barriers to medication adherence in the elderly: looking beyond cost and regimen complexity. Am J Geriatr Pharmacother 2011;9:11-23. [Crossref] [PubMed]
- Osterberg L, Blaschke T. Adherence to medication. N Engl J Med 2005;353:487-97. [Crossref] [PubMed]
- Steiner JF. Can we identify clinical predictors of medication adherence... and should we? Med Care 2010;48:193-5. [Crossref] [PubMed]
- Lauffenburger JC, Franklin JM, Krumme AA, et al. Predicting Adherence to Chronic Disease Medications in Patients with Long-term Initial Medication Fills Using Indicators of Clinical Events and Health Behaviors. J Manag Care Spec Pharm 2018;24:469-77. [Crossref] [PubMed]
- Deo RC. Machine Learning in Medicine. Circulation 2015;132:1920-30. [Crossref] [PubMed]
- Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. [Crossref] [PubMed]
- Debray TP, Damen JA, Snell KI, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ 2017;356:i6460. [Crossref] [PubMed]
- Moons KG, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med 2014;11:e1001744. [Crossref] [PubMed]
- Ma J, Ovalle A, Woodbridge DM. Medhere: A Smartwatch-based Medication Adherence Monitoring System using Machine Learning and Distributed Computing. Annu Int Conf IEEE Eng Med Biol Soc 2018;2018:4945-8. [Crossref] [PubMed]
- Koesmahargyo V, Abbas A, Zhang L, et al. Accuracy of machine learning-based prediction of medication adherence in clinical research. Psychiatry Res 2020;294:113558. [Crossref] [PubMed]
- Huang Y, Talwar A, Chatterjee S, et al. Application of machine learning in predicting hospital readmissions: a scoping review of the literature. BMC Med Res Methodol 2021;21:96. [Crossref] [PubMed]
- Franklin JM, Shrank WH, Lii J, et al. Observing versus Predicting: Initial Patterns of Filling Predict Long-Term Adherence More Accurately Than High-Dimensional Modeling Techniques. Health Serv Res 2016;51:220-39. [Crossref] [PubMed]
- Lucas JE, Bazemore TC, Alo C, et al. An electronic health record based model predicts statin adherence, LDL cholesterol, and cardiovascular disease in the United States Military Health System. PLoS One 2017;12:e0187809. [Crossref] [PubMed]
- Bourdès V, Ferrières J, Amar J, et al. Prediction of persistence of combined evidence-based cardiovascular medications in patients with acute coronary syndrome after hospital discharge using neural networks. Med Biol Eng Comput 2011;49:947-55. [Crossref] [PubMed]
- Zullig LL, Jazowski SA, Wang TY, et al. Novel application of approaches to predicting medication adherence using medical claims data. Health Serv Res 2019;54:1255-62. [Crossref] [PubMed]
- Hu F, Warren J, Exeter DJ. Predicting Lipid-Lowering Medication Persistence after the First Cardiovascular Disease Hospitalization. Methods Inf Med 2020;59:61-74. [Crossref] [PubMed]
- Lee SK, Kang BY, Kim HG, et al. Predictors of medication adherence in elderly patients with chronic diseases using support vector machine models. Healthc Inform Res 2013;19:33-41. [Crossref] [PubMed]
- Aziz F, Malek S, Mhd Ali A, et al. Determining hypertensive patients' beliefs towards medication and associations with medication adherence using machine learning methods. PeerJ 2020;8:e8286. [Crossref] [PubMed]
- Gao W, Liu H, Ge C, et al. A Clinical Prediction Model of Medication Adherence in Hypertensive Patients in a Chinese Community Hospital in Beijing. Am J Hypertens 2020;33:1038-46. [Crossref] [PubMed]
- Galozy A, Nowaczyk S. Prediction and pattern analysis of medication refill adherence through electronic health records and dispensation data. J Biomed Inform 2020;112S:100075. [Crossref] [PubMed]
- Taranik M, Kopanitsa G. Using Machine Learning for Personalized Patient Adherence Level Determination. Stud Health Technol Inform 2019;261:174-8. [PubMed]
- Karanasiou GS, Tripoliti EE, Papadopoulos TG, et al. Predicting adherence of patients with HF through machine learning techniques. Healthc Technol Lett 2016;3:165-70. [Crossref] [PubMed]
- Son YJ, Kim HG, Kim EH, et al. Application of support vector machine for prediction of medication adherence in heart failure patients. Healthc Inform Res 2010;16:253-9. [Crossref] [PubMed]
- Li Y, Jasani F, Su D, et al. Decoding Nonadherence to Hypertensive Medication in New York City: A Population Segmentation Approach. J Prim Care Community Health 2019;10:2150132719829311. [Crossref] [PubMed]
- PQA. Adherence measures. Pharmacy Quality Alliance, 2021. Available online: https://www.pqaalliance.org/adherence-measures
- Anghel LA, Farcas AM, Oprean RN. An overview of the common methods used to measure treatment adherence. Med Pharm Rep 2019;92:117-22. [Crossref] [PubMed]
- Zhu VJ, Tu W, Rosenman MB, et al. A Comparison of Data Driven-based Measures of Adherence to Oral Hypoglycemic Agents in Medicaid Patients. AMIA Annu Symp Proc 2014;2014:1294-301. [PubMed]
- Kumamaru H, Lee MP, Choudhry NK, et al. Using Previous Medication Adherence to Predict Future Adherence. J Manag Care Spec Pharm 2018;24:1146-55. [Crossref] [PubMed]
- Figueroa RL, Zeng-Treitler Q, Kandula S, et al. Predicting sample size required for classification performance. BMC Med Inform Decis Mak 2012;12:8. [Crossref] [PubMed]
- Heeb RM, Kreuzberg V, Grossmann V. Physicians’ Assessment of Medication Adherence: A Systematic Review. J Pharm Care Heal Syst 2019;6:1-18.
- Lee YM, Kim RB, Lee HJ, et al. Relationships among medication adherence, lifestyle modification, and health-related quality of life in patients with acute myocardial infarction: a cross-sectional study. Health Qual Life Outcomes 2018;16:100. [Crossref] [PubMed]
- Stirratt MJ, Dunbar-Jacob J, Crane HM, et al. Self-report measures of medication adherence behavior: recommendations on optimal use. Transl Behav Med 2015;5:470-82. [Crossref] [PubMed]
- Shao J. Linear Model Selection by Cross-validation. J Am Stat Assoc 1993;88:486-94. [Crossref]
Cite this article as: Zakeri M, Sansgiry SS, Abughosh SM. Application of machine learning in predicting medication adherence of patients with cardiovascular diseases: a systematic review of the literature. J Med Artif Intell 2022;5:5.