A systematic literature review of explainable risk assessment models for bronchial asthma

Introduction

Background

Bronchial asthma (BA) is a chronic respiratory disease that is prevalent in the lives of an estimated 300 million people worldwide and the cause of around 1,000 deaths each day (1). This long-term condition impacts the lives of individuals living with asthma, where there is a reduced quality of life or even the risk of death. Improper management of an individual’s asthma condition, such as ignoring symptoms, may put that person at risk of experiencing asthma exacerbations, which are severe cases of asthma symptoms that make it difficult to breathe due to the narrowing of the lung’s airways, inflammation, swelling, and increased production of (2). Each patient would have a Written Asthma Action Plan written specifically for them by their doctor to allow the patients to monitor their own symptoms and react accordingly (1). A Written Asthma Action Plan would have the dose of medication for a patient and the next action to be taken by the patient, such as going to the nearest hospital if the condition worsens. It also allows the patients to check whether they are in the green zone, the yellow zone, and the red zone, with green being in “good condition”, yellow being in “getting worse”, and red indicating that patients should be alert as their condition is even worse. Inattentiveness towards asthma symptoms can be life-threatening. Artificial intelligence (AI) has been broadly used in the medical field, such as in medical imaging, healthcare tasks (3), and utilised in the care of BA, where it is used for the diagnosis and screening of asthma in patients, classifying patients, managing and monitoring asthma, and for the treatment of asthma (4). It is expected to enhance medical care using machine learning (ML) or deep learning (DL) models. However, AI is still confronted with the issues regarding transparency, interpretability, and explainability, which could affect the users’ trust towards the AI model (5). The complex inference mechanisms of the AI models make it incomprehensible to its users. This limitation in AI is known as the “Black box”, where the AI model processes input data through mathematical algorithms and produces output data, but it is not always known how the AI model reached the decision or prediction it has made. There are certain limitations in AI that research in explainable AI (XAI) wishes to address. Especially within the medical sector, where risk prediction models are required to be accurate, trustworthy, and precise.

Rationale and knowledge gap

The increasing interest of using AI for risk prediction within a clinical setting extends the need for the models to be explainable. The scarcity of recent publications for XAI application in risk prediction for BA is highlighted within this literature review. Although many studies have used AI for diagnosis of asthma, lung-related diseases, and prediction for other diseases in the healthcare sector, there is a lack of literatures particularly on XAI for prediction of risk in BA. This paper aims to review the existing literature and find connections between them by sectioning and examining the selected literatures to procure a detailed insight regarding XAI-based risk assessment models by examining the models used, their architecture, their performances, the input data, and the outputs of the models before concluding with the potential suitable XAI risk assessment model for BA.

Objective

The objectives for this review were to analyse the current XAI risk assessment models for BA and their performances, the optimum explanation technique, and to conclude which XAI risk assessment model would work better for risk assessment in BA. This includes any risks to patients that are related to BA, including predicting the risk of the next asthma exacerbation, predicting the risk of being a higher or lower risk patient, which would indicate the need for alertness towards patients’ need for hospitalization, and anything related to assessing risk in BA. We present this article in accordance with the PRISMA reporting checklist (available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-204/rc).

Methods

The systematic literature review was conducted in eight steps using the methods in a paper by Yu and Watson (6) for conducting a systematic literature review as a guide. Figure 1 shows the summary of the systematic literature review search process for the selection of XAI models. Figure 2 shows the PRISMA flow chart of the systematic literature review process. The search process is divided into three different stages, where the first stage, indicated by the blue box, comprises of the planning stage with steps 1 and 2 of the systematic literature review, which are addressing the research problem and specifying the review protocols. The second stage, indicated by the green box, is the process of the literature review itself with steps 3 to 7, which are literature identification, inclusion screening, eligibility, extracting data, and analysing the data. Finally, in the last stage indicated by the yellow box, is step 8, reporting the findings.

Figure 1 Systematic literature search process. AI, artificial intelligence; XAI, explainable artificial intelligence.

Figure 2 PRISMA flow chart of the systematic literature review process. ACM, Association for Computing Machinery.

Step 1: addressing the research problem

The first step in this systematic literature review is to detail the research problem, which would in turn steer the direction of the review (7). It is rudimentary to set a clear boundary and scope to narrow down on the problem at hand. A major problem in AI is the lack of interpretability or also known as the black box problem. The demand for interpretability in AI models is increasing, especially in the medical field (8). This research provides us with an important opportunity to contribute to the field of XAI for risk assessment in BA by exploring current related literatures, the data used for feature selection, the performance of models used, the current explanation models used, and drawing a conclusion for the next step in future research.

Step 2: specifying the review protocols

The review protocols are specified before starting the literature search. They are constructed by considering the purpose of the study, the research questions, the inclusion criteria for literatures, the search strategies, how the screening of the literature would be conducted, procedure of data procurement, analysis, and reporting the findings (9,10). To start the process of determining the review protocols, an initial general search was done using broad keywords such as “artificial intelligence in asthma”, “artificial intelligence in healthcare”, “explainable artificial intelligence in asthma”, and “explainable intelligence in healthcare”. It was then decided that the term AI would be removed from the search terms and only XAI would be kept to narrow down the search. However, some literatures that did not distinctively specify that they were applying XAI still showed up in the search and were included in the review after a thorough screening. In addition to that, the domain of the search was limited to asthma and other diseases within the medical field, where XAI techniques are applied for risk assessment. The decision to limit the domain to only asthma and other diseases was set because limiting it to only asthma would not yield enough literature, while not limiting it to be within only the medical field would yield too many results. It was also decided that literatures using different terms, such as explainable ML and explainable DL instead of XAI were accepted. Apart from this, literatures with both ad-hoc and post-hoc types of XAI were included.

The inclusion criteria are as follows: (I) literature that included XAI in their research, both ML and DL, and post-hoc and ad-hoc, and literature that did not explicitly specify research for XAI; however, the model used in their research is considered under explainable models. (II) Literature that uses XAI within the asthma or medical field, or using XAI for risk assessment. (III) Literature from the years 2019 until 2023 was included in this literature review to construct a review based on recent publications. (IV) Literature written in English. (V) Only articles with available full text.

The exclusion criteria are as follows: (I) literature that was not directly relevant to the research topic. (II) Not written in English. (III) Unavailable full-text literature.

The search strategy used was to search through selected databases using the formulated search terms with consideration toward the research topic. The five databases used were Science Direct, Association for Computing Machinery (ACM) Digital Library, Springer, PubMed, and Scopus. The initial search terms used before narrowing it down were “artificial intelligence”, “explainable artificial intelligence”, “risk assessment”, “risk assessment model”, “asthma”, “healthcare”, and the combinations and permutations of the search terms such as “artificial intelligence and healthcare”, “artificial intelligence and asthma”, “explainable artificial intelligence and healthcare”, “explainable artificial intelligence and asthma”, “explainable artificial intelligence” AND “risk assessment” AND “health”, and “explainable artificial intelligence” AND “risk assessment” AND “asthma”. The search terms that provided broad results were further narrowed down and combined to create more specific search terms. The search terms were then refined to only include a search on XAI. In conclusion, the four decided search terms were “explainable artificial intelligence” AND “asthma”, “explainable artificial intelligence” AND “risk assessment” AND “health OR asthma”, “explainable artificial intelligence” AND “risk assessment model”, and “explainable artificial intelligence” AND “risk assessment model” AND “asthma”.

The screening of the literature would then be conducted by manually reading through all the identified literatures after filtering them to determine which literature would be relevant based on the set inclusion criterion. The procurement of data would be carried out by listing all the selected literature in an Excel file, separated by their titles, abstracts, and keywords. Several categories would then be created to try to find similarities between the chosen literatures. Any categories that could be combined would then be combined. The analysis of the data is then carried out by deciding how to categorise and find similarities between the literatures. Followed by a discussion to report the findings of the literature.

Step 3: finding the literature

The systematic review was then conducted by searching through literature concentrating on XAI techniques for risk assessment in healthcare or asthma research using the determined searched terms through the five databases. The screening of the literatures was done by one reviewer. The literatures that were included were published between January 1, 2019 and October 26, 2023. The determined search terms, “explainable artificial intelligence” AND “asthma”, “explainable artificial intelligence” AND “risk assessment” AND “health OR asthma”, “explainable artificial intelligence” AND “risk assessment model”, and “explainable artificial intelligence” AND “risk assessment model” AND “asthma” was all used in the same manner for all the databases following the rules of the Boolean operators to specify the search.

The search was conducted through five databases, which were Science Direct, ACM Digital Library, Springer, PubMed, and Scopus by one reviewer. A search using the combined search term “explainable artificial intelligence” AND “asthma” through the databases Science Direct, ACM, Springer, PubMed, and Scopus initially yielded a total of 211 studies, not including preview-only content for Springer. The search was then refined by using the filters to only include work from 2019 until 2023, English written literature, free full text, full text and abstracts, research articles, books, and conference proceedings, which yielded a total of 151 studies. For the combined search term “explainable artificial intelligence” AND “risk assessment” AND “health OR asthma” through the five databases, it returned an initial total of 281 studies. After being filtered with the same constraints as above, the search yielded a total of 191 studies. Furthermore, the combined search term “explainable artificial intelligence” AND “risk assessment model” yielded an initial of 191 studies. After being filtered, a total of 108 studies were left. Finally, the combined search term “explainable artificial intelligence” AND “risk assessment model” AND “asthma” returned a total of six studies, and with five left after being filtered using the same constraints used for the other search terms. The initial screening of the literature using the search terms and filters produced a total of 455 research papers from the combined five databases.

Step 4: literature screening for inclusion

The titles and abstracts of the screened studies were then read by one reviewer to determine their relevance to this research on XAI risk assessment model for BA. After manually going through the titles and abstracts of the literatures combined from the 5 databases to exclude non-relevant literatures and removing 11 duplicates, a total of 51 studies were determined to be related to the research of XAI risk assessment for asthma including a few articles on XAI risk assessment models for health conditions other than asthma such as chronic cough, cardiovascular disease (CVD), acute kidney injury (AKI), and allergies.

Step 5: assessing the eligibility of literature

The 51 literatures selected based on their titles and abstracts were then further evaluated to determine their relevance and eligibility by going through and reading the full text of the papers. Eight articles were excluded because the full-text writing of the articles could not be retrieved. A total of 43 research papers were included in this research. The 43 research papers left were sorted using Microsoft Excel, listing their titles, keywords, and abstracts for easier categorization. The literature search process is summarised in Figure 1.

Step 6: extracting data from the literature

Information was then withdrawn from each research paper for the purpose of generalization, categorization, and meta-analysis. The information was manually extracted by one reviewer from each research paper by reading through them and taking notes on what each paper had, such as if it focused on asthma, lungs, or other healthcare-related topics. Mendeley was also used to help with organizing and sorting the reference papers. Tags in Mendeley were manually created to help with the sortation of the papers. The papers were tagged by whether they implemented ML or DL, whether they were asthma, lung, or other healthcare-related papers, whether they used predictive or diagnostic models, and what kind of data they were using. Any literature with missing data or that is an ongoing study was kept, but noted as ongoing. The results were then recorded in tables for further categorisation. It was determined that the selected literature could then be sorted into several categories under three main sections. It was decided that the three main sections would be application, architecture, and their models, under which the papers will be further sorted into categories based on their shared characteristics. This provides an insight on current research trends on XAI-based risk assessment models in BA or healthcare and overall performance of the models in improving the quality of care by means of risk assessment within their category. Each paper may belong to one or more categories.

Step 7: analysing the data

In this step, we examine the selected literatures to procure a considerable detailed insight regarding XAI-based risk assessment models. A possible cause of heterogeneity would be the scarcity of literatures focused solely on XAI-based risk assessment models within the domain of BA. The search, if limited to the specific research topic, procured limited results, and thus the need to expand the search topic to include XAI risk assessment models in healthcare as well. However, similarities were found between them. The selected literatures were sorted under three sections, which are application, architecture, and the type of models being used in the selected literature. The author(s), the data used, the architecture of the algorithms; whether they are ML or DL or others, the models used, its specific application, and the performance metrics used is the information extracted from the literatures and listed in the tables as a way of analysing the data. The input data used by the literatures were sought to find an insight regarding the features that the XAI models used in their application domain. The performance results of the models used in the studies were analysed to eventually draw a conclusion on which XAI model would be suitable for the risk assessment of BA.

Step 8: reporting the findings

Step 8 of reporting the findings was incorporated throughout steps 1 to 7; therefore, step 8 was completed through the documentation of steps 1 to 7.

Results

A total of 43 literatures were found to be within the inclusion criteria. The 43 literatures were then sorted according to their similarities by exploring the XAI techniques used to assess risk, the data used by the models, their performance metrics, and their level of relevance towards this study, where some papers are more relevant than others. Although there are several studies that researched XAI techniques for diagnosis, risk prediction, and identification of levels of risk in other health issues, only a limited number of literatures were specifically examining XAI for asthma risk prediction. The 43 literatures are sorted under three sections: application, architecture, and the type of models.

The first section, application, refers to where the AI algorithms are being applied. The application section is separated into three categories, which are literatures that have an application in asthma, lungs, or others. The other category includes applications in the healthcare sector that are not related to asthma or lungs, such as the prediction of sepsis, risk of mortality or heart attacks, and other healthcare-related literatures. The application section was also separated into these three categories to set apart literatures that would be more relevant to this research, which would be literatures related to XAI and asthma. The next most relevant category would be literatures that employ XAI models in application to lung-related problems or diseases. Some literatures may also belong to one or more categories. Tables 1-3 list the literatures that have applications in asthma, lungs, or other sectors, respectively.

Section 1: application

Literature in the asthma application category

The asthma application category under the application section highlights the scarcity of literatures directly relevant to risk assessment for asthma using XAI as shown in Table 1. Literature with research in AI models related to asthma was listed in this category. Five out of the 43 selected literatures were within the asthma domain. The literatures by Tomita et al. (11), Ray et al. (12), and Lotfata et al. (15) covered random forest (RF), making it the algorithm that showed up the most among the literatures within this category. In two of the literatures, extreme gradient boosting (XGBoost) is used, namely by Tomita et al. (11) and Luo et al. (13), making it the second most commonly used algorithm in this category. Model performance will be discussed in the “Discussion” section.

Literature in the lung application category

Due to the limited literatures specifically on XAI risk assessment for asthma, literatures relating to XAI for risk prediction within the healthcare field were also included. We first look at XAI for lung-related diseases, which may contain similarities in features needed for an asthma XAI risk assessment model, as depicted in Table 2. Seven out of 43 literatures were categorised under the lung application category. Six of these literatures distinctively specified research in explainable or interpretable AI techniques. After an analysis of the literature, it was found that the most used algorithm by the literatures under the lung application category was RF, which appeared five times across several literatures, and was either used as a model or as a comparison for their own model. This was followed closely by the support vector machine (SVM) algorithm with four appearances and logistic regression (LR) and XGBoost algorithm, that was each found to be used in three of the literatures. The performances of the models would be discussed in the discussion section.

Literatures in the other application category

Due to the scarcity of literatures specifically implementing XAI models for the prediction of the risks in BA, we look at studies that utilize predictive modelling to predict the risks in other medical-related problems. This part of the study aims to analyse the models used in other literatures. Table 3 lists the models used by each literature. Thirty-one of the 43 selected literatures were categorised under this category. Twenty-one distinctively specified the employment of XAI techniques under Table 3. Although some of the literatures did not explicitly mention or focus on XAI, the articles still showed up in the systematic literature review search. Either they used interpretable models or have stated or mentioned the use of post hoc XAI methods in their work. The focus is on explainable models that could be suitable for predicting the risks of asthma with precision and accuracy, whether it is predicting the risk or able to classify the patients based on the level of patient care needed. Subsequently, there are two literatures that were related to healthcare, but were studies on how the environment affects a patient. They were categorised under the environment subsection for their use of environmental data, such as daily or hourly measured meteorological and air pollution data for cause-specific mortality, such as in the literature written by Lee et al. (53), and socioeconomic and environmental determinants prevalence in asthma by Lotfata et al. (15).

The most used model within this category is RF with 13 mentions of being used, followed by XGBoost with nine and LR with eight. The most popular post hoc method is SHapley Additive exPlanations (SHAP) with 12 literatures using it. Implementing XAI in risk prediction is crucial for a user’s understanding.

Section 2: architecture

In section 2, architecture, the literatures are separated into two categories, which are ML or DL. The literatures are categorized based on the use of ML or DL techniques within their research, which is then further split into the sub-categories of explainable ML or explainable DL. This is because ML models are relatively known to be more interpretable compared to DL models; however, it still depends on the complexity of the models themselves. The literatures that would be most relevant to this research would be literatures that implement explainability features and have focus on asthma applications. In Tables 4,5 only the citations to the literatures are listed. Information extracted from the literatures are available in Tables 1-3, including the models or algorithms used in each research.

Table 4 lists nine literatures that use ML and 21 literatures with explainable ML. Whereas Table 5 lists seven literatures that use DL and seven literatures with DL and explainability features. Some of the literatures implement both ML and DL algorithms and are categorized into both categories.

Section 3: models

This section categorizes the literatures based on their AI models. There are several types of AI models, such as descriptive models, prescriptive models, classification models, natural language processing (NLP) models, and generative models. The literature was sorted according to how the authors used the models in their literature in Tables 6-8. The models used in the selected literatures were found to be mostly used as predictive models and diagnostic models, such as listed in Tables 6-8, list the literature that was not predictive or diagnostic models. However, the algorithms used may be capable of performing more than one task depending on how it is applied, such as in the case of XGBoost, which can perform classification, regression, or survival analysis tasks. Table 9 is a comparison between related work that shows the similarities or differences of the applied models in the papers. Table 10 compares the performance of the models used and the number of patients in the samples. A table, Table 11, was added to provide more context on the medical tasks performed, such as diagnosis of asthma, forecasting of hospital visits, predicting diseases, and so on, and what algorithm is applied and the algorithm task in each literature. For example, an author may have used an algorithm to diagnose patients, making it a diagnosis model, but the task performed to diagnose the patients may have been a classification task that classifies a patient into categories based on the patients’ parameters. Some literatures were categorised into more than one category because the purpose of the model in the literature was more than only one, for example, both a predictive model and a diagnosis model. The literature most relevant to this research would be the literatures that have application towards asthma, implements explainability, and are predictive models. However, the diagnosis of a disease can also be included as a feature for prediction, and the classification task can be used to classify the level of care a patient may need. The relevance of a predictive model is that in the risk assessment of BA, it would be required to identify patterns and predict future outcomes, such as a patient being at risk of asthma exacerbations or being in the red zone.

Discussion

Key findings

The risk assessment methods used in each paper were compared to each other. The papers are not all on the same asthma-related task; however, our focus is on the algorithm tasks that are carried out by the algorithms for risk assessment, preferably in the BA domain. The focus of the study is on explainable risk assessment models. Due to the scarcity in material, we included literatures in lung and healthcare, with focus on the performance of the risk assessment tasks accomplished by the algorithms and the performance of the explainability models. We analysed and compared the XAI algorithms and the algorithm tasks performed for risk assessment. The algorithm tasks we refer to are tasks, such as classification tasks, regression tasks, or survival analysis tasks, among others, that may be used to assess risk. The connections and similarities were found between them.

Although this study also included literature on ML, DL and XAI methods that were developed for other diseases, such as chronic obstructive pulmonary disease (COPD) (17), diabetes (25), sepsis (23), CVD (26), gestational diabetes (27), kidney injury (28), venous thromboembolism (VTE) (31), malnutrition (33), myocardial infarction (MI) (34), viral infections (35), risk of emergency visits (36), complications after heart surgery (38), mortality prediction (24), and so on, they all involve assessing risk using patient data. The models also rely on similar input data, such as electronic health records (EHR), clinical history and data, medical history and data, laboratory data, patient vital signs, or demographic data to predict risk. Since risk assessment in asthma also uses similar inputs in data, the adaptations of the methods used for application in other diseases is feasible. The similarities in the input data and research goals, such as predicting the risk of the progression of a disease, identifying clinical risk, or predicting worsening condition are all similar to the needs of risk assessment in BA, such as the risks of exacerbation or hospitalization. The use of multidimensional patient data and the approaches used in other applications are all equally relevant in risk assessment of asthma. The cases in each study all centre around prediction of clinical risk and carry out algorithm tasks, such as classification task, forecasting, survival analysis, which are all equally relevant in the risk assessment of BA, such as the risk of exacerbations or hospitalization related to asthma. Out of the 43 selected literatures, 12 literatures were decidedly more relevant than the others due to their similarities in requirements for their research. Five out of the 12 literatures were research in explainable ML or DL in asthma application (11-15), and the remaining seven were in lung application for prediction of lung-related diseases by (16-22). In three of the literatures, XGBoost was compared to other models. These three literatures were the papers by Tomita et al. (11), de Lima et al. (20), and Kor et al. (21), where in the first two papers, XGBoost performed better than the other models compared in the papers, with an accuracy of 81% and area under the curve (AUC) of 85% in the paper by Tomita et al. (11) and an AUC of more than 90% in the paper by de Lima et al. (20), but in the paper by Kor et al. (21), XGBoost had an AUC of 0.7703 and performed less than gradient boosting machine (GBM) and SVM. All three papers used the XGBoost to carry out classification tasks. The results, sample size, and performance of the algorithms can be referred in Table 10. Table 11 lists the medical tasks, algorithm applied, and the algorithm task of each literature. The paper by Tomita et al. (11) used XGBoost classifier to diagnose adult asthma, with a sample size of 566 patients. The paper by de Lima et al. (20) diagnosed respiratory abnormalities in sarcoidosis with a sample size of 72 patients and compared between 10 algorithms as listed in Table 10. The paper by Kor et al. (21) predicted first-time acute exacerbation of COPD (AECOPD) with a sample size of 509 patients. The model that had better performance when compared to the other models in the 12 most relevant literature was XGBoost, with it having better performance 2 out of the 3 times (66.67%) it was compared to other models. Five of the 12 literatures applied XGBoost as their model, making it the most chosen model (41.67%); however, two of the literatures by Zeng et al. (17) and Luo et al. (13), by the same group of researchers, conducted research on their own automatic explanation method for ML models by using XGBoost as their model. The other literatures in this study have adopted models that have a possibility of being directly interpretable on its own, or have adopted post-hoc techniques, such as SHAP (14 literatures), local interpretable model-agnostic explanation (LIME) (three literatures), and gradient-weighted class activation mapping (Grad-CAM) (four literatures).

Strengths and limitations

Strengths

This study extends the dialogue in XAI for BA and has identified the explainable ML or DL techniques that researchers have included in their recent work. This includes ante-hoc or post-hoc explainable models, the most used model among them, the most commonly used post-hoc model, and the model that performs better than other models in studies that include comparison. This work has identified that XGBoost may be worth looking into as a model for BA risk prediction.

Limitations

There are several limitations confining this study. The first limitation is that the literatures included in this study was limited to five databases, namely Science Direct, ACM Digital Library, Springer, PubMed, and Scopus. Any possible relevant research outside of these five databases could have been excluded. Second, the search criteria for literatures may have omitted any literatures that did not explicitly mention the use of XAI, risk assessment, risk assessment model, asthma, and health. Other than that, the inclusion of literatures written only using the English language may have excluded relevant literatures written in other languages, resulting in language bias. Furthermore, some literatures had unattainable full-text and were consequently excluded from the review.

Moreover, there was a disproportion between literatures with explainable ML algorithms and explainable DL algorithms with ML XAI being a majority. This puts the review in focus on ML XAI while under-representing DL XAI.

In this review, we did not focus on the feature extraction or selection for an XAI model but on the model itself. This review also insufficiently explores the relation between XAI and asthma, with asthma being a global issue, due to the scarcity of papers written for that exact topic.

Comparison with similar researches

Comparison between similar researches is presented in Tables 9,10.

Explanations of findings

A comparison was done between the 12 most relevant literature, determined by their application in asthma and lungs in Tables 9,10. Table 9 was a comparison between what was done in each study, and Table 10 compared the performances of the models with the dataset used, number of patients, and type of patients. It is worth noting that only studies that explicitly mentioned research XAI in their studies were given a tick, although all the studies are considered using explainable models. Table 11 lists the medical tasks, algorithm applied, and algorithm task of each literature to give a clearer definition of the medical tasks, the algorithms used to solve the medical tasks, and the algorithm task of each algorithm. For example, the classification task of the XGBoost algorithm was used to solve the medical task of diagnosing adult asthma (11).

It was found that XGBoost was the most used model within the 12 literatures and the better performing model, with it chosen to be included in the studies five out 12 times and having a better performance two out of the three times it was compared to other models. There were some studies that did comparisons between their own models with other models, but did not include XGBoost as one of those models, such as in the work by AlSaad et al. (14), Luo et al. (16), and Hernández-Pereira et al. (22). Therefore, it is difficult to deduce whether XGBoost would have performed better than the selected models. It is also worth noting that the size or generalizability of each dataset used may contribute to the performance in the outputs of the model as shown in Table 10. Moving on, although being frequently used may not be an indicator of it being the better model, but it may be the desirable choice due to it being a tree-based ensemble method that, if not too complex, may be relatively interpretable. Next, the XGBoost model has built-in feature importance metrics that can be used to identify the most important features and gain insight on how the features may affect the patterns of the predictions.

Implications and actions needed

Research in XAI for risk prediction is seeing rapid development. However, there is still a need for more literatures on the use of XAI for predicting risks in asthma. Successful integration of XAI for clinical use in risk prediction of asthma will require clinical confirmation and research.

Conclusions

In conclusion, this review has analysed the XAI models used for asthma, lung, and healthcare. It has presented an overview on XAI models currently being used for risk assessment within the healthcare field. XAI for risk assessment in BA that is accurate, trustworthy, and interpretable by the users may soon contribute to the quality of life for asthma patients.

Acknowledgments

The authors declare that during the writing process of the work, AI-assisted technology ChatGPT and Monica were used only to improve clarity, enhance the overall quality of the writing, and were solely used as a tool to optimize the writing in the work. The authors edited and reviewed the work as needed.

Footnote

Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-204/rc

Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-204/prf

Funding: This research was supported by Universiti Malaysia Sarawak under the Vice Chancellor High Impact Grant (UNI/F08/VC-HIRG/85481/P01-03/2022).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-204/coif). All authors report a grant from Universiti Malaysia Sarawak outside the submitted work (UNI/F08/VC-HIRG/85481/P01-03/2022). The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Global Initiative for Asthma (GINA). 2024 GINA Main Report. 2024 [cited 2024 Jun 25]. Available online: https://ginasthma.org/2024-report/
2. National Heart, Lung, and Blood Institute. What Is Asthma? 2021. Available online: https://www.nhlbi.nih.gov/health/asthma
3. Chaddad A, Peng J, Xu J, et al. Survey of Explainable AI Techniques in Healthcare. Sensors (Basel) 2023; [Crossref]
4. Exarchos KP, Beltsiou M, Votti CA, et al. Artificial intelligence techniques in asthma: a systematic review and critical appraisal of the existing literature. Eur Respir J 2020;56:2000521. [Crossref] [PubMed]
5. Antoniadi AM, Du Y, Guendouz Y, et al. Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review. Applied Sciences 2021;11:5088.
6. Yu X, Watson M. Guidance on conducting a systematic literature review. Journal of Planning Education and Research 2017;39:93-112.
7. Kitchenham B, Charters SM. Guidelines for performing systematic literature reviews in Software Engineering. EBSE Technical Report. 2007. Available online: https://www.researchgate.net/publication/302924724_Guidelines_for_performing_Systematic_Literature_Reviews_in_Software_Engineering_Guidelines_for_performing_Systematic_Literature_Reviews_in_Software_Engineering
8. Sheu RK, Pardeshi MS. A Survey on Medical Explainable AI (XAI): Recent Progress, Explainability Approach, Human Interaction and Scoring System. Sensors (Basel) 2022; [Crossref]
9. Gates S. Review of methodology of quantitative reviews using meta-analysis in ecology. Journal of Animal Ecology 2002;71:547-57.
10. Gomersall JS, Jadotte YT, Xue Y, et al. Conducting systematic reviews of economic evaluations. Int J Evid Based Healthc 2015;13:170-8. [Crossref] [PubMed]
11. Tomita K, Yamasaki A, Katou R, et al. Construction of a Diagnostic Algorithm for Diagnosis of Adult Asthma Using Machine Learning with Random Forest and XGBoost. Diagnostics (Basel) 2023; [Crossref]
12. Ray A, Das J, Wenzel SE. Determining asthma endotypes and outcomes: Complementing existing clinical practice with modern machine learning. Cell Rep Med 2022;3:100857. [Crossref] [PubMed]
13. Luo G, Nau CL, Crawford WW, et al. Generalizability of an Automatic Explanation Method for Machine Learning Prediction Results on Asthma-Related Hospital Visits in Patients With Asthma: Quantitative Analysis. J Med Internet Res 2021;23:e24153. [Crossref] [PubMed]
14. AlSaad R, Malluhi Q, Janahi I, et al. Interpreting patient-Specific risk prediction using contextual decomposition of BiLSTMs: application to children with asthma. BMC Med Inform Decis Mak 2019;19:214. [Crossref] [PubMed]
15. Lotfata A, Moosazadeh M, Helbich M, et al. Socioeconomic and environmental determinants of asthma prevalence: a cross-sectional study at the U.S. County level using geographically weighted random forests. Int J Health Geogr 2023;22:18. [Crossref] [PubMed]
16. Luo X, Gandhi P, Zhang Z, et al. Applying interpretable deep learning models to identify chronic cough patients using EHR data. Comput Methods Programs Biomed 2021;210:106395. [Crossref] [PubMed]
17. Zeng S, Arjomandi M, Luo G. Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study. JMIR Med Inform 2022;10:e33043. [Crossref] [PubMed]
18. Islam MK, Rahman M, Ali MS, et al. Enhancing lung abnormalities detection and classification using a Deep Convolutional Neural Network and GRU with explainable AI: A promising approach for accurate diagnosis. Machine Learning With Applications 2023;14:100492.
19. Casiraghi E, Malchiodi D, Trucco G, et al. Explainable Machine Learning for Early Assessment of COVID-19 Risk Prediction in Emergency Departments. IEEE Access 2020;8:196299-325.
20. de Lima AD, Lopes AJ, do Amaral JLM, et al. Explainable machine learning methods and respiratory oscillometry for the diagnosis of respiratory abnormalities in sarcoidosis. BMC Med Inform Decis Mak 2022;22:274. [Crossref] [PubMed]
21. Kor CT, Li YR, Lin PR, et al. Explainable Machine Learning Model for Predicting First-Time Acute Exacerbation in Patients with Chronic Obstructive Pulmonary Disease. J Pers Med 2022; [Crossref]
22. Hernández-Pereira E, Fontenla-Romero O, Bolón-Canedo V, et al. Machine learning techniques to predict different levels of hospital care of CoVid-19. Appl Intell (Dordr) 2022;52:6413-31. [Crossref] [PubMed]
23. Nesaragi N, Patidar S, Veerakumar T. A correlation matrix-based tensor decomposition method for early prediction of sepsis from clinical data. Biocybernetics and Biomedical Engineering 2021;41:1013-24.
24. Kim SY, Kim S, Cho J, et al. A deep learning model for real-time mortality prediction in critically ill children. Crit Care 2019;23:279. [Crossref] [PubMed]
25. Wang YC, Chen TCT, Chiu MC. A systematic approach to enhance the explainability of artificial intelligence in healthcare with application to diagnosis of diabetes. Healthcare Analytics 2023;3:100183.
26. Ordikhani M, Saniee Abadeh M, Prugger C, et al. An evolutionary machine learning algorithm for cardiovascular disease risk prediction. PLoS One 2022;17:e0271723. [Crossref] [PubMed]
27. Du Y, Rafferty AR, McAuliffe FM, et al. An explainable machine learning-based clinical decision support system for prediction of gestational diabetes mellitus. Sci Rep 2022;12:1170. [Crossref] [PubMed]
28. Zhang Y, Yang D, Liu Z, et al. An explainable supervised machine learning predictor of acute kidney injury after adult deceased donor liver transplantation. J Transl Med 2021;19:321. [Crossref] [PubMed]
29. Roseiro M, Henriques J, Paredes S, et al. An interpretable machine learning approach to estimate the influence of inflammation biomarkers on cardiovascular risk assessment. Comput Methods Programs Biomed 2023;230:107347. [Crossref] [PubMed]
30. Zadeh A, Broach C, Nosoudi N, et al. Building analytical models for predicting de novo malignancy in pancreas transplant patients: A machine learning approach. Expert Systems with Applications 2024;237:121584.
31. Qatawneh Z, Alshraideh M, Almasri N, et al. Clinical decision support system for venous thromboembolism risk classification. Applied Computing and Informatics 2019;15:12-8.
32. Alzoubi H, Alzubi R, Ramzan N. Deep Learning Framework for Complex Disease Risk Prediction Using Genomic Variations. Sensors (Basel) 2023; [Crossref]
33. Yalçın N, Kaşıkcı M, Çelik HT, et al. Development and validation of machine learning-based clinical decision support tool for identifying malnutrition in NICU patients. Sci Rep 2023;13:5227. [Crossref] [PubMed]
34. Singh A, Miller RJH, Otaki Y, et al. Direct Risk Assessment From Myocardial Perfusion Imaging Using Explainable Deep Learning. JACC Cardiovasc Imaging 2023;16:209-20. [Crossref] [PubMed]
35. Choubey S, Barde S, Badholia A. Enhancing the prediction efficiency of virus borne diseases using enhanced backpropagation with an artificial neural network. Measurement: Sensors 2022;24:100505.
36. George R, Ellis B, West A, et al. Ensuring fair, safe, and interpretable artificial intelligence-based prediction tools in a real-world oncological setting. Commun Med (Lond) 2023;3:88. [Crossref] [PubMed]
37. Petrauskas V, Jasinevicius R, Damuleviciene G, et al. Explainable Artificial Intelligence-Based Decision Support System for assessing the Nutrition-Related Geriatric Syndromes. Applied Sciences 2021;11:11763.
38. Zeng X, Hu Y, Shu L, et al. Explainable machine-learning predictions for complications after pediatric congenital heart surgery. Sci Rep 2021;11:17244. [Crossref] [PubMed]
39. Giuste FO, Sequeira R, Keerthipati V, et al. Explainable synthetic image generation to improve risk assessment of rare pediatric heart transplant rejection. J Biomed Inform 2023;139:104303. [Crossref] [PubMed]
40. Islam MS, Awal MA, Laboni JN, et al. HGSORF: Henry Gas Solubility Optimization-based Random Forest for C-Section prediction and XAI-based cause analysis. Comput Biol Med 2022;147:105671. [Crossref] [PubMed]
41. Liu X, Hu P, Yeung W, et al. Illness severity assessment of older adults in critical illness using machine learning (ELDER-ICU): an international multicentre study with subgroup bias evaluation. Lancet Digit Health 2023;5:e657-67. [Crossref] [PubMed]
42. Wang K, Tian J, Zheng C, et al. Interpretable prediction of 3-year all-cause mortality in patients with heart failure caused by coronary heart disease based on machine learning and SHAP. Comput Biol Med 2021;137:104813. [Crossref] [PubMed]
43. Tu KC, Tau ENT, Chen NC, et al. Machine Learning Algorithm Predicts Mortality Risk in Intensive Care Unit for Patients with Traumatic Brain Injury. Diagnostics (Basel) 2023; [Crossref]
44. Kavya R, Christopher J, Panda S, et al. Machine Learning and XAI approaches for Allergy Diagnosis. Biomedical Signal Processing and Control 2021;69:102681.
45. Garcia-Canadilla P, Isabel-Roquero A, Aurensanz-Clemente E, et al. Machine Learning-Based Systems for the Anticipation of Adverse Events After Pediatric Cardiac Surgery. Front Pediatr 2022;10:930913. [Crossref] [PubMed]
46. Shin H, Noh G, Choi BM. Photoplethysmogram based vascular aging assessment using the deep convolutional neural network. Sci Rep 2022;12:11377. [Crossref] [PubMed]
47. Aghamohammadi M, Madan M, Hong JK, et al. Predicting heart attack through explainable artificial intelligence. In: International Conference on Computational Science. Cham: Springer International Publishing; 2019:633-45.
48. Tseng PY, Chen YT, Wang CH, et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Crit Care 2020;24:478. [Crossref] [PubMed]
49. Li W, Song Y, Chen K, et al. Predictive model and risk analysis for diabetic retinopathy using machine learning: a retrospective cohort study in China. BMJ Open 2021;11:e050989. [Crossref] [PubMed]
50. Wang X, Ezeana CF, Wang L, et al. Risk factors and machine learning model for predicting hospitalization outcomes in geriatric patients with dementia. Alzheimers Dement (N Y) 2022;8:e12351. [Crossref] [PubMed]
51. Kovalchuk SV, Kopanitsa GD, Derevitskii IV, et al. Three-stage intelligent support of clinical decision making for higher trust, validity, and explainability. J Biomed Inform 2022;127:104013. [Crossref] [PubMed]
52. Twick I, Zahavi G, Benvenisti H, et al. Towards interpretable, medically grounded, EMR-based risk prediction models. Sci Rep 2022;12:9990. [Crossref] [PubMed]
53. Lee W, Lim YH, Ha E, et al. Forecasting of non-accidental, cardiovascular, and respiratory mortality with environmental exposures adopting machine learning approaches. Environ Sci Pollut Res Int 2022;29:88318-29. [Crossref] [PubMed]