User engagement and functionality in chatbot-assisted intervention for adolescents and young adults with mental disorders: a narrative review
Introduction
Mental health disorders are highly prevalent worldwide, affecting individuals across all age groups. Notably, the prevalence of mental health disorders among children and adolescents is approximately 15% globally, highlighting a significant public health concern (1). With evidence indicating that 50% of mental health disorders manifest by age 14 and 75% by age 24 years, the early onset of these conditions highlights the critical need for effective intervention during childhood and adolescence (2). Patients with mental disorders also experienced diminished health-related quality of life (HRQoL) and substantial economic burden (3,4). Among children and adolescents, mental illness is associated with significant disease burden and long-term consequences, including developmental delays, poor academic performance, and an increased risk of suicidal ideation (5). The increasing incidence rates of mental illness among children and adolescents over the past decade have heightened awareness of this critical public health issue (6). Despite this, only a limited number of children and adolescents utilized mental health services or seek professional support (7). This gap can be attributed to various factors identified in previous literature, including stigma, lack of health insurance, limited access to care, and hesitancy toward medication (7). Consequently, there is an increasing need for accessible interventions tailored to children and adolescents to improve mental health care.
In recent years, the widespread application of artificial intelligence (AI) in healthcare research has yielded encouraging outcomes across various aspects and areas. One prominent example of AI implementation is the chatbot, which has become a widely used form of Human-Computer Interaction (8). A chatbot is a computer program designed to communicate with humans via text or voice (9). Chatbots offer numerous benefits across diverse fields, including automating responses and processes, thereby enhancing consistency, availability, accessibility, and cost-efficiency in user experiences. These advantages make chatbots a valuable tool in supporting mental health care, particularly for children and adolescents who may face barriers to traditional services. Accordingly, AI-driven chatbots could play a crucial role in enabling early detection, personalized treatment plans, and continuous monitoring of mental health conditions in young populations. Previous reviews show the effectiveness of chatbots in improving mental health and promoting healthy behavioral changes (10,11). However, while previous studies have documented the effectiveness of chatbots in improving individuals’ mental health in general population, no studies specifically focused on adolescents and young adults when evaluating the use of mental health chatbots. There remains a knowledge gap, as the psychological and social needs of younger individuals may differ from those of adults.
Therefore, the objective of this study is to review and summarize the available studies that implemented the chatbot as a mental health intervention for adolescents and young adults who experienced mental illness. The review aims to identify current trends and gaps in this field, and suggest future research in digital mental health solutions to better support younger populations. We present this article in accordance with the Narrative Review reporting checklist (available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-188/rc).
Methods
We performed a literature search in PubMed, MEDLINE, Web of Science, Scopus, Embase, IEEE Xplore, and ACM Digital Library for articles on user engagement with chatbots for adolescents with mental disorders (search strategy shown in Table S1) during January 01, 2010–August 23, 2023. Given the rapid advancement of chatbots in recent years, this review is limited to studies published in 2010 or later. The article lists were generated by searching relevant keywords, including mental health, chatbot, user engagement, and study population in the databases. We only included English-published articles in this review.
The selection process included two parts: title and abstract screening and full-text review. For title and abstract screening, two reviewers (T.W. and J. Li) focused on the relevance of the article, and articles that lacked any aspects of the topic were excluded (for example, some articles discussed the usage of chatbots for individuals’ mental healthcare, but they focused on adults aged 18 years or older). During the full-text review, the eligible article was selected for our narrative review only if: (I) the study’s targeted populations were adolescents and young adults (between ages 10 and 20 years) with any type of mental disorders; (II) the study involved the implementation and use of chatbots; and (III) outcomes of the study evaluated users’ engagement with the chatbot. The summary of search strategy is shown in Table 1.
Table 1
| Item | Specification |
|---|---|
| Date of search | August 25, 2023 |
| Databases and other sources searched | PubMed, MEDLINE, Web of Science, Scopus, Embase, IEEE Xplore, and ACM Digital Library |
| Search terms used | Free text search term: mental illness, mental health, mental disorder, psychiatry, psychological wellbeing, mental wellbeing, depression, depressive, depress, mood disorder, mood, affective disorder, anxiety, anxious, panic disorder, phobia, bipolar, psychosis, schizophrenia, conversational agent, conversational AI, intelligent assistant, intelligent agent, chatbot, social bot, digital assistant, conversational UI, conversational interface, conversation system, dialog agent, AI agent, usability, user experience, user evaluation, adoption, qualitative, user perspective, barrier, interview, focus group, adhere, maintain, retention, sustain, satisfaction, user preference, user acceptance, reliable, involvement, user interface, UI, feasible, adolescents, young, child, youth, teenager, pediatric, juvenile |
| Time frame | January 1, 2010–July 31, 2023 |
| Inclusion criteria | Inclusion: (I) the study’s targeted populations were adolescents and young adults (between ages 10 and 20) with any type of mental disorders; (II) English-published manuscripts; (III) the study involved the implementation and use of chatbots; (IV) outcomes of the study evaluated users’ engagement with the chatbot |
| Selection process | Two researchers (T.W. and J. Li) independently identified the relevant published studies |
Findings
Based on the literature search across all databases, 1,263 manuscripts were imported into Covidence. After removing the duplicates, 1,055 manuscripts were left for title and abstract screening. Among 43 manuscripts that were eligible for full-text review, 7 studies were selected for this study (Figure 1). Among seven eligible studies, four studies reported the mean age of their participants (range, 14.7 to 22.2 years) (12-15) (Table 2). Most of the studies reported the distribution of sex (n=6, percentage of male participants: 12.0–63.3%), and two studies reported race and ethnicity (13,15). The study sample size varied across the seven studies, which ranged from 13 to 70 participants. Three studies were conducted in the United States (42.9%), followed by Australia (2 studies), China (1 study), and New Zealand (1 study). Depression was the most prevalent mental health issue reported in all included studies (71.4%). Other mental health conditions, including anxiety and mood disorders, were also frequently reported in the study population.
Table 2
| Study | Country | Sample size | Type of mental health concerns | Age group of participants (years), range | Age (years), mean (SD) | Sex (female) |
|---|---|---|---|---|---|---|
| Dosovitsky and Bunge, 2022 (12) | United States | 23 | Depression | 13–18 | 14.96 (1.49) | 45.5% |
| Elmasri and Maeder, 2016 (16) | Australia | 17 | Alcohol abuse | 18–25 | NR | 41.2% |
| Fitzpatrick et al., 2017 (13) | United States | 70 | Anxiety and depression | 18–28 | 22.2 (2.33) | 67% |
| He et al., 2022 (14) | China | 49 | Depression | 17–34 | 18.80 (0.89) | 36.7% |
| Nicol et al., 2022 (15) | United States | 17 | Moderate depressive symptoms | 13–17 | 14.7 (1.7) | 88% |
| Williams et al., 2021 (17) | New Zealand | 64 | Self-identified “stressed” | 18–24 | NR | NR |
| Wrightson-Hester et al., 2023 (18) | Australia | 13 | Symptoms of anxiety, depression, or low mood | 16–24 | NR | 53.85% |
NR, not reported in the study; SD, standard deviation.
User experiences
To evaluate user experiences using chatbots, the eligible studies applied both quantitative approaches (assessments, Likert scale, and questionnaires) and qualitative assessments (open-ended questions, interviews, and focus groups) to collect participants’ responses (Table 3). Various existing questionnaires, surveys, and measures were used to assess different aspects of user experiences:
- Satisfaction: the 8-item Client Satisfaction Survey (16);
- Acceptability: the Acceptability Scale (14), the 4-item Acceptability of Intervention Measure (15);
- Feasibility: the 4-item Feasibility of Intervention Measure (15);
- Usability: the Usability Metric for User Experience-LITE (14), The 10-item System Usability Scale (15,18);
- Engagement: the 3-item User Engagement Survey (18).
Table 3
| Authors, year of publication | Type of chatbot | Description of chatbot | User experience | Suggestions or issues on chatbot’s functionality | Mental health-related outcomes |
|---|---|---|---|---|---|
| Dosovitsky and Bunge, 2022 (12) | BethBot | Text-based chat bot delivered via Facebook Messenger | Chatbot’s utility: positive attitude: 54.5%; negative attitude: 31.8%; neutral/ambiguous: 13.6% | Technical issues: 66.0%; personalization: 60.0%; content issues: 53.0% | 61.1% believe it could help people improve their mental health symptoms |
| Elmasri and Maeder, 2016 (16) | Prototype chatbot | Implemented by AIML as for ALICE | Positive: amount of knowledge provided (71.4%); ease of conversation (71.4%); quick response time (38.1%); clear to comprehend (19.0%); ease of use (19.0%) | Recognising more keywords: 6/21; voice recognition: 1/21 | NR |
| Negative: amount of knowledge provided (23.8%); ease of conversation (14.3%); undesirable interface (23.8%); incorrect/inappropriate response (4.8%) | |||||
| Fitzpatrick et al., 2017 (13) | Woebot | Text-based conversational agent | Level of satisfaction: chatbot group (4.3); information control group (3.4) | Process violations (n=15): not being able to converse naturally (n=10), repetitive (n=2), miscl (n=1) | Significantly reduced their symptoms of depression over the study period by the PHQ-9 |
| Emotional awareness: chatbot group (4.0); information control group (3.4) | Technical problems (n=8): glitches (n=4), looping (n=4) | ||||
| Learned something new: chatbot group (100%); information control group (77%) | Content (n=8): emoticons (n=2), interaction too short (n=2), videos too long (n=2) | ||||
| He et al., 2022 (14) | XiaoE | Mental health chatbot | The chatbot group has higher scores on: working alliance questionnaire (3.407); acceptability scale (4.322); content satisfaction (5.093); emotional awareness (3.636); learning new knowledge (4.330); relevance to daily life (4.834) | Content (n=120): “inflexible, “response, “tedious, “repetitive, and “mechanical; technology (n=28): “glitches, “lag, “system, “crash, and “inflexible. Suggestions: hope for a more fluent process of dialogue, more emotional response and interaction, and server upgrade | Depressive symptoms significantly reduced more among participants in the XiaoE group |
| Nicol et al., 2022 (15) | Woebot | Fully automated and relational conversational agent | Mean scores: acceptability [16.6 (1–20)]; feasibility [17 (range, 1–20)]; usability [21.4 (range, 5–25)] | Potential issues of the chatbot: efficacy of the app, the potential safety risks of missing reports of suicidal ideation, overreliance on technology and reduced self-efficacy for adaptive help-seeking, increased screen time, and privacy and confidentiality of sensitive information (note: from provider participants, instead of targeted users) | Mean PHQ-9 scores at 4 weeks decreased by 3.3 points in the Woebot group |
| Williams et al., 2021 (17) | 21 Day Stress Detox Chatbot | Designed to provide content in daily instalments to teach and reinforce coping strategies for dealing with stress and anxiety | Rate chatbot as “Okay or Great”: over 90% participants | Irrelevant content, inappropriate language used; not acting like a real person, doesn’t listen to the user/patronizing; time consuming; same questions/responses | Improvement in PSS-10 and WHO-5 |
| Wrightson-Hester et al., 2023 (18) | MYLO | An artificially intelligent chatbot that emulates the method of levels therapy | System usability scale: during-testing survey (71.59); post-testing survey (75.75); most participants expressed satisfaction with the chatbot | Some participants (3/8, 38%) felt that the chatbot had difficulties understanding them because of how they were typing (resolve the problem by adjusting the language); some participants (4/8, 50%) also found it difficult to explain their feelings; some participants (4/8, 50%) had trouble understanding some of MYLO’s questions, so they struggled to answer them; the repetition of questions or the use of very similar questions that made participants feel they were repeating themselves (no stats) | No significant difference was observed |
MYLO, Manage Your Life Online; NR, not reported in the study; PHQ-9, Patient Health Questionnaire; PSS-10, Perceived Stress Scale; WHO-5, The World Health Organization-Five Well-Being Index.
Three studies (42.9%) asked open-ended questions to collect free-text responses and comments from participants. Meanwhile, three studies conducted interviews after the survey, and among those three studies, Wrightson-Hester et al. also involved focus groups to collect more detailed feedback from participants (15,16,18). Given only less than half of the reviewed studies included in-depth qualitative insights into user perceptions (interviews and/or focus groups) while others were mainly relying on survey results, not all studies were equally robust. Comparing to interviews, surveys may not capture the nuances of individual experiences and perspectives, which were important factors in this context (evaluating young adults’ personal experiences on using mental health chatbots).
Overall, adolescents and young adults expressed positive attitudes toward the usage of mental health chatbots. Studies showed that over half of the participants provided positive feedback (12), found it acceptable (15), or had a satisfactory experience with the chatbot after using it. Especially when compared with other approaches such as general chatbots (without specification on mental healthcare) or e-books, participants using mental health chatbots had higher levels of satisfaction, greater amount of emotional awareness, and learned more new knowledge than the control group (13,14). In addition, it is possible that users’ satisfaction levels could increase as they continue using mental health chatbots. Wrightson-Hester et al. found that participants provided higher ratings on the usability of the chatbot during the post-testing survey than during the test-testing survey (18).
Pros and cons of using chatbots
Overall, 6 out of 7 studies explicitly addressed positive results on using chatbots to help adolescents and young adults with mental disorders (12-14,16-18). Two studies found that chatbots showed reliability and accountability (e.g., daily check-in feature) as a mental health intervention tool (13,16). Compared with traditional methods, chatbots are convenient and easy to use (14,17). It also helps individuals to explore their problems by asking novel questions (18) and using simple language (16). The human-like conversations provided by chatbots also received positive feedback from the study participants (12-14,16). Chatbots not only have user-friendly interaction (13,16) but are also able to express engaging interaction and provide supportive care when the user is experiencing loneliness or depression (13,14). In addition, participants also pointed out that features implemented on the chatbots could be helpful, such as videos (13), games (13), weekly graphs (13), and aesthetic appeal (17).
Although the chatbots impressed participants in many aspects, there were also concerns regarding the functions and features provided by chatbots. When applying chatbots to assist adolescents and young adults with mental disorders, two major issues were identified in the previous studies: technical issues and content issues.
Technical issues: 6 out of 7 (85.7%) studies mentioned that participants experienced technical issues when having conversations with the chatbots, such as receiving no response because the chatbot didn’t understand the questions or recognize the keywords (12-14,16,18), glitches (13,14), time-consuming because of lags (14,17), looping conversations (13), and system crash (14).
Content issues: 5 out of 7 (71.4%) studies mentioned that participants were questioning the content provided by chatbots when having the conversations. The content-wise issues include irrelevant content (12,13,17), repetitive responses to different questions (13,14,17,18), inappropriate language used in the responses (15), and unable to act like a real person with emotional responses (12-14,17). Because of those issues, especially the lack of personalized care, Wrightson-Hester et al. mentioned that participants found it difficult to express their feelings to the chatbots (18). However, a few participants from the same study indicated that the lack of human factors improved their experiences since it made them feel less judged (18).
However, only one study explicitly pointed out the issue of privacy and confidentiality of sensitive information (e.g., mental health status) shared when communicating with the chatbots (15). Considering these chatbots were dealing with sensitive mental health and other types of personal information, concerns around data privacy, confidentiality, and safeguards may be underexplored.
Benefits of mental health chatbots
Although studies considered different measurements when reporting the effectiveness of chatbots, improvement of mental health symptoms among participants was commonly observed in most of the studies. Out of the four eligible studies that evaluated and reported the post-intervention mental health condition, 100% of the studies found that some level of involvement of mental health chatbots could improve some of the participants’ mental health symptoms (13-15,17). Examined on the post-intervention Patient Health Questionnaire (PHQ-9) metrics, three studies identify that participant significantly reduced their mental illness symptoms (100%) (13-15). Williams et al. reported that both patients who experienced anxiety and those without anxiety diagnosis during the pre-intervention period experienced lower General Anxiety Disorder (GAD-7) symptoms (17). Meanwhile, participants who were clinically diagnosed with anxiety reported significantly lower Perceived Stress Scale (PSS-10) and The World Health Organization-Five Well-Being Index (WHO-5) compared to their counterparts.
Limitations in functionality
Several limitations in study design also need to be considered when interpreting the study findings. Firstly, the small sample size observed and reported in studies might impact the generalizability and limit the interpretation of the study findings in larger and more diverse populations, such as people living in rural areas with limited access to mental health care. The small sample size might further impact the power of the statistical power of the analysis when examining the clinical significance of the mental health chatbots. Also, the length of the follow-up period (range between 1 to 12 weeks) was relatively short in current studies, so future studies are needed to evaluate the long-term outcomes to provide a comprehensive review of the effectiveness of mental health chatbots.
All included studies reported obtaining ethical approval from institutional review boards or committees and described informed consent procedures. No study reported ethical concerns specific to the chatbot format. Meanwhile, this review did not assess the safety protocols implemented in the interventions, which represents an important area for future research to evaluate and minimize potential risks associated with chatbot-based interventions.
Conclusions
To the best of our knowledge, this is the first study that reviewed the application of chatbot-assisted intervention for adolescents and young adults with mental diseases. Findings from this review showed satisfactory user engagement and encouraged symptom improvement with chatbot intervention. Considering the increasing prevalence of mental health challenges among young people globally, chatbot-assisted intervention might be considered as a potential approach to improve patient care and eventually achieve optimal mental health goals.
Acknowledgments
None.
Footnote
Reporting Checklist: The authors have completed the Narrative Review reporting checklist. Available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-188/rc
Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-188/prf
Funding: None.
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-24-188/coif). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Polanczyk GV, Salum GA, Sugaya LS, et al. Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. J Child Psychol Psychiatry 2015;56:345-65. [Crossref] [PubMed]
- Kessler RC, Berglund P, Demler O, et al. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005;62:593-602. [Crossref] [PubMed]
- Arias D, Saxena S, Verguet S. Quantifying the global burden of mental disorders and their economic value. EClinicalMedicine 2022;54:101675. [Crossref] [PubMed]
- Defar S, Abraham Y, Reta Y, et al. Health related quality of life among people with mental illness: The role of socio-clinical characteristics and level of functional disability. Front Public Health 2023;11:1134032. [Crossref] [PubMed]
- WHO. Mental health of adolescents. 2021. Available online: https://www.who.int/news-room/fact-sheets/detail/adolescent-mental-health. Accessed March 1 2024.
- Bitsko RH, Claussen AH, Lichstein J, et al. Mental Health Surveillance Among Children - United States, 2013-2019. MMWR Suppl 2022;71:1-42. [Crossref] [PubMed]
- Radez J, Reardon T, Creswell C, et al. Why do children and adolescents (not) seek and access professional help for their mental health problems? A systematic review of quantitative and qualitative studies. Eur Child Adolesc Psychiatry 2021;30:183-211. [Crossref] [PubMed]
- Adamopoulou E, Moussiades L. An overview of chatbot technology. In: Maglogiannis I, Iliadis L, Pimenidis, E. (eds) Artificial Intelligence Applications and Innovations. AIAI 2020. IFIP Advances in Information and Communication Technology. Springer; 2020.
- Dahiya M. A tool of conversation: Chatbot. International Journal of Computer Sciences and Engineering 2017;5:158-61.
- Abd-Alrazaq AA, Rababeh A, Alajlani M, et al. Effectiveness and Safety of Using Chatbots to Improve Mental Health: Systematic Review and Meta-Analysis. J Med Internet Res 2020;22:e16021. [Crossref] [PubMed]
- Aggarwal A, Tam CC, Wu D, et al. Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review. J Med Internet Res 2023;25:e40789. [Crossref] [PubMed]
- Dosovitsky G, Bunge E. Development of a chatbot for depression: adolescent perceptions and recommendations. Child Adolesc Ment Health 2023;28:124-7. [Crossref] [PubMed]
- Fitzpatrick KK, Darcy A, Vierhile M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Ment Health 2017;4:e19. [Crossref] [PubMed]
- He Y, Yang L, Zhu X, et al. Mental Health Chatbot for Young Adults With Depressive Symptoms During the COVID-19 Pandemic: Single-Blind, Three-Arm Randomized Controlled Trial. J Med Internet Res 2022;24:e40719. [Crossref] [PubMed]
- Nicol G, Wang R, Graham S, et al. Chatbot-Delivered Cognitive Behavioral Therapy in Adolescents With Depression and Anxiety During the COVID-19 Pandemic: Feasibility and Acceptability Study. JMIR Form Res 2022;6:e40242. [Crossref] [PubMed]
- Elmasri D, Maeder A. A conversational agent for an online mental health intervention. In: Ascoli G, Hawrylycz M, Ali H, et al. (eds) Brain Informatics and Health. BIH 2016. Lecture Notes in Computer Science. Springer; 2016.
- Williams R, Hopkins S, Frampton C, et al. 21-day stress detox: open trial of a universal well-being chatbot for young adults. Soc Sci 2021;10:416.
- Wrightson-Hester AR, Anderson G, Dunstan J, et al. An Artificial Therapist (Manage Your Life Online) to Support the Mental Health of Youth: Co-Design and Case Series. JMIR Hum Factors 2023;10:e46849. [Crossref] [PubMed]
Cite this article as: Wang T, Li J, Liu J, Liu G. User engagement and functionality in chatbot-assisted intervention for adolescents and young adults with mental disorders: a narrative review. J Med Artif Intell 2026;9:6.
