Application of artificial intelligence for the assessment of mucosal healing and inflammation

Artificial intelligence (AI), also referred to as machine intelligence, has been increasingly entering all avenues of our lives (1-5). AI has enabled facial, object, speech, gesture and writing recognition, language translation, autonomous cars, internet searches, cyber and home security and many other areas. It has revolutionized diverse aspects of medical care, including electronic health records, guidance in medical diagnosis and treatment decisions, medical statistics, analysis of X-rays, CT-scans, MRIs, electrocardiograms (EKGs), evaluation of endoscopic and histologic images, robotics, and cellular and molecular biology including arrays and genome-, proteome- and metabolome- “omics”.

AI has been defined as “a system’s ability to correctly interpret external data, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation” (5). In this capacity, a machine mimics “cognitive” functions of humans including “learning” and “problem-solving” (2). AI has been classified as analytical, human-inspired, and humanized AI (5). Analytical AI includes cognitive intelligence—depiction of the world and uses learning from the past experience to generate decisions. Computer vision is a component of AI that enables computers to identify, process and interpret a variety of visual data and applies deep learning and pattern recognition and identification to interpret the content of an image. An image and pattern recognition system analyzes an image as input and outputs and can only understand objects details and classes it has learned. The pattern recognition allows on the automatic discovery of regularities in data through the use of computer algorithms and classifying the data into different categories (2). AI has been employed in logistics, data mining, medical diagnosis and other areas (2,6). AI’s tremendous growth was facilitated and enabled by the development of powerful computers and massive data storage capacity.

In response to the invitation by the editors to contribute an editorial and comment on the recent article by Maeda et al. on the identification of ulcerative colitis (UC)-associated inflammation using AI, which was published in February 2019 issue of Gastrointestinal Endoscopy (7) we commented on this paper. Moreover, we used this opportunity to elaborate, based on our own experience, on the endoscopic assessment of mucosal healing in UC using new state-of-the-art technologies: endocytoscopy (EC) and confocal laser endomicroscopy (CLE) systems that enable visualization of colonic mucosal structures and cells in vivo at high ~1,000× magnification during ongoing endoscopy.

Recent clinical trials in patients with UC and Crohn’s disease have used endoscopic assessment of mucosal healing as an indicator of disease activity, and important prognostic and therapeutic gauge, and as an endpoint (8-14). Reliable and reproducible assessment of inflammation and mucosal healing in UC may guide appropriate treatment aimed to decrease or prevent relapses and complications and to improve quality of life. This topic has been extensively reviewed before (8-14). Completeness of mucosal healing in UC determined endoscopically is associated with sustained clinical remission and reduced colectomy risk (12-14). In UC patients, assessment of mucosal healing by regular colonoscopy includes presence or absence of ulcers, abnormal blood vessels, erythema, edema, and nodularity (12-14). The use of magnifying chromoendoscopy and narrow band imaging has further improved the assessment of mucosal healing in UC and Crohn’s disease (15-17). However, a considerable proportion of UC patients have relapses despite exhibiting endoscopically “healed” colonic mucosa. This suggests that macroscopic healing of the mucosa determined by standard endoscopy, which only inspects the mucosal surface, is not sufficient because standard endoscopy cannot detect abnormalities and inflammation below the surface epithelium.

Endocytoscopy (EC) and CLE are novel technologies that enable to visualize during ongoing endoscopy not only mucosal surface, but also subepithelial mucosa, including cells, blood vessels and connective tissue at high 750× to 1,000× magnification (virtual biopsy) (17). EC and CLE images of colonic mucosa closely resemble microscopic images obtained from conventional histological biopsies (16,17). CLE has been used to evaluate disease activity and to predict relapse in patients with UC (14,15) and Crohn’s disease (16). EC and CLE have also successfully visualized and detect dysplasia and cancer in patients with UC. EC, which is based on the principle of contact light microscopy, enables in vivo microscopic visualization, imaging, and analysis of superficial mucosal structures. In contrast, CLE using, e.g., Cellvizio fluorescence imaging (Mauna Kea Technology, Paris France) allows analysis of mucosal structures up to 250 µm in depth, and with the use of needle probe the depth is unlimited. CLE requires intravenous infusion of fluorescein that visualizes mucosal blood vessels, and later also epithelial cells.

In a recent paper published in Gastrointestinal Endoscopy (7), Maeda and co-authors from Kanagawa, Nagoya, and Tokyo, Japan successfully detected UC-associated histologic inflammation in vivo using EC and a computer-aided diagnosis (CAD) system developed in collaboration with Cybernet Systems (Tokyo, Japan). This group has also previously published other important papers on AI assisted identification of diminutive polyps during colonoscopy and AI-assisted polyp detection for colonoscopy (18,19).

The authors evaluated their CAD system to detect presence of histologic inflammation in colonic mucosa using colonoscopy and EC in a retrospective study of 187 patients with UC from whom conventional biopsy samples from several colorectal segments were also obtained following EC and were evaluated histologically to determine activity. The CAD system was directly connected to the endoscopic instrument. For the histologic assessment, experienced pathologists blinded to the endoscopic results, evaluated the biopsy samples using the Geboes score, a grading score that predicts disease relapse and is used in clinical trials and routine clinical practice. The samples were divided into “histologically active” and “histologically healing”. The author’s definition of histologically active samples with persistent histologic inflammation was a score of ≥3.1 and that of histologically “healing” samples (indicating no persistent histologic inflammation) was a score of ≤3.0. A total of 22,835 EC images were collected for machine learning and validation and were tagged according to the histologic criteria, considered as the standard.

The main outcome measure was the diagnostic ability of CAD to identify presence of histologic inflammation. The authors used their CAD system to evaluate EC images and were able to identify persistent histologic inflammation with high degree (>90%) of specificity, accuracy and reproducibility. They concluded that their CAD system could be used for automated identification of UC-associated persistent histologic inflammation. For this study, they used a previously described CAD system that allows automated differential diagnosis between neoplastic and non-neoplastic polyps (18,19). They also used a support vector machine, the most common classifier in machine learning, and output a 2-category diagnosis (“active” or “healing”) based on the extracted features. The predicted histologic inflammatory status was displayed together with the probability of the output. In summary, the authors developed a novel, powerful CAD system that uses AI and advanced EC instrument for automated evaluation of UC-related mucosal inflammation. This CAD system has the potential to reduce the number of required standard biopsy samples.

We read paper by Maeda et al. (7) with interest and commend the authors on their thorough, well-conceived investigation. Their paper appears to be the first report of the development, validation, and application of a CAD system in patients with UC to identify presence of histologic inflammation. Based on our own experience with both EC and CLE in the assessment of quality of mucosal healing and inflammation in UC in vivo during colonoscopy, we feel that the latter instrument, especially Cellvizio CLE system (Mauna Kea Technology, Paris France) with a needle probe is superior to EC, because it allows visualization and analysis of mucosal structures in a greater depth level and also allows an excellent visualization of mucosal vessels filled with fluorescein, their abnormalities, increased vascular permeability, persistent inflammation and impaired and distorted crypt regeneration. Moreover, it allows molecular imaging, e.g., cyclooxygenase2 (COX2) and tumor necrosis factor alpha (TNF-α) expression and mitochondrial gene mutation (mtDNA) (20-22). In our previous publication (21) we showed that normal-appearing colonic mucosa in patients with UC-in remission visualized by standard colonoscopy has impaired crypt regeneration, persistent inflammation, distinct abnormalities in angioarchitecture and increased vascular permeability when examined using CLE (Figure 1). Furthermore, molecular imaging showed increased COX2 and mtDNA mutations. Therefore, CLE and to some degree EC might serve as a new standard for determining mucosal healing in UC.

Figure 1 Standard histology and CLE images of colonic mucosa in a control patient (A) and in a patient with UC in remission (UC-IR) (B,C). Colonic mucosa of control patient (A) has a normal mucosal structure with round crypts (*) and normal size microvessels (arrows). In patients with UC-IR (B), colonic mucosa has irregular and distorted crypts (*), increased spaces between the crypts, enlarged blood microvessels between crypts (arrows), and leakage of fluorescein into the extravascular space (arrowheads). CLE imaging of colonic mucosa of patient with UC-IR (C) shows increased inflammation marked by 6.6-fold increased number of infiltrating inflammatory cells (arrows). (modified and reproduced with permission from Ref. 21). CLE, confocal laser endomicroscopy.

A recent paper by de Lange, Halvorsen and Riegler discussed strategies, challenges and pitfalls related to development of machine learning for improving performance in gastrointestinal endoscopy (23). They stressed the importance of large databases, the quality and completeness of data and correct annotation of all images for creating algorithms for image analysis (24).

In our previous studies we analyzed expression of growth factors, quantify angiogenesis, mucosal healing, imaged the autonomous neural system in the esophagus and stomach, mitochondrial receptors and mitochondrial potential, and analyzed other features. For these determinations, we have used sophisticated computational intelligence in the form of image analysis software such as Image J system (NIH) and MetaMorph 7.0 (Molecular Devices, Downington, PA, USA), but we did not use the term AI. The Metamorph Imaging system analyses and categorizes objects into user-definable groups based on various parameters, such as shape, size, or intensity and offers customization using journals—macros that enable and automate a series of tasks. The Metamorph 7.0 software allows the use of functions for simplification of system operations, automating the acquisition of image and controlling the device, and for sequencing the events. All of these different software systems have advantages and disadvantages that no doubt will evolve as the field continues to advance.

In conclusion, advanced endoscopic imaging technologies such as EC and confocal laser endoscopic systems in combination with AI in the form of CAD system are feasible, safe, and useful tools for detailed in vivo virtual histologic diagnosis of mucosal inflammation and possible detection of dysplasia and neoplasia. AI-assisted in vivo imaging has many advantages—it is more objective, faster and more precise, and importantly is not critically dependent on the expertise of endoscopist and pathologist. The development of a large open dataset in the future will enable potential standardization of endoscopic and histologic diagnoses at the national and global levels. Another critical frontier in the future will take advantage of the acceleration of computer processing speeds to do such analyses in real time, rather than retrospectively as we (21,22), Maeda et al. (7), and others have done in the past. We can imagine a future in which the computer “watches” over the endoscopist’s shoulder in real time, bracketing areas for inspection of endoscopic appearance or biopsy and synergistically improving the endoscopist’s diagnostic acumen!

Acknowledgments

Funding: This work was supported by Merit Review Award # I01 BX000626-05A2 from the United States (U.S.) DVA Biomedical Laboratory Research and Development Service (to AS Tarnawski).

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, Journal of Medical Artificial Intelligence. The article did not undergo external peer review.

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jmai.2019.03.02). The authors have no conflicts of interest to declare.

Disclaimer:The contents do not represent the views of the U.S. Department of Veterans Affairs or the US Government.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Poole D, Mackworth A, Goebel R. Computational Intelligence: A Logical Approach. New York: Oxford University Press, 1998.
2. Russell SJ, Norvig P. Artificial Intelligence: A Modern Approach (3rd ed). Upper Saddle River, New Jersey: Prentice Hall, 2009.
3. Nilsson N. Artificial Intelligence: A New Synthesis. San Francisco, CA, USA: Morgan Kaufmann, 1998.
4. Legg S, Hutter M. A collection of definitions of intelligence. Goertzel B, Wang P (eds). Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms. Proceedings of the AGI Workshop, Amsterdam IOS, 2007:17-24.
5. Kaplan A, Haenlein M. Siri, Siri, in my hand: Who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business Horizons 2019;62:15-25. [Crossref]
6. Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 2017;69S:S36-40. [Crossref] [PubMed]
7. Maeda Y, Kudo SE, Mori Y, et al. Fully automated diagnostic system with artificial intelligence using endocytoscopy to identify the presence of histologic inflammation associated with ulcerative colitis (with video). Gastrointest Endosc 2019;89:408-15. [Crossref] [PubMed]
8. Lichtenstein GR, Rutgeerts P. Importance of mucosal healing in ulcerative colitis. Inflamm. Bowel Dis 2010;16:338-46. [Crossref] [PubMed]
9. Mazzuoli S, Guglielmi FW, Antonelli E, et al. Definition and evaluation of mucosal healing in clinical practice. Dig Liver Dis 2013;45:969-77. [Crossref] [PubMed]
10. Peyrin-Biroulet L, Bressenot A, Kampman W. Histologic Remission: The Ultimate Therapeutic Goal in Ulcerative Colitis? Clin Gastroenterol Hepatol 2014;12:929-34.e2. [Crossref] [PubMed]
11. Seidelin JB, Coskun M, Nielsen OH. Mucosal healing in ulcerative colitis: pathophysiology and pharmacology. Adv Clin Chem 2013;59:101-23. [Crossref] [PubMed]
12. Pineton de Chambrun G, Peyrin-Biroulet L, Lémann M, et al. Clinical implications of mucosal healing for the management of IBD. Nat Rev Gastroenterol Hepatol 2010;7:15-29. [Crossref] [PubMed]
13. Neurath MF, Travis SP. Mucosal healing in inflammatory bowel diseases: a systematic review. Gut 2012;61:1619-35. [Crossref] [PubMed]
14. Colombel JF, Rutgeerts P, Reinisch W, et al. Early mucosal healing with infliximab is associated with improved long-term clinical outcomes in ulcerative colitis. Gastroenterology 2011;141:1194-201. [Crossref] [PubMed]
15. Li CQ, Xie XJ, Yu T, et al. Classification of inflammation activity in ulcerative colitis by confocal laser endomicroscopy. Am J Gastroenterol 2010;105:1391-6. [Crossref] [PubMed]
16. Neumann H, Vieth M, Atreya R, et al. Assessment of Crohn’s disease activity by confocal laser endomicroscopy. Inflamm Bowel Dis 2012;18:2261-9. [Crossref] [PubMed]
17. Kiesslich R, Goetz M, Lammersdorf K, et al. Chromoscopy-guided endomicroscopy increases the diagnostic yield of intraepithelial neoplasia in ulcerative colitis. Gastroenterology 2007;132:874-82. [Crossref] [PubMed]
18. Mori Y, Kudo SE, Wakamura K, et al. Novel computer-aided diagnostic system for colorectal lesions by using endocytoscopy (with videos). Gastrointest Endosc 2015;81:621-9. [Crossref] [PubMed]
19. Mori Y, Kudo SE, Chiu PW, et al. Impact of an automated system for endocytoscopic diagnosis of small colorectal lesions: an international web-based study. Endoscopy 2016;48:1110-8. [Crossref] [PubMed]
20. Goetz M, Wang TD. Molecular imaging in gastrointestinal endoscopy. Gastroenterology 2010;138:828-33.e1. [Crossref] [PubMed]
21. Macé V, Ahluwalia A, Coron E, et al. Confocal laser endomicroscopy: a new gold standard for the assessment of mucosal healing in ulcerative colitis. J Gastroenterol Hepatol 2015;30:85-92. [Crossref] [PubMed]
22. Samarasena JB, Ahluwalia A, Shinoura S, et al. In vivo imaging of porcine gastric enteric nervous system using confocal laser endomicroscopy &molecular neuronal probe. J Gastroenterol Hepatol 2016;31:802-7. [Crossref] [PubMed]
23. de Lange T, Halvorsen P, Riegler M. Methodology to develop machine learning algorithms to improve performance in gastrointestinal endoscopy. World J Gastroenterol 2018;24:5057-62. [Crossref] [PubMed]
24. Pogorelov K, Randel KR, Griwodz C, et al. Kvasir: A multiclass image dataset for computer-aided gastrointestinal disease detection. ACM on Multimedia Systems Conference 2017:164-9.