The top 100 most cited articles in medical artificial intelligence: a bibliometric analysis
Artificial intelligence (AI) is a general term used to describe machines and computers performing tasks usually requiring human intelligence (1). There has been interest in the application of AI in medicine since the 1950s (2). Since then, computer scientists and medical researchers have continued to investigate the use of AI in almost every field of medicine (2). Proponents of medical AI argue that it can help the clinician in all aspects of medicine, from formulating a diagnosis to making therapeutic decisions and predicting patient outcomes (3,4).
An extensive body of literature has become available regarding the use of AI in medicine, with considerable variation in the quality of articles, as well as in the study methodologies and AI techniques used. This can lead to difficulty in identifying articles of significance, particularly for clinicians with limited technical knowledge in AI. A bibliometric analysis of the most highly cited publications relating to medical AI may provide a better understanding of the progress made to date in this field, as well as identify areas for future research efforts (5-7). Citation frequency is a method of bibliometric analysis that involves examining the publications that have been most cited by other researchers. Citation count serves as an indicator of the influence and quality of a scientific publication (8).
A number of bibliometric analyses from various fields of medicine have been published (9-13). While the international interest and academic output relating to medical AI has risen significantly in recent years, a bibliometric analysis of the citation classics in this field, to the best of our knowledge, has not yet been performed. As such, this study aimed to identify and examine the characteristics of the top 100 most cited articles relating to the application of AI in medicine.
A retrospective bibliometric analysis of the 100 most cited peer-reviewed journal articles related to the use of AI in medicine was performed in April 2019. Articles from the MEDLINE® (U.S. National Library of Medicine, Bethesda, USA) database were identified using the Web of Science (Clarivate Analytics, Philadelphia, USA) citation indexing service (14,15). The MEDLINE® database indexes more than 5,000 journals, comprising more than 25 million references in medicine and life sciences published since 1950. MEDLINE® uses Medical Subject Headings (MeSH), which imposes uniformity and consistency to the indexing of biomedical literature.
The following search strategy was used: [“artificial intelligence” OR “machine learning” OR “deep learning” OR “natural language processing” OR “support vector machine” OR “naïve bayes” OR “bayesian learning” OR “artificial neural network” OR “random forest” OR “machine intelligence” OR “k-nearest neighbor” OR “decision tree learning” OR “data mining” OR “fuzzy” OR “computational intelligence” OR “computer reasoning”] AND [“medicine” OR “medical” OR “surgery” OR “surgical” OR “healthcare”]. Articles were identified if they included these search terms in either its title, abstract or MeSH terms. There were no restrictions on language or year of publication. A total of 16,025 articles were returned from this search. The top 100 articles ranked by citation count were identified and downloaded to a local database.
Analysis of individual articles was performed by three reviewers (S Sreedharan, M Mian, RA Robertson) in order to extract relevant information relating to year of publication, authorship, journal title and impact factor (IF), institution, country of origin, article type, key words and field of medicine. Journal IFs were obtained from the SCImago Journal Rank (Elsevier, Amsterdam, Netherlands) issued in 2017, the most recent year of available data at the time of search. Institution and country of origin were based on the corresponding author’s affiliations. Article type was dichotomised into original research article or review article. The original articles were further dichotomised into clinical or non-clinical, where clinical papers were defined as those where the primary research question was clinical and the study involved human participants or data. Keywords were recorded for each article by screening their titles, abstracts and MeSH terms for the search terms used. Articles that could not be assigned to a specific field of medicine were grouped in a “general” category. The “medical image analysis” category included any papers relating to the analysis of non-radiological medical images, such as histopathological images or clinical photographs. The average citations per year for each article was also included as an adjunctive measure of overall article impact. Average citation per year was calculated by dividing each article’s total number of citations by the number of years since that article had been published.
The top 100 articles relating to the use of AI in medicine were ranked according to citation count (Table 1). The median [IQR] number of citations was 238 [205–347]. The median [IQR] number of citations per year was 21 [16–41]. “The American College of Rheumatology preliminary diagnostic criteria for fibromyalgia and measurement of symptom severity”, published in Arthritis Care & Research in 2010, was ranked first with 1,475 citations. When ranked according to citations per year (Table 2), it became the fourth highest ranked article with an average of 164 citations per year.
|Rank||Article||Citations||Citations per year|
|1||Wolfe F, et al. The American College of Rheumatology preliminary diagnostic criteria for fibromyalgia and measurement of symptom severity. Arthritis Care & Research, 2010;62(5):600-10.||1,475||163.89|
|2||Radke RJ, et al. Image change detection algorithms: a systematic survey. IEEE Transactions on Image Processing: a publication of the IEEE Signal Processing Society, 2005;14(3):294-307.||930||66.43|
|3||Strobl C, et al. An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 2009;14(4):323-48.||730||73.00|
|4||Aronson AR. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proceedings. AMIA Symposium, 2001:17-21.||665||36.94|
|5||Heimann T, Meinzer HP. Statistical shape models for 3D medical image segmentation: a review. Medical Image Analysis, 2009;13(4):543-63.||608||60.80|
|6||Tu JV. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. Journal of Clinical Epidemiology, 1996;49(11):1225-31.||587||25.52|
|7||Gillies RJ, et al. Radiomics: Images Are More than Pictures, They Are Data. Radiology, 2016;278(2):563-77.||582||194.00|
|8||Noble JA, Boukerroui D. Ultrasound image segmentation: a survey. IEEE Transactions on Medical Imaging, 2006;25(8):987-1010.||552||42.46|
|9||Rosse C, Mejino JL Jr. A reference ontology for biomedical informatics: the Foundational Model of Anatomy. Journal of Biomedical Informatics, 2003;36(6):478-500.||539||33.69|
|10||Yao X, Liu Y. A new evolutionary system for evolving artificial neural networks. IEEE Transactions on Neural Networks, 1997;8(3):694-713.||505||22.95|
|11||Savova GK, et al. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. Journal of the American Medical Informatics Association, 2010;17(5):507-13.||497||55.22|
|12||Gulshan V, et al. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA, 2016;316(22):2402-2410.||485||161.67|
|13||Reyna VF, et al. How numeracy influences risk comprehension and medical decision making. Psychological Bulletin, 2009;135(6):943-73.||483||48.30|
|14||Benenson Y, et al. An autonomous molecular computer for logical control of gene expression. Nature, 2004;429(6990):423-9.||482||32.13|
|15||Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA, 2013;309(13):1351-2.||429||71.50|
|16||Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. Journal of Biomedical Informatics, 2002;35(5-6):352-9.||427||25.12|
|17||Mitra S, Hayashi Y. Neuro-fuzzy rule generation: survey in soft computing framework. IEEE Transactions on Neural Networks, 2000;11(3):748-68.||427||22.47|
|18||Litjens G, et al. A survey on deep learning in medical image analysis. Medical Image Analysis, 2017;42:60-88.||403||201.50|
|19||Baxt WG. Application of artificial neural networks to clinical medicine. The Lancet, 1995;346(8983):1135-8.||398||16.58|
|20||Lopes R, Betrouni N. Fractal and multifractal analysis: a review. Medical Image Analysis, 2009;13(4):634-49.||388||38.80|
|21||Saeed M, et al. Multiparameter Intelligent Monitoring in Intensive Care II: a public-access intensive care unit database. Critical Care Medicine, 2011;39(5):952-60.||374||46.75|
|22||Qian L, et al. Neural network computation with DNA strand displacement cascades. Nature, 2011;475(7356):368-72.||370||46.25|
|23||Li K, et al. Optimal surface segmentation in volumetric images--a graph-theoretic approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006;28(1):119-34.||361||27.77|
|24||Iizuka N, et al. Oligonucleotide microarray for prediction of early intrahepatic recurrence of hepatocellular carcinoma after curative resection. The Lancet, 2003;361(9361):923-9.||360||22.50|
|25||Chapman WW, et al.A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of Biomedical Informatics, 2001;34(5):301-10.||349||19.39|
|26||Lu, G, Fei B. Medical hyperspectral imaging: a review. Journal of Biomedical Optics, 2014;19(1):10901.||346||69.20|
|27||Kononenko I. Machine learning for medical diagnosis: history, state of the art and perspective. Artificial Intelligence in Medicine, 2001;23(1):89-109.||332||18.44|
|28||Ott PA, et al. An immunogenic personal neoantigen vaccine for patients with melanoma. Nature, 2017;547(7662):217-221.||330||165.00|
|29||Xiong HY, et al. RNA splicing. The human splicing code reveals new insights into the genetic determinants of disease. Science, 2015;347(6218):1254806.||328||82.00|
|30||Friedman C, et al. A general natural-language text processor for clinical radiology. Journal of the American Medical Informatics Association, 1994;1(2):161-74.||325||13.00|
|31||Forbes SA, et al. The Catalogue of Somatic Mutations in Cancer (COSMIC). Current Protocols in Human Genetics, 2008;Chapter 10:Unit 10.11.||323||29.36|
|32||Szarfman A, et al. Use of screening algorithms and computer systems to efficiently signal higher-than-expected combinations of drugs and events in the US FDA's spontaneous reports database. Drug Safety, 2002;25(6):381-92.||314||18.47|
|33||Muller HM, et al. Textpresso: an ontology-based information retrieval and extraction system for biological literature. PLoS Biology, 2004;2(11):e309.||306||20.40|
|34||Bellazzi R, Zupan B. Predictive data mining in clinical medicine: current issues and guidelines. International Journal of Medical Informatics, 2008;77(2):81-97.||298||27.09|
|35||Tajbakhsh N, et al. Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? IEEE Transactions on Medical Imaging, 2016;35(5):1299-1312.||294||98.00|
|36||Kamnitsas K, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis, 2017;36:61-78.||286||143.00|
|37||Markelj P, et al. A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis, 2012;16(3):642-61.||286||40.86|
|38||El-Naqa I, et al. A support vector machine approach for detection of microcalcifications. IEEE Transactions on Medical Imaging, 2002;21(12):1552-63.||284||16.71|
|39||Lanitis A, et al. Comparing different classifiers for automatic age estimation. IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics: a publication of the IEEE Systems, Man, and Cybernetics Society, 2004;34(1):621-8.||280||18.67|
|40||Rice TW, et al. Cancer of the esophagus and esophagogastric junction: data-driven staging for the seventh edition of the American Joint Committee on Cancer/International Union Against Cancer Cancer Staging Manuals. Cancer, 2010;116(16):3763-73.||277||30.78|
|41||Sluimer I, et al. Computer analysis of computed tomography scans of the lung: a survey. IEEE Transactions on Medical Imaging, 2006;25(4):385-405.||274||21.08|
|42||Shahar Y, et al. The Asgaard project: a task-specific framework for the application and critiquing of time-oriented clinical guidelines. Artificial Intelligence in Medicine, 1998;14(1-2):29-51.||272||12.95|
|43||Denny JC, et al. Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data. Nature Biotechnology, 2013;31(12):1102-10.||265||44.17|
|44||Zhang DQ, Chen SC. A novel kernelized fuzzy C-means algorithm with application in medical image segmentation. Artificial Intelligence in Medicine, 2004;32(1):37-50.||258||17.20|
|45||Zhang H, et al. Discontinuation of statins in routine care settings: a cohort study. Annals of Internal Medicine, 2013;158(7):526-34.||256||42.67|
|46||Bates DW, et al. Detecting adverse events using information technology. Journal of the American Medical Informatics Association, 2003;10(2):115-28.||254||15.88|
|47||Dayhoff JE, DeLeo JM. Artificial neural networks: opening the black box. Cancer, 2001;91(8 Suppl):1615-35.||253||14.06|
|48||Uzuner O, et al. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 2011;18(5):552-6.||243||30.38|
|49||Cios KJ, Moore GW. Uniqueness of medical data mining. Artificial Intelligence in Medicine, 2002;26(1-2):1-24.||241||14.18|
|50||Erho N, et al. Discovery and validation of a prostate cancer genomic classifier that predicts early metastasis following radical prostatectomy. PloS One, 2013;8(6):e66855.||239||39.83|
|51||Iorio F, et al. A Landscape of Pharmacogenomic Interactions in Cancer. Cell, 2016; 166(3):740-754.||238||79.33|
|52||Obermeyer Z, Emanuel EJ. Predicting the Future - Big Data, Machine Learning, and Clinical Medicine. New England Journal of Medicine, 2016;375(13):1216-9.||235||78.33|
|53||Lisboa, PJ. A review of evidence of health benefit from artificial neural networks in medical intervention. Neural Networks, 2016;375(13):1216-9.||233||13.71|
|54||Pauker SG, et al. Towards the simulation of clinical cognition. Taking a present illness by computer. The American Journal of Medicine, 1976;60(7):981-96.||233||5.42|
|55||Muller FJ, et al. Regulatory networks define phenotypic classes of human stem cell lines. Nature, 2008;455(7211):401-5.||232||21.09|
|56||Meystre SM, et al. Extracting information from textual documents in the electronic health record: a review of recent research. Yearbook of Medical Informatics, 2008:128-44.||229||20.82|
|57||Nauck D, Kruse R. Obtaining interpretable fuzzy classification rules from medical data. Artificial Intelligence in Medicine, 1999;16(2):149-69.||229||11.45|
|58||Crum WR, et al. Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Transactions on Medical Imaging, 2006;25(11):1451-61.||228||17.54|
|59||Saad ZS, et al. A new method for improving functional-to-structural MRI alignment using local Pearson correlation. NeuroImage, 2009;44(3):839-48.||225||22.50|
|60||Mazurowski MA, et al. Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Networks, 2008;21(2-3):427-36.||224||20.36|
|61||Bigio IJ, et al. Diagnosis of breast cancer using elastic-scattering spectroscopy: preliminary clinical results. Journal of Biomedical Optics, 2000;5(2):221-8.||224||11.79|
|62||Barnett GO, et al. DXplain. An evolving diagnostic decision-support system. JAMA, 1987; 258(1):67-74.||223||6.97|
|63||Woodhams R, et al. ADC mapping of benign and malignant breast tumors. Magnetic Resonance in Medical Sciences, 2005;4(1):35-42.||221||15.79|
|64||Libbrecht MW, Noble WS. Machine learning applications in genetics and genomics. Nature Reviews Genetics, 2015;16(6):321-32.||220||55.00|
|65||Pedersen T, et al. Measures of semantic similarity and relatedness in the biomedical domain. Journal of Biomedical Informatics, 2007;40(3):288-99.||220||18.33|
|66||Xu Q, et al. Low-dose X-ray CT reconstruction via dictionary learning. IEEE Transactions on Medical Imaging, 2012;31(9):1682-97.||218||31.14|
|67||Acampora G, et al. A Survey on Ambient Intelligence in Health Care. Proceedings of the IEEE, 2013; 101(12):2470-2494.||215||35.83|
|68||Rizk NP, et al. Optimum lymphadenectomy for esophageal cancer. Annals of Surgery, 2010; 251(1):46-50.||212||23.56|
|69||Lisboa PJ, Taktak AF. The use of artificial neural networks in decision support in cancer: a systematic review. Neural Networks, 2006;19(4):408-15.||212||16.31|
|70||Terwee CB, et al. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Quality of Life Research, 2009;18(8):1115-23.||210||21.00|
|71||Friedman C, et al. Automated encoding of clinical documents based on natural language processing. Journal of the American Medical Informatics Association, 2004; 11(5):392-402.||210||14.00|
|72||Nanni L, et al. Local binary patterns variants as texture descriptors for medical image analysis. Artificial Intelligence in Medicine, 2010;49(2):117-25.||208||23.11|
|73||Rebhan M, et al. GeneCards: a novel functional genomics compendium with automated data mining and query reformulation support. Bioinformatics, 1998;14(8):656-64.||208||9.90|
|74||Gurcan MN, et al. Histopathological image analysis: a review. IEEE Reviews in Biomedical Engineering, 2009;2:147-71.||207||20.70|
|75||Madhavan S, et al. Rembrandt: helping personalized medicine become a reality through integrative translational research. Molecular Cancer Research, 2009;7(2):157-67.||205||20.50|
|76||Spackman KA, et al. SNOMED RT: a reference terminology for health care. Proceedings: a conference of the American Medical Informatics Association. AMIA Fall Symposium, 1997:640-4.||205||9.32|
|77||Hripcsak G, et al. Unlocking clinical data from narrative reports: a study of natural language processing. Annals of Internal Medicine, 1995;122(9):681-8.||203||8.46|
|78||Wolf I, et al. The medical imaging interaction toolkit. Medical Image Analysis, 2005;9(6):594-604.||202||14.43|
|79||Cruz JA, Wishart DS. Applications of machine learning in cancer prediction and prognosis. Cancer Informatics, 2007;2:59-77.||197||16.42|
|80||Chuang LY, et al. Improved binary PSO for feature selection using gene expression data. Computational Biology and Chemistry, 2008;32(1):29-37.||194||17.64|
|81||Rindflesch TC, Fiszman M. The interaction of domain knowledge and linguistic structure in natural language processing: interpreting hypernymic propositions in biomedical text. Journal of Biomedical Informatics, 2003;36(6):462-77.||192||12.00|
|82||Nadkarni PM, et al. Chapman, Natural language processing: an introduction. Journal of the American Medical Informatics Association, 2011; 18(5):544-51.||191||23.88|
|83||Xu H, et al. MedEx: a medication information extraction system for clinical narratives. Journal of the American Medical Informatics Association, 2010;17(1):19-24.||190||21.11|
|84||Grabner G, et al.Symmetric atlasing and model based segmentation: an application to the hippocampus in older adults. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2006;9(Pt 2):58-66.||190||14.62|
|85||Rossini PM, et al. Double nerve intraneural interface implant on a human amputee for robotic hand control. Clinical Neurophysiology, 2010;121(5):777-83.||188||20.89|
|86||Ritchie MD, et al. Robust replication of genotype-phenotype associations across multiple diseases in an electronic medical record. American Journal of Human Genetics, 2010; 86(4):560-72.||188||20.89|
|87||Shen D, et al. Deep Learning in Medical Image Analysis. Annual Review of Biomedical Engineering, 2017;19:221-248.||187||93.50|
|88||Wein W, et al. Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention. Medical Image Analysis, 2008;12(5):577-85.||187||17.00|
|89||Abbass HA. An evolutionary artificial neural networks approach for breast cancer diagnosis. Artificial Intelligence in Medicine, 2002;25(3):265-81.||186||10.94|
|90||Zang Y, et al. Flexible suspended gate organic thin-film transistors for ultra-sensitive pressure detection. Nature Communications, 2015;6:6269.||184||46.00|
|91||Li BN, et al. Integrating spatial fuzzy clustering with level set methods for automated medical image segmentation. Computers in Biology and Medicine, 2011; 41(1):1-10.||183||22.88|
|92||Heinrich MP, et al. MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Medical Image Analysis, 2012;16(7):1423-35.||182||26.00|
|93||Bodenreider O, Stevens R. Bio-ontologies: current trends and future directions. Briefings in Bioinformatics, 2006;7(3):256-74.||182||14.00|
|94||Friedman C, et al. GENIES: a natural-language processing system for the extraction of molecular pathways from journal articles. Bioinformatics, 2001;17 Suppl 1:S74-82.||182||10.11|
|95||Fox J, et al. Disseminating medical knowledge: the PROforma approach. Artificial Intelligence in Medicine, 1998;14(1-2):157-81.||181||8.62|
|96||Musen MA. Dimensions of knowledge sharing and reuse. Computers and Biomedical Research, 1992;25(5):435-67.||180||6.67|
|97||Duda RO, Shortliffe EH. Expert Systems Research. Science, 1983;220(4594):261-8.||180||5.00|
|98||Sharma N, Aggarwal LM. Automated medical image segmentation techniques. Journal of Medical Physics, 2010;35(1):3-14.||177||19.67|
|99||Sorzano CO, et al. Elastic registration of biological images using vector-spline regularization. IEEE Transactions on Biomedical Engineering, 2005; 52(4):652-63.||177||12.64|
|100||Hripcsak G, et al. Rationale for the Arden Syntax. Computers and Biomedical Research, 1994;27(4):291-324.||176||7.04|
|Rank||Article||Citations per year||Citations||Rank by citations|
|1||Litjens G, et al. A survey on deep learning in medical image analysis. Medical Image Analysis, 2017;42:60-88.||201.50||403||18|
|2||Gillies RJ, et al. Radiomics: Images Are More than Pictures, They Are Data. Radiology, 2016;278(2):563-77.||194.00||582||7|
|3||Ott PA, et al. An immunogenic personal neoantigen vaccine for patients with melanoma. Nature, 2017;547(7662):217-221.||165.00||330||28|
|4||Wolfe F, et al. The American College of Rheumatology preliminary diagnostic criteria for fibromyalgia and measurement of symptom severity. Arthritis Care & Research, 2010;62(5):600-10.||163.89||1475||1|
|5||Gulshan V, et al. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA, 2016;316(22):2402-2410.||161.67||485||12|
|6||Kamnitsas K, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis, 2017;36:61-78.||143.00||286||36|
|7||Tajbakhsh N, et al. Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? IEEE Transactions on Medical Imaging, 2016; 35(5):1299-1312.||98.00||294||25|
|8||Shen D, et al. Deep Learning in Medical Image Analysis. Annual Review of Biomedical Engineering, 2017;19:221-248.||93.50||187||87|
|9||Xiong HY, et al. RNA splicing. The human splicing code reveals new insights into the genetic determinants of disease. Science, 2015;347(6218):1254806.||82.00||328||29|
|10||Iorio F, et al. A Landscape of Pharmacogenomic Interactions in Cancer. Cell, 2016;166(3):740-754.||79.33||238||51|
|11||Obermeyer Z, Emanuel EJ. Predicting the Future - Big Data, Machine Learning, and Clinical Medicine. The New England Journal of Medicine, 2016; 375(13):1216-9.||78.33||235||52|
|12||Strobl C, et al. An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 2009;14(4):323-48.||73.00||730||3|
|13||Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA, 2013;309(13):1351-2.||71.50||429||15|
|14||Lu G, Fei B. Medical hyperspectral imaging: a review. Journal of Biomedical Optics, 2014;19(1):10901.||69.20||346||26|
|15||Radke RJ, et al. Image change detection algorithms: a systematic survey. IEEE transactions on image processing: a publication of the IEEE Signal Processing Society, 2005; 14(3):294-307.||66.43||930||2|
|16||Heimann T, Meinzer HP. Statistical shape models for 3D medical image segmentation: a review. Medical Image Analysis, 2009;13(4):543-63.||60.80||608||5|
|17||Savova GK, et al. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. Journal of the American Medical Informatics Association, 2010;17(5):507-13.||55.22||497||11|
|18||Libbrecht MW, Noble WS. Machine learning applications in genetics and genomics. Nature reviews. Genetics, 2015;16(6):321-32.||55.00||220||64|
|19||Reyna VF, et al. How numeracy influences risk comprehension and medical decision making. Psychological Bulletin, 2009; 135(6):943-73.||48.30||483||13|
|20||Saeed M, et al. Multiparameter Intelligent Monitoring in Intensive Care II: a public-access intensive care unit database. Critical Care Medicine, 2011;39(5):952-60.||46.75||374||21|
Only 15 articles were published prior to 2000. Thirty-five articles were published within the last decade, and the majority of articles were published between 2000 and 2010 (Table 3). The oldest article on the list was “Towards the stimulation of clinical cognition. Taking a present illness by computer” published in The American Journal of Medicine in 1976. This original research article presented a novel computer software that was developed to take a clinical history for a patient presenting with oedema in order to determine the most likely underlying illness.
|Decade||No of articles|
A total of 55 journals contributed articles to the top 100 list, with 15 contributing two or more articles. With eight articles each, Artificial Intelligence in Medicine (IF: 3.62) and Medical Image Analysis (IF: 6.50) contributed the most articles, followed by Journal of the American Medical Informatics Association (IF: 3.97) and Journal of Biomedical Informatics (IF: 7.22) with seven articles each. The journals with the highest IFs were New England Journal of Medicine (IF: 42.18) and Nature Reviews Genetics (IF: 38.94), each contributing one article.
There were 16 institutions contributing two or more articles each to the top 100 list. Harvard Medical School was the biggest contributor with five articles, followed by Vanderbilt University  and the National Library of Medicine . The United States of America was the leading country of origin with 55 articles, followed by the United Kingdom  and Canada . A further nine countries contributed two or more articles each (Figure 1).
The top 100 articles list comprised 60 original research articles and 40 review articles. The 60 original research articles included 11 clinical studies (Figure 2). Of note, 8 of the 11 clinical papers were published in the last decade, and 6 of the 11 clinical studies were related to oncology. The most frequently represented field in the top 100 list was medical informatics , followed by radiology , oncology  and non-radiological medical image analysis  (Figure 3). Papers relating to medical image analysis had the highest average citations per year at 69.03, followed by radiology (43.15) and genetics (42.16). The 11 clinical papers were from oncology , ophthalmology , internal medicine , critical care  and radiology .
“Artificial intelligence” , “natural language processing” , “machine learning” , “data mining”  and “artificial neural network”  were the most frequently used keywords among the top 100 list (Table 4). Eighteen of the 20 articles that included the keyword “natural language processing” were from the field of medical informatics.
|Institution||No of articles|
|Natural language processing||20|
|Artificial neural network||13|
|Support vector machine||4|
There has been a recent surge of interest in the use of AI in medicine. However, many of the top 100 articles identified in this study were of low-level evidence and limited clinical significance, comprised of review articles and commentaries providing only a general overview of medical AI. Original research articles investigating the use of AI in clinical populations were lacking, with only 11 of the top 100 articles identified as clinical studies. The lack of highly cited clinical studies demonstrates the need for more studies investigating the integration of AI into clinical medicine. High-level evidence, including randomised controlled trials and meta-analyses, is required for clinicians to gain confidence in the capabilities of AI. Improved collaboration between clinicians and computer scientists or streamlined pathways for dual qualification in health and computer science may be feasible strategies to facilitate improved medical AI research.
Medical informatics contributed the most articles to the top 100 list. With the implementation of electronic medical records (EMR), medical informatics is becoming increasingly important. In order to reach their full potential, EMRs require automated applications to manage the huge amounts of clinical information available in them. Within the field of medical informatics, “natural language processing (NLP)” was the most frequently used keyword. NLP began in the 1950s and represents the intersection of AI and linguistics (16). Clinical information is often not coded or structured but recorded in natural language text that may not be readily accessible by informaticians. NLP may be able to overcome this problem by extracting text-based information and structuring it into useful data (17).
Radiology was the leading field of clinical medicine represented in the citation classics, contributing just over one fifth of the top 100 articles. While medical informatics had the highest number of articles, the average annual citations per article in radiology was more than double that of medical informatics, demonstrating the substantial interest in the use of AI in radiology. Radiology is a unique field of medicine in that it encompasses many of the common applications of AI. AI techniques have been experimented with in various aspects of radiology, from assisting clinicians to determine the most appropriate imaging procedure, to image interpretation and computer-assisted diagnosis, and lastly results reporting and the extraction of information from radiologist reports (18).
It is worth noting that there is an overlap in the approaches used in applying AI to radiological and non-radiological medical images. Non-radiological medical image analysis was the fourth leading field in our study. This category refers to histopathological slides and clinical photographs, such as endoscopic images or images of dermatological lesions (19). The separation of this category from radiology was made in order to recognise solely radiological studies from those involving wider medical image analysis. As with radiology, the application of AI in medical image analysis relies on the ability to collect and use large datasets in order to train AI systems in pattern recognition. It is likely that both fields will continue to be a major focus of medical AI in the future (19).
The majority of the clinical studies in the top 100 list were oncological papers. This is likely due to a number of factors. First, cancer is among the leading causes of mortality in developed countries (20), driving a scientific interest to innovate in this field. AI also has the potential to improve outcomes in oncology because of the variety of cancer types and presentations, and the risk of patients being asymptomatic until late and severe stages of disease. Lastly, oncology relies on a range of data rich modalities such as genomics and metabolomics, which enables the generation of large clinical datasets useful in the building and validation of AI models (21). In contrast, AI may be harder to implement in fields with fewer objective investigations and data available, such as psychiatry. Interestingly, cardiovascular disease did not feature in the top 100 list despite being the leading cause of mortality globally (20). This highlights a potential mismatch between disease burden and AI research efforts.
There are several limitations of our study that should be considered. A source of bias in the use of citation analysis is that older articles are more likely to be cited, independent of quality of the article. Further, total citation count does not provide information about the temporal profile of citations for each paper. We addressed this by including an average annual citation number as another indicator for article impact and contemporary influence (13). The use of journal IFs from 2017 only does not account for changes in journal IFs over time and does not account for the journal IF at the time of article publication. Another limitation is that the search terms used in our study may have excluded some relevant or highly cited articles (12). We attempted to mitigate this risk by including both general search terms, such as “artificial intelligence” or “machine learning”, as well as detailed search terms of specific AI techniques, such as “random forest” or “neural network”. It should be noted that applying these search terms to only titles, abstracts and MeSH keywords meant that articles may have not employed the mentioned AI technique, but only discussed it. However, this still highlights the AI techniques most frequently discussed across the highly cited literature in the field of medical AI.
This study provides a comprehensive overview of the top 100 most cited articles relating to the use of AI in medicine over the past 70 years. It highlights that the current citation classics are largely in the non-clinical, experimental phase and have yet to progress to the clinical, integration phase of medical AI. While medical informatics and radiology were heavily featured in the citation classics, oncology was the leading field with clinical integration of AI. There is an apparent mismatch between disease burden and AI research efforts, with a lack of representation of cardiovascular medicine in the top 100 list despite cardiovascular disease being the leading cause of mortality globally. These results offer important insights into current research trends in medical AI and could help direct future research in this highly active and exciting field.
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jmai.2019.11.04). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
- Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 2017;69S:S36-S40. [Crossref] [PubMed]
- Ramesh A, Kambhampati C, Monson JR, et al. Artificial intelligence in medicine. Ann R Coll Surg Engl 2004;86:334. [Crossref] [PubMed]
- Schwartz WB, Patil RS, Szolovits P. Artificial intelligence in medicine. N Engl J Med 1987;316:685-8. [Crossref] [PubMed]
- Szolovits P, Patil RS, Schwartz WB. Artificial intelligence in medical diagnosis. Ann Intern Med 1988;108:80-7. [Crossref] [PubMed]
- Choudhri AF, Siddiqui A, Khan NR, et al. Understanding bibliometric parameters and analysis. Radiographics 2015;35:736-46. [Crossref] [PubMed]
- Abramo G, D’Angelo CA. Evaluating research: from informed peer review to bibliometrics. Scientometrics 2011;87:499-514. [Crossref]
- Goodall A. The place of citations in today's academy. International Higher Education 2015. doi: https://doi.org/
- Godin B. On the origins of bibliometrics. Scientometrics 2006;68:109-33. [Crossref]
- Kelly J, Glynn R, O’Briain D, et al. The 100 classic papers of orthopaedic surgery: a bibliometric analysis. J Bone Joint Surg Br 2010;92:1338-43. [Crossref] [PubMed]
- Yoon DY, Yun EJ, Ku YJ, et al. Citation classics in radiology journals: the 100 top-cited articles, 1945–2012. AJR Am J Roentgenol 2013;201:471-81. [Crossref] [PubMed]
- Brandt JS, Downing AC, Howard DL, et al. Citation classics in obstetrics and gynecology: the 100 most frequently cited journal articles in the last 50 years. Am J Obstet Gynecol 2010;203:355.e1-7. [Crossref] [PubMed]
- Mohammed MF, Chahal T, Gong B, et al. Trends in CT colonography: bibliometric analysis of the 100 most-cited articles. Br J Radiol 2017;90:20160755. [Crossref] [PubMed]
- Maingard J, Phan K, Ren Y, et al. The 100 most cited articles in the endovascular management of intracranial aneurysms. J Neurointerv Surg 2018;10:859-68. [Crossref] [PubMed]
- Kulkarni AV, Aziz B, Shams I, et al. Comparisons of Citations in Web of Science, Scopus, and Google Scholar for Articles Published in General Medical Journals. JAMA 2009;302:1092-6. [Crossref] [PubMed]
- Falagas ME, Pitsouni EI, Malietzis GA, et al. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J 2008;22:338-42. [Crossref] [PubMed]
- Nadkarni PM, Ohno-Machado L, Chapman WW. Natural language processing: an introduction. J Am Med Inform Assoc 2011;18:544-51. [Crossref] [PubMed]
- Friedman C, Hripcsak G. Natural language processing and its future in medicine. Acad Med 1999;74:890-5. [Crossref] [PubMed]
- Kahn CE Jr. Artificial intelligence in radiology: decision support systems. Radiographics 1994;14:849-61. [Crossref] [PubMed]
- Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. [Crossref] [PubMed]
- World Health Organization. Global health estimates 2016: disease burden by cause, age, sex, by country and by region, 2000–2016. Geneva: World Health Organization, 2018.
- Ramaswamy S. Translating cancer genomics into clinical oncology. N Engl J Med 2004;350:1814. [Crossref] [PubMed]
Cite this article as: Sreedharan S, Mian M, Robertson RA, Yang N. The top 100 most cited articles in medical artificial intelligence: a bibliometric analysis. J Med Artif Intell 2020;3:3.