Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter
  • Published:

Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence

Abstract

Artificial intelligence (AI)-based methods have emerged as powerful tools to transform medical care. Although machine learning classifiers (MLCs) have already demonstrated strong performance in image-based diagnoses, analysis of diverse and massive electronic health record (EHR) data remains challenging. Here, we show that MLCs can query EHRs in a manner similar to the hypothetico-deductive reasoning used by physicians and unearth associations that previous statistical methods have not found. Our model applies an automated natural language processing system using deep learning techniques to extract clinically relevant information from EHRs. In total, 101.6 million data points from 1,362,559 pediatric patient visits presenting to a major referral center were analyzed to train and validate the framework. Our model demonstrates high diagnostic accuracy across multiple organ systems and is comparable to experienced pediatricians in diagnosing common childhood diseases. Our study provides a proof of concept for implementing an AI-based system as a means to aid physicians in tackling large amounts of data, augmenting diagnostic evaluations, and to provide clinical decision support in cases of diagnostic uncertainty or complexity. Although this impact may be most evident in areas where healthcare providers are in relative shortage, the benefits of such an AI system are likely to be universal.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Workflow diagram of our AI pediatric diagnosis framework.
Fig. 2: Hierarchy of the diagnostic framework in a large pediatric cohort.

Similar content being viewed by others

Data availability

We have made available the Jupyter notebook that we used in constructing and validating the hierarchical logistic regression models: https://s3.cn-north-1.amazonaws.com.cn/ped.emr/Data/hierachical_logistic_regression.ipynb. To protect patient confidentiality, we have deposited de-identified aggregated patient data in a secured and patient confidentiality compliant cloud in China in concordance with data security regulations. Data access can be requested by writing to the corresponding authors. All data access requests will be reviewed and (if successful) granted by the Data Access Committee.

References

  1. Hu, J., Perer, A. & Wang, F. Data Driven Analytics for Personalized Healthcare. (Springer Internatonal Publishing, Switzerland, Healthcare Information Management Systems: Cases, Strategies, and Solutions, 2016).

  2. Nezhad, M.Z., Zhu, D.X., Sadati, N., Yang, K. & Levy, P. SUBIC: A supervised bi-clustering approach for precision medicine. 2017 16th Ieee International Conference on Machine Learning and Applications (Icmla). Preprint at https://arxiv.org/pdf/1709.09929.pdf (2017).

  3. Hornberger, J. Electronic health records: a guide for clinicians and administrators. JAMA 301, 110–110 (2009).

    Article  CAS  Google Scholar 

  4. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

    Article  CAS  Google Scholar 

  5. Kermany, D. S. et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172, 1122–1131 (2018).

    Article  CAS  Google Scholar 

  6. Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).

    Article  Google Scholar 

  7. Erickson, B. J., Korfiatis, P., Akkus, Z. & Kline, T. L. Machine learning for medical imaging. Radiographics 37, 505–515 (2017).

    Article  Google Scholar 

  8. Wang, F., Zhang, P., Qian, B., Wang, X. & Davidson, I. Clinical risk prediction with multilinear sparse logistic regression. In Proc. 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.145–154 (2014).

  9. Turchin, A. et al. Using regular expressions to abstract blood pressure and treatment intensification information from the text of physician notes. J. Am. Med. Inform. Assoc. 13, 691–695 (2006).

    Article  Google Scholar 

  10. Halevy, A., Norvig, P. & Pereira, F. The unreasonable effectiveness of data. IEEE Intelligent Systems 24, 8–12 (2009).

    Article  Google Scholar 

  11. Banko, M. & Brill, E. Scaling to very very large corpora for natural language disambiguation. In Proc. 39th Annual Meeting Association for Computational Linguistics. 26–33 (Association for Computational Linguistics, Stroudsburg, 2001).

  12. Tsui, B. Y., et al. Creating a scalable deep learning based named entity recognition model for biomedical textual data by repurposing biosample free-text annotations. Preprint at https://www.biorxiv.org/content/biorxiv/early/2018/09/12/414136.full.pdf (2018).

  13. Rajkomar, A. et al. Scalable and accurate deep learning with electronic health records. NPJ Digital Medicine 1, 18 (2018).

    Article  Google Scholar 

  14. Wilkinson, M. D. et al. Comment: the fair guiding principles for scientific data management and stewardship. Sci. Data 3, 160018 (2016).

    Article  Google Scholar 

  15. Liang, Y., Chen, Z., Huang, X. & Zeng, L. Analysis of the disease spectrum of hospitalized children in guangdong province. Chin. Med. J. (Engl) 1, 414–418 (2013).

    Google Scholar 

  16. WHO. International Statistical Classification of Diseases and Related Health Problems. (World Health Organization, 2004).

  17. English–Chinese Medical Dictionary (英汉医学大词典) (Shanghai Scientific and Technical Publishers (上海科学技术出版社), 2015).

  18. Lindberg, D. A. B., Humphreys, B. L. & Mccray, A. T. The unified medical language system. Methods Inf. Med. 32, 281–291 (1993).

    Article  CAS  Google Scholar 

  19. Tweedie, F. J., Singh, S. & Holmes, D. I. Neural network applications in stylometry: the federalist papers. Computers and the Humanities 30, 1–10 (1996).

    Article  Google Scholar 

  20. Luong, M.-T., Pham, H. & Manning, C. D. Effective approaches to attention-based neural machine translation. Preprint at https://arxiv.org/abs/1508.04025 (2015).

  21. Lipton, Z.C., Kale, D.C. & Wetzel, R.C. Phenotyping of clinical time series with LSTM recurrent neural networks. Preprint at https://arxiv.org/pdf/1510.07641.pdf (2015).

  22. Peng, X.B., Andrychowicz, M., Zaremba, W. & Abbeel, P. Sim-to-real transfer of robotic control with dynamics randomization. IEEE International Conference on Robotics and Automation (ICRA) 3803–3810 (2018).

  23. Graves, A. & Schmidhuber, J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks 18, 602–610 (2005).

    Article  Google Scholar 

  24. Yeung, K.Y. & Ruzzo, W.L. Details of the adjusted rand index and clustering algorithms supplement to the paper ‘an empirical study on Principal Component Analysis for clustering gene expression data. Available at http://faculty.washington.edu/kayee/pca/supp.pdf (2011).

Download references

Acknowledgements

This study was funded by the National Key Research and Development Program of China (2017YFC1104600 to H.L.), National Natural Science Foundation of China (81771629 to H.X. and 81700882 to J.X.), Guangzhou Women and Children’s Medical Center, Guangzhou Regenerative Medicine and Health Guangdong Laboratory (Innovation and Startup Talents Program 2018GZR031001 to L.Z. and R.H.).

Author information

Authors and Affiliations

Authors

Contributions

H.L., B.T., H.N., W.C., S.L.B., G. Liu, D.S.K., X. S., C.C.S.V., P.T., H.S., J.C., L. H., J.Z., L.Z., R.H., S.H., G. Li, P.L., X.Z., Z.Z., L.P., H.C., R.L., S.L., Y.C., S.T., H.Y., X.H., W. He, W.L., Q.Z., J.J., W.Y., J.G., W.O., Y. Deng, Q.H., B. Wang, C.Y., Y.L., S.Z., Y. Duan, R.Z., S.G., C.L.Z., O.L., E.D.Z., G.K., X.W., C.W., N.N., J.X., W.X., B. Wang, W.W., J.L., B.P., C.B., D.X., W. He, S.H., Y.Z., W. Haw, M.G., A.T., C.-N.H., H.C., L.Z., H.X. and K.Z. collected and analyzed the data. X.H. and K.Z. conceived the project. K.Z., S.L.B., B.T., H.L., and H.X. wrote the manuscript. All authors discussed the results and reviewed the manuscript.

Corresponding authors

Correspondence to Kang Zhang or Huimin Xia.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data 1 Unsupervised clustering of NLP extracted textual features from pediatric diseases.

The diagnostic system analyzed the EHRs in the absence of a defined classification system. This grouping structure reflects the detection of trends in clinical features without pre-defined labeling or human input. The clustered blocks are marked with the boxes with grey lines.

Extended Data 2 Design of the natural language processing information extraction model.

Segmented sentences from the raw text of the EHR were embedded using word2vec. The LSTM model then generated the structured records in a query–answer format. This schematic illustrates the process using the free-text ‘lesion in the upper left lobe of patient’s lung’ as an example.

Supplementary Information

Reporting Summary

Supplementary Tables

Supplementary Tables 1–9

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, H., Tsui, B.Y., Ni, H. et al. Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence. Nat Med 25, 433–438 (2019). https://doi.org/10.1038/s41591-018-0335-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-018-0335-9

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing