Techniques for Collaborative Development of AI Models in the Age of COVID

Nat Med. 2021 Oct;27(10):1735-1743. doi: 10.1038/s41591-021-01506-3. Epub 2021 Sep 15. | PubMed

Nat Commun. 2020 Aug 14;11(1):4080. doi: 10.1038/s41467-020-17971-2. | PubMed

Res Sq. 2021 Jun 4;rs.3.rs-571332. doi: 10.21203/rs.3.rs-571332/v1. Preprint | PubMed

Med Image Anal. 2021 May;70:101992. doi: 10.1016/j.media.2021.101992. Epub 2021 Feb 6. | PubMed

arXiv:2104.10195

Ittai Dayan¹, Holger R Roth², Aoxiao Zhong^3,4, Ahmed Harouni², Amilcare Gentili⁵, Anas Z Abidin², Andrew Liu², Anthony Beardsworth Costa⁶, Bradford J Wood^7,8, Chien-Sung Tsai⁹, Chih-Hung Wang^10,11, Chun-Nan Hsu¹², C K Lee², Peiying Ruan², Daguang Xu², Dufan Wu³, Eddie Huang², Felipe Campos Kitamura¹³, Griffin Lacey², Gustavo César de Antônio Corradi¹³, Gustavo Nino¹⁴, Hao-Hsin Shin¹⁵, Hirofumi Obinata¹⁶, Hui Ren³, Jason C Crane¹⁷, Jesse Tetreault², Jiahui Guan², John W Garrett¹⁸, Joshua D Kaggie¹⁹, Jung Gil Park²⁰, Keith Dreyer^1,21, Krishna Juluru¹⁵, Kristopher Kersten², Marcio Aloisio Bezerra Cavalcanti Rockenbach²¹, Marius George Linguraru^22,23, Masoom A Haider^24,25, Meena AbdelMaseeh²⁵, Nicola Rieke², Pablo F Damasceno¹⁷, Pedro Mario Cruz E Silva², Pochuan Wang^26,27, Sheng Xu^7,8, Shuichi Kawano¹⁶, Sira Sriswasdi^28,29, Soo Young Park³⁰, Thomas M Grist³¹, Varun Buch²¹, Watsamon Jantarabenjakul^32,33, Weichung Wang^26,27, Won Young Tak³⁰, Xiang Li³, Xihong Lin³⁴, Young Joon Kwon⁶, Abood Quraini², Andrew Feng², Andrew N Priest³⁵, Baris Turkbey^8,36, Benjamin Glicksberg³⁷, Bernardo Bizzo²¹, Byung Seok Kim³⁸, Carlos Tor-Díez²², Chia-Cheng Lee³⁹, Chia-Jung Hsu³⁹, Chin Lin^40,41,42, Chiu-Ling Lai⁴³, Christopher P Hess¹⁷, Colin Compas², Deepeksha Bhatia², Eric K Oermann⁴⁴, Evan Leibovitz²¹, Hisashi Sasaki¹⁶, Hitoshi Mori¹⁶, Isaac Yang², Jae Ho Sohn¹⁷, Krishna Nand Keshava Murthy¹⁵, Li-Chen Fu⁴⁵, Matheus Ribeiro Furtado de Mendonça¹³, Mike Fralick⁴⁶, Min Kyu Kang²⁰, Mohammad Adil², Natalie Gangai¹⁵, Peerapon Vateekul⁴⁷, Pierre Elnajjar¹⁵, Sarah Hickman¹⁹, Sharmila Majumdar¹⁷, Shelley L McLeod^48,49, Sheridan Reed^7,8, Stefan Gräf⁵⁰, Stephanie Harmon^8,51, Tatsuya Kodama¹⁶, Thanyawee Puthanakit^32,33, Tony Mazzulli^52,53,54, Vitor Lima de Lavor¹³, Yothin Rakvongthai⁵⁵, Yu Rim Lee³⁰, Yuhong Wen², Fiona J Gilbert¹⁹, Mona G Flores⁵⁶, Quanzheng Li³

1. MGH Radiology and Harvard Medical School, Boston, MA, USA.
2. NVIDIA, Santa Clara, CA, USA.
3. Center for Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
4. School of Engineering and Applied Sciences, Harvard University, Boston, MA, USA.
5. San Diego VA Health Care System, San Diego, CA, USA.
6. Department of Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
7. Radiology & Imaging Sciences/Clinical Center, National Institutes of Health, Bethesda, MD, USA.
8. National Cancer Institute, National Institutes of Health, Bethesda, MD, USA.
9. Division of Cardiovascular Surgery, Department of Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan.
10. Department of Otolaryngology-Head and Neck Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan.
11. Graduate Institute of Medical Sciences, National Defense Medical Center, Taipei, Taiwan.
12. Center for Research in Biological Systems, University of California, San Diego, CA, USA.
13. DasaInova, Diagnósticos da América SA, Barueri, Brazil.
14. Division of Pediatric Pulmonary and Sleep Medicine, Children's National Hospital, Washington, DC, USA.
15. Memorial Sloan Kettering Cancer Center, New York, NY, USA.
16. Self-Defense Forces Central Hospital, Tokyo, Japan.
17. Center for Intelligent Imaging, Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA.
18. Departments of Radiology and Medical Physics, The University of Wisconsin-Madison School of Medicine and Public Health, Madison, WI, USA.
19. Department of Radiology, NIHR Cambridge Biomedical Resource Centre, University of Cambridge, Cambridge, UK.
20. Department of Internal Medicine, Yeungnam University College of Medicine, Daegu, South Korea.
21. Center for Clinical Data Science, Massachusetts General Brigham, Boston, MA, USA.
22. Sheikh Zayed Institute for Pediatric Surgical Innovation, Children's National Hospital, Washington, DC, USA.
23. Departments of Radiology and Pediatrics, The George Washington University School of Medicine and Health Sciences, Washington, DC, USA.
24. Joint Dept. of Medical Imaging, Sinai Health System, University of Toronto, Toronto, Ontario, Canada.
25. Lunenfeld-Tanenbaum Research Institute, Toronto, Ontario, Canada.
26. MeDA Lab Institute of Applied Mathematical Sciences, National Taiwan University, Taipei, Taiwan.
27. Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan.
28. Research Affairs, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand.
29. Center for Artificial Intelligence in Medicine, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand.
30. Department of Internal Medicine, School of Medicine, Kyungpook National University, Daegu, South Korea.
31. Departments of Radiology, Medical Physics, and Biomedical Engineering, The University of Wisconsin-Madison School of Medicine and Public Health, Madison, WI, USA.
32. Department of Pediatrics, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand.
33. Thai Red Cross Emerging Infectious Diseases Clinical Center, King Chulalongkorn Memorial Hospital, Bangkok, Thailand.
34. Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA.
35. Department of Radiology, NIHR Cambridge Biomedical Resource Centre, Cambridge University Hospital, Cambridge, UK.
36. Department of Radiology and Imaging Sciences, National Institutes of Health, Bethesda, MD, USA.
37. Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai and Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
38. Department of Internal Medicine, Catholic University of Daegu School of Medicine, Daegu, South Korea.
39. Planning and Management Office, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan.
40. School of Medicine, National Defense Medical Center, Taipei, Taiwan.
41. School of Public Health, National Defense Medical Center, Taipei, Taiwan.
42. Graduate Institute of Life Sciences, National Defense Medical Center, Taipei, Taiwan.
43. Medical Review and Pharmaceutical Benefits Division, National Health Insurance Administration, Taipei, Taiwan.
44. Department of Neurosurgery, NYU Grossman School of Medicine, New York, NY, USA.
45. MOST/NTU All Vista Healthcare Center, Center for Artificial Intelligence and Advanced Robotics, National Taiwan University, Taipei, Taiwan.
46. Division of General Internal Medicine and Geriatrics (Fralick), Sinai Health System, Toronto, Ontario, Canada.
47. Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok, Thailand.
48. Schwartz/Reisman Emergency Medicine Institute, Sinai Health, Toronto, Ontario, Canada.
49. Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada.
50. Department of Medicine and NIHR BioResource for Translational Research, NIHR Cambridge Biomedical Research Centre, University of Cambridge, Cambridge, UK.
51. Clinical Research Directorate, Frederick National Laboratory for Cancer, National Cancer Institute, Frederick, MD, USA.
52. Department of Microbiology, Sinai Health/University Health Network, Toronto, Ontario, Canada.
53. Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada.
54. Public Health Ontario Laboratories, Toronto, Ontario, Canada.
55. Chulalongkorn University Biomedical Imaging Group and Division of Nuclear Medicine, Department of Radiology, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand.
56. NVIDIA, Santa Clara, CA, USA. mflores@nvidia.com.

Abstract

Federated learning (FL) is a method used for training artificial intelligence models with data from multiple sources while maintaining data anonymity, thus removing many barriers to data sharing. Here we used data from 20 institutes across the globe to train a FL model, called EXAM (electronic medical record (EMR) chest X-ray AI model), that predicts the future oxygen requirements of symptomatic patients with COVID-19 using inputs of vital signs, laboratory data and chest X-rays. EXAM achieved an average area under the curve (AUC) >0.92 for predicting outcomes at 24 and 72 h from the time of initial presentation to the emergency room, and it provided 16% improvement in average AUC measured across all participating sites and an average increase in generalizability of 38% when compared with models trained at a single site using that site's data. For prediction of mechanical ventilation treatment or death at 24 h at the largest independent test site, EXAM achieved a sensitivity of 0.950 and specificity of 0.882. In this study, FL facilitated rapid data science collaboration without data exchange and generated a model that generalized across heterogeneous, unharmonized datasets for prediction of clinical outcomes in patients with COVID-19, setting the stage for the broader use of FL in healthcare.

Presented By Holger Roth | ORCID iD