Advertisement
Canadian Journal of Cardiology

The Role of Machine Learning in Cardiovascular Pathology

  • Author Footnotes
    ‡ These authors contributed equally to this work.
    Carolyn Glass
    Correspondence
    Corresponding author: Dr Carolyn Glass, 217AM Davison Building, Box 3712, 40 Duke Medicine Circle, Department of Pathology, Duke University Medical Center, Durham, North Carolina 27710, USA.
    Footnotes
    ‡ These authors contributed equally to this work.
    Affiliations
    Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA

    Department of Pathology, Duke University Medical Center, Durham, North Carolina, USA
    Search for articles by this author
  • Author Footnotes
    ‡ These authors contributed equally to this work.
    Kyle J. Lafata
    Footnotes
    ‡ These authors contributed equally to this work.
    Affiliations
    Department of Radiology, Duke University School of Medicine, Durham, North Carolina, USA

    Department of Radiation Oncology, Duke University School of Medicine, Durham, North Carolina, USA

    Department of Electrical and Computer Engineering, Duke Pratt School of Engineering, Duke University, Durham, North Carolina, USA
    Search for articles by this author
  • William Jeck
    Affiliations
    Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA

    Department of Pathology, Duke University Medical Center, Durham, North Carolina, USA
    Search for articles by this author
  • Roarke Horstmeyer
    Affiliations
    Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA

    Department of Biomedical Engineering, Duke Pratt School of Engineering, Durham, North Carolina, USA
    Search for articles by this author
  • Colin Cooke
    Affiliations
    Department of Electrical and Computer Engineering, Duke Pratt School of Engineering, Duke University, Durham, North Carolina, USA
    Search for articles by this author
  • Jeffrey Everitt
    Affiliations
    Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA

    Department of Pathology, Duke University Medical Center, Durham, North Carolina, USA
    Search for articles by this author
  • Matthew Glass
    Affiliations
    Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA

    Department of Anesthesiology, Duke University Medical Center, Durham, North Carolina, USA
    Search for articles by this author
  • David Dov
    Affiliations
    Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA

    Tel-Aviv Sourasky Medical Center, Tel Aviv, Israel
    Search for articles by this author
  • Michael A. Seidman
    Affiliations
    Laboratory Medicine Program, University Health Network, Toronto, Ontario, Canada

    Department of Laboratory Medicine and Pathobiology, University of Toronto, Ontario, Canada
    Search for articles by this author
  • Author Footnotes
    ‡ These authors contributed equally to this work.
Published:November 19, 2021DOI:https://doi.org/10.1016/j.cjca.2021.11.008

      Abstract

      Machine learning has seen slow but steady uptake in diagnostic pathology over the past decade to assess digital whole-slide images. Machine learning tools have incredible potential to standardise, and likely even improve, histopathologic diagnoses, but they are not yet widely used in clinical practice. We describe the principles of these tools and technologies and some successful preclinical and pretranslational efforts in cardiovascular pathology, as well as a roadmap for moving forward. In nonhuman animal models, one proof-of-principle application is in rodent progressive cardiomyopathy, which is of particular significance to drug toxicity studies. Basic science successes include screening the quality of differentiated stem cells and characterising cardiomyocyte developmental stages, with potential applications for research and toxicology/drug safety screening using derived or native human pluripotent stem cells differentiated into cardiomyocytes. Translational studies of particular note include those with success in diagnosing the various forms of heart allograft rejection. For fully realising the value of these tools in clinical cardiovascular pathology, we identify 3 essential challenges. First is image quality standardisation to ensure that algorithms can be developed and implemented on robust, consistent data. The second is consensus diagnosis; experts don’t always agree, and thus “truth” may be difficult to establish, but the algorithms themselves may provide a solution. The third is the need for large-enough data sets to facilitate robust algorithm development, necessitating large cross-institutional shared image databases. The power of histopathology-based machine learning technologies is tremendous, and we outline the next steps needed to capitalise on this power.

      Résumé

      Au cours de la dernière décennie, l’apprentissage automatique a connu une adoption lente, mais constante, dans l’analyse des images numérisées de lamelle entière pour le diagnostic de maladies. Les outils d’apprentissage automatique ont un fabuleux potentiel de standardisation, voire d’amélioration, des diagnostics histopathologiques; ils ne sont toutefois pas encore largement utilisés en pratique clinique. Cet article vise à expliquer les principes de ces outils et technologies et à présenter certaines expériences précliniques et prétranslationnelles fructueuses en pathologie cardiovasculaire, de même qu’une feuille de route pour aller de l’avant. Dans les modèles animaux non humains, la cardiomyopathie évolutive chez le rongeur constitue une démonstration de principe qui revêt une importance particulière pour le succès des études toxicologiques. Les réussites en sciences fondamentales comprennent la sélection de cellules souches différenciées de qualité et la caractérisation des étapes du développement des cardiomyocytes, de même que leurs applications potentielles en recherche et en analyse toxicologique/de l’innocuité des médicaments à l’aide de cellules souches pluripotentes humaines ou dérivées se différenciant en cardiomyocytes. Les études translationnelles notables comprennent celles ayant permis de diagnostiquer les diverses formes de rejet d’allogreffe cardiaque. Pour pleinement comprendre la valeur de ces outils en pathologie clinique cardiovasculaire, nous avons repéré trois défis essentiels. Le premier est la standardisation de la qualité des images pour garantir que les algorithmes soient élaborés et mis en œuvre à partir de données solides et cohérentes. Le second est le diagnostic consensuel; les experts ne sont pas toujours d’accord, et par conséquent, il peut être difficile d’établir la « vérité », mais les algorithmes pourraient eux-mêmes fournir une solution. Le troisième est la nécessité de constituer des ensembles de données suffisantes pour favoriser l’élaboration d’algorithmes solides, ce qui nécessite de vastes bases de données d’images partagées entre les établissements. La puissance des technologies d’apprentissage automatique fondées sur l’histopathologie est immense; nous décrirons les prochaines étapes qui nous permettront de tirer profit de leur puissance.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Canadian Journal of Cardiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Dov D.
        • Kovalsky S.Z.
        • Assaad S.
        • et al.
        Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images.
        Med Image Anal. 2021; 67: 101814
        • Elliott Range D.D.
        • Dov D.
        • Kovalsky S.Z.
        • et al.
        Application of a machine learning algorithm to predict malignancy in thyroid cytopathology.
        Cancer Cytopathol. 2020; 128: 287-295
        • Janowczyk A.
        • Madabhushi A.
        Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases.
        J Pathol Inform. 2016; 7: 29
        • Coudray N.
        • Ocampo P.S.
        • Sakellaropoulos T.
        • et al.
        Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning.
        Nat Med. 2018; 24: 1559-1567
        • Goldenberg S.L.
        • Nir G.
        • Salcudean S.E.
        A new era: artificial intelligence and machine learning in prostate cancer.
        Nat Rev Urol. 2019; 16: 391-403
        • Osareh A.
        • Shadgar B.
        Machine learning techniques to diagnose breast cancer. 2010 5th International Symposium on Health Informatics and Bioinformatics (HIBIT), Ankara, Turkey, April 20-22, 2010.
        IEEE, 2010: 114-120 (Available at: https://doi.org/10.1109/HIBIT.2010.5478895. Accessed October 19, 2021.)
        • Lee G.
        • Sparks R.
        • Ali S.
        • et al.
        Co-occurring gland angularity in localized subgraphs: predicting biochemical recurrence in intermediate-risk prostate cancer patients.
        PLoS One. 2014; 9e97954
        • Saltz J.
        • Gupta R.
        • Hou L.
        • et al.
        Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images.
        Cell Rep. 2018; 23: 181-193.e7
        • Heindl A.
        • Sestak I.
        • Naidoo K.
        • et al.
        Relevance of spatial heterogeneity of immune infiltration for predicting risk of recurrence after endocrine therapy of ER+ breast cancer.
        J Natl Cancer Inst. 2018; 110: 166-175
        • Ali S.
        • Lewis J.
        • Madabhushi A.
        Spatially aware cell cluster(spACC1) graphs: predicting outcome in oropharyngeal pl6+ tumors, Medical image computing and computer-assisted intervention.
        Med Image Comput Comput Assist Interv. 2013; 16: 412-419
        • Ehteshami Bejnordi B.
        • Veta M.
        • Johannes van Diest P.
        • et al.
        Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.
        JAMA. 2017; 318: 2199-2210
        • Silva L.M.
        • Pereira E.M.
        • Salles P.G.
        • et al.
        Independent real-world application of a clinical-grade automated prostate cancer detection system.
        J Pathol. 2021; 254: 147-158
        • US Food and Drug Administration
        FDA authorizes software that can help identify prostate cancer. FDA news release, September 21, 2021.
        (Available at:) (Accessed October 19, 2021)
        • Campanella G.
        • Hanna M.G.
        • Geneslaw L.
        • et al.
        Clinical-grade computational pathology using weakly supervised deep learning on whole slide images.
        Nat Med. 2019; 25: 1301-1309
        • Pantanowitz L.
        • Quiroga-Garza G.M.
        • Bien L.
        • et al.
        An artificial intelligence algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies: a blinded clinical validation and deployment study.
        Lancet Digit Health. 2020; 2: e407-e416
        • Wong G.L.
        • Yuen P.
        • Ma A.J.
        • et al.
        Artificial intelligence in prediction of non-alcoholic fatty liver disease and fibrosis.
        J Gastroenterol Hepatol. 2021; 36: 543-550
        • Barisoni L.
        • Lafata K.J.
        • Hewitt S.M.
        • Madabhushi A.
        • Balis U.G.J.
        Digital pathology and computational image analysis in nephropathology.
        Nat Rev Nephrol. 2020; 16: 669-685
        • Hermsen M.
        • de Bel T.
        • den Boer M.
        • et al.
        Deep learning–based histopathologic assessment of kidney tissue.
        J Am Soc Nephrol. 2019; 30: 1968-1979
        • van der Laak J.A.
        • Litjens G.J.S.
        • Ciompi F.
        Deep learning in histopathology: the path to the clinic.
        Nat Med. 2021; 27: 775-784
        • García-Rojo M.
        • de Mena D.
        • Muriel-Cueto P.
        • et al.
        New European Union regulations related to whole slide image scanners and image analysis software.
        J Pathol Inform. 2019; 10: 2
        • Herrmann M.
        • Clunie D.
        • Fedorov A.
        • et al.
        Implementing the DICOM standard for digital pathology.
        J Pathol Inform. 2018; 9: 37
        • Yang H.
        • Lai C.K.
        • Gjertson D.W.
        • et al.
        Has the 2004 revision of the International Society of Heart and Lung Transplantation grading system improved the reproducibility of the diagnosis and grading of cardiac transplant rejection?.
        Cardiovasc Pathol. 2009; 18: 198-204
        • Crespo-Leiro M.G.
        • Zuckermann A.
        • Bara C.
        • et al.
        Concordance among pathologists in the second Cardiac Allograft Rejection Gene Expression Observational Study (CARGO II).
        Transplantation. 2012; 94: 1172-1177
        • Bulten W.
        • Balkenhol M.C.
        • Belinga J.A.
        • et al.
        Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists.
        Mod Pathol. 2021; 34: 660-671
        • Tizhoosh H.R.
        • Diamandis P.
        • Campbell C.J.V.
        • et al.
        Searching images for consensus: can AI remove observer variability in pathology?.
        Am J Pathol. 2021; 191: 1702-1708
        • Neitzel U.
        • Maack I.
        • Günther-Kohfahl S.
        Image quality of a digital chest radiography system based on a selenium detector.
        Med Phys. 1994; 21: 509-516
        • Antonuk L.E.
        • Yorkston J.
        • Huang W.
        • et al.
        A real-time, flat-panel, amorphous silicon, digital x-ray imager.
        Radiographics. 1995; 15: 993-1000
        • Zhao W.
        • Rowlands J.A.
        X-ray imaging using amorphous selenium: feasibility of a flat panel self-scanned detector for digital radiology.
        Med Phys. 1995; 22: 1595-1604
        • Kandarakis I.
        • Cavouras D.
        • Panayiotakis G.S.
        • Nomicos C.D.
        Evaluating x-ray detectors for radiographic applications: a comparison of ZnSCdS:Ag with Gd2O2S:Tb and Y2O2S:Tb screens.
        Phys Med Biol. 1997; 42: 1351-1373
        • Zhang W.
        • Doi K.
        • Giger M.L.
        • et al.
        Computerized detection of clustered microcalcifications in digital mammograms using a shift-invariant artificial neural network.
        Med Phys. 1994; 21: 517-524
        • Lafata K.J.
        • Wang Y.
        • Konkel B.
        • Yin F.F.
        • Bashir M.R.
        Radiomics: a primer on high-throughput image phenotyping [e-pub ahead of print].
        Abdom Radiol (NY). 2021 Aug 25; https://doi.org/10.1007/s00261-021-03254-x
        • Chang Y.
        • Lafata K.
        • Wang C.
        • et al.
        Digital phantoms for characterizing inconsistencies among radiomics extraction toolboxes.
        Biomed Phys Eng Express. 2020; 6025016
        • Chang Y.
        • Lafata K.
        • Segars W.P.
        • Yin F.
        • Ren L.
        Development of realistic multi-contrast textured XCAT (MT-XCAT) phantoms using a dual-discriminator conditional-generative adversarial network (D-CGAN).
        Phys Med Biol. 2020; 65065009
        • Jaggi A.
        • Mattonen S.A.
        • McNitt-Gray M.
        • Napel S.
        Stanford DRO Toolkit: digital reference objects for standardization of radiomic features.
        Tomography. 2020; 6: 111-117
        • Zwanenburg A.
        • Vallières M.
        • Abdalah M.A.
        • et al.
        The Image Biomarker Standardization Initiative: standardized quantitative radiomics for high-throughput image-based phenotyping.
        Radiology. 2020; 295: 328-338
        • Janowczyk A.
        • Zuo R.
        • Gilmore H.
        • Feldman M.
        • Madabhushi A.
        HistoQC: an open-source quality control tool for digital pathology slides.
        JCO Clin Cancer Inform. 2019; 3: 1-7
        • Gennaro G.
        The “perfect” reader study.
        Eur J Radiol. 2018; 103: 139-146
        • Lafata K.
        • Zhou Z.
        • Liu J.
        • Yin F.
        Data clustering based on Langevin annealing with a self-consistent potential.
        Q Appl Math. 2018; 77: 591-613
        • Lafata K.J.
        • Chang Y.
        • Wang C.
        • et al.
        Intrinsic radiomic expression patterns after 20 Gy demonstrate early metabolic response of oropharyngeal cancers.
        Med Phys. 2021; 48: 3767-3777
        • Lafata K.J.
        • Corradetti M.N.
        • Gao J.
        • et al.
        Radiogenomic analysis of locally advanced lung cancer based on CT imaging and intratreatment changes in cell-free DNA.
        Radiol Imaging Cancer. 2021; 3e200157
        • Fu X.
        • Liu T.
        • Xiong Z.
        • et al.
        Segmentation of histological images and fibrosis identification with a convolutional neural network.
        Comput Biol Med. 2018; 98: 147-158
        • Turner O.C.
        • Aeffner F.
        • Bangari D.S.
        • et al.
        Society of Toxicologic Pathology Digital Pathology and Image Analysis Special Interest Group article: Opinion on the application of artificial intelligence and machine learning to digital toxicologic pathology.
        Toxicol Pathol. 2020; 48: 277-294
        • Berridge B.R.
        • Hoffmann P.
        • Turk J.R.
        • et al.
        Integrated and translational nonclinical in vivo cardiovascular risk assessment: Gaps and opportunities.
        Regul Toxicol Pharmacol. 2013; 65: 38-46
        • Hailey J.R.
        • Maleeff B.E.
        • Thomas H.C.
        • et al.
        A diagnostic approach for rodent progressive cardiomyopathy and like lesions in toxicology studies up to 28 days in the Sprague Dawley rat (part 1 of 2).
        Toxicol Pathol. 2017; 45: 1043-1054
        • Hailey J.R.
        • Maleeff B.E.
        • Thomas H.C.
        • et al.
        A diagnostic approach for rodent progressive cardiomyopathy and like lesions in toxicology studies up to 28 days in the Sprague Dawley rat (part 2 of 2).
        Toxicol Pathol. 2017; 45: 1055-1066
        • Tokarz D.A.
        • Steinbach T.J.
        • Lokhande A.
        • et al.
        Using artificial intelligence to detect, classify, and objectively score severity of rodent cardiomyopathy.
        Toxicol Pathol. 2021; 49: 888-896
        • Orita K.
        • Sawada K.
        • Koyama R.
        • Ikegaya Y.
        Deep learning–based quality control of cultured human-induced pluripotent stem cell–derived cardiomyocytes.
        J Pharm Sci. 2019; 140: 313-316
        • Gorospe G.
        • Zhu R.
        • Millrod M.A.
        • et al.
        Automated grouping of action potentials of human embryonic stem cell–derived cardiomyocytes.
        IEEE Trans Biomed Eng. 2014; 61: 2389-2395
        • Skrabanek P.
        • Zahradnikova A.
        Automatic assessment of the cardiomyocyte development stages from confocal microscopy images using deep convolutional networks.
        PLoS One. 2019; 14e0216720
        • Nirschl J.J.
        • Janowczyk A.
        • Peyster E.G.
        • et al.
        A deep-learning classifier identifies patients with clinical heart failure using whole-slide images of H&E tissue.
        PLoS One. 2018; 13e0192726
        • Peyster E.
        • Madabhushi A.
        • Margulies K.
        Advanced morphologic analysis for diagnosing allograft rejection: the case of cardiac transplant rejection.
        Transplantation. 2018; 102: 1230-1239
        • Peyster E.G.
        • Arabyarmohammadi S.
        • Janowczyk A.
        • et al.
        An automated computational image analysis pipeline for histological grading of cardiac allograft rejection.
        Eur Heart J. 2021; 42: 2356-2369
        • Stewart S.
        • Winters G.L.
        • Fishbein M.C.
        • et al.
        Revision of the 1990 Working Formulation for the Standardization of Nomenclature in the Diagnosis of Heart Rejection.
        J Heart Lung Transplant. 2005; 24: 1710-1720
        • Glass C.
        • Davis R.
        • Xiong B.
        • Dov D.
        • Glass M.
        The use of artificial intelligence (AI) machine learning to determine myocyte damage in cardiac transplant acute cellular rejection.
        J Heart Lung Transplant. 2020; 39: S59
        • Davis H.
        • Carney J.
        • Dibernardo l.
        • et al.
        Detecting cardiac transplant antibody mediated rejection by artificial intelligence: a novel deep learning approach.
        Mod Pathol. 2020; 39: S501-S502
        • Colvin M.
        • Cook J.
        • Chang P.
        • et al.
        Antibody-mediated rejection in cardiac transplantation: emerging knowledge in diagnosis and management: a scientific statement from the American Heart Association.
        Circulation. 2015; 131: 1608-1639
        • Cooper L.T.
        • Baughman K.L.
        • Feldman A.M.
        • et al.
        The role of endomyocardial biopsy in the management of cardiovascular disease: a scientific statement from the American Heart Association, the American College of Cardiology, and the European Society of Cardiology. Endorsed by the Heart Failure Society of America and the Heart Failure Association of the European Society of Cardiology.
        J Am Coll Cardiol. 2007; 50: 1914-1931
        • Giarraputo A.
        • Barison I.
        • Fedrigo M.
        • et al.
        A changing paradigm in heart transplantation: an integrative approach for invasive and non-invasive allograft rejection monitoring.
        Biomolecules. 2021; 11: 201
        • Deng M.C.
        • Eisen H.J.
        • Mehra M.R.
        • et al.
        Noninvasive discrimination of rejection in cardiac allograft recipients using gene expression profiling.
        Am J Transplant. 2006; 6: 150-160
        • Crespo-Leiro M.G.
        • Stypmann J.
        • Schulz U.
        • et al.
        Clinical usefulness of gene-expression profile to rule out acute rejection after heart transplantation: CARGO II.
        Eur Heart J. 2016; 37: 2591-2601
        • Castellani C.
        • Burrello J.
        • Fedrigo M.
        • et al.
        Circulating extracellular vesicles as noninvasive biomarker of rejection in heart transplant.
        J Heart Lung Transplant. 2020; 39: 1136-1148
        • CareDx
        Product pipeline: innovation is coming.
        (Available at:)
        https://caredx.com/company/pipeline/
        Date accessed: October 19, 2021
        • Stewart B.J.
        • Clatworthy M.R.
        Applying single-cell technologies to clinical pathology: progress in nephropathology.
        J Pathol. 2020; 250: 693-704
        • Aichler M.
        • Walch A.
        MALDI imaging mass spectrometry: current frontiers and perspectives in pathology research and practice.
        Lab Invest. 2015; 95: 422-431
        • Shilo S.
        • Rossman H.
        • Segal E.
        Axes of a revolution: challenges and promises of big data in healthcare.
        Nat Med. 2020; 26: 29-38
        • Moulin P.
        • Grünberg K.
        • Barale-Thomas E.
        • van der Laak J.A.W.M.
        IMI-Bigpicture: a central repository for digital pathology.
        Toxicol Pathol. 2021; 49: 711-713
        • Innovative Medicines Initiative
        BIGPICTURE: central repository for digital pathology.
        (Available at:)
        • Association for Molecular Pathology Clinical Practice Committee
        Molecular diagnostic assay validation. October 2009.
        (Available at:) (Accessed October 19, 2021)