Canadian Journal of Cardiology

Health Technology Assessment for Cardiovascular Digital Health Technologies and Artificial Intelligence: Why Is It Different?

  • Dominique Vervoort
    Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA

    Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    Search for articles by this author
  • Derrick Y. Tam
    Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada

    Schulich Heart Program, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
    Search for articles by this author
  • Harindra C. Wijeysundera
    Corresponding Author: Dr Harindra C. Wijeysundera, Schulich Heart Program, Sunnybrook Health Sciences Centre, University of Toronto, 2075 Bayview Ave, Suite A202, Toronto, Ontario M4N 3M5, Canada. Tel.: +1-416-480-6066; fax: +1-416-480-4657.
    Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada

    Schulich Heart Program, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
    Search for articles by this author
Published:August 27, 2021DOI:


      Innovations in health care are growing exponentially, resulting in improved quality of and access to care, as well as rising societal costs of care and variable reimbursement. In recent years, digital health technologies and artificial intelligence have become of increasing interest in cardiovascular medicine owing to their unique ability to empower patients and to use increasing quantities of data for moving toward personalised and precision medicine. Health technology assessment agencies evaluate the money spent on a health care intervention or technology to attain a given clinical impact and make recommendations for reimbursement considerations. However, there is a scarcity of economic evaluations of cardiovascular digital health technologies and artificial intelligence. The current health technology assessment framework is not equipped to address the unique, dynamic, and unpredictable value considerations of these technologies and highlight the need to better approach the digital health technologies and artificial intelligence health technology assessment process. In this review, we compare digital health technologies and artificial intelligence with traditional health care technologies, review existing health technology assessment frameworks, and discuss challenges and opportunities related to cardiovascular digital health technologies and artificial intelligence health technology assessment. Specifically, we argue that health technology assessments for digital health technologies and artificial intelligence applications must allow for a much shorter device life cycle, given the rapid and even potentially continuously iterative nature of this technology, and thus an evidence base that maybe less mature, compared with traditional health technologies and interventions.


      Les innovations dans le domaine de la santé sont en croissance exponentielle, ce qui se traduit par une amélioration de l'accès aux soins et de leur qualité, mais aussi par une augmentation des coûts sociétaux des soins et des remboursements variables. Ces dernières années, les technologies de santé numérique et l'intelligence artificielle ont suscité un intérêt croissant en médecine cardiovasculaire en raison de leur capacité unique à responsabiliser les patients et à utiliser des quantités croissantes de données pour avancer vers une médecine personnalisée et de précision. Les agences d'évaluation des technologies de santé quantifient les sommes dépensées pour une intervention ou une technologie de soins de santé en vue d'obtenir une retombée clinique donnée et formulent des recommandations sur des considérations de remboursement. Cependant, les évaluations économiques des technologies de santé numériques cardiovasculaires et de l'intelligence artificielle sont rares. Le cadre d'évaluation actuel des technologies de la santé n'est pas outillé pour prendre en considération l'aspect original, dynamique et imprévisible de ces technologies, soulignant la nécessité de mieux aborder le processus d'évaluation des technologies numériques de la santé et de l'intelligence artificielle. Dans cette étude, nous comparons les technologies numériques de la santé et l'intelligence artificielle par rapport aux technologies traditionnelles des soins de santé, nous examinons les dispositifs d'évaluation des technologies de santé existants et nous discutons des défis et des possibilités liés à l'évaluation des techno-logies numériques de la santé et de l'intelligence artificielle dans le domaine cardiovasculaire. Plus précisément, nous soutenons que les évaluations des technologies de la santé pour les technologies numériques de la santé et les applications de l'intelligence artificielle doivent tenir compte d'un cycle de vie des dispositifs beaucoup plus court, étant donné la nature rapide et même potentiellement itérative et continue de ces technologies, constituant alors une base de données factuelles peut-être moins mature, en comparaison avec les technologies et interventions traditionnelles de santé.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Canadian Journal of Cardiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Vervoort D.
        • Marvel F.A.
        • Isakadze N.
        • Kpodonu J.
        • Martin S.S.
        Digital cardiology: opportunities for disease prevention.
        Curr Cardiovasc Risk Rep. 2020; 14: 10
        • Johnson K.W.
        • Torres Soto J.
        • Glicksberg B.S.
        • et al.
        Artificial intelligence in cardiology.
        J Am Coll Cardiol. 2018; 71: 2668-2679
        • Krittanawong C.
        • Zhang H.
        • Wang Z.
        • Aydar M.
        • Kitai T.
        Artificial intelligence in precision cardiovascular medicine.
        J Am Coll Cardiol. 2017; 69: 2657-2664
        • Alami H.
        • Lehoux P.
        • Auclair Y.
        • et al.
        Artificial intelligence and health technology assessment: anticipating a new level of complexity.
        J Med Internet Res. 2020; 22e17707
        • Contreras I.
        • Vehi J.
        Artificial intelligence for diabetes management and decision support: literature review.
        J Med Internet Res. 2018; 20e10775
        • Weaver C.G.
        • McAlister F.A.
        Machine-learning, predictive analytics, and the emperor’s new clothes: why artificial intelligence hasn’t yet replaced conventional approaches.
        Can J Cardiol. 2021; 37: 1156-1158
        • Quer G.
        • Arnaout R.
        • Henne M.
        • Arnaout R.
        Machine learning and the future of cardiovascular care: JACC state-of-the-art review.
        J Am Coll Cardiol. 2021; 77: 300-313
        • Stone J.A.
        Mobile medicine: digital dynamo or virtual vaporware.
        Can J Cardiol. 2017; 33: 216-218
        • Cowie M.R.
        • Lam C.S.P.
        Remote monitoring and digital health tools in CVD management.
        Nat Rev Cardiol. 2021; 18: 457-458
        • Tam D.Y.
        • Miranda R.N.
        • Elbatarny M.
        • Wijeysundera H.C.
        Real-world health-economic considerations around aortic valve replacement in a publicly funded health system.
        Can J Cardiol. 2021; 37: 992-1003
        • Sud M.
        • Tam D.Y.
        • Wijeysundera H.C.
        The economics of transcatheter valve interventions.
        Can J Cardiol. 2017; 33: 1091-1098
        • Anderson J.L.
        • Heidenreich P.A.
        • Barnett P.G.
        • et al.
        ACC/AHA statement on cost/value methodology in clinical practice guidelines and performance measures: a report of the American College of Cardiology/American Heart Association Task Force on Performance Measures and Task Force on Practice Guidelines.
        Circulation. 2014; 129: 2329-2345
        • Porter M.E.
        What is value in health care?.
        N Engl J Med. 2010; 363: 2477-2481
        • Singh S.M.
        • Micieli A.
        • Wijeysundera H.C.
        Economic evaluation of percutaneous left atrial appendage occlusion, dabigatran, and warfarin for stroke prevention in patients with nonvalvular atrial fibrillation.
        Circulation. 2013; 127: 2414-2423
        • Jiang X.
        • Ming W.-K.
        • You J.H.
        The cost-effectiveness of digital health interventions on the management of cardiovascular diseases: systematic review.
        J Med Internet Res. 2019; 21: e13166
        • Phillips K.A.
        • Douglas M.P.
        • Trosman J.R.
        • Marshall D.A.
        “What goes around comes around”: lessons learned from economic evaluations of personalized medicine applied to digital medicine.
        Value Health. 2017; 20: 47-53
        • Liu X.
        • Faes L.
        • Kale A.U.
        • et al.
        A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis.
        Lancet Digit Health. 2019; 1: e271-e297
        • Gandhi S.
        • Chen S.
        • Hong L.
        • et al.
        Effect of mobile health interventions on the secondary prevention of cardiovascular disease: systematic review and meta-analysis.
        Can J Cardiol. 2017; 33: 219-231
        • Sampedro-Gómez J.
        • Dorado-Díaz P.I.
        • Vicente-Palacios V.
        • et al.
        Machine learning to predict stent restenosis based on daily demographic, clinical, and angiographic characteristics.
        Can J Cardiol. 2020; 36: 1624-1632
        • Gupta S.
        • Ko D.T.
        • Azizi P.
        • et al.
        Evaluation of machine learning algorithms for predicting readmission after acute myocardial infarction using routinely collected clinical data.
        Can J Cardiol. 2020; 36: 878-885
        • Tam D.Y.
        • Azizi P.M.
        • Fremes S.E.
        • et al.
        The cost-effectiveness of transcatheter aortic valve replacement in low surgical risk patients with severe aortic stenosis.
        Eur Heart J Qual Care Clin Outcomes. 2021; 28: 556-563
        • Whitehead S.J.
        • Ali S.
        Health outcomes in economic evaluation: the QALY and utilities.
        Br Med Bull. 2010; 96: 5-21
        • Somers C.
        • Grieve E.
        • Lennon M.
        • et al.
        Valuing mobile health: an open-ended contingent valuation survey of a national digital health program.
        JMIR Mhealth Uhealth. 2019; 7: e3
        • Beattie A.
        • Bagnall A.
        • Burn J.
        • O’Leary R.
        Adoption of HeartFlow FFRCT at the Newcastle upon Tyne Hospitals NHS Foundation Trust in a rapid access chest pain clinic setting.
        (Available at:)
      1. Royal United Hospitals Bath NHS Foundation Trust. Adoption of HeartFlow in Royal United Hospitals Bath NHS Foundation Trust. December 2019. Available at: Accessed June 18, 2021.

        • Roberts M.
        • Driggs D.
        • Thorpe M.
        • et al.
        Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans.
        Nat Mach Intell. 2021; 3: 199-217
        • El Hechi M.
        • Ward T.M.
        • An G.C.
        • et al.
        Artificial intelligence, machine learning, and surgical science: reality versus hype.
        J Surg Res. 2021; 264: A1-A9
        • Benjamens S.
        • Dhunnoo P.
        • Meskó B.
        The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database.
        NPJ Digit Med. 2020; 3: 118
        • Wu E.
        • Wu K.
        • Daneshjou R.
        • Ouyang D.
        • Ho D.E.
        • Zou J.
        How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals.
        Nat Med. 2021; 27: 582-584
        • Love-Koh J.
        • Peel A.
        • Rejon-Parrilla J.C.
        • et al.
        The future of precision medicine: potential impacts for health technology assessment.
        Pharmacoeconomics. 2018; 36: 1439-1451
        • Perez M.V.
        • Mahaffey K.W.
        • Hedlin H.
        • et al.
        Large-scale assessment of a smartwatch to identify atrial fibrillation.
        N Engl J Med. 2019; 381: 1909-1917
        • Vis C.
        • Bührmann L.
        • Riper H.
        • Ossebaard H.C.
        Health technology assessment frameworks for eHealth: a systematic review.
        Int J Technol Assess Health Care. 2020; 36: 204-216
      2. National Institute for Health and Care Excellence. Evidence Standards Framework for Digital Health Technologies. December 10, 2018; updated April 23, 2021. Available at: Accessed June 18, 2021.

        • Unsworth H.
        • Dillon B.
        • Collinson L.
        • et al.
        The NICE Evidence Standards Framework for digital Health and Care Technologies—developing and maintaining an innovative evidence framework with global impact.
        Digit Health. 2021; 7 (20552076211018617)
        • Haverinen J.
        • Keränen N.
        • Falkenbach P.
        • et al.
        Digi-HTA: health technology assessment framework for digital healthcare services.
        Finn J eHealth eWelfare. 2019; 11: 326-341
      3. APACMed. Overview of country health technology assessment (HTA) guidelines for digital health technologies (DHTs).
        (Available at:)
        • Haas J.S.
        • Moskowitz E.J.
        Health technology assessment in Canada and the United States: the case of biologics.
        Biotechnol Healthc. 2007; 4: 47-51
        • Shemer J.
        • Shani M.
        • Tamir O.
        • Siebzehner M.I.
        Health technology management in Israel: HTA in action.
        Int J Technol Assess Health Care. 2009; 25 Suppl 1: 134-139
      4. Digital Health Division, Israel Ministry of Health. Digital health technology evaluation for health organizations: an evaluation framework for early-stage technologies. June 2021. Available at: Accessed June 18, 2021.

        • Connecting Canadians, Government of Canada
        Connecting Canadians: Digital Canada 150.
        (Available at:)
        • Canadian Radio-television Telecommunications Commission, Government of Canada
        Broadband Fund: closing the digital divide in Canada.
        (Available at:)
        • Government of Canada
        Responsible use of artificial intelligence (AI).
        (Available at:)
        • McCradden M.D.
        • Baba A.
        • Saha A.
        • et al.
        Ethical concerns around use of artificial intelligence in health care research from the perspective of patients with meningioma, caregivers and health care providers: a qualitative study.
        CMAJ Open. 2020; 8: E90-E95
        • Canadian Agency for Drugs and Technologies in Health
        Canada’s First Digital Health Evaluation Network. March 23, 2021.
        (Available at:)