If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Blalock-Taussig-Thomas Pediatric and Congenital Heart Center, Department of Pediatrics, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA
Blalock-Taussig-Thomas Pediatric and Congenital Heart Center, Department of Pediatrics, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland, USADepartment of Cardiovascular Sciences, KU Leuven, Leuven, Belgium
Blalock-Taussig-Thomas Pediatric and Congenital Heart Center, Department of Pediatrics, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA
Ted Rogers Centre for Heart Research, Peter Munk Cardiac Centre, University Health Network, Department of Medicine, University of Toronto, Toronto, Ontario, Canada
The artificial intelligence (AI) revolution is well underway, including in the medical field, and has dramatically transformed our lives. An understanding of the basics of AI applications, their development, and challenges to their clinical implementation is important for clinicians to fully appreciate the possibilities of AI. Such a foundation would ensure that clinicians have a good grasp and realistic expectations for AI in medicine and prevent discrepancies between the promised and real-world impact. When quantifying the track record for AI applications in cardiology, we found that a substantial number of AI systems are never deployed in clinical practice, although there certainly are many success stories. Successful implementations shared the following: they came from clinical areas where large amount of training data was available; were deployable into a single diagnostic modality; prediction models generally had high performance in external validation; and most were developed as part of collaborations with medical device manufacturers who had substantial experience with implementation of new clinical technology. When looking into the current processes used for developing AI-based systems, we suggest that expanding the analytic framework to address potential deployment and implementation issues at project outset will improve the rate of successful implementation, and will be a necessary next step for AI to achieve its full potential in cardiovascular medicine.
Résumé
La révolution de l’intelligence artificielle (IA) qui va bon train, y compris dans le domaine médical, a radicalement transformé nos vies. Pour pleinement saisir les possibilités de l’IA, il est important que les cliniciens comprennent les principes fondamentaux des applications d’IA et de leur mise au point, et des défis liés à leur mise en œuvre dans la pratique clinique. Une telle base permettrait de garantir que les cliniciens ont une bonne compréhension de l’IA en médecine et des attentes réalistes à l’égard de celle-ci, et d’éviter des décalages entre ses promesses et ses répercussions réelles. Lors de l’évaluation quantitative relative aux applications de l’IA en cardiologie, nous avons observé qu’un nombre important de systèmes d’IA ne sont jamais employés en pratique clinique, malgré de nombreuses réussites. Les exemples de mises en œuvre fructueuses partagent les caractéristiques suivantes : elles se sont déroulées dans des secteurs cliniques où une grande quantité de données de formation était accessible; elles pouvaient être déployées dans le cadre d’une méthode diagnostique unique; les modèles de prédiction affichaient généralement un haut rendement de validation externe; et la plupart étaient élaborées dans le cadre de collaborations avec des fabricants de dispositifs médicaux ayant une expérience considérable dans la mise en œuvre de nouvelles technologies cliniques. Au regard des processus utilisés actuellement pour élaborer des systèmes fondés sur l’IA, nous pensons que l’élargissement du cadre analytique permettant de régler les problèmes potentiels liés à leur déploiement et à leur mise en œuvre dès le début du projet améliorera le taux de mise en œuvre fructueuse et sera une prochaine étape nécessaire pour tirer pleinement parti du potentiel de l’IA en médecine cardiovasculaire.
Artificial Intelligence in Medicine
The artificial intelligence (AI) revolution is well underway and has already dramatically transformed our lives in countless respects. Nowadays, most people interact with AI systems in routine daily activities, whether they realise it or not, and the fundamental impact that AI has on our world is undeniable. AI is also of great interest to medicine, with great promise and an unprecedented amount of resources devoted to the development of AI-based technologies in a variety of medical specialties, including cardiology. In many respects, the development and deployment of AI systems in medicine is still in the early stages compared with many other industries which started heavily investing in those technologies a few decades ago, first with expert systems and later with progressively more sophisticated algorithms as methods and computational resources improved. As such, comparisons between the status of AI in medicine and in other industries highlight substantial gaps both in the capacity of the technology and in the number of successful implementations.
There has also been a growing recognition that the expectations regarding the current capabilities of AI in medicine, when extrapolated from other areas of activities, might have been somewhat overstated compared with their current state of development,
To prevent a period of disillusion for AI in medicine due to inflated expectations, medical practitioners need a basic understanding of AI systems in medicine to be able to properly assess its capabilities and have realistic expectations. Therefore, this review will cover the following topics: 1) an introduction into AI theory to make the clinician familiar with some of the central concepts of this technology; 2) an overview of the process of building and evaluating AI systems, focusing on model training, validation, and quantifying model performance; 3) a discussion of the track record, barriers, and opportunities for implementation of AI models in clinical cardiology, with a review of implementation successes; and 4) a perspective on what can be done in the future to improve the rate of successful deployment and hopefully help AI to fulfill its promises for medicine and cardiology. Given our focus on clinical implementation in cardiology, we will not address the current state and latest stages of development of AI-based technology in cardiology subspecialties, but rather direct the reader to general
AI is broadly defined as the development of computer systems that can display machine intelligence, that is, that are able to extract and use information from their environment to achieve a specific goal in a process that mimics human intelligence but without human intervention. Although the concept of AI has been around ever since the design of the precursors of modern computers by Alan Turing in the 1950's, real-life applications were limited until the long-awaited “thinking machines” envisioned by Turing became technically feasible.
The exponential growth in large volumes of available data, advancements of computational power and cloud computing, and the advent of machine learning (ML) have all been instrumental in the explosion of AI use in many fields, including medicine.
Early efforts in the field of AI focused on so-called strong AI, the generation of systems that use high-level processes to mimic human reasoning and apply those processes to new problems. This approach was expected to eventually lead to general AI, through which an AI system would be able to accomplish any task within the realm of human capability without needing task-specific training. These early efforts were very successful in some respects, particularly through the development of expert systems for situations that are rule dependent (eg, playing checkers
) but tended to perform poorly for simpler tasks that are intuitive to humans but not necessarily rule based, and this approach also proved to have an upper limit in the level of complexity of the problems it can handle. Because of these limitations, the field of AI moved from general AI to narrower task-based AI (sometimes referred to as weak AI). Task-based AI systems are at the center of the current AI revolution. Roughly speaking, tasks performed by AI systems can be related to perception (computer vision, natural language processing), information processing (reasoning/problem solving, knowledge representation, learning), and performative tasks (task planning/optimisation, data generation including language and images, object manipulation and motion). By far, the majority of AI systems being developed in medicine now are specifically targeting learning tasks, but nonmedical applications routinely use the full constellation of AI tasks. AI systems can combine these tasks to increase the complexity of the situation or problem they can address.
Although there is a growing consensus that task-based AI is unlikely to lead to general AI, which is now a separate area of research, the capabilities of current task-based AI systems have far exceeded systems built using other approaches.
In addition to the tasks they perform, AI systems can also be classified based on their intended use, which has important considerations on how those systems are built, evaluated, monitored, and regulated. AI systems can be designed to be automatic, assisting, augmenting, or autonomous (Fig. 1). Automatic systems, designed to handle simple repetitive tasks, and assisting systems, designed to provide the user with more information (eg, diagnostic, features, etc) that they can use in decision making, are lower-complexity systems. Augmenting systems, on the other hand, help the user to improve overall performance, either by providing insights that would not otherwise be available, such as patient-level predictions, or by performing part of a task directly (eg, robotic assistance). Finally, autonomous systems are those systems that perform complex tasks without human intervention. There are 2 major concepts underlying the intended use framework, first the degree of complexity and persistence (a function of how the task[s] being performed change[s] over time) and second the degree of human involvement in accomplishing the tasks. Automatic and assisting systems generally have low complexity and requirement for adaptability over time (ie, the extent to which the tasks or the underlying environment changes over time), whereas augmenting and autonomous systems handle tasks with a high degree of complexity that have substantial adaptability requirements. Assisting and augmenting AI systems still require a large amount of human involvement and oversight, whereas automatic and autonomous systems work independently.
Matheny M. Israni S.T. Ahmed M. Whicher D. Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril. National Academy of Medicine,
Washington2019
Unsurprisingly, automatic and some assisting systems currently have the highest rate of successful implementations, and augmenting applications are still in development. Autonomous AI is not yet a reality in medicine, but autonomous systems do exist in full deployment in many other industries.
Figure 1Classification of AI systems by purpose, complexity, and level of independence. Current and future examples of applications in cardiovascular medicine are indicated in each of the quadrants, with green indicating US Food and Drug Administration–approved applications, orange indicating applications currently being research (at various levels), and red indicating use cases. AI, artificial intelligence; ECG, electrocardiography; EHR, electronic health record; NLP, natural language processing.
With the emerging role of AI in daily medical practice, it is important for clinicians to understand some basic terminology related to AI in this context (Fig. 2). As previously mentioned, in the medical realm, the overwhelming majority of applications focus on the learning task and use ML as the primary methodology. Machine learning encompasses a group of algorithms that learn to accomplish a task without being explicitly programmed to do so. These algorithms use sample data to learn patterns that are then used for predictions or classification. Within the realm of ML, there are multiple learning approaches (supervised and unsupervised learning being the most common approaches) based on the purpose and type of training data available, and each approach has a number of algorithms available. Deep learning, which uses neural networks, is one such algorithm among many other options, not all of which are based on neural networks. These terms are different from data science, data mining, and big data. Data science is the discipline concerned with applied ML (in contrast to computer science, which is concerned with the development of ML algorithms), and data mining is a subdiscipline of data science concerned with exploratory data analysis (as opposed to predictive analytics in the context of ML or traditional statistical testing). On the other hand, big data refers to the field of technologies and methods used to handle data sets larger than one typical computational platforms can handle. In some areas, and particularly in mainstream contexts, the term big data is used as an umbrella term that incorporates data mining, data science, and the wide implementation of ML-based algorithms, but this formulation should be avoided in academic and technical settings in the same way that the terms defined in this section should not be used interchangeably or as synonyms.
Figure 2The landscape of health data and analytics. CNN, convolutional neural network; RNN, recurrent neural network.
As noted above, the 2 most common approaches to ML are supervised and unsupervised learning, both of which have different objectives (Fig. 3). Supervised learning is primarily used for classification and prediction problems. Supervised learning methods require a labelled data set (also known as “ground truth”), which is typically a diagnosis, membership in a group or class, or an outcome. The algorithm is then trained, using example data, to recognise patterns of features (ie, known facts about the patient) associated with a specific label and thereafter to predict those labels on new cases. There are multiple categories of supervised learning algorithms, the most common of which include regression models, decision trees, support vector machines, and neural networks (the underlying algorithm for deep learning). Each of these families of algorithms include multiple permutations/modifications to apply to specific situations. A brief introduction to deep learning is warranted here given the popularity of those methods. Deep learning is a method that uses a layered structure of algorithms called an artificial neural network (ANN). Mimicking the activity of biological networks between neurons in the human neocortex, these models lead to a process of learning that can reach far beyond the capabilities of other ML models. An ANN consists of many simple connected neurons, each of which produces a sequence of activations. Input neurons receive the initial input to the model, whereas subsequent neurons become activated through weighted connections from previously active neurons. As the input travels through the network, it is modified by these weights to eventually generate the output. The learning element in ANN thus involves the search for the weights that make the model exhibit the desired behaviour. Subtypes of ANN include convolutional neural networks, such as those used for object recognition from imaging data, and recurrent neural networks (RNNs), which are applied in natural language processing (NLP).
Unsupervised learning, in contrast, works with unlabelled data sets. This type of model attempts to detect the natural patterns present within a set of scattered data points without dependent variables. Unsupervised learning can be used to identify like cases or to identify groups of cases sharing similarities (eg, clustering methods such as k-means, hierarchic clustering) as well as for dimension reductions (eg, principal component analysis). Unsupervised learning methods are often used to generate features that are then included in supervised learning algorithms (eg, cluster membership or principal component loads), but they can also be used for fundamental discovery. Cluster membership can reveal novel patterns or subphenotypes that were not previously clinically recognised and change our understanding of the spectrum associated with a clinical entity. For example, the application of unsupervised clustering on data sets from patients with heart failure with preserved ejection fraction has led to the discovery of 3 distinct phenogroups, each with a different pathophysiology and likely to respond to different therapeutic regimens.
There are other methods to train ML algorithms, including semisupervised learning, where algorithms are trained using a small amount of labelled data with a large amount of unlabeled data, and reinforcement learning, where algorithms are trained through a reward/penalty system for correct/incorrect predictions, but these are still relatively new and there are very few examples of application of these methods in a clinical context, although reinforcement learning has been used to optimise treatment of patients with sepsis in the intensive care unit.
The Process of Building and Evaluating ML Systems for Clinical Use
A major requirement for ML systems to be sustainable and clinically useful is their continuous assessment and evaluation at all points in their life cycle. This evaluation goes beyond the assessment of model performance to include an evaluation of the needs for ML systems to address a specific clinical question, an assessment of potential future implementation challenges, and continuous evaluation of model performance and changes in outcomes after deployment. At the outset, the question of whether ML is a good solution or even the best solution for a specific problem should always be asked. Although ML has been used to create highly accurate algorithms, there is now a recognition that in some circumstances traditional statistical methods perform just as well as ML-based algorithms.
These include situations where the size of the available training data is too small or where the feature space is limited (small number of mostly binary and normally distributed continuous inputs). The use of ML should also be considered carefully in situations where a high degree of explainability and transparency is required,
Model training should only start when the appropriateness of ML in a specific situation has been established.
Model training
Model training first requires the selection of a broad analytic approach, the choice is often task dependent and based on the nature of the input data. In some cases, multiple type of algorithms are potential candidates and the choice is based on a comparison of model performance between the different options. Algorithm training often requires the selection and tuning of hyperparameters: ie the options specific to each type of algorithms that will yield the best performing model. Hyperparameters are often adjusted empirically, that is, by trial and error until the best combination is found. Although the details of algorithm training provided in scientific paper can seem complex for the uninitiated, clinicians should remember that no single type of algorithm works in all situations and ML studies should report the considerations that led to the choice of a specific analytic approach. However, as part of model training, there are 2 additional considerations that are common to all studies and will be addressed below: establishing ground truth and the representativity of the training sample.
With supervised learning still representing the majority of ML models in medicine, the establishment of labels, also known as ground truth, remains a major point of attention. Establishing ground truth is the process by which labels (the target of the prediction algorithm) are assigned to training cases. This is a critical component of algorithm training, given that mislabelled training cases result in a poorly performing algorithm. The process of properly labelling training cases is resource intensive and often requires substantial amount of work from content experts.
In some situations, high-quality training data are readily available; for example, ML algorithms in signal processing can rely on large public databases for training, which have become widely available given the large number of monitoring devices with digital storage and improved computing capacity. In fact, electrocardiographic (ECG) data were the first to be digitised and shared on platforms such as the PhysioNet database and the MIT-BIH arrhythmia database.
Clinical risk models present additional challenges. They often rely on sensitive patient information that cannot be completely anonymised, and therefore cannot be released publicly, although some data generated in clinical trials or in large research initiatives such as the UK Biobank can be accessed by qualified investigators.
Therefore, public databases for these purposes are somewhat limited and data collection remains costly and time intensive. This is particularly prevalent for relatively rare conditions such as congenital heart disease, myocarditis, or cardiomyopathy, where the issue of limited data availability is further compounded and multicentre collaboration in large, and costly, registries is necessary. Further compounding the problem, many variables that are of interest to clinical cardiology are contained in administrative data or within patient health records. As a general rule, administrative data are not an optimal solution to obtain high-quality training data, especially when they are derived from claims data, which are generally not coded by content experts. Indeed, multiple studies have shown a substantial degree of disagreement between expert review of medical records and administrative data in cardiology and beyond. Although the increased adoption of electronic health records over the past decade has resulted in more point-of-care clinical data being available,
a lot of this data is unstructured free text and largely inaccessible. NLP has been investigated as one potential strategy to access these data for analysis.
have reported some initial examples applied to heart failure, where NLP allowed identifying heart failure cases in uncoded data with a positive predictive value (PPV) of 0.914 and extracting heart failure medications with an overall precision of 0.955. This being said, even the most sophisticated NLP algorithms are still facing major technical challenges, including differences in terminology, local conventions/peculiarities, and homonymy. With the current state of the technology, the use of NLP to generate high-quality training data is unlikely to be a robust solution given both the introduction of mislabelled training cases and the fact that NLP models themselves rely on large data sets of ground truth, which also require chart verification and annotation. Furthermore, NLP models are highly prone to failure when applied outside the training environment, because they are highly context specific and have limited ability to retrain themselves after changes in the deployment environment.
The other important consideration in generating and assessing the quality training data set is its representativeness. As a general rule, the characteristics of the population in the training data set should be representative of the population in which the model will eventually be deployed. One should also be aware that training data can reflect historical racial, societal, and economic disparities that could be perpetuated or even exacerbated by implementing the model,
Therefore, when critically appraising prediction models, particularly those created with the use of supervised learning, it is important to consider the quality of the labels, how they were obtained, the likelihood of them being misclassified, and the likelihood of ascertainment bias. As far as representativity, one should consider the source and characteristics of the patients included in the training data set. For clinicians, it is best to think of the quality of the labelling and the representativeness of the training data set with the same care and importance as, respectively, the primary outcome and study population of a clinical trial.
Validation strategies
At the outset of ML model development, the original data set is usually partitioned randomly into 2 parts: one for training and another for testing (or validation/hold-out). The training data set is the part of the data set from which the ML algorithm learns the relationship between features and the target variable (label). The testing data set is the part of the input data to which the algorithm, after it has been taught based on the training data set, is applied to validate whether it accurately performs the task. This validation strategy is feasible and favoured for very large data sets. In smaller data sets, however, using this method is not always possible. In those cases, another validation strategy known as k-fold cross-validation is used. It involves the data set being randomly split up into k groups, where each group is used, in turn, as the validation cohort for a model trained on all of the other groups (k − 1 groups). The process of splitting, model development, and testing is repeated k times such that every group has been the testing data set once. It is worth noting that k-fold cross-validation can also be used during model training, in which case the model is trained k times using cross-validation, with the learners merged at the end of the training process and then further evaluated on the hold-out data set. This method has been shown to improve model generalisability.
While both of these approaches validate whether the ML model accurately performs the task that it has been trained for, they still report on the performance of the model in the same population it was derived from, which would thus be expected to have characteristics very similar to the training data set. This process is therefore referred to as internal validation. In contrast, external validation aims to assess generalisability of the ML model when applied to a different population. As a general rule, all prediction models should be evaluated through cross-validation and performance metrics should be reported for the validation sets. The hold-out method is not always possible in smaller samples, because it reduces the amount of data available for training, and it has been argued that unless the available sample size is extremely large, cross-validation should be preferred to hold-out validation given that they essentially provide the same information.
Given that internal validation is nearly universal as part of model development, internal performance metrics are generally included in scientific publications.
On the other hand, reporting of external validation used to be limited in scientific publications, although it is now a requirement for more and more scientific journals. A recent review by Wessler et al.
revealed that only 58% of cardiovascular clinical prediction models in the Tufts Predictive Analytics and Comparative Effectiveness Clinical Prediction Model Registry had undergone external validation, leaving much room for improvement.
External validation of prediction models is important because they tend to perform worse outside their training data set. The review by Wessler et al.
found that in 81% of external validations of cardiovascular clinical prediction models, the area under the receiver operating characteristic curve (AUC) showed discrimination below that reported in the training data set. The AUC value decreased by a median percentage of −11.1% (interquartile range −32.4% to +2.7%). Model performance often decreases in external validation cohorts for multiple reasons, including potential differences, even small ones, between the characteristics of the populations. Another culprit, particularly affecting prediction models based on electronic health records, is a difference in the quality of data curation between the highly curated training data and the generally messier real-world validation data. But the most important reason is often related to model overfitting. This happens when the model learns patterns that are specific to the training data but are irrelevant to the underlying relationship between features and the target (ie, the model has become adapted to noise rather than signal). Thus overfitted models are unable to generalise to new observations. If the model has a high accuracy in the training data set but performs poorly in the testing data set, this may indicate overfitting. This is particularly prevalent in studies with small sample sizes, rare outcomes, and reporting of performance only from the original training data set. One solution to prevent overfitting is using more training data, although this is not always practically feasible. A second solution is to use cross-validation, thus reducing the capacity of the model to memorise training data and increasing the chance of incorporation of signal into the model. Other options include limiting the number of features, early stopping (stopping the training process before overfitting occurs), regularisation (forcing the model to be simpler), and ensembling (combining predictions from multiple separate models). This can be done by using different ML methods, using only a subset of the variables in each of the training sets (bagging), or using only a subset of the observations in each training set (bootstrapping). Thus, the critical appraisal of prediction models should always include considerations for both internal and external validation of model performance.
Assessment of model performance
Model performance can be defined by several measures, none of which capture all aspects of model performance, nor should they be interpreted in isolation.
The first aspect of model performance is discrimination, the ability of a model to accomplish the task it is designed to do (eg, accurately separating patients who experienced an event and those who did not). Good discrimination is achieved when the predicted probabilities of the different groups have very little overlap with each other, thus accurately delineating membership in each group. The most common measure of discriminative ability is the AUC, also known as the C-statistic or C-index (for concordance between model-based estimates and observed events). The AUC represents the probability that, given one individual with the event and another without, the model will yield a higher predicted probability for the first than for the second. This value generally ranges from 0.5 (random concordance) to 1 (perfect concordance). It is important to understand that AUC provides a more balanced estimate of the discriminative ability of a model than accuracy (the sum of true positives and true negatives divided by the total number of patients). At low prevalence, accuracy obscures the outcome, whereas at high prevalence it weights the outcome too much. To illustrate this, a prediction model for risk of endocarditis that would predict “no risk of infective endocarditis” for everyone regardless of their characteristics would have an accuracy of > 99.9% (because the incidence is only 11-15/100,000 population), whereas the AUC would be 0.500.
Although the AUC is more informative than raw accuracy, a drawback is that it equally values false positives and false negatives, which is antithetical to clinical medicine. Alternative metrics are the PPV (proportion of patients with predicted event who actually have an event) and the negative predictive value (NPV; proportion of patients for whom the model does not predict an event and who actually do not experience the event), both of which are measures of a model’s misclassification. Both measures can be useful in situations where the consequences of wrong (prediction-based) management can involve serious harm for the patient. One such example is the decision to proceed with cardiac surgery in high-risk patients. If the model predicts a higher risk of major adverse cardiac events with conservative management and the model has a high PPV, the surgeon may reasonably rely on the model’s prediction and advise surgical treatment; however, if the model has had a low PPV, there would still have been a considerable chance that a patient who might have fared better with conservative management has wrongfully been exposed to the risk of surgery (eg, if the PPV was 60%, this would account for 40% of the decisions for surgery). Conversely, 1 − NPV gives a good indication of the patient’s chances of having an event even though the model predicted they would not. There are multiple additional metrics that assess various aspects of model performance, but at a minimum, all models should be reported with AUC and at least 1 additional set of metrics addressing the differential value of false positives and false negatives.
Beyond discrimination, the other component of model performance is calibration, a measure of how accurate an estimate of the risk prediction is in the targeted population. This measure is often presented in a calibration plot, which depicts the observed values (on the y-axis) vs the predicted values (on the x-axis). The 45 degrees line in this plot denotes perfect calibration, meaning that the predicted value exactly matches the observed value in the data set. From the calibration plot, it is easy to assess the range of values for which the model has poor calibration (Fig. 4). Model fit, or internal consistency, can also be assessed with the use of goodness-of-fit tests, such as the Hosmer-Lemeshow test. However, while a significant P value (P ≤ 0.05) implies poor model fit, the opposite is not true: a model with P > 0.05 can still have poor calibration. Furthermore, a simple goodness-of-fit test does not provide the same detailed information as a calibration plot. For these reasons, the latter is usually preferred. Unfortunately, model calibration has not consistently been reported in the past, although it is now a requirement for more and more scientific journals.
Figure 4Examples of calibration plots for prediction models: (A) example of good calibration, (B) example of imprecise prediction model, (C) a situation where a prediction model overestimates risk in low-risk patients, and (D) a situation where a prediction model underestimates risk in high-risk patients.
One important final note about the evaluation of prediction models is that model performance is a dynamic concept, not a static one. Because medicine is a continuously evolving field, prediction models become stale and training data sets obsolete. Therefore, performance should be consistently monitored and models should regularly be updated to reflect the current state of the art.
What Is the Implementation Success Rate for ML-Based Technology in Cardiology?
After reviewing the complexities that go in the development of ML-based algorithms, we will now discuss issues surrounding implementation. One of the most common outcomes for clinical prediction models, whether ML-based or not, is the low proportion of those models that move from development to actual use in clinical settings despite excellent performance in a sandbox environment. Although there are multiple reasons for this, some of which will be investigated in further detail here, one of the most important culprits is the limited focus on future implementation challenges as part of model development to prevent future deployment failure.
To illustrate this point, we performed a review of the literature on ML and AI in cardiology from 1990 to 2020. The methodology that was used for that review included an NLP algorithm and has been described previously by Quer et al.
Briefly, bibliometric information was retrieved on all publications identified through a PubMed search using combinations of the search terms “clinical decision,” “decision support,” “algorithms” with either “card∗ abnormalities,” “card∗ disease∗,” “card∗ surg∗,” “heart disease,” “card∗,” and “cardiology.” The publications were reviewed and annotated according to whether they were original research articles, whether they used ML methods, and the disease and modalities they studied.
In the period 1990-2020, a total of 11,118 articles on algorithms in cardiology were published in PubMed, including 6276 articles using ML models and 4,842 applying “traditional” regression models (Fig. 5A ). Whereas the latter stabilised in the past decade, the numbers of publications using ML models continued to increase at a rate of ∼10%/year, representing tremendous growth. In 2020, 2 ML articles were published for each article using traditional regression models for prediction problems. The heatmaps in Figure 5B show the number of articles by disease and modality, revealing that computed tomography (CT), ECG, and echocardiography in the setting of atherosclerosis, as well as ECG in the setting of arrhythmias, were the most commonly studied topics. On the other hand, many other topics have received less attention and remain underrepresented in the literature. These trends were comparable between ML and traditional regression, although the use of ML models was 3 times more common in articles about arrhythmia detection based on ECG than were traditional regression models.
Figure 5Trends in articles related to machine learning and artificial intelligence in cardiology during 1990-2020. (A) Number of articles per year, grouped into machine learning models and ‘traditional’ regression models. (B) Heatmap representing the number of publications by disease and modality. CT, computed tomography; ECG, electrocardiography; EHR, electronic health record; MRI, magnetic resonance imaging. (C) Number of publications with the keywords “implementation,” “evaluation,” “integration,” “application,” or “validation” (solid) compared with the total number of publications (shaded) per year.
Subsequently, we used specific key words to estimate the overall implementation rate of these articles. We found that 437 articles mentioned the key word “implementation,” 1292 mentioned “evaluation,” 139 mentioned “integration,” 1016 mentioned “application,” and 860 mentioned “validation,” suggesting that a total of 2587 articles (23.3%) included some consideration of implementation of their algorithm. Figure 1C reveals that this percentage remained stable over time, while the absolute number of articles reporting implementation increased.
Because the above analysis was primarily an automated informal survey of the literature intended to identify global patterns, we conducted a manual review of articles published in the year 2018 (2018 was selected because it gave at least 2 years of post-publication time for implementation). We identified a total of 97 articles that reported ML-based clinical risk prediction models. Here, we assessed implementation rate with the use of more strict criteria: The model had to have received US Food and Drug Administration (FDA) clearance,
or there had to be very clear evidence for implementation in real-life clinical practice available in the article itself and/or elsewhere on the internet. Using these criteria, only 3 articles (3.1%) were found to have had true implementation success. This number should be interpreted carefully, however, given that many ML-based systems for cardiovascular disease that were cleared by the FDA came from the industry (as opposed to academia), with little previous publication in the scientific literature.
Why Are ML-Based Technologies Failing Implementation at Such a High Rate in Clinical Settings?
Implementation challenges with new technology and with predictive models in clinical medicine are not new. In a systematic review of clinical prediction models developed in the field of cardiovascular diseases from 1990 to 2012, Wessler et al. identified 796 models. The majority (667, 84%) of these models were based on logistic and Cox regression, and presented lack of standardisation, generally poor quality of reporting, limited penetration in clinical practice, and little impact on patient care.
Some of those challenges have been partially addressed but they still exist and affect ML models in the same way that they affect clinical prediction models created with more traditional statistical methods. Other implementation challenges are either unique to ML models or at least are amplified because of some of the unique features of those algorithms.
Mismatched expectations
The first major challenge to implementation in ML models is one of mismatched expectations and poor fit between the current capabilities of ML systems and their intended use. In the lead to the current hype toward ML in medicine, there have been many claims about the capabilities of these systems that were either unrealistic or at least premature. However, it is important to realise that in its current form, ML is generally an extension of traditional modelling approaches, with their own limitations.
Many medical applications of ML systems have been envisioned beyond these limitations and as such invariably fall short of expectations.
Part of the hype around ML in medicine is the idea that ML-models are somewhat of a magic bullet that can solve a variety of problems that have affected the development and implementation of clinical prediction models. The phrase “using an ML approach,” as though it will lead to some new transformational approach, clearly highlights that there is a high degree of unnecessary use of ML. Unfortunately, ML models remains affected by sample size, poor quality, and inherent bias in the training data, poor study design, and mislabelling of outcomes. Indeed a recent meta-analysis showed no improved performance for clinical prediction models created with ML than logistic regression when based on traditional data elements extracted from the electronic health record.
However, one area where ML-based models have shown an improvement over traditional methods is that of clinical prediction models that include features derived from diagnostic modalities (eg, imaging, ECG, physiologic data). In a comprehensive review of ML-based prediction models published through 2019, the average AUC for published models was > 0.90 in most clinical areas. In comparison, the AUC was ∼0.80 from the models included in Wessler et al.
More than 80% of the clinical prediction models had been created with logistic or Cox regression, and those incorporating diagnostic-derived features performed better than those without those features.
The discrepancy between the intended use of ML in medicine and its actual capabilities is one of the major reason for the high implementation failure rate of ML-based applications in medicine.
Model performance
The second limitation is poor model performance outside of the training settings. Clinical prediction models tend to be developed in a sandbox environment with curated data and few, if any, implementation challenges. Once deployed in the real world, these ML models tend to perform much more poorly then their published performance. There are many reasons for the decrease in performance,
most notably: 1) reliance on medical records for model training, which tend to be incomplete and prone to erroneous data, reflect current medical practice whether or not it is optimal, and have depreciating representativeness over time; 2) lack of external replication studies before deployment; 3) bias in model training reflecting historical racial, societal, and economic disparities
; 4) institution-specific bias, where local peculiarities are learned as rules by the algorithm, which is particularly prominent in single-centre imaging studies
; and 5) model overfitting and lack of generalisability when the characteristics of the population in which a model is deployed do not match the characteristics of the population in which the model was trained. Poor model performance is one of the major factors explaining the reluctance of some physicians to use clinical prediction models for decision support.
The next reason has to do with lack of interpretability of the predictions associated with many ML methods. For many ML methods, including those based on neural networks, there is no direct way to track down how the model generates a prediction from the inputted features; those are colloquially referred to as “black box” models. Whereas interpretability is less important for task-focused algorithms, whose performance against a gold standard is easily measured and monitored, it is of utmost importance for clinical predictions that health care providers rely on as part of augmenting algorithms. Poor interpretability is the most often cited reason for the lack of trust of some physicians in ML-based prediction models.
The final set of reasons for the poor track record of deployment of ML models in cardiology is also the least discussed: the lack of infrastructure and framework for deployment and post-deployment model management. Deployment infrastructure is an important consideration. ML models often require a large number of features from various data sources, which requires an integration point for raw data, a data-processing script that often needs to be locally adapted, and a high-performance computer engine, none of which are generally available in standard electronic health record platforms. In addition to technological impediments to deployment, there are also organisational ones that contribute to the poor track record. The regulatory framework around clinical prediction models is in its relative infancy, with many questions regarding ethics, liability, security, and reimbursement still unresolved. The lack of clear guidance on the regulatory framework around those algorithms make planning for future deployment and implementation much more difficult. Guidance on the regulatory status of software as a medical device
There is still a lack of research on how clinicians use and understand clinical predictions, human factors considerations, and how augmentation algorithms affect the organisation of health care organisations’ and clinicians’ cognitive processes. Furthermore, there is no mechanism to manage the unintended consequences of large-scale algorithm deployment. For example, the deployment of smart devices that detect atrial fibrillation led to concerns with overdiagnosis, unnecessary testing secondary to false positive readings, and overdetection of nonpathologic forms of diseases that would otherwise not have caused complications or have required treatment.
Finally, there is no consensus on the long-term management of algorithms, including the continuous monitoring of an algorithm’s performance and patient outcomes. As evidenced above, there are multiple barriers to the implementation of ML-based models in clinical settings, some preexisting and some specific to ML. These barriers are a substantial disincentive to algorithm implementation and together explain in large part the high failure rate for the clinical deployment of such models.
What Do Successful ML Implementations Have in Common?
Despite those barriers, ML already has had an undeniable impact in clinical cardiology, and many algorithms have been successfully deployed and/or are in the latter stages of deployment. Of the 343 ML-based applications cleared by the FDA as of June 2021, 41 were in the cardiovascular realm and 241 were in radiology (some with use in cardiac imaging). A collection of representative examples of FDA-cleared ML-based applications and other applications that are being used in clinical practice are presented in Table 1.
Estimating one-year risk of incident chronic kidney disease: retrospective development and validation study using electronic medical record data from the state of Maine.
These can be divided into 1) signal processing and diagnostics, 2) image processing and diagnosis, and 3) clinical risk stratification and information retrieval from electronic health records through NLP. The area of cardiology with the highest number of successfully implemented ML-based algorithms by far is signal processing and diagnosis, with many applications in use to detect various arrhythmias (and more recently heart failure
ML-based systems have also been used successfully for image processing and diagnostics with the 3 main modalities in cardiology: echocardiography, CT, and magnetic resonance imaging. Successfully implemented applications are used for automated view identification, segmentation of cardiac chambers, quantification of cardiac structures and function, and disease detection.
successfully applied a supervised ML algorithm, XGBoost, to develop and validate a risk-prediction model of incident essential hypertension within the following year. In addition, ML-based applications for heart failure case finding and automated extraction of medication-related information deployed into clinical practice have been reported.
Automated detection of AF based on Apple Watch Series 2 or 3 with KardioBand (AliveCor)
SmartRhythm 2.0, a convolutional neural network
Data of heart rate, activity level, and ECGs from 7500 AliveCor users (training), and data from 24 patients with insertable cardiac monitor and history of paroxysmal AF (validation)
Episode sensitivity 97.5%, duration sensitivity 97.7%, patient sensitivity 83.3% overall and 100% during time worn
Improvement beyond SmartRhythm 1.0, which was validated in 100 patients undergoing cardioversion
then ended sales in 2019 as native ECG functionality was coming to the Apple Watch Series 463; Alivecor continues to invest in their KardioMobile technology
Detection of ventricular fibrillation and ventricular tachycardia (shockable rhythms) to improve shock advice algorithms in automated external defibrillators
Convolutional neural network as a feature extractor and boosting classifier
1135 shockable segments and 5185 nonshockable segments from 57 records in public databases (Creighton University Ventricular Tachyarrhythmia Database and the MIT-BIH Malignant Ventricular Arrhythmia Database)
Accuracy 99.3%, sensitivity 97.1%, specificity 99.4% (compared with sensitivity 90% and specificity 95% in current American Heart Association algorithms)
Similar results have been reported in models developed by Picon et al.
Automated ECG interpretation, including view identification, segmentation of cardiac chambers across 5 common views, quantification of structures and function, and disease detection
Convolutional neural networks
14,035 echocardiograms spanning a 10-year period
Identification of views in > 95%, median absolute deviation 15%-17% for structure and < 10% for function, detection of hypertrophic cardiomyopathy, cardiac amyloidosis, and pulmonary disease with C-statistics of 0.93, 0.87, and 0.85, respectively
EchoCV web-based fully automated echocardiogram interpretation system has been patented and made available online
Decreasing computational demand of FFR calculation by developing a ML-based model as an alternative to computational fluid dynamics approaches
Deep neural network
125 lesions in 87 patient-specific anatomic models generated from CT data using image segmentation
Excellent correlation (0.9994; P < 0.001) and no systematic bias in Bland-Altman analysis: FFR ≤ 0.80 was predicted with sensitivity 81.6%, specificity 83.9%, and accuracy 83.2%
Risk prediction model of incident essential hypertension within the following year
Feature selection and generation of ensemble of classification trees with the use of XGBoost
Data from individual patient electronic health records as part of the Health Information Exchange data set of Maine (823,627 for retrospective cohort and 680,810 for prospective cohort)
C-statistics of 0.917 in the retrospective cohort and 0.870 in the prospective cohort
Deployed in the state of Maine, where similar risk models have been implemented for chronic kidney disease
Estimating one-year risk of incident chronic kidney disease: retrospective development and validation study using electronic medical record data from the state of Maine.
Development of CHIEF to automatically extract LV function mentions and values, congestive HF medications, and documented reasons for a patient not receiving these medications
Combination of rules, dictionaries, and ML methods
Various clinical notes from 1083 Veterans Health Administration patients
High recall and precision for HF medications and EF (> 0.960), while only reaching fair recall and precision for reasons for not prescribing HF medications (< 0.400)
Additional improvements of the model have been reported
Successful implementation examples in cardiology have many points in common. First, they tend to come from clinical areas where large amounts of training data are available owing to the early digitisation of the information in an easily retrievable format (eg, imaging and signal processing). These applications also can be deployed into a single diagnostic modality or with the acquisition of only a few clinical features and do not rely on a complex data integration infrastructure. Next, those models are highly predictive, generally with AUC > 0.90 in external validation cohorts and clinical trials. Finally, and perhaps most importantly, most of these applications have been developed by or in collaboration with large medical device manufacturers who have decades of experience with approval, deployment, and implementation of novel technology in the clinical area, as opposed to academic researchers who are generally more focused on fundamental discoveries.
The Path Forward: How to Improve the Rate of Successful Implementation in the Future
From this review of ML systems implementation successes and failures in cardiology, it is clear that successful applications are limited to those where either the implementation issues are contained or simple (eg, single-modality deployment, risk scores) or where the implementation matters have been integrated into the technology development cycle (eg, wearables). Multimodality ML systems and those that include large amounts of between-centre variations (eg, natural language) have lower rates of successful implementation.
Based on the observations above, the path forward to improve the transition rate of ML systems in cardiology from showcase projects to real applications with full deployment that have a direct impact on patient care will require both deep environmental changes and a new technology development framework that addresses implementation issues at the project onset. From a medical environment point of view, improving the successful transition rate will require rethinking sharing and access to large-scale multicentre training data. As highlighted, the easy availability of large repositories of ECG data is one of the main reasons why many of the success stories for ML systems in cardiology are in the arrhythmia domain.
but there are few of these publicly available resources for clinical data, particularly when deep phenotypic information is needed. The second major environmental change will need to be in the creation and management of medical data. The current reliance of free text with nonsystematic terminology makes it very onerous to generate high-quality data that can be used in algorithms, and the current practice of using health care providers as data entry clerks is not sustainable. New strategies that incorporate real-time data acquisition, systematic terminology, information modelling, and data standards incorporated in electronic health record platforms will be necessary, particularly for the successful implementation of clinical risk prediction models that are not based on a single-modality diagnostic system.
Finally, education of health care providers on AI will be necessary for the future end-user of ML systems to fully understand the benefits but also the limitations of the technology and have reasonable expectations toward ML systems.
Beyond environmental changes, there is also a need to adopt a more comprehensive framework in the development of ML systems for cardiology and for medicine in general. Part of this framework is already emerging with new guidelines for the reporting of clinical prediction models in the scientific literature,
Proposed Requirements for Cardiovascular Imaging–Related Machine Learning Evaluation (PRIME): a checklist: reviewed by the American College of Cardiology Healthcare Innovation Council.
However, the data presented in this review suggest that an even more comprehensive implementation framework might be needed for the development of AI systems that can be successfully deployed for the benefit of patients with heart disease. In this framework, the process would start with a comprehensive analysis of the opportunity and the appropriateness of AI systems to provide a suitable solution and would include the design of a technical clinical implementation plan focusing specifically on availability of source data and data integration strategy. This framework would obviously include proper algorithm design, reporting, and evaluation, the latter including a deep evaluation of model performance in an external cohort, with varying levels of missing data, and in specific patient populations that might be more prone to biased predictions. Clinical deployment should include both a full human factor evaluation and shadow testing (testing under real-life conditions but without the results being made available to clinicians or patients). Finally, the full implementation framework should include continuous monitoring of the algorithms after deployment, including performance, indication of biased behaviour (both known and unintended), impact on patient outcomes, and unintended consequences.
Conclusion
ML in cardiology is associated with substantial promise, great interest, and a large amount of work and resources dedicated to it, and there have been undeniable successes that are already having an impact on patient care. However, the rate of successful implementation of ML systems in clinical practice to date has been limited, and as such, the full impact on patients remains to be realised. This being said, as more systems get successfully implemented in clinical practice and in consumer-oriented and medical-grade wearable devices,
the impact of ML technology in cardiology will surely deepen over time. In this review, we have highlighted some of the considerations and challenges necessary to build and implement ML systems in cardiology. Because ML in cardiology has only just begun to return dividends, this will be an important step to prevent a future decoupling in the size of the clinical impact of ML in cardiology vs the amount of resources that are dedicated to its development, which could lead to disappointment and ultimately disinvestment in this potentially revolutionising technology. We also highlighted that ML in cardiology will reach its full potential and positive impact on patient health only by leveraging a framework where technology development and implementation planning occur in parallel as opposed to implementation being a secondary consideration.
Acknowledgements
We thank Grace Howard and Madeline Cheshire for their contribution to the literature review. Jef van den Eynde is supported by the Belgian American Educational Foundation.
Funding Sources
The authors have no funding sources to declare.
Disclosures
Dr Kutty is a consultant for GE Healthcare. The other authors have no conflicts of interest to disclose.
References
UNESCO
The race against time for smarter development. UNESCO science report.
Matheny M. Israni S.T. Ahmed M. Whicher D. Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril. National Academy of Medicine,
Washington2019
Estimating one-year risk of incident chronic kidney disease: retrospective development and validation study using electronic medical record data from the state of Maine.
Proposed Requirements for Cardiovascular Imaging–Related Machine Learning Evaluation (PRIME): a checklist: reviewed by the American College of Cardiology Healthcare Innovation Council.
00903-X&id=gr1.jpg){kind=link}
00903-X&id=gr2.jpg){kind=link}
00903-X&id=gr3.jpg){kind=link}
00903-X&id=gr4.jpg){kind=link}
00903-X&id=gr5.jpg){kind=link}