Abstract
Background
Machine learning (ML) has arrived in medicine to deliver individually adapted medical
care. This study sought to use ML to discriminate stent restenosis (SR) compared with
existing predictive scores of SR. To develop an easily applicable model, we performed
our predictions without any additional variables other than those obtained in daily
practice.
Methods
The dataset, obtained from the Grupo de Análisis de la Cardiopatía Isquémica Aguda (GRACIA)-3 trial, consisted of 263 patients with demographic, clinical, and angiographic
characteristics; 23 (9%) of them presented with SR at 12 months after stent implantation.
A methodology to work with small imbalanced datasets, based in cross-validation and
the precision/recall (PR) plots, was used, and state-of-the-art ML classifiers were
trained.
Results
Our best performing model (0.46, area under the PR curve [AUC-PR]) was developed with
an extremely randomized trees classifier, which showed better performance than chance
alone (0.09 AUC-PR, corresponding to the 9% of patients presenting SR in our dataset)
and 3 existing scores; Prevention of Restenosis With Tranilast and its Outcomes (PRESTO)-1 (0.31 AUC-PR), PRESTO-2 (0.27 AUC-PR), and Evaluation of Drug-Eluting
Stents and Ischemic Events (EVENT) (0.18 AUC-PR). The most important variables ranked
according to their contribution to the predictions were diabetes, ≥2 vessel-coronary
disease, post-percutaneous coronary intervention thrombolysis in myocardial infarction
(PCI TIMI)-flow, abnormal platelets, post-PCI thrombus, and abnormal cholesterol.
To counteract the lack of external validation for our study, we deployed our ML algorithm
in an open source calculator, in which the model would stratify patients of high and
low risk as an example tool to determine generalizability of prediction models from
small imbalanced sample size.
Conclusions
Applied immediately after stent implantation, a ML model better differentiates those
patients who will present with SR over current discriminators.
Résumé
Contexte
L’avènement de l’apprentissage automatique (AA) en médecine permet de fournir des
soins personnalisés. Cette étude portait sur l’utilisation de l’AA comme outil prédictif
du risque de resténose après l’implantation d’une endoprothèse comparativement aux
scores prédictifs existants. Pour mettre au point un modèle facilement applicable,
nous avons effectué nos prévisions sans autres variables que celles obtenues dans
la pratique quotidienne.
Méthodologie
L’ensemble de données, issu de l’essai Grupo de Análisis de la Cardiopatía Isquémica Aguda (GRACIA)-3, portait sur 263 patients dont les caractéristiques démographiques, cliniques
et angiographiques étaient consignées; 23 (9 %) d’entre eux avaient présenté une resténose
dans les 12 mois suivant l’implantation d’une endoprothèse. Une méthodologie permettant
de travailler avec de petits ensembles de données déséquilibrées, fondée sur la validation
croisée et la courbe précision-rappel (PR), a été utilisée et des classificateurs
d’AA de pointe ont été entraînés.
Résultats
Notre modèle le plus performant (aire sous la courbe PR [ASC-PR] : 0,46) a été mis
au point à l’aide d’un classificateur d’arbres extrêmement aléatoires, présentant
une meilleure performance que le hasard seul (ASC-PR : 0,09, correspondant aux 9 %
de patients présentant une resténose dans notre ensemble de données), et à l’aide
de trois scores existants, à savoir : Prevention of Restenosis With Tranilast and its Outcomes (PRESTO)-1 (ASC-PR : 0,31), PRESTO-2 (ASC-PR : 0,27) et Evaluation of Drug-Eluting Stents and Ischemic Events (EVENT) (ASC-PR : 0,18). Les variables les plus importantes, classées en fonction
de leur contribution aux prévisions, étaient le diabète, la coronaropathie multitronculaire
(≥ 2), le flux TIMI (thrombolyse de l’infarctus du myocarde) après une intervention
coronarienne percutanée, les anomalies plaquettaires, la thrombose après une intervention
coronarienne percutanée et un taux de cholestérol anormal. Pour contrebalancer le
manque de validation externe dans le cadre de notre étude, nous avons déployé notre
algorithme d’AA dans un calculateur libre et ouvert, où le modèle opérait la stratification
des patients très à risque et peu à risque à titre d’outil d’exemplification en vue
de déterminer la généralisabilité des modèles prédictifs à partir d’un échantillon
déséquilibré de petite taille.
Conclusions
Un modèle d’AA appliqué immédiatement après l’implantation d’une endoprothèse cerne
mieux les patients qui présenteront une resténose que les indicateurs actuels.
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Canadian Journal of CardiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Current reperfusion strategies for ST elevation myocardial infarction: a Canadian perspective.Can J Cardiol. 2004; 20: 525-533
- The evolution of coronary stents: a brief review.Can J Cardiol. 2014; 30: 35-45
- Restenosis after coronary angioplasty: an overview.J Am Coll Cardiol. 1991; 17: 2B-13B
- In-stent restenosis in the drug-eluting stent era.J Am Coll Cardiol. 2010; 56: 1897-1907
- Machine learning in medicine.N Engl J Med. 2019; 380: 1347-1358
- Artificial intelligence and machine learning in cardiology.JACC Cardiovasc Interv. 2019; 12: 1312-1314
- Applications of artificial intelligence in cardiology: the future is already here.Rev Esp Cardiol. 2019; 72: 1065-1075
- The role of physicians in the era of big data.Can J Cardiol. 2019; 36: 19-21
- Clinical and angiographic predictors of restenosis after percutaneous coronary intervention: insights from the Prevention of Restenosis With Tranilast and Its Outcomes (PRESTO) trial.Circulation. 2004; 109: 2727-2731
- Predicting restenosis of drug-eluting stents placed in real-world clinical practice: derivation and validation of a risk model from the EVENT registry.Circ Cardiovasc Interv. 2010; 3: 327-334
- Defining coronary restenosis: newer clinical and angiographic paradigms.Circulation. 1993; 88: 1310-1323
- Role of the paclitaxel-eluting stent and tirofiban in patients with ST-elevation myocardial infarction undergoing postfibrinolysis angioplasty: the GRACIA-3 randomized clinical trial.Circ Cardiovasc Interv. 2010; 3: 297-307
- Sensitivity analysis of kappa-fold cross validation in prediction error estimation.IEEE Trans Pattern Anal Mach Intell. 2010; 32: 569-575
- Scikit-learn: machine learning in Python.J Mach Learn Res. 2011; 12: 2825-2830
- Bias in error estimation when using cross-validation for model selection.BMC Bioinformatics. 2006; 7: 91
- Random search for hyper-parameter optimization.J Mach Learn Res. 2012; 13: 281-305
- Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.Ann Intern Med. 2015; 162: W1-W73
- The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets.PLoS One. 2015; 10e0118432
- The relationship between precision-recall and ROC curves.Proceedings of the 23rd International Conference on Machine Learning (ICML-06). 2006; : 233-240
- An experimental comparison of cross-validation techniques for estimating the area under the ROC curve.Comput Stat Data Anal. 2011; 55: 1828-1844
- Choosing between two learning algorithms based on calibrated tests.Proceedings of the 20th International Conference on Machine Learning (ICML-03). 2003; : 51-58
- Random forests.Mach Learn. 2001; 45: 5-32
- Drug-eluting stents versus bare-metal stents for acute coronary syndrome.Cochrane Database Syst Rev. 2017; 8: CD012481
- Impact of scheduled angiographic follow-up in patients treated with primary percutaneous coronary intervention for ST-segment elevation myocardial infarction.J Interv Cardiol. 2013; 26: 319-324
- Plasma phospholipids and sphingolipids identify stent restenosis after percutaneous coronary intervention.JACC Cardiovasc Interv. 2017; 10: 1307-1316
Article info
Publication history
Published online: February 07, 2020
Accepted:
January 14,
2020
Received:
October 8,
2019
Footnotes
See editorial by Avram et al., pages 1574—1576 of this issue.
See page 1631 for disclosure information.
Identification
Copyright
© 2020 Canadian Cardiovascular Society. Published by Elsevier Inc. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- The Rise of Open-Sourced Machine Learning in Small and Imbalanced Datasets: Predicting In-Stent RestenosisCanadian Journal of CardiologyVol. 36Issue 10
- PreviewAfter the successful placement of a coronary stent and revascularization, in-stent restenosis (ISR) occurs in up to 12% of patients and represents the gradual failure of the stent by lumen renarrowing.1 Clinically, ISR is important to recognize because it is usually associated with recurrent angina symptoms, a higher risk for acute coronary syndrome, and increased mortality.1 The ability to accurately predict ISR would enable closer monitoring of those patients at higher risk of ISR or the consideration of alternate therapies.
- Full-Text
- Preview
