Nomogram for prediction of in-hospital mortality rate in children with congenital heart disease in pediatric intensive care: establishment and external validation

Introduction

Congenital heart disease (CHD) is a global pediatric concern, affecting approximately 1% of newborns (1). In recent years, advancements in perinatal care, diagnosis, medical management, and surgical repair techniques have significantly improved the prognosis of affected children (2,3). However, the heterogeneity of CHD phenotypes and its complex, multifactorial etiology make it challenging to uncover precise pathogenesis, hindering prenatal interventions for CHD patients. Consequently, despite the advancements (4,5), the incidence of CHD has remained unchanged.

Additionally, CHD is a common reason for children to be admitted to the intensive care unit (ICU) (6). CHD children admitted to the ICU tend to have more severe and complex conditions, leading to relatively higher in-hospital mortality rates. However, there is limited research on risk factors for in-hospital mortality in CHD children, and no specific models have been developed to predict their in-hospital mortality rates. The in-hospital mortality rate among CHD patients may be influenced by various factors, such as heart rate, blood pressure, and oxygen saturation; however, the relative contributions of each factor remain unclear. Nomograms, as a visualization tool, can quantify the weights of variables in a model and calculate the probability of outcomes (7), greatly enhancing interpretability.

Therefore, this study aimed to identify risk factors for in-hospital mortality in CHD children in the ICU and develop a predictive nomogram model utilizing these risk factors to estimate the in-hospital mortality rate. We present this article in accordance with the TRIPOD reporting checklist (available at https://tp.amegroups.com/article/view/10.21037/tp-2024-506/rc).

Methods

Study population and data

The study population was derived from the paediatric intensive care unit (PICU) database. The PICU database is a large, pediatric-specific, single-center database maintained by the Laboratory for Computational Physiology at the Massachusetts Institute of Technology (8). This database contains information related to children admitted to the ICU of a large children’s hospital in China from 2010 to 2018, including vital sign measurements, medications, laboratory measurements, fluid balance records, diagnostic codes, demographic information, etc. (9). Patient information in the PICU database has been anonymized, eliminating the need for informed consent.

Inclusion criteria were as follows: (I) children diagnosed with CHD were diagnosed according to the International Classification of Diseases, ninth and tenth editions (ICD-9 and ICD-10); (II) intensive care treatment was required, with definitive admission to the ICU; (III) the medical record was complete, including follow-up data and all medical records from the hospital stay. Exclusion criteria were: (I) patients who were not being treated in an ICU; (II) lack of complete follow-up records to provide subsequent clinical data; (III) patients who died within 24 hours of birth; (IV) patients with shock; (V) age >12 years. For patients with multiple admissions, data at the first admission were used. The Medical Information Mart for Intensive Care III (MIMIC-III) dataset is consistent with the PICU dataset. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments.

Patient characteristics

Structured query language (SQL) was used to extract patient demographics, comorbidities, surgical history, laboratory indicators, in-hospital mortality rates, and other data from the PICU database. Demographic information included gender, age in months; comorbidities included prematurity, low birth weight, respiratory failure (RF), sepsis, respiratory distress syndrome (RDS), pneumonia, intracranial haemorrhage; surgical history included ventricular septal defect (VSD) repair, atrial septal defect (ASD) repair, patent ductus arteriosus (PDA) repair (without extracorporeal circulation), tetralogy of Fallot repair, cardiac angiography, total anomalous pulmonary venous drainage (TAPVC); laboratory markers included adenosine deaminase, serum albumin-globulin ratio (ALB/GLB), albumin, alkaline phosphatase, albumin, alanine transaminase (ALT), amylase, anion gap, aspartate aminotransferase (AST), brain natriuretic peptide (BNP), base excess, basophils, direct bilirubin, indirect bilirubin, total bilirubin, calcium, bicarbonate, carboxyhemoglobin, chloride, creatine kinase MB (CKMB), creatine kinase, creatinine, cystatin C, D-dimer, eosinophils, fibrinogen, γ-glutamyltransferase, globulin, hematocrit, hemoglobin, lactate dehydrogenase, lactate, lymphocytes, magnesium ion, mean corpuscular hemoglobin (MCH), mean corpuscular hemoglobin concentration (MCHC), mean corpuscular volume (MCV), mean platelet volume, monocytes, neutrophils, oxygen saturation, partial pressure of carbon dioxide (pCO₂), plateletcrit (PCT), pH, phosphate, platelet distribution width (RDW), platelet count, oxygen partial pressure (pO₂), red blood cells, sodium, total cholic acid, triglycerides, urea, uric acid, and white blood cells (WBC). Variables with missing values exceeding 30% were excluded. If the patient had multiple admissions, the first admission was analysed. For laboratory indicators, data were extracted within 48 hours of the patient’s admission, and the first measurement was analysed if there were multiple measurements within 48 hours.

Statistical analysis

Normally distributed continuous variables were presented as mean ± standard deviation and were compared using the t-test, non-normally distributed continuous variables were presented as median with interquartile range and compared using the Mann-Whitney U test. Categorical variables were presented as frequencies and percentages, and between-group comparison was performed using Pearson’s Chi-squared test or Fisher’s exact test. To prevent overfitting, the total study cohort was randomly divided into a training cohort and a validation cohort. The training cohort was used for model training, and the validation cohort was used to validate the performance of the model. Variable selection was performed using single-factor Cox regression and stepwise Cox regression based on Akaike information criterion (AIC) in the training cohort. Variables with statistical significance in univariate analysis were included in a stepwise Cox regression. The significant variables from the stepwise Cox regression were then incorporated into the final Cox regression model and visualized as a nomogram. Calibration curves and receiver operator characteristic (ROC) curves were generated by comparing observed values with predicted values from the nomogram to assess the predictive performance of the constructed nomogram. To avoid multicollinearity between variables, variables with a variance inflation factor (VIF) ≥5 were excluded. All analyses were conducted using R 4.3.0, and a two-tailed P value <0.05 was considered statistically significant.

Results

Study population

A total of 2,274 children with CHD were identified from the PICU database. After excluding patients with missing data (n=34) and patients with a follow-up time <1 day (n=9), 2,231 patients were finally included in the analysis, with a mean age of 25.61 months. In the total study cohort, 77 (3.45%) patients died in the hospital. There were 1,070 male patients (48.0%) and 1,161 female patients (52.0%). The patients were randomly divided into a training cohort (n=1,562) and a validation cohort (n=669) at a 7:3 ratio. The two cohorts were comparable without statistically significant differences in patient characteristics (Table 1). A total of 601 children with congenital heart disease in the MIMIC-III dataset were included in the study.

Model building

In-hospital mortality was used as the outcome measure. Feature selection was performed through single-factor Cox regression and stepwise Cox regression (Table 2). According to the single-factor Cox regression, such variables as patient age, albumin, amylase, ASD repair, AST, base excess, direct bilirubin, indirect bilirubin, total bilirubin, BNP, calcium ion, chloride ion, total cholesterol, creatine kinase, creatinine, cystatin C, D-dimer, eosinophils, fibrinogen, glutamyltransferase, globulin, hematocrit, hemoglobin, lactate, lactate dehydrogenase, lymphocytes, magnesium ion, MCH, MCHC, MCV, neutrophils, oxygen saturation, pCO₂, PCT, pH, phosphate, platelet count, PDW, pO₂, potassium, prealbumin, total protein, partial thromboplastin time (PTT), TAPVC repair, VSD repair showed statistical significance and were considered as potential predictive factors. These variables were further screened using stepwise Cox regression. Lymphocyte percentage [hazard ratio (HR): 1.097, 95% confidence interval (CI): 1.038–1.160], magnesium ion (HR: 1.002, 95% CI: 1.001–1.002), neutrophil percentage (HR: 1.111, 95% CI: 1.050–1.175), pO₂ (HR: 0.987, 95% CI: 0.981–0.993), PTT (HR: 1.033, 95% CI: 1.020–1.047), and VSD repair (HR: 0.117, 95% CI: 0.028–0.494) were identified as statistically significant predictors of patient prognosis. These variables were included in the final Cox proportional hazards model and visualized as a nomogram (Figure 1). At the same time, we extracted the variables from the column-line diagrams in the MIMIC-III database for external validation. The total points on the nomogram were the sum of the scores assigned to each variable and vertically corresponded to the scales on the predictor (7-, 14-, and 30-day), which represent the survival probability of patients at specific time intervals.

Figure 1 Nomogram of the Cox proportional risk model. pO₂, oxygen partial pressure; PTT, partial thromboplastin time; VSD, ventricular septal defect.

Model validation

ROC curves and calibration curves were plotted to validate the performance of the model. The ROC curve analysis showed that the model had a favorable discriminative ability with AUCs of 0.940, 0.857, and 0.776 for predicting in-hospital mortality at 7, 14, and 30 days, respectively, in the training cohort, and AUCs of 0.921, 0.858, and 0.699 in the validation cohort (Figure 2). The calibration curves demonstrated good consistency of the model without significant over- or underestimation of patient mortality risk (Figure 3). In the external dataset, the AUC of the model for predicting 7-, 14-, and 30-day in-hospital mortality were 0.732, 0.722, and 0.629, respectively (Figure 4A). The calibration curve also shows equally good consistency (Figure 4B). The model exhibited excellent predictive performance and could provide adequate prognostic information.

Figure 2 ROC curves of the model. (A) Training set; (B) validation set. AUC, area under the curve; CI, confidence interval; ROC, receiver operator characteristic.

Figure 3 Calibration curves of the model. (A) Training set; (B) validation set.

Figure 4 ROC curve (A) and calibration curve (B) of external validation. AUC, area under the curve; CI, confidence interval; ROC, receiver operator characteristic.

Discussion

The final model constructed in this study included six predictive factors: lymphocyte percentage, magnesium ion, neutrophil percentage, pO₂, PTT, and VSD repair. These indicators in the model are clinically accessible and simple. The analysis of ROC curves and calibration curves revealed that the model had favorable predictive performance and could provide effective prognostic information to aid in clinical decision making.

Among the significant predictive factors, our study results indicated that elevated levels of lymphocyte percentage (HR: 1.097, 95% CI: 1.038–1.160, P=0.0010), neutrophil percentage (HR: 1.111, 95% CI: 1.050–1.175, P=0.0002), magnesium ion (HR: 1.002, 95% CI: 1.001–1.002, P<0.0001), and PTT (HR: 1.033, 95% CI: 1.020–1.047, P<0.0001) significantly increased the risk of in-hospital mortality in CHD patients. Based on the results of this study, elevated neutrophil percentages, lymphocyte percentages, and decreased partial pO₂ were found to be significantly associated with the risk of death in children with CHD. Specifically, elevated neutrophil percentages may reflect the body’s stress response to inflammation or infection, which is particularly common in children with congenital heart disease in intensive care. As the main effector cells of the inflammatory response, an increase in the percentage of neutrophils often signals an increase in the inflammatory state, which in turn leads to deterioration of cardiac function and poor prognosis (10). In addition, an elevated percentage of lymphocytes likewise reveals a poor prognosis. This may be related to the role of lymphocytes in the immune response. Although lymphocytes play an important role in normal immune function, their abnormal elevation may indicate the presence of chronic inflammation or infection, which negatively affects the survival of children with congenital heart disease (11,12). In addition, the results also showed that pO₂ (HR: 0.987, 95% CI: 0.981–0.993, P<0.0001) and VSD repair surgery (HR: 0.117, 95% CI: 0.028–0.494, P=0.0035) were significant protective factors for in-hospital mortality. In patients with CHD, pO₂ reflects the oxygen content in the blood and provides information about the patient’s oxygenation status (13). For CHD patients, a lower pO₂ than the normal range indicates worsening cardiac condition. Furthermore, high-oxygen therapy is considered a potential prenatal intervention for genetic heart abnormalities. The study by Doll et al. has shown that intermittent mild hyperoxia during pregnancy can partially rescue genetic heart abnormalities in mouse models (14). A decrease in the partial pO₂, on the other hand, is a direct reflection of the deterioration of the oxygenation status of the child. Our findings suggest that low pO₂ is significantly associated with poor prognosis in children with CHD in the ICU. This finding is consistent with previous studies that hypoxemia may be a manifestation of acute RDS, while hypoxemia may increase the risk of death in children by triggering systemic inflammatory response and multi-organ dysfunction (15). In addition, low pO₂ may further exacerbate cardiac load and lead to deterioration of cardiac function, thus creating a vicious circle (16,17). Therefore, early recognition and correction of hypoxemia are essential to improve the prognosis of children with CHD. Magnesium, the most abundant intracellular divalent cation, plays a key role in regulating neuronal excitation, cardiac conduction, and myocardial contraction by modulating various ion transporters, including potassium and calcium channels, in the heart. Therefore, magnesium may have significant implications for the pathogenesis and progression of cardiovascular diseases (18). Some studies have indicated that intraoperative magnesium intake can reduce the risk of postoperative arrhythmias in congenital heart surgery (19,20), but have not shown a direct effect of elevated magnesium ions in the blood on patient mortality. In our study cohort, the mean and standard deviation of blood magnesium ions were 0.88 (0.12) mmol/L, and there were 175 patients (7.84% of the total cohort) with higher magnesium ion levels (>1.0 mmol/L). This suggested that the blood magnesium levels in the majority of patients were within the normal range. The elevation of blood magnesium ions may be attributed to prophylactic intake during or before surgery, and patients who require magnesium intake for arrhythmia prevention have a higher risk of mortality. However, the possibility that higher than normal magnesium levels increase the risk of mortality in patients with CHD cannot be ruled out. Furthermore, the results indicated that an elevated PTT significantly increased the risk of patient mortality. High PTT levels are typically caused by deficiencies in clotting factors or excessive anticoagulant factors (21). Patients with CHD may experience two types of coagulation disorders. On one hand, these patients may develop secondary erythrocytosis, leading to increased blood viscosity and a higher risk of thrombus formation. On the other hand, the reduced plasma volume results in severe deficiencies of various clotting proteins, including platelets, fibrinogen, and other clotting factors, which increases the risk of perioperative bleeding (22). Therefore, monitoring the coagulation status during the perioperative period is crucial for CHD patients.

In this study, we utilized a nomogram to visualize the Cox proportional hazards model. The nomogram is intuitive, capable of presenting multivariable relationships, and easy to calculate. By presenting the relationships between multiple variables graphically, the nomogram makes complex mathematical relationships visually accessible and easily understood, eliminating the need for users to delve into mathematical details while extracting information from the nomogram. Additionally, it enables the display of relationships among multiple variables, facilitating the analysis of multifactorial influences. Within the nomogram, total points can be calculated by simply adding the points for each variable, allowing for the estimation of patients’ survival probabilities at different time intervals without complex calculations (23). Compared with complex machine learning models, the nomogram is highly interpretable (24).

In summary, our model exhibits strong interpretability, ease of use, data accessibility, and favorable predictive performance. However, there are several limitations in this study. Firstly, our model was established based on a single-center database from China, and its applicability to populations with significant genetic differences may be limited. Second, due to the restrictive nature of ICD-9 and ICD-10 codes, some diseases cannot be clearly distinguished, such as ASD and patent foramen ovale. In addition, we did not include some variables in the analysis, such as echocardiographic examination data, due to a high level of missing data in the database. Finally, due to the limitations of the variables in the database, some variables of interest to us were not included in the analysis, such as the Glenn and Fontan surgeries, and the history of drug, tobacco and alcohol use of the parents of the children.

Conclusions

We have developed a nomogram model based on Cox regression, which has incorporated six predictive factors, including lymphocyte percentage, magnesium ion, neutrophil percentage, pO₂, PTT, and VSD repair. The model has favorable performance and can assist in clinical decision-making by providing clinicians with effective predictive information.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://tp.amegroups.com/article/view/10.21037/tp-2024-506/rc

Peer Review File: Available at https://tp.amegroups.com/article/view/10.21037/tp-2024-506/prf

Funding: None.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tp.amegroups.com/article/view/10.21037/tp-2024-506/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All analyses were based on previous published studies, and thus no ethical approval and patient consent are required. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Tsao CW, Aday AW, Almarzooq ZI, et al. Heart Disease and Stroke Statistics-2022 Update: A Report From the American Heart Association. Circulation 2022;145:e153-639. Erratum in: Circulation 2022;146:e141. [Crossref] [PubMed]
2. Lynn MM, Salemi JL, Kostelyna SP, et al. Lesion-Specific Congenital Heart Disease Mortality Trends in Children: 1999 to 2017. Pediatrics 2022;150:e2022056294. [Crossref] [PubMed]
3. Ison HE, Griffin EL, Parrott A, et al. Genetic counseling for congenital heart disease - Practice resource of the National Society of Genetic Counselors. J Genet Couns 2022;31:9-33. [Crossref] [PubMed]
4. Pierpont ME, Brueckner M, Chung WK, et al. Genetic Basis for Congenital Heart Disease: Revisited: A Scientific Statement From the American Heart Association. Circulation 2018;138:e653-711. [Crossref] [PubMed]
5. Cerrone M, Remme CA, Tadros R, et al. Beyond the One Gene-One Disease Paradigm: Complex Genetics and Pleiotropy in Inheritable Cardiac Disorders. Circulation 2019;140:595-610. [Crossref] [PubMed]
6. Dempsey EM, El-Khuffash AF. Objective cardiovascular assessment in the neonatal intensive care unit. Arch Dis Child Fetal Neonatal Ed 2018;103:F72-7. [Crossref] [PubMed]
7. Wang X, Lu J, Song Z, et al. From past to future: Bibliometric analysis of global research productivity on nomogram (2000-2021). Front Public Health 2022;10:997713. [Crossref] [PubMed]
8. Reyna MA, Sadr N, Alday EAP, Gu A, Shah AJ, Robichaux C, et al., editors. Will two do? Varying dimensions in electrocardiography: the PhysioNet/Computing in Cardiology Challenge 2021. 2021 Computing in Cardiology (CinC); 2021: IEEE.
9. Li H, Zeng X, Yu G. Paediatric Intensive Care database (version 1.1.0). (2020). PhysioNet. doi: 10.13026/32x9-wv38.10.13026/32x9-wv38
10. Hardee I, Wright L, McCracken C, et al. Maternal and Neonatal Outcomes of Pregnancies in Women With Congenital Heart Disease: A Meta-Analysis. J Am Heart Assoc 2021;10:e017834. [Crossref] [PubMed]
11. Neunert CE, Arnold DM, Grace RF, et al. The 2022 review of the 2019 American Society of Hematology guidelines on immune thrombocytopenia. Blood Adv 2024;8:3578-82. [Crossref] [PubMed]
12. Taksande A, Jameel PZ. Critical Congenital Heart Disease in Neonates: A Review Article. Curr Pediatr Rev 2021;17:120-6. [Crossref] [PubMed]
13. Temel HH, Kumbasar U, Büber E, et al. Comparison of antioxidant reserve capacity of children with acyanotic & cyanotic congenital heart disease. Indian J Med Res 2020;152:626-32. [Crossref] [PubMed]
14. Doll CF, Pereira NJ, Hashimi MS, et al. Gestational intermittent hyperoxia rescues murine genetic congenital heart disease in part. Sci Rep 2021;11:6608. [Crossref] [PubMed]
15. Chang CC, Lin YC, Chen TC, et al. High-Flow Nasal Cannula Therapy in Children With Acute Respiratory Distress With Hypoxia in A Pediatric Intensive Care UnitA Single Center Experience. Front Pediatr 2021;9:664180. [Crossref] [PubMed]
16. Berman W Jr, Wood SC, Yabek SM, et al. Systemic oxygen transport in patients with congenital heart disease. Circulation 1987;75:360-8. [Crossref] [PubMed]
17. Taka H, Douguchi T, Miyamoto A, et al. Oxygen Delivery During Cardiopulmonary Bypass in Pediatric Patients With Congenital Heart Disease: Association With Postoperative Acute Kidney Injury. J Cardiothorac Vasc Anesth 2025;39:702-10. [Crossref] [PubMed]
18. Tangvoraphonkchai K, Davenport A. Magnesium and Cardiovascular Disease. Adv Chronic Kidney Dis 2018;25:251-60. [Crossref] [PubMed]
19. Khalil AAMA, Ali ASA, Soliman AMRA, et al. Amiodarone and Magnesium Sulphate in Arrhythmia Prophylaxis in Pediatric Congenital Heart Diseases Patients Undergoing Cardiopulmonary Bypass. The Egyptian Journal of Hospital Medicine 2021;83:882-8. [Crossref]
20. He D, Aggarwal N, Zurakowski D, et al. Lower risk of postoperative arrhythmias in congenital heart surgery following intraoperative administration of magnesium. J Thorac Cardiovasc Surg 2018;156:763-770.e1. [Crossref] [PubMed]
21. Adcock DM, Moore GW, Montalvão SL, et al. Activated Partial Thromboplastin Time and Prothrombin Time Mixing Studies: Current State of the Art. Semin Thromb Hemost 2023;49:571-9. [Crossref] [PubMed]
22. Zabala LM, Guzzetta NA. Cyanotic congenital heart disease (CCHD): focus on hypoxemia, secondary erythrocytosis, and coagulation alterations. Paediatr Anaesth 2015;25:981-9. [Crossref] [PubMed]
23. Balachandran VP, Gonen M, Smith JJ, et al. Nomograms in oncology: more than meets the eye. Lancet Oncol 2015;16:e173-80. [Crossref] [PubMed]
24. Guidotti R, Monreale A, Ruggieri S, et al. A survey of methods for explaining black box models. ACM computing surveys (CSUR) 2018;51:1-42. [Crossref]