Evaluation of radiomics as an assistant tool for radiologists in the diagnosis of necrotizing enterocolitis

Introduction

Necrotizing enterocolitis (NEC) is a serious gastrointestinal emergency that primarily affects premature infants, and it is one of the foremost causes of neonatal mortality. Despite advances in perinatal care and neonatal management, preterm infants are still at risk of NEC due to their underdeveloped intestinal function, making them more susceptible to gut microbiota infections and intestinal wall damage (1). Once NEC develops, it can progress rapidly, posing a significant risk to the newborn’s life and potentially resulting in long-lasting complications such as short bowel syndrome, intestinal narrowing, and neurological developmental abnormalities (2). Therefore, timely diagnosis of NEC can help physicians to make appropriate measures to reduce the risk of serious complications, decrease mortality, and improve the effectiveness of NEC treatment (3).

Imaging examination plays an important role in the diagnosis of NEC and in the follow-up of disease changes. In recent years, many studies have shown that abdominal ultrasound has a high positive rate in the early diagnosis of NEC (4,5), can dynamically monitor and evaluate the condition of children, and has high clinical value (6). Nonetheless, abdominal ultrasound is not widely used in the clinical diagnosis and monitoring of NEC, and abdominal radiograph is still the imaging method of choice for evaluating children with suspected NEC. Doctors rely on clinical manifestations and radiograph examinations to diagnose and stage NEC (2). However, the early clinical symptoms and imaging features of NEC are often nonspecific and can be easily mistaken for other gastrointestinal diseases. Such misidentification can lead to delays in treatment, potentially leading to severe health repercussions for the patient. To address this issue, researchers are exploring biomarkers as a tool for predicting NEC (7). Radiomics is a promising tool for improving the diagnosis, treatment, and management of diseases (8). By analyzing the radiomics features of imaging data, including complex patterns that are difficult for the human eye to recognize or quantify, it can discover new disease characteristics and identify biomarkers (8,9). Through the analysis of radiomics features, there is an opportunity to discover new diagnostic and prognostic markers for NEC, which can then be used to develop targeted therapies that selectively inhibit these pathways or processes, potentially leading to more effective treatments with fewer side effects. This allows doctors to intervene early, providing more precise treatment and better health management for affected children, reducing the occurrence of severe complications associated with NEC.

Currently, there are some radiomics studies based on NEC radiographs. Some studies (10,11) have shown that extracting radiomics features from radiographic images of children with NEC and constructing a radiomics model helped physicians to predict whether a patient with NEC needed surgical treatment. However, their radiomics study did not take portal venous gas on radiographs into account when considering the region of interest (ROI), an imaging sign that is important in the actual radiologist’s work (12). Portal venous gas is a highly significant imaging marker in NEC, occurring in 10–30% of cases. It is regarded as an extension of intramural gas in NEC and serves as a specific indicator of panintestinal necrosis. It demonstrates high specificity but low sensitivity for diagnosing NEC stage ≥II (13). In addition to this, their research primarily assessed the model’s sensitivity and specificity, often seeking direct comparison with radiologists’ performance. However, it suffered from a lack of external validation and did not assess the value of radiomics as assist tool when integrated with radiologists for diagnostic purposes.

Herein, our research employed three datasets (training set, test set and temporal validation set) to develop and validate a radiomics model. In the temporal validation set, we conducted a parallel test using radiologists’ diagnoses and radiomics diagnoses to evaluate the assistant diagnostic role of the radiomics model. We present this article in accordance with the TRIPOD reporting checklist (available at https://tp.amegroups.com/article/view/10.21037/tp-2024-496/rc).

Methods

The workflow chart in this study is shown in Figure 1. The steps included model design, statistics generation.

Figure 1 Workflow for radiomics feature extraction, model building, and model validation. NEC, necrotizing enterocolitis.

Data source

Our study was a retrospective analysis approved by the institutional ethics committee of the Children’s Hospital of Fudan University (No. 2023-196). The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. Informed consent was obtained from all the patients’ legal guardians, and the data were anonymized to ensure confidentiality. We collected abdominal radiographs of NEC patients at the time they were diagnosed in accordance with the modified Bell staging system (14) and included them in the NEC group. We reviewed the electronic medical records and abdominal radiograph requests of hospitalized NEC patients, and selected the abdominal radiograph at the time clinicians suspected NEC-related symptoms. Meanwhile, we included in the non-NEC group patients within six months of age who were hospitalized during the same period, had not been diagnosed with NEC, and did not suffer from other intestinal diseases. Finally, we collected 484 images (Table 1) from NEC group (n=262) and non-NEC group (n=222), who underwent abdominal radiography examinations from 2016–2022. The data from 2018-2022 were divided into a training set (n=246) and a test set (n=105) in a ratio of 7:3, with the data from 2016–2017 serving as a temporal validation set (n=133). In order to understand the diagnostic situation of NEC by radiologists, we analyzed the radiological reports of 86 NEC patients in the temporal validation set to see if the conclusions indicated NEC.

Imaging segmentation

All patients had undergone abdominal imaging on CR or DR. We acquired images from the picture archiving and communication system (PACS). We performed segmentation of radiograph image ROI using 3D Slicer software (version 5.2.2, www.slicer.org) (15). Due to the complexity of NEC lesions on radiograph images and the need for meticulous image segmentation by experienced diagnostic physicians (11), which is time-consuming, we opted for a simple rectangular segmentation approach for the abdominal area (Figure 2). The upper boundary of the rectangle is aligned with the horizontal level of the right diaphragm, and the lower boundary is aligned with the upper edge of the hip joint, encompassing the entire abdomen while making an effort to exclude the femoral heads. The work of image segmentation task was performed by a physician with 1 year of working experience in radiology supervised by radiologists with 18 and 28 years of working experience.

Figure 2 Abdominal segmentation for radiograph.

Feature extraction

We utilized the open-source software FeAture Explorer (FAE) V.0.5.5 (16) to extract features from ROI. This software employs Pyradiomics as the backend for feature extraction. Before extracting radiomics features, we normalized all images by scaling their intensities to a range of 0 to 1,000 and resampling them to the same resolution (1 mm × 1 mm × 1 mm) to mitigate heterogeneity biases among the data. Subsequently, we performed radiomics feature extraction on the FAE platform. These images encompassed 9 types, namely “Original”, “Wavelet Transform”, “Laplacian of Gaussian”, “Square”, “Square Root”, “Logarithm”, “Exponential”, “Gradient”, and “LBP2D”. From these 9 types of images, we extracted 7 classes of features, including first order, shape-based, gray level co-occurrence matrix (GLCM), gray level run length matrix (GLRLM), gray level size zone matrix (GLSZM), gray level dependence matrix (GLDM), neighbourhood gray tone difference matrix (NGTDM). The definitions and formulas of the extracted radiomic features were obtained from the official pyradiomics documentation (https://pyradiomics.readthedocs.io/).

Features screening and model development

We standardized radiomics features using the Z-score method. In the training set, we initially applied the minimum redundancy maximum relevance algorithm (mRMR) to select the top 20 features (17). Subsequently, we employed ten-fold cross-validation for the least absolute shrinkage and selection operator regression (LASSO). The mRMR algorithm is a feature selection method designed to identify the most representative subset of features from high-dimensional data. Its core principle is to select features that are highly correlated with the target variable while maintaining low redundancy among themselves. This approach ensures that the selected features are both informative and non-redundant, improving model performance and interpretability. The fundamental principle of the LASSO algorithm involves introducing a penalty term based on the L1 norm, which is the sum of the absolute values of all regression coefficients, to shrink the regression coefficients of all variables. As the penalty parameter (λ) increases, more coefficients are shrunk to zero, while variables not shrunk to zero are retained for model training (18). Finally, we used the selected features to construct the diagnostic model. We calculated weights for these features and their regression coefficients to obtain the predicted probability and evaluated the model on different datasets.

Statistical analysis

We employed Chi-square tests or Fisher’s exact tests to compare categorical variables. For the comparison of quantitative variables, t-tests or Mann-Whitney U tests were utilized. Logistic regression models were applied for analysis, and we calculated 95% confidence interval (CI). Receiver operating characteristic (ROC) curve analysis and the calculation of the area under the curve (AUC) were performed to assess the predictive performance of image features. The cutoff value was determined by Youden index, and accuracy, sensitivity, and specificity were calculated accordingly. All tests were two-tailed, and P<0.05 was considered statistically significant. Statistical analyses were conducted using R software (version 4.2.2, www.rproject.org) and SPSS 20. P values less than 0.05 were deemed to indicate statistically significant differences.

Results

Demographic information analysis

The final dataset included 484 patients. The clinical characteristics of these patients are summarized in Table 1. Among the 262 NEC patients, the distribution across stages was as follows: 128 cases at Stage IIA, 57 cases at Stage IIB, 59 cases at Stage IIIA, and 18 cases at Stage IIIB. Significant differences were observed in age, gestational age, birth weight and Apgar score between the NEC group and the non-NEC group. Additionally, gestational age and birth weight showed significant differences across all three datasets. Across the three datasets, there were no significant differences in patient sex (P=0.99) or age (P=0.37). There were more boys than girls in all three datasets.

Radiomics model performance

In this study, a total of 1,037 features were initially extracted, and after eliminating irrelevant features, 983 features remained. Through mRMR and LASSO feature dimensionality reduction (Figure 3), 18 features were selected for constructing the radiomics model, with the corresponding LASSO coefficients for radiomics model presented in Table 2. Among the 18 radiomic features selected in abdominal radiographs of NEC, the distribution can be categorized into four main types. Texture features represent the largest proportion with 10 features (55.6%), including GLCM-based (4 features), GLRLM-based (2 features), and GLSZM-based metrics (4 features). Wavelet features constitute the second-largest group with 5 features (27.8%). First-order features account for 2 features (11.1%), while shape features represent the smallest category with only 1 feature (5.6%). Figure 4 illustrates the heatmap of feature correlations in the training set, ranging from −0.44 to 0.52. The AUCs of the radiomics model (Table 3) on the training set, test set, and temporal validation set were 0.82, 0.74, and 0.71, respectively; the sensitivities were 0.75, 0.57, and 0.69, respectively. Figure 5 shows the ROC curve and calibration curve of different datasets.

Figure 3 LASSO regression analysis for feature selection and coefficient determination. (A) The curve illustrates the change in binomial deviance for the subset of features with the highest predictability as the parameter λ varies in the LASSO regression model. The horizontal axis represents the logarithmic values of λ, and the vertical axis represents binomial deviance. The upper numbers indicate the number of selected features, and the optimal value of λ, where the binomial deviance of the model is minimized, is denoted by a vertical dashed line. (B) The trend chart illustrates the variation of coefficients for radiomics features obtained through LASSO regression as log(λ) changes. LASSO, least absolute shrinkage and selection operator.

Figure 4 The heatmap of 18 radiomic features correlations in the training set (n=246).

Figure 5 ROC curves and calibration curves of radiomics models in different datasets. (A) The ROC curve for the radiomics model in different datasets. (B) Calibration curve of training set. (C) Calibration curve of test set. (D) Calibration curve of temporal validation set. ROC, receiver operating characteristic.

Validation in temporal validation set

In the temporal validation set of 86 NEC patients, radiologists missed five cases due to the absence of typical features such as pneumatosis intestinalis or portal venous gas. However, the radiomics model successfully identified three of these five missed cases, demonstrating its potential to detect subtle imaging patterns that may be overlooked in conventional radiological assessments.

The Chi-square test showed that the combined approach (parallel testing) of radiologists and the radiomics model was better than the diagnosis made by the radiologist alone (P=0.003). Table 4 shows the results of the chi-square test of this comparison. The combined diagnostic approach of radiologists and radiomics outperformed the radiologist-only assessment, showing improved diagnostic accuracy and fewer missed diagnoses. This result highlights the value of integrating radiomics with radiologist interpretation, potentially enhancing the detection rate and reducing the likelihood of missed cases compared to radiologist assessments alone.

Discussion

In this study, we employed a rectangular ROI segmentation to extract radiomics features, resulting in a conventional radiomics model with certain predictive performance. This model not only exhibited certain performance on both the training and test sets, but also showed consistent stability when applied to the temporal validation set, with AUC values of 0.82, 0.74, and 0.71, respectively. The combination of radiomics models and radiologists in parallel diagnosis has increased the sensitivity for diagnosing NEC and reduced the rate of missed diagnoses, which illustrates the value of radiomics models as an assist tool.

Compared to the Duke score (12), our use of rectangular segmentation and radiomics quantified image information, reducing reliance on physician subjectivity. In contrast to previous radiomics studies (10,11), our study included the liver in the ROI, because radiologists should take into account the possibility of dendritic portal venous gas in the liver area in real-world situations when diagnosing NEC. From this perspective, our approach may have more meaningful implications for translational application in medical practice.

Previous studies have shown a 90% agreement in the value of plain abdominal radiographs for confirming the diagnosis of NEC, providing guidance for decision-making regarding surgery and abdominal radiography as the preferred treatment modality and the most important radiographic sign, respectively (6). However, relatively few studies have applied machine learning (ML) and artificial intelligence (AI) to NEC. There is a study that use radiological features to predict NEC surgery (10), constructing a joint model based on four radiomics features and clinical signs when abdominal radiographs do not suggest an absolute surgical indication for free pneumoperitoneum. Additionally, there is a study that use a multimodal AI system built with feature engineering, ML, and deep learning (DL) for NEC diagnosis and surgery prediction using plain abdominal radiographs, and find that NEC diagnostic efficacy reaches 0.93, and then this efficacy is a combination of 40 abdominal imaging features and 18 clinical parameters (11). This study, on the other hand, used only abdominal radiomics features and also achieved a stable performance in the test set and temporal validation set.

The distribution of radiomic features in our study provides insights into the characterization of NEC radiographic patterns. Firstly, the predominance of texture features (55.6%) highlights the importance of spatial relationships and gray-level patterns in identifying NEC-related changes (8). Particularly, the balanced distribution among GLCM (4 features), GLRLM (2 features), and GLSZM (4 features) captures various aspects of image heterogeneity, from local pixel relationships to regional intensity patterns (19,20). Secondly, the substantial proportion of wavelet features (27.8%) suggests the significance of multi-scale analysis in detecting subtle radiographic changes at different resolutions. Finally, the smaller proportions of first-order (11.1%) and shape features (5.6%) indicate that while basic intensity statistics and morphological characteristics contribute to the analysis, they play a supplementary role compared to the more sophisticated texture and wavelet-based features in NEC assessment (8,21). In summary, texture and wavelet features comprise 83.4% of all features, indicating that the complex patterns and multi-scale characteristics of radiographs in NEC can be captured by these advanced image analysis methods (22). Notably, the wavelet.LH_firstorder_Skewness feature has the largest coefficient (−0.541) in the model, indicating its significant role in the model’s performance as the most important and specific sign. Skewness quantifies the asymmetry of voxel intensity distributions within medical images. The wavelet.LH_firstorder_Skewness feature may be sensitive to these changes and can help capture the complex patterns and multi-scale characteristics of radiographic images in NEC. In the context of radiomics, it can reflect the asymmetry of the gray level distribution in the image. In the radiological manifestations of NEC, the presence of pneumoperitoneum, pneumatosis intestinalis, and portal venous gas, which appear as low-density areas, mixed with the soft tissue density from thickened bowel loops, results in heterogeneous imaging findings. We hypothesized that these radiological features may be captured by these radiomic features. Previous researchers have hypothesized that the transform-filtered statistics may have the potential to suppress noise and highlight details in the original image, so that they can extract regions featuring progressively coarser texture patterns in a more adaptable way (10). However, current studies do not well integrate these physiology and pathology of the disease with radiomic features.

Our study demonstrated that parallel diagnosis combining radiologists and radiomics models improved the sensitivity of NEC diagnosis. For severe neonatal conditions like NEC that progress rapidly and have high mortality rates, timely detection and early intervention are crucial. Studies have shown that delayed diagnosis can lead to expanded intestinal necrosis, increasing surgical risks and mortality rates (23). Therefore, improving sensitivity in NEC diagnosis has special clinical significance, and the trade-off with slightly reduced specificity may be worthwhile. The advantages of this parallel diagnostic approach have been validated in multiple studies. A systematic review found that combining AI with clinicians significantly improved disease detection rates while maintaining high specificity (24). This advantage stems from the complementarity between AI systems and clinicians: radiomics models can capture subtle features difficult for human eyes to identify, while clinicians possess rich experience and comprehensive clinical thinking. One research confirmed that the combination of AI and radiologists could reduce missed diagnoses in breast cancer screening, effectively creating a double-check mechanism (25). Additionally, other research reported that AI-assisted diagnosis could improve radiologists’ work efficiency by helping them focus on suspicious cases (26). Our study also validates this point, as the parallel diagnostic strategy helps overcome potential subjective judgment differences and fatigue-related errors that may occur when relying solely on radiologists. AI should not be viewed as a tool to replace doctors but as a powerful aid to clinical decision-making (27). This aligns with our research philosophy of pursuing human-machine collaboration and complementary advantages to jointly improve diagnostic accuracy. Future research could further explore how to optimize the parallel diagnostic process and better integrate AI into clinical practice.

Nonetheless, there are certain limitations to this study. First, this is a single-center study, and although a temporal validation set was used to simulate the real situation, data from external centers are still needed for further validation. Second, our research primarily depends on single-source medical imaging data. However, clinical diagnosis of NEC often involves a combination of clinical symptoms, such as vomiting, abdominal distension, and bloody stools, along with radiological signs like pneumoperitoneum, pneumatosis intestinalis, and portal venous gas (3,14), and other studies have included clinical features, genetic markers, metabolomics, and fecal microbiota (11,28,29), therefore multimodal data such as ultrasound, and electronic medical record data can be combined with radiomics in the future. Third, while our results from the temporal validation set show no signs of overfitting, the number of features in the model increases concerns about overly optimistic results when the number of features in the model is high relative to the number of cases in a category. Augmenting the data may help in the training of the model. Fourth, this study utilized only the images associated with the initial diagnosis or suspected diagnosis of NEC from the radiological diagnostic reports. Further analysis of follow-up imaging from the case group may contribute to enhancing the effectiveness of radiomics in NEC diagnosis.

Conclusions

Radiomics shows promise as a valuable tool to complement traditional NEC diagnostics. By providing quantitative, reproducible data, it reduces subjectivity and enhances diagnostic precision, particularly by improving sensitivity and reducing missed diagnoses in equivocal cases. Future multi-center studies should validate these findings and explore combining radiomics with other diagnostic modalities.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://tp.amegroups.com/article/view/10.21037/tp-2024-496/rc

Data Sharing Statement: Available at https://tp.amegroups.com/article/view/10.21037/tp-2024-496/dss

Peer Review File: Available at https://tp.amegroups.com/article/view/10.21037/tp-2024-496/prf

Funding: None.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tp.amegroups.com/article/view/10.21037/tp-2024-496/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the institutional ethics committee of the Children’s Hospital of Fudan University (No. 2023-196). Informed consent was obtained from all the patients’ legal guardians, and the data were anonymized to ensure confidentiality.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Bell EF, Hintz SR, Hansen NI, et al. Mortality, In-Hospital Morbidity, Care Practices, and 2-Year Outcomes for Extremely Preterm Infants in the US, 2013-2018. JAMA 2022;327:248-63. [Crossref] [PubMed]
2. Lueschow SR, Boly TJ, Jasper E, et al. A critical evaluation of current definitions of necrotizing enterocolitis. Pediatr Res 2022;91:590-7. Erratum in: Pediatr Res 2022;91:711. [Crossref] [PubMed]
3. Hu X, Liang H, Li F, et al. Necrotizing enterocolitis: current understanding of the prevention and management. Pediatr Surg Int 2024;40:32. [Crossref] [PubMed]
4. Hwang M, Tierradentro-García LO, Dennis RA, et al. The role of ultrasound in necrotizing enterocolitis. Pediatr Radiol 2022;52:702-15. [Crossref] [PubMed]
5. Kim JH. Role of Abdominal US in Diagnosis of NEC. Clin Perinatol 2019;46:119-27. [Crossref] [PubMed]
6. Ahle M, Ringertz HG, Rubesova E. The role of imaging in the management of necrotising enterocolitis: a multispecialist survey and a review of the literature. Eur Radiol 2018;28:3621-31. [Crossref] [PubMed]
7. Gephart SM, Gordon PV, Penn AH, et al. Changing the paradigm of defining, detecting, and diagnosing NEC: Perspectives on Bell’s stages and biomarkers for NEC. Semin Pediatr Surg 2018;27:3-10. [Crossref] [PubMed]
8. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016;278:563-77. [Crossref] [PubMed]
9. Tomaszewski MR, Gillies RJ. The Biological Meaning of Radiomic Features. Radiology 2021;298:505-16. [Crossref] [PubMed]
10. Li Y, Wu K, Yang H, et al. Surgical prediction of neonatal necrotizing enterocolitis based on radiomics and clinical information. Abdom Radiol (NY) 2024;49:1020-30. [Crossref] [PubMed]
11. Gao W, Pei Y, Liang H, et al. Multimodal AI System for the Rapid Diagnosis and Surgical Prediction of Necrotizing Enterocolitis. IEEE Access 2021;9:51050-64.
12. Coursey CA, Hollingsworth CL, Gaca AM, et al. Radiologists’ agreement when using a 10-point scale to report abdominal radiographic findings of necrotizing enterocolitis in neonates and infants. AJR Am J Roentgenol 2008;191:190-7. Erratum in: AJR Am J Roentgenol 2008;191:931. [Crossref] [PubMed]
13. Chen Y, Duan Y, Wei B, et al. Value of portal venous gas and a nomogram for predicting severe neonatal necrotizing enterocolitis. Pediatr Res 2024; Epub ahead of print. [Crossref] [PubMed]
14. Patel RM, Ferguson J, McElroy SJ, et al. Defining necrotizing enterocolitis: current difficulties and future opportunities. Pediatr Res 2020;88:10-5. [Crossref] [PubMed]
15. Fedorov A, Beichel R, Kalpathy-Cramer J, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging 2012;30:1323-41. [Crossref] [PubMed]
16. Song Y, Zhang J, Zhang YD, et al. FeAture Explorer (FAE): A tool for developing and comparing radiomics models. PLoS One 2020;15:e0237587. [Crossref] [PubMed]
17. Xia TY, Zhou ZH, Meng XP, et al. Predicting Microvascular Invasion in Hepatocellular Carcinoma Using CT-based Radiomics Model. Radiology 2023;307:e222729. [Crossref] [PubMed]
18. Jia H, Li R, Liu Y, et al. Preoperative Prediction of Perineural Invasion and Prognosis in Gastric Cancer Based on Machine Learning through a Radiomics-Clinicopathological Nomogram. Cancers (Basel) 2024;16:614. [Crossref] [PubMed]
19. Aerts HJ, Velazquez ER, Leijenaar RT, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014;5:4006. [Crossref] [PubMed]
20. Yip SS, Aerts HJ. Applications and limitations of radiomics. Phys Med Biol 2016;61:R150-66. [Crossref] [PubMed]
21. Lambin P, Rios-Velazquez E, Leijenaar R, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 2012;48:441-6. [Crossref] [PubMed]
22. van Griethuysen JJM, Fedorov A, Parmar C, et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res 2017;77:e104-7. [Crossref] [PubMed]
23. Patel EU, Wilson DA, Brennan EA, et al. Earlier re-initiation of enteral feeding after necrotizing enterocolitis decreases recurrence or stricture: a systematic review and meta-analysis. J Perinatol 2020;40:1679-87. [Crossref] [PubMed]
24. Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 2019;1:e271-97. [Crossref] [PubMed]
25. Rodriguez-Ruiz A, Lång K, Gubern-Merida A, et al. Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists. J Natl Cancer Inst 2019;111:916-22. [Crossref] [PubMed]
26. McKinney SM, Sieniek M, Godbole V, et al. International evaluation of an AI system for breast cancer screening. Nature 2020;577:89-94. Erratum in: Nature 2020;586:E19. [Crossref] [PubMed]
27. Göndöcs D, Dörfler V. AI in medical diagnosis: AI prediction & human judgment. Artif Intell Med 2024;149:102769. [Crossref] [PubMed]
28. Hooven TA, Lin AYC, Salleb-Aouissi A. Multiple Instance Learning for Predicting Necrotizing Enterocolitis in Premature Infants Using Microbiome Data. Proc ACM Conf Health Inference Learn (2020) 2020;2020:99-109.
29. Lin YC, Salleb-Aouissi A, Hooven TA. Interpretable prediction of necrotizing enterocolitis from machine learning analysis of premature infant stool microbiota. BMC Bioinformatics 2022;23:104. [Crossref] [PubMed]