Development and validation of a machine learning model for predicting co-infection of Mycoplasma pneumonia in pediatric patients
Original Article

Development and validation of a machine learning model for predicting co-infection of Mycoplasma pneumonia in pediatric patients

Xiaohan Liu1,2,3#, Wenbei Xu1,2,3#, Lingjian Meng4, Juan Long1,2,3, Xiaonan Sun1,2,3, Qiang Li5, Haiquan Kang6, Yiping Mao7, Chunfeng Hu1,2,3, Kai Xu1,2,3, Yankai Meng1,2,3

1Department of Radiology, the Affiliated Hospital of Xuzhou Medical University, Xuzhou, China; 2College of Medical Imaging, Xuzhou Medical University, Xuzhou, China; 3Jiangsu Medical Imaging and Digital Medical Engineering Research Center, Xuzhou Medical University, Xuzhou, China; 4Department of Pediatrics, the Affiliated Hospital of Xuzhou Medical University, Xuzhou, China; 5Department of Infectious Diseases, the Affiliated Hospital of Xuzhou Medical University, Xuzhou, China; 6Department of Clinical Lab, the Affiliated Hospital of Xuzhou Medical University, Xuzhou, China; 7Department of Infection Management, the Affiliated Hospital of Xuzhou Medical University, Xuzhou, China

Contributions: (I) Conception and design: Y Meng, X Liu, W Xu; (II) Administrative support: C Hu, K Xu, Y Meng, H Kang, Y Mao; (III) Provision of study materials or patients: Y Meng, L Meng; (IV) Collection and assembly of data: X Liu, W Xu; (V) Data analysis and interpretation: X Liu, W Xu, Q Li; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

#These authors contributed equally to this work.

Correspondence to: Yankai Meng, MD. Department of Radiology, the Affiliated Hospital of Xuzhou Medical University, No. 99 Huaihai West Road, Xuzhou, China; College of Medical Imaging, Xuzhou Medical University, Xuzhou, China; Jiangsu Medical Imaging and Digital Medical Engineering Research Center, Xuzhou Medical University, Xuzhou, China. Email: mengyankai@126.com.

Background: Mycoplasma pneumoniae pneumonia (MPP) is endemic in China, while Mycoplasma co-infection with other pathogens (Co-MPP) linked to severe outcomes. Despite radiomics and machine learning potential in pneumonia, pediatric Co-MPP differentiation remains underexplored. This study aimed to bridge this gap by evaluating machine learning models, particularly radiomics features derived from high-resolution computed tomography (HRCT) scans, to differentiate between MPP and Co-MPP, and to compare their predictive performance with traditional clinical models.

Methods: We conducted a retrospective analysis of hospitalized pediatric pneumonia patients from June to December 2023 at Affiliated Hospital of Xuzhou Medical University. Chest computed tomography (CT) scans were performed using a multi-slice CT scanner with over 64 detectors. Fluorescent quantitative polymerase chain reaction (PCR) was used to detect 14 pathogens in bronchoalveolar lavage (BAL) fluid. The most recent laboratory results prior to BAL were included in multifactorial logistic regression (LR) analysis, selecting variables with P<0.05 for constructing the clinical model. The largest cross-section of the lesion was selected, and image segmentation was performed using ITK-SNAP software. Radiomics features were extracted with Pyradomics. Features were filtered using t-tests, Mann-Whitney U tests, and Spearman rank correlation coefficients. The least absolute shrinkage and selection operator (LASSO) regression and ten-fold cross-validation were used for feature selection and to construct the radiomics model, optimizing the dimensionality of the dataset. Eight different machine learning models [LR, support vector machine (SVM), K-nearest neighbor (KNN), RandomForest, ExtraTrees, eXtreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and multi-layer perceptron (MLP)] were trained with the selected features, with five-fold cross-validation yielding the final radiomics model. The clinical and radiomics models were combined to create a nomogram model. Data analysis was performed using R software and SPSS 26.0.

Results: A total of 124 cases of MPP and children with Co-MPP were included. The extracted radiomics features consisted of first-order signal intensity features (n=360), morphological features (n=14), and texture features (n=1,460). LASSO regression and ten-fold cross-validation identified 23 non-zero correlation coefficient features for constructing Radscore. The LR model demonstrated superior predictive performance for Co-MPP in the validation cohort, with an area under the curve (AUC) of 0.951, sensitivity of 0.778, and specificity of 0.875. The nomogram model combining clinical and radiomics labels significantly outperformed the clinical model (P=0.004). Calibration curve analysis indicated that the nomogram model exhibited the best agreement with actual values. Both the radiomics and nomogram models provided greater clinical net benefits compared to the clinical model.

Conclusions: The radiomics model trained using machine learning effectively predicts Co-MPP in children, while the combined clinical and radiomics nomogram model offers the best predictive performance.

Keywords: Mycoplasma pneumoniae (MP); Mycoplasma pneumoniae pneumonia (MPP); co-infection; machine learning; computed tomography (CT)


Submitted Dec 06, 2024. Accepted for publication May 07, 2025. Published online Jun 25, 2025.

doi: 10.21037/tp-2024-562


Highlight box

Key findings

• The combined clinical and radiomics nomogram model based on machine learning effectively predicts Mycoplasma co-infection with other pathogens (Co-MPP) in children.

What is known and what is new?

• Mycoplasma pneumoniae pneumonia (MPP) is endemic in China, with Co-MPP linked to severe outcomes. Polymerase chain reaction/metagenomic next-generation sequencing (PCR/mNGS) enhance pathogen detection but face cost or access barriers. High-resolution computed tomography (HRCT) aids diagnosis with operator-dependent variability. Radiomics based on machine learning are utilized in pneumonia, yet pediatric Co-MPP differentiation remains underexplored.

• Novel radiomics-clinical nomogram (area under the curve 0.951) for pediatric Co-MPP prediction using least absolute shrinkage and selection operator-optimized 23-feature radiomics, overcoming HRCT/PCR limitations.

What is the implication, and what should change now?

• Machine learning predicts pediatric Co-MPP effectively yet requires expanded samples, automated segmentation, and multicenter validation.


Introduction

Pneumonia caused by Mycoplasma pneumoniae (MP), referred to as Mycoplasma pneumoniae pneumonia (MPP), is a prevalent condition in China (1,2). MPP is frequently complicated by co-infection with other pathogens, including both viral and bacterial agents, leading to what is termed Mycoplasma co-infection with other pathogens (Co-MPP) (3,4). These co-infections can result in more severe forms of pneumonia, including refractory cases, thereby prolonging illness duration and complicating treatment in pediatric patients.

Early and accurate detection of Co-MPP is of substantial clinical importance. Polymerase chain reaction (PCR) has emerged as a sensitive and specific diagnostic tool capable of simultaneously identifying MP and other pathogens. However, the requirement for specialized equipment limits its use in primary care settings (5). Metagenomic next-generation sequencing (mNGS) provides even broader pathogen detection but is constrained by higher costs and longer processing times (5,6).

High-resolution computed tomography (HRCT) has been shown to help differentiate MPP from Co-MPP by identifying specific morphological characteristics. However, its diagnostic accuracy is often dependent on the interpreting physician’s expertise, leading to variability in results (7-10).

Radiomics, a technique that extracts high-dimensional quantitative features from medical images, is gaining attention as a promising diagnostic tool. In recent years, machine learning models have been increasingly utilized to integrate radiomics with clinical data to predict and classify various medical conditions, including pneumonia (11). These models show great promise in automating diagnosis, improving predictive accuracy, and aiding clinicians in making informed treatment decisions (12). Notably, Li et al. (13) have shown that radiomics can effectively distinguish MPP from community-acquired pneumonia (CAP) caused by other pathogens. In addition, Wong et al. (14) have also reported promising results using machine learning to differentiate between various pneumonia pathogens. However, despite the advances in imaging and machine learning, the application of such technologies in distinguishing between MPP and Co-MPP in pediatric population remains underexplored.

This study aims to bridge this gap by evaluating machine learning models, particularly radiomics features derived from HRCT scans, to differentiate between MPP and Co-MPP, and to compare their predictive performance with traditional clinical models. Given the increasing interest and application of artificial intelligence in healthcare, this research represents an important step in integrating machine learning into the clinical management of pediatric pneumonia, with the goal of enhancing diagnostic accuracy and treatment outcomes. We present this article in accordance with the TRIPOD reporting checklist (available at https://tp.amegroups.com/article/view/10.21037/tp-2024-562/rc).


Methods

Clinical data

This study involved the collection of clinical and HRCT data from pediatric patients hospitalized for pneumonia at Affiliated Hospital of Xuzhou Medical University between June and December 2023. This single-center investigation was conducted in accordance with the ethical principles of the Declaration of Helsinki and its subsequent amendments and approved by the Institutional Review Board (IRB) of Affiliated Hospital of Xuzhou Medical University (No. XYFY2024-KL438-01). Retrospective analysis used anonymized clinical and imaging data. Informed consent was waived by the IRB due to the non-interventional, observational nature of the study. Patients were categorized into two distinct groups: MPP and Co-MPP. The MPP group included patients diagnosed solely with MPP due to MP infection, while the Co-MPP group consisted of patients with MPP accompanied by infections from other pathogens.

The inclusion criteria for this study were as follows: (I) all patients underwent bronchoalveolar lavage (BAL) alongside 14-panel pathogen detection, confirming positive MP detection; (II) chest HRCT was performed prior to BAL, with a maximum allowable interval of ≤7 days. Patients were excluded based on the following criteria: (I) contraindications for BAL (e.g., febrile seizures, congenital airway malformations, critical illness); (II) poor-quality chest computed tomography (CT) images characterized by significant motion artifacts; (III) previous BAL within 1 week; or (IV) the presence of congenital disorders, hematological diseases, or autoimmune diseases complicating MPP.

Chest CT scanning protocol

Chest CT scans were performed using a multi-slice CT scanner with over 64 detectors. Patients were positioned supine with the head entering first, covering the area from the lung apex to the lung base. The exposure parameters were adjusted based on the patient’s age: for those under 1 year old, the tube voltage was set to 80 kV with a tube current of 50 mA; for patients aged 1 to 7 years, the tube voltage was 100 kV with a tube current of 60 mA; and for patients older than 7 years, the tube voltage was 120 kV with a tube current of 80 mA. Additionally, for cooperative children, scans were performed during inspiratory breath-holding. The tube rotation time was set to 0.5 seconds, with a pitch of 1.2–1.5, slice thickness of 5 mm, slice interval of 5 mm, reconstruction slice thickness of 1 mm, and reconstruction interval of 1 mm. Iterative reconstruction techniques were used, with lung or bone high-resolution algorithms applied for image reconstruction.

BAL procedure

Prior to BAL, preoperative evaluations were performed to confirm the absence of surgical contraindications. The bronchoscope was directed toward the most inflamed lung segment for the procedure. Sterile saline at 37 ℃ was instilled into the affected lung segment based on the patient’s weight (1 mL/kg), and the lavage fluid was subsequently aspirated at a negative pressure of 100 mmHg (1 mmHg =0.133 kPa) before being sent for 14-panel pathogen detection.

Fluorescent quantitative PCR was employed to detect the 14 pathogens present in the BAL samples, utilizing nucleic acid extraction and reagent kits sourced from BioKangxin (Tianjin) Biotechnology Co., Ltd. (Tianjin, China).

Clinical model construction

The most recent laboratory results obtained prior to BAL were included in the statistical analysis. Baseline clinical and laboratory data from all enrolled patients underwent multivariate logistic regression (LR) analysis; variables with a P value <0.05 were selected for the construction of the clinical model.

Image segmentation

DICOM images were imported into ITK-SNAP software (version 4.0.2) for analysis. A senior radiologist specializing in thoracic imaging identified the largest cross-sectional area of the lesion on the HRCT images for image segmentation. Subsequently, a radiologist with five years of thoracic imaging experience delineated the image along the lesion’s boundary on the selected slice, ensuring comprehensive inclusion of the entire lesion.

Radiomics feature extraction and selection

Radiomics feature extraction was performed using Pyradiomics (version 3.1.0rc2.post5). Quantitative image features from manual segmentation were categorized into three primary groups: (I) geometric features; (II) intensity features; and (III) texture features.

To ensure robustness and minimize the risk of overfitting, we employed three different statistical tests for feature selection: t-tests, Mann-Whitney U tests, and Spearman correlation analysis. The use of these three methods allowed for a comprehensive filtering of irrelevant features and ensured that the selected features were both statistically significant and highly predictive. This multi-step approach enhanced the quality of the features used for model training, ultimately improving the performance and reliability of the predictive model. The radiomics feature selection process consisted of three methods: (I) retaining only features with a P value <0.05, assessed through statistical t-tests and Mann-Whitney U tests; (II) applying Spearman correlation analysis to identify features with a correlation coefficient >0.9, retaining only one from each correlated pair. Specifically, when selecting features in cases of strong correlation, we prioritized features based on their predictive power, statistical significance, and clinical relevance, ensuring that only the most informative and non-redundant features were retained; and (III) further dimensionality reduction using least absolute shrinkage and selection operator (LASSO) regression. Ten-fold cross-validation was employed to determine the optimal λ value that minimized cross-validation error. Non-zero coefficient features were retained for machine learning training to construct the radiomics signature, with LASSO regression modeling performed using the Python scikit-learn package.

Radiomics model training

The radiomics features selected through LASSO regression were utilized in eight distinct machine learning models: LR, support vector machine (SVM), K-nearest neighbor (KNN), RandomForest, ExtraTrees, eXtreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and multi-layer perceptron (MLP), maintaining an 80:20 training-to-validation split ratio. In our study, we set the number of neighbors to 5 for the KNN algorithm based on cross-validation optimization. The final radiomics model was established through 5-fold cross-validation. Additionally, we used grid search with cross-validation to optimize the hyperparameters of each machine learning model. Various hyperparameter combinations were evaluated, and the combination that performed best on the validation set was selected, ensuring optimal model performance. Furthermore, we employed a nonlinear SVM model with a radial basis function (RBF) kernel, which is widely used for handling complex, nonlinear data relationships and enhances the model’s generalization capability.

Nomogram model development

A nomogram model was developed by integrating both clinical and radiomics features. This model combines the predictive power of clinical data, such as laboratory results and patient demographics, with radiomics features extracted from high-resolution CT images. The clinical and radiomics models were fused to provide a comprehensive, composite prediction. This integration allows for a more accurate and personalized prediction of co-infection, leveraging both traditional clinical indicators and advanced imaging features. The predictive performance of this model was evaluated using receiver operating characteristic (ROC) curves, while calibration curves were employed to assess calibration performance. The Hosmer-Lemeshow goodness-of-fit results provided insights into the model’s calibration capability, and decision curve analysis (DCA) was used to evaluate the clinical benefit of the model.

To evaluate the performance of the models, we used several key metrics including the area under the curve (AUC), sensitivity, specificity, accuracy, and precision. These metrics were calculated for both the training and validation cohorts. Additionally, we performed DCA to assess the clinical net benefit of each model. The AUC provides an overall measure of the model’s ability to distinguish between MPP and co-infection, while sensitivity and specificity offer insights into the model’s ability to correctly identify positive and negative cases, respectively. The DCA helped us determine the clinical usefulness of the models by comparing the potential benefits of the models over various threshold values. Figure 1 illustrates the flowchart for radiomics image segmentation, feature extraction, selection, model construction, and evaluation.

Figure 1 Flowchart of the radiomics feature extraction, selection, and model construction process. Red area: manually segmented ROI on the largest lesion slice of pediatric pneumonia CT. AUC, area under the curve; CI, confidence interval; ROI, region of interest.

Statistical analysis

Statistical analyses were conducted using R software (version 4.3.2) and SPSS 26.0. The Kolmogorov-Smirnov test was employed to assess the normality of quantitative data. For normally distributed data, results were presented as mean ± standard deviation (x¯±s), while non-normally distributed data were expressed as median (interquartile range). Categorical data were reported as proportions or percentages (%). Comparisons of quantitative data were made using t-tests or Mann-Whitney U tests, while categorical data were analyzed using the χ2 test or Fisher’s exact test. Intra-class correlation coefficients (ICC) were calculated to evaluate consistency. Stepwise multivariate LR was employed to identify clinically significant variables, comparing their predictive value for distinguishing between MPP and Co-MPP. The comparison involved calculating odds ratios (ORs) and confidence intervals (CIs), with P values used to assess statistical significance, and Spearman’s rank correlation was used for correlation analysis. A P value of less than 0.05 was considered statistically significant. The nomogram was utilized to visualize the study results.


Results

Demographic characteristics

This study included 124 pediatric patients diagnosed with MPP and Co-MPP, who were randomly divided into training (n=99) and validation (n=25) sets in an 80:20 ratio. A total of 78 (62.9%) cases were diagnosed with MPP and 46 (37.1%) cases with Co-MPP. The average time interval between chest HRCT and BAL was 3.6±1.9 days. Among the 124 patients, 103 (83.1%) had CT dose reports, with computed tomography dose index (CTDI), dose-length product (DLP), and effective dose (ED) values of 2.07 (95% CI: 1.20, 2.77) mGy, 49.57 (95% CI: 31.34, 74.15) mGy·cm and 0.97 (95% CI: 0.58, 1.35) mSv, respectively. The mean age of patients in the training set was 6.85±2.53 years, while in the validation set it was 6.40±2.60 years, showing no statistically significant difference (P>0.99).

Clinical model construction

Univariate and multivariate LR analyses indicated that length of hospital stay and procalcitonin levels were independent predictors of Co-MPP, with ORs of 1.034 and 1.774, respectively (Table 1).

Table 1

Multivariate analysis of clinical characteristics in the training cohort (N=99)

Characteristics OR 95% CI P value
Gender 0.982 0.831–1.161 0.86
Age (years) 0.993 0.962–1.025 0.72
Cough 1.464 0.823–2.606 0.27
Sputum production 0.881 0.703–1.105 0.36
Length of hospital stay (days) 1.034 1.012–1.055 0.01*
Admission temperature (℃) 1.060 0.966–1.163 0.30
Duration from symptom onset to admission (days) 0.991 0.959–1.023 0.62
Fibrinogen degradation products (<5 mg/mL) 0.99 0.970–1.010 0.34
D-dimer (0–0.5 µg/mL) 0.997 0.909–1.093 0.96
Activated partial thromboplastin time (25–31.3 s) 0.995 0.962–1.028 0.79
Fibrinogen (2–4 g/L) 0.996 0.887–1.119 0.95
Procalcitonin (0.02–0.046 ng/mL) 1.774 1.105–2.849 0.047*
White blood cell count (×109) (3.5–9.5 ×109/L) 1.02 0.997–1.042 0.15
Neutrophil percentage (%) (40–75%) 1.003 0.995–1.010 0.55
C-reactive protein (mg/L) (0–5) 1.003 0.997–1.009 0.41

*, significance of differences. CI, confidence interval; OR, odds ratio.

Radiomics feature extraction and selection

In this study, we extracted radiomics features, including 360 first-order signal intensity features, 14 morphological features, and 1,460 texture features. The results of the consistency analysis indicated an ICC greater than 0.75.

Statistical tests and correlation analysis identified 81 relevant radiomics features. We employed LASSO regression and ten-fold cross-validation for dimensionality reduction (Figure 2A,2B), ultimately selecting 23 features with non-zero correlation coefficients for constructing the Radscore (Figure 3).

Figure 2 LASSO regression results showing feature selection and the determination of the optimal λ value (A) and the correlation coefficients of the selected features (B). Radscore = 0.37096774193548376 − 0.003410 * exponential_glcm_ClusterProminence + 0.035694 * exponential_glrlm_RunEntropy + 0.005861 * exponential_glszm_ZoneEntropy − 0.013506 * lbp_3D_k_firstorder_Minimum − 0.014402 * lbp_3D_k_glszm_SmallAreaEmphasis − 0.040841 * lbp_3D_k_glszm_SmallAreaLowGrayLevelEmphasis − 0.041107 * lbp_3D_m1_glcm_ClusterShade + 0.018958 * lbp_3D_m2_glcm_InverseVariance −0.071762 * lbp_3D_m2_ngtdm_Contrast − 0.001909 * log_sigma_1_0_mm_3D_firstorder_Kurtosis − 0.040344 * log_sigma_3_0_mm_3D_ngtdm_Coarseness + 0.061064 * original_glcm_ClusterShade + 0.030447 * squareroot_glcm_Correlation + 0.028943 * wavelet_HLH_glcm_ClusterShade + 0.015726 * wavelet_HLL_firstorder_RootMeanSquared + 0.003242 * wavelet_HLL_glcm_ClusterShade + 0.108930 * wavelet_LHL_firstorder_Mean +0.033621 * wavelet_LLH_glcm_MaximumProbability − 0.079245 * wavelet_LLL_firstorder_Minimum + 0.032921 * wavelet_LLL_glcm_Correlation − 0.038249 * wavelet_LLL_glcm_Imc2 + 0.031427 * wavelet_LLL_glszm_SmallAreaHighGrayLevelEmphasis + 0.121468 * wavelet_LLL_ngtdm_Strength. LASSO, least absolute shrinkage and selection operator; MSE, mean squared error.
Figure 3 Histogram of correlation coefficients of radiomics features with Co-MPP, demonstrating the relationship between feature values and disease outcome. Co-MPP, Mycoplasma co-infection with other pathogens.

Comparison of predictive performance among different machine learning models

In the validation set, the LR model demonstrated superior efficacy in predicting Co-MPP compared to other models, with an AUC, sensitivity, and specificity of 0.951, 0.778, and 0.875, respectively (Table 2, Figure 4A,4B). The radiomics model trained with the LR model was utilized to construct a nomogram that integrates clinical data for predicting Co-MPP.

Table 2

Predictive performance of different machine learning models in the training and validation sets (N=124)

Model AUC SE SP PPV NPV Accuracy Precision Recall F1 Threshold
Training set
   LR 0.930 0.865 0.823 0.744 0.911 0.838 0.744 0.865 0.800 0.347
   SVM 0.978 0.892 0.935 0.892 0.935 0.919 0.892 0.892 0.892 0.386
   KNN 0.884 0.432 0.984 0.941 0.744 0.778 0.941 0.432 0.593 0.600
   RandomForest 0.970 0.865 0.935 0.889 0.921 0.909 0.889 0.865 0.877 0.419
   ExtraTrees 0.937 0.892 0.839 0.767 0.929 0.859 0.767 0.892 0.825 0.369
   XGBoost 1.000 0.973 1.000 1.000 0.984 0.990 1.000 0.973 0.986 0.531
   LightGBM 0.950 0.865 0.855 0.780 0.914 0.859 0.780 0.865 0.821 0.390
   MLP 0.938 0.865 0.855 0.780 0.914 0.859 0.780 0.865 0.821 0.360
Validation set
   LR 0.951 0.778 0.875 0.778 0.875 0.840 0.778 0.778 0.778 0.303
   SVM 0.854 0.556 1.000 1.000 0.800 0.840 1.000 0.556 0.714 0.500
   KNN 0.781 0.000 1.000 0.000 0.640 0.640 0.000 0.000 N/A 0.800
   RandomForest 0.743 0.444 0.937 0.800 0.750 0.760 0.800 0.444 0.571 0.468
   ExtraTrees 0.681 0.444 0.937 0.800 0.750 0.760 0.800 0.444 0.571 0.481
   XGBoost 0.792 0.556 0.875 0.714 0.778 0.760 0.714 0.556 0.625 0.350
   LightGBM 0.771 0.444 0.937 0.800 0.750 0.760 0.800 0.444 0.571 0.501
   MLP 0.778 0.556 0.937 0.833 0.789 0.800 0.833 0.556 0.667 0.556

AUC, area under the curve; KNN, K-nearest neighbor; LightGBM, light gradient boosting machine; LR, logistic regression; MLP, multi-layer perceptron; N/A, not applicable; NPV, negative predictive value; PPV, positive predictive value; SE, sensitivity; SP, specificity; SVM, support vector machine; XGBoost, eXtreme gradient boosting.

Figure 4 Comparison of ROC curves for different machine learning models in the training (A) and validation (B) sets. AUC, area under the curve; CI, confidence interval; KNN, K-nearest neighbor; LightGBM, light gradient boosting machine; LR, logistic regression; MLP, multi-layer perceptron; nan, not a number; ROC, receiver operating characteristic; SVM, support vector machine; XGBoost, eXtreme gradient boosting.

Comparison of prediction performance for Co-MPP among clinical model, radiomics model, and nomogram model

In the training set, the radiomics model demonstrated superior predictive performance for Co-MPP compared to the clinical labels and the nomogram model (Figure 5A).

Figure 5 Predictive performance of clinical, radiomics, and nomogram models for Co-MPP in the training (A) and validation (B) sets. AUC, area under the curve; CI, confidence interval; Co-MPP, Mycoplasma co-infection with other pathogens.

In the validation set, the nomogram model yielded AUC, sensitivity, and specificity of 0.951, 0.778, and 0.937, respectively (Table 3, Figure 5B).

Table 3

Predictive performance of different models in the training and validation sets (N=124)

Model AUC SE SP PPV NPV Accuracy Precision Recall F1 Threshold
Training set
   Clinical model 0.654 0.595 0.629 0.489 0.722 0.616 0.489 0.595 0.537 0.355
   Radiomics model 0.930 0.865 0.823 0.744 0.911 0.838 0.744 0.865 0.800 0.347
   Nomogram model 0.926 0.838 0.839 0.756 0.897 0.838 0.756 0.838 0.795 0.349
Validation set
   Clinical model 0.562 0.111 0.937 0.500 0.652 0.640 0.500 0.111 0.182 0.544
   Radiomics model 0.951 0.778 0.875 0.778 0.875 0.840 0.778 0.778 0.778 0.303
   Nomogram model 0.951 0.778 0.937 0.875 0.882 0.880 0.875 0.778 0.824 0.359

AUC, area under the curve; NPV, negative predictive value; PPV, positive predictive value; SE, sensitivity; SP, specificity.

Statistical analysis revealed that the predictive performance of the nomogram model for Co-MPP was significantly better than that of the clinical labels (P=0.004). However, there was no statistically significant difference in predictive performance between the nomogram model and the radiomics model (P>0.99). Nonetheless, the nomogram model exhibited higher accuracy and precision in predicting Co-MPP compared to the radiomics model.

Calibration and DCA: calibration curve

Results indicate that the nomogram model aligns best with actual values in the validation set (Figure 6A).

Figure 6 Calibration, clinical utility, and application of the nomogram model. Calibration curve (A): optimal agreement between nomogram predictions and actual values in the validation set. DCA (B): radiomics and nomogram models provide higher net clinical benefit than clinical labels. Clinical application (C): example of nomogram-guided diagnostic decision-making. DCA, decision curve analysis.

DCA results demonstrate that both the radiomics model and the nomogram model provide greater clinical net benefit compared to clinical labels, with good curve fitting (Figure 6B).

Clinical application of the nomogram model is illustrated, showcasing its utility in practice (Figure 6C).


Discussion

This study critically evaluates the predictive performance of various machine learning models applied to pediatric patients with Co-MPP and develops a nomogram that effectively integrates clinical and radiomics features. The findings suggest that the clinical net benefit derived from both the radiomics features and the nomogram model surpasses that of the traditional clinical model.

In a related study, Guo et al. developed a clinical model to differentiate between mixed pneumonia (MPP) and viral pneumonia, utilizing clinical and laboratory data, which achieved an efficacy of 0.788 in the validation cohort. Their DCA indicated a commendable alignment with actual patient outcomes. In our investigation, however, the efficacy of the clinical model in distinguishing between MPP and Co-MPP was notably lower. This discrepancy may be attributed to the inclusion of patients undergoing BAL, which prolonged the duration of illness and resulted in more severe lesions, thus contributing to the overlap in clinical and laboratory findings. Furthermore, some laboratory results were obtained a significant time after the BAL procedure, and the potential impact of immune-related indicators on the differential diagnosis was not considered, which may have further influenced the predictive efficacy of the clinical model.

Several studies have successfully integrated HRCT morphological features with clinical laboratory results to differentiate MPP (7,9). Li et al. (15) constructed a nomogram based on clinical and imaging data aimed at predicting refractory MPP (RMPP), identifying chest imaging scores as independent predictive factors. Additionally, other researchers have developed scoring systems based on HRCT imaging characteristics to distinguish MPP from CAP caused by various pathogens, reporting high sensitivity and specificity (16). It is essential to note, however, that subjective assessments of imaging features may be influenced by the evaluators’ expertise.

In comparison to previous studies (13,14) employing radiomics and machine learning to differentiate MPP from bacterial pneumonia, our study demonstrates superior predictive performance for both MPP and Co-MPP. This enhancement may result from the strategic selection of the most representative maximum layer of lesions and the employment of eight distinct machine learning models to be trained with the selected radiomics features, thereby augmenting model robustness. Nonetheless, the necessity for manual segmentation increases the clinical workload, highlighting the imperative for automated segmentation methodologies based on image contrast to improve operational efficiency.

While our study offers valuable insights into the performance of various machine learning models for predicting Co-MPP, we acknowledge some limitations. One notable observation is that overfitting occurred in models other than LR. This is likely due to the inherent complexity of the models, the relatively small sample size, or the high dimensionality of the feature set. Although no adjustments were made to the training process or hyperparameters, we recognize that applying techniques like regularization or enhancing cross-validation procedures could help mitigate overfitting in future work.

Another point to address is the unusual result where validation set performance exceeded training set performance. Typically, we expect the validation set performance to be slightly lower, as the model is fine-tuned to the training data. This discrepancy might be attributed to several factors: overfitting on the training set, where the model became overly specific to the training data; the test set characteristics, which may have included samples that were easier to predict or more representative of the overall distribution; or the cross-validation process used for hyperparameter tuning, which may have helped the model generalize better to the test set. We have reviewed the data to ensure there was no data leakage between the sets. In future work, we plan to continue monitoring these aspects and make necessary adjustments to improve generalization and reduce overfitting.

The limitations of this study are noteworthy: (I) a relatively small sample size; (II) the absence of an analysis regarding the time interval between laboratory tests and BAL on the results; (III) manual segmentation performed exclusively on the maximum layer of lesions; (IV) a retrospective analysis involving images from multiple CT scanners; and (V) the lack of external validation from multiple centers.


Conclusions

In conclusion, radiomics models trained using machine learning techniques have been proven to be effective in predicting Co-MPP in children, with the integrated clinical and radiomics nomogram model demonstrating the highest predictive performance.


Acknowledgments

We are particularly grateful to all the people who have given us help with our article.


Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://tp.amegroups.com/article/view/10.21037/tp-2024-562/rc

Data Sharing Statement: Available at https://tp.amegroups.com/article/view/10.21037/tp-2024-562/dss

Peer Review File: Available at https://tp.amegroups.com/article/view/10.21037/tp-2024-562/prf

Funding: This study was supported from the National Natural Science Foundation of China (No. 81771904), the Jiangsu Traditional Chinese Medicine Science and Technology Development Plan Project (grant No. MS2021100), and the Jiangsu Province Senior Health Research Project (No. LKM2022018).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tp.amegroups.com/article/view/10.21037/tp-2024-562/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments and was approved by the Affiliated Hospital of Xuzhou Medical University Ethics Committee (No. XYFY2024-KL438-01). Due to the retrospective nature of the data collection and analysis, the need for written informed consent was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Xie XY, Zhou RY, Ding SA, et al. Emerging trends and concerns in Mycoplasma pneumoniae pneumonia among Chinese pediatric population. Pediatr Res 2024;95:1388-90. [Crossref] [PubMed]
  2. Zhang XB, He W, Gui YH, et al. Current Mycoplasma pneumoniae epidemic among children in Shanghai: unusual pneumonia caused by usual pathogen. World J Pediatr 2024;20:5-10. [Crossref] [PubMed]
  3. Li T, Chu C, Wei B, et al. Immunity debt: Hospitals need to be prepared in advance for multiple respiratory diseases that tend to co-occur. Biosci Trends 2024;17:499-502. [Crossref] [PubMed]
  4. Ding G, Zhang X, Vinturache A, et al. Challenges in the treatment of pediatric Mycoplasma pneumoniae pneumonia. Eur J Pediatr 2024;183:3001-11. [Crossref] [PubMed]
  5. Wang YS, Zhou YL, Bai GN, et al. Expert consensus on the diagnosis and treatment of macrolide-resistant Mycoplasma pneumoniae pneumonia in children. World J Pediatr 2024;20:901-14. [Crossref] [PubMed]
  6. Zhang D, Yang A, Sheng K, et al. Application of the second-generation sequencing technology of metagenomics in the detection of pathogens in respiratory patients. J Microbiol Methods 2024;225:107021. [Crossref] [PubMed]
  7. Wang J, Xia C, Sharma A, et al. Chest CT Findings and Differential Diagnosis of Mycoplasma pneumoniae Pneumonia and Mycoplasma pneumoniae Combined with Streptococcal Pneumonia in Children. J Healthc Eng 2021;2021:8085530. [Crossref] [PubMed]
  8. Wang D, Zhao J, Zhang R, et al. The value of CT radiomic in differentiating mycoplasma pneumoniae pneumonia from streptococcus pneumoniae pneumonia with similar consolidation in children under 5 years. Front Pediatr 2022;10:953399. [Crossref] [PubMed]
  9. Sui DX, Ma HC, Wang CC, et al. Diagnostic significance of HRCT imaging features in adult mycoplasma pneumonia: a retrospective study. Sci Rep 2024;14:153. [Crossref] [PubMed]
  10. Huang X, Gu H, Wu R, et al. Chest imaging classification in Mycoplasma pneumoniae pneumonia is associated with its clinical features and outcomes. Respir Med 2024;221:107480. [Crossref] [PubMed]
  11. Li J, Si J, Yang Y, et al. The value of CT-based radiomics for differentiation of pleural effusions in bacterial pneumonia and Mycoplasma pneumoniae pneumonia in children. Transl Pediatr 2025;14:70-9. [Crossref] [PubMed]
  12. Singh K, Kaur N, Prabhu A. Combating COVID-19 Crisis using Artificial Intelligence (AI) Based Approach: Systematic Review. Curr Top Med Chem 2024;24:737-53. [Crossref] [PubMed]
  13. Li H, Li T, Cai Q, et al. Development and Validation of a Radiomics Nomogram for Differentiating Mycoplasma Pneumonia and Bacterial Pneumonia. Diagnostics (Basel) 2021;11:1330. [Crossref] [PubMed]
  14. Wong PK, Yan T, Wang H, et al. Automatic detection of multiple types of pneumonia: Open dataset and a multi-scale attention network. Biomed Signal Process Control 2022;73:103415. [Crossref] [PubMed]
  15. Li M, Wei X, Zhang SS, et al. Recognition of refractory Mycoplasma pneumoniae pneumonia among Myocoplasma pneumoniae pneumonia in hospitalized children: development and validation of a predictive nomogram model. BMC Pulm Med 2023;23:383. [Crossref] [PubMed]
  16. Iwanaga Y, Yamasaki K, Nemoto K, et al. Combined Radiographic Features and Age Can Distinguish Mycoplasma pneumoniae Pneumonia from Other Bacterial Pneumonias: Analysis Using the 16S rRNA Gene Sequencing Data. J Clin Med 2022;11:2201. [Crossref] [PubMed]
Cite this article as: Liu X, Xu W, Meng L, Long J, Sun X, Li Q, Kang H, Mao Y, Hu C, Xu K, Meng Y. Development and validation of a machine learning model for predicting co-infection of Mycoplasma pneumonia in pediatric patients. Transl Pediatr 2025;14(6):1201-1212. doi: 10.21037/tp-2024-562

Download Citation