Development and Validation of an In-Hospital Mortality Prediction Mode | COPD

Introduction

Chronic obstructive pulmonary disease (COPD) is a prevalent respiratory condition and has emerged as the third leading cause of death globally.¹ According to incomplete statistics, approximately 600 million individuals worldwide are affected by COPD, with millions of deaths attributed to the disease each year.² Factors such as smoking, environmental pollution, and population aging contribute to the increasing number of deaths associated with COPD, further burdening the treatment of this condition.¹ However, due to its inherent heterogeneity, the development of personalized treatment plans for COPD remains limited. Consequently, there is an urgent need to discover effective strategies for identifying high-risk patients with a propensity for mortality.

With the development of molecular biology, research has discovered that certain biomarkers contribute to the prognosis evaluation of patients with acute exacerbations of chronic obstructive pulmonary disease (AECOPD). For example, C-reactive protein (CRP), red cell distribution width (RDW), and N-terminal pro-brain natriuretic peptide (NT-proBNP).^3–5 However, the accuracy of these biomarkers in assessing the risk of mortality in AECOPD patients across different populations and disease severities appears to be inconsistent. For instance, Chen found that an elevated level of blood urea nitrogen (BUN) is associated with increased in-hospital mortality in AECOPD patients.⁶ However, in the study by Li,⁵ BUN was not identified as an independent risk factor for mortality in AECOPD patients. Such inconsistencies are common in research on multiple risk factors for mortality in AECOPD patients. Hence, relying solely on one clinical characteristic for evaluating the prognosis of AECOPD patients is unreliable. The use of a predictive model built on multiple risk factors enables the scoring of patients across different dimensions, where higher scores denote an elevated risk of in-hospital mortality. This approach enhances result reliability to a certain extent, providing a more intuitive and convenient methodology.

Several prediction models for in-hospital mortality of AECOPD patients have been proposed previously, but the variables included in their model constructions are not entirely the same.^7–10 Several factors could underlie this discrepancy, including variations in study populations across different regions, differences in study designs, diverse statistical analysis methods, and constraints on the number of clinical features in studies influenced by local healthcare economics. The study cohort observed mainly consists of Chinese AECOPD patients, and through analysis of other similar studies, it was found that there is still a lack of research on in-hospital mortality risk prediction models for Chinese AECOPD patients. For example, some previous studies had insufficient effective sample sizes,^8,9 while others focused on severely ill patients,¹⁰ which could potentially reduce the accuracy and applicability of the models.

Therefore, The study intends to analyze risk factors linked to in-hospital mortality among AECOPD patients using a sufficient sample size and readily available clinical data. Additionally, it seeks to develop a user-friendly nomogram model for predicting mortality risk during hospitalization. The goal is for this research to offer a scientific foundation and valuable guidance to clinicians in identifying high-risk patients for timely intervention.

Materials and Methods

Study Design and Subjects

This retrospective clinical study involved a total of 1224 hospitalized patients who were diagnosed with AECOPD at the Second People’s Hospital of Wuhu City between January 2013 and December 2022. The study included patients who met the following inclusion criteria: (1) The COPD diagnosis is definitive, and the patient was hospitalized this time due to an acute exacerbation. The diagnostic code for the AECOPD is “J44.100”; (2) AECOPD characterized by worsening dyspnea and/or cough and sputum symptoms in COPD patients, with symptom deterioration occurring within 14 days, potentially accompanied by shortness of breath and/or tachycardia.¹¹ The exclusion criteria were as follows: (1) patients admitted solely for respiratory distress caused by pulmonary embolism or acute heart failure; (2) patients with severely incomplete laboratory data; (3) patients with multiple readmissions; (4) patients aged ≤40 years.

Study Outcomes

The primary outcome of this study is the occurrence of in-hospital died.

Predictive Factors

We obtained the demographic characteristics, duration of illness, smoking index, comorbidities, complications, heart rate (HR), respiratory rate (R), and laboratory indicators of patients from the electronic medical record system. The demographic characteristics include patient gender and age. The smoking index (SI) is calculated by multiplying the number of cigarettes smoked per day by the number of years smoked. We identified acute respiratory failure (ARF) as a complication, using whether the patient’s admission diagnosis included respiratory failure as a screening criterion. The comorbidities comprise hypertension (HTN), coronary heart disease (CHD), diabetes, bronchial asthma, bronchiectasis, and lung cancer, which were diagnosed before or after admission. We collected data on the first laboratory tests following admission, including red blood cell count (RBC), hemoglobin (HB), RDW, platelet count (PLT), platelet distribution width (PDW), absolute lymphocyte count (ALC), absolute neutrophil count (ANC), eosinophil count (EOS), D-dimer (DD), fibrinogen (FIB), albumin (ALB), BUN, creatinine (Cr), interleukin-6 (IL-6), procalcitonin (PCT), serum potassium, serum sodium, serum chloride, and serum calcium.

Missing Values

In the dataset of this study, there are missing values, with a maximum proportion of missingness reaching 5.47%. To handle these missing values, we applied the “missForest” package available in R software and utilized the random forest imputation method. The imputation process resulted in an NRMSE value of 0.129 for the OOBerror, along with a PFC value of 0.¹²

Statistical Analysis

Summary of baseline characteristics of patients was conducted using descriptive statistical methods. Continuous variables were expressed as mean ± standard deviation or median (interquartile range [IQR]), while categorical variables were presented as numbers (percentages). Student’s t-test and Mann–Whitney U-test were used for intergroup comparisons of continuous variables with normal and non-normal distributions, respectively. Pearson’s chi-square test or Fisher’s exact test was employed for the analysis of categorical variables.

To develop a robust predictive model, we utilized the Least Absolute Shrinkage and Selection Operator (LASSO) to identify candidate variables with potential predictive significance. Variable selection was based on the lambda.1se in LASSO regression cross-validation, resulting in the establishment of Model A. LASSO regression, characterized by the imposition of penalties and continual coefficient compression, aims to curb overfitting and collinearity by reducing the model’s variable count. To enhance model simplicity and clinical applicability, we applied multiple logistic regression to analyze the variables chosen through LASSO regression and retained those with P

All statistical analyses were conducted using R version 4.3.0 (www.r-project.org), SPSS version 26, and EmpowerStats (www.empowerstats.com). Differences with a two-sided p-value of less than 0.05 were deemed statistically significant.

Results

Baseline Characteristics

A total of 1224 patients were included in the study, of whom 98 (8%) died during hospitalization. The study flowchart is presented in Figure 1. Patients were categorized into two groups based on in-hospital died. Table 1 displays the baseline characteristics of the study population. The two groups differed in terms of age, heart rate, respiratory rate, comorbidities, and laboratory indicators. Deceased patients had a higher average age, faster resting heart rate and respiratory rate, and a higher proportion of ARF upon admission compared to non-deceased patients. In terms of laboratory indicators, deceased patients exhibited lower levels of RBC, HB, PDW, ALC, EOS, ALB, serum sodium, serum potassium, and serum calcium, while they had higher levels of RDW, ANC, DD, BUN, IL-6, PCT, and serum potassium.

Figure 1 Patients inclusion flowchart.

Model Establishment and Validation

Based on LASSO regression and tenfold cross-validation, 11 variables were selected at one standard error (lambda.1se), including ARF, Lung Cancer, HR, HB, ANC, ALB, PCT, serum chloride, DD, BUN, and IL-6 (Figure 2a and b). Using these variables, model A was developed and its performance assessed for discrimination, calibration, and clinical utility (Table 2). Figure 3a illustrates the discriminative capability of the model, revealing a C-index of 0.859 (95% CI: 0.820, 0.898) for predicting in-hospital mortality risk among AECOPD patients. In Figure 4a, the calibration curve of the model displays some deviation from the optimal line, with a maximum deviation of 0.045 and minimum deviation of 0.007. Nonetheless, The P-value of the goodness-of-fit test for the Hosmer-Lemeshow statistic is greater than 0.05, specifically 0.48. The DCA curve in Figure 5 indicates that for threshold probabilities ranging from 0.02 to 0.93, as compared to “treat all” or “no treatment”, the model offers a net benefit to patients, with a maximum value of 0.07.

Table 2 Model A

Figure 2 The vertical axis in the figure represents the values of the coefficients, the top horizontal axis represents the number of non-zero coefficients in the model, and each line represents a variable (A). The vertical axis represents the cross-validationerror, and the top horizontal axis corresponds to the number of variables for different λ values. The dashed line on the left corresponds to the number of variables corresponding to the minimum standard error, while the dashed line on the right corresponds to the number of variables corresponding to 1 times the standard error (B).

Figure 3 ROC of model A (A) and model B (B).

Figure 4 The gray line in the middle of the chart (45-degree diagonal line) represents the ideal curve, the solid black line represents the curve of the current model, and the dashed line represents the calibration curve. It can be seen from the figure that the fitting line of the calibration curve of Model B is very close to the ideal line, indicating good model fit (A and B).

Figure 5 The x-axis represents the threshold probability, and the y-axis represents the net benefit. The red solid line represents the column chart model. The decision curve indicates that both result in a similar peak net benefit for patients, with a threshold probability of 2–92% for Model A, and 2–72% for Model B.

Following the multivariable logistic regression analysis, the variables DD, PCT, and IL-6 were excluded based on a significance level of 0.05. Subsequently, model B was constructed using eight variables: RF, Lung Cancer, HR, HB, ANC, ALB, BUN, and serum chloride (Table 3). The C-index of Model B for predicting in-hospital mortality risk among AECOPD patients was 0.858 (95% CI 0.819, 0.897) (Figure 3b), signifying the effective discriminatory power of the model. Examination of the calibration curve in Figure 4b revealed a close alignment of the fitted curve with the ideal line, demonstrating a maximum deviation of 0.022 and a minimum deviation of 0.005. The goodness-of-fit test yielded a P value of 0.520 with a chi-square value of 8.133. Furthermore, the DCA of model B indicated a net benefit for patients within a threshold probability range from 0.02 to 0.73, with a peak value near 0.07 (Figure 5).

Table 3 Model B

By comprehensively considering discrimination, calibration, and clinical utility, we conducted a multi-faceted comparison between Model A and Model B. Initially, there was no significant difference in discrimination between the two models, with only a minimal difference of 0.001 in the C-index. Regarding calibration, Model B was notably superior to Model A, as the calibration curve in Model A deviated significantly from the ideal line. In terms of clinical utility, Results indicated a higher threshold probability for Model A compared to Model B. However, in line with clinical practicality, we found that the variables in Model B were more easily obtainable compared to the time-consuming acquisition of PCT and IL-6 data in Model A. Therefore, we ultimately selected Model B and conducted internal validation using the bootstrap method with 500 resamplings, resulting in a C-index of 0.851 (95% CI: 0.805, 0.893) (Figure 6). Ultimately, we visually presented model B through a a nomogram (Figure 7).

Figure 6 Figure 6 shows the internal validation of Model B using the bootstrap method, with the blue shaded area indicating the estimated 95% confidence interval.

Figure 7 This figure shows the nomogram model established in this study for predicting in-hospital mortality risk of AECOPD patients. The top section corresponds to the predicted scores for different variable values, while the middle section represents the predictive factors. By assigning values based on the patient’s clinical data, the total assigned score corresponds to the predicted probability of in-hospital mortality at the bottom. The larger the total score, the higher the in-hospital mortality risk for AECOPD patients.

Discussion

Key Findings

Using the data from this study, we identified a strong association between the clinical characteristics of ARF, Lung Cancer, HR, HB, ANC, ALB, BUN, and serum chloride with in-hospital mortality among AECOPD patients. Further analysis revealed that these eight variables encompass various essential aspects of AECOPD patients, including complications (ARF, serum chloride), comorbidities (Lung Cancer, HB), and inflammation markers (ANC, ALB). This suggests that the selected variables may serve as pivotal features in this AECOPD patient cohort. We developed a l nomogram model to predict in-hospital mortality risk for AECOPD patients based on these variables and evaluated its predictive capabilities. The findings demonstrated that this model provides effective discrimination, calibration, and clinical utility. Internal cross-validation further confirmed its reliability. Moreover, the clinical data used in our model are commonly available and can be promptly obtained following patient admission.

Comparison with Other Studies and Interpretation of the Model

Firstly, regarding the type of clinical research, we adopted a retrospective study approach, which is consistent with the majority of previous studies.7–10 With the development of electronic medical records, clinical data are now more easily stored and accessed. However, due to regional differences in healthcare economic conditions, the types and quantities of variables included in the analysis may vary across different studies. Secondly, upon analyzing previous studies, it was found that some studies had insufficient sample sizes, such as in Dong’s study,⁸ where a total of 29 deaths occurred among AECOPD patients, and Chen’s study,⁹ where a total of 19 deaths occurred among AECOPD patients. This could potentially affect the stability of the final model. Therefore, we evaluated the effective sample size in this study based on the 10EPV (events per variable) principle.^13,14 The model we established in this study includes 8 variables, which according to the 10EPV principle would require a minimum of 80 positive samples. However, in our study, a total of 98 AECOPD patients died during hospitalization. This indicates the credibility of the model we constructed.

In addition, regarding the discrimination ability of the model, Yu established a model with a C-index of 0.929, indicating excellent performance.¹⁵ The C-index of other models was 0.745, 0.785, 0.82, and 0.85^10,16–18 respectively. Our model achieved a C-index of 0.858 (95% CI 0.819, 0.897), positioning it above average compared to previous models. Additionally, the C-index calculated through internal validation using the bootstrap method was 0.851, showing minimal fluctuation between the two indices. In fact, there is a contradiction between the number of variables in the model and the desired efficacy of the model. Tabak argues that limiting the total number of variables is more important than achieving the maximum discriminative ability.¹⁹ We agree with this viewpoint and emphasize that including variables that capture key features of AECOPD will make the model more clinically meaningful.

In the course of AECOPD, the oxygen deficiency in patients exacerbates, frequently resulting in the development of ARF. ARF represents a complex and severe syndrome characterized by physiological and metabolic disruptions, predominantly arising from inadequate lung ventilation among AECOPD patients. Similar to studies conducted by Dong, Chen, and others,^8–10 we assessed whether patients developed RF and found that the presence of ARF is an independent risk factor for in-hospital mortality in AECOPD patients. Additionally, the presence of comorbidities often exacerbates the severity of RF, such as pneumonia, acute heart failure, arrhythmias, and lung cancer.²⁰ In our study, lung cancer was identified as an independent risk factor for mortality in AECOPD patients, which is similar to the findings of Chen et al.¹⁷ Anemia is also a common comorbidity in COPD patients, with reports indicating its prevalence as high as 7.53%.²¹ The decrease in hemoglobin levels further reduces the available oxygen in the blood for AECOPD patients. A study by Cireli demonstrated that AECOPD patients with concomitant anemia have shorter survival times, consistent with our research findings.²²

Respiratory tract infection, on the other hand, is the most common cause of acute exacerbation in COPD patients, during which inflammatory cells and inflammatory biomarkers levels escalate significantly. The established model in this study encompasses the former. Firstly, regarding ANC, consistent with the study conducted by Chen,¹⁰ our results indicate that ANC is an effective predictive factor for the prognosis of AECOPD patients. Secondly, during the acute inflammatory phase, levels of certain negative acute-phase reactants (APRs) decrease, such as albumin (ALB). Its predictive role in in-hospital mortality of AECOPD patients has been confirmed in several studies.^10,15,18 Similar to ALB, BUN is currently regarded as a predictive factor for in-hospital mortality in AECOPD patients.^6,23 In the study by Chen et al,⁶ the optimal cut-off value for BUN was 7.63, which is very close to the median value of BUN in our death cohort (7.49).

Electrolyte imbalance is also common in hospitalized AECOPD patients. In our study, hypochloremia was found to be associated with in-hospital mortality in AECOPD patients, whereas previous studies did not include this variable.^9,15,16 There has been limited research on the relationship between hypochloremia and AECOPD prognosis. Existing studies suggest that hypochloremia is associated with poor outcomes in patients with acute heart failure²⁴ and pulmonary arterial hypertension,²⁵ both of which are common complications in AECOPD patients. Further research is needed to explore the association between hypochloremia and the prognosis of AECOPD patients in the future.

The baseline heart rate elevation in AECOPD patients is associated with multiple factors such as hypoxia and infection. Byrd analyzed the baseline heart rate of 16,485 COPD patients and found a correlation between high baseline heart rate and overall mortality in COPD patients, showing a linear relationship. This is consistent with the results of our study.²⁶

Limitations

The study has several Limitations. Firstly, a substantial amount of data was lost due to the constraints of a retrospective design, resulting in the exclusion of vital clinical data like PaO2, PaCO2, CRP, and NTpro-BNP due to a high degree of missing information. Secondly, the choice of a single-center approach restricted the assessment of the model’s external generalizability. Moreover, potential biases in some variables may stem from the inherent limitations of a retrospective study. For instance, incomplete documentation of patients’ smoking habits impeded a more thorough investigation.

Conclusions

In summary, our study constructs a model to predict the risk of mortality in AECOPD patients after admission based on 8 variables: ARF, Lung Cancer, HR, HB, ANC, ALB, BUN, and Serum Chloride. These variables can be obtained within a relatively short period of time, and the model is presented in a column line graph format, facilitating clinical assessment of AECOPD patients’ condition by Clinical doctors.

Declarations

This study is in accordance with the Helsinki Declaration.

Ethical Approval

This study has been approved by the Medical Ethics Committee of Wuhu Second People’s Hospital (Ethics number: 2023-KY-026). It is a retrospective study, and we have anonymized the patients. The review committee has waived the requirement for written informed consent.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Funding

This research received financial support from a project grant provided by the city of Wuhu, identified by project number 2020rkx4-3.

Disclosure

The authors of this study declare no conflicts of interest.

References

1. Lozano R, Naghavi M, Foreman K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380(9859):2095–2128. doi:10.1016/S0140-6736(12)61728-0

2. Wang C, Xu J, Yang L, et al. Prevalence and risk factors of chronic obstructive pulmonary disease in China (the China Pulmonary Health [CPH] study): a national cross-sectional study. Lancet. 2018;391(10131):1706–1717. doi:10.1016/S0140-6736(18)30841-9

3. Sneh A, Pawan T, Randeep G, et al. Acute phase proteins as predictors of survival in patients with acute exacerbation of chronic obstructive pulmonary disease requiring mechanical ventilation. COPD. 2020;17(1):22–28. doi:10.1080/15412555.2019.1698019

4. Hu GP, Zhou YM, Wu ZL, et al. Red blood cell distribution width is an independent predictor of mortality for an acute exacerbation of COPD. Int J Tuberc Lung Dis. 2019;23(7):817–823. doi:10.5588/ijtld.18.0429

5. Li H, Zeng Z, Cheng J, et al. Prognostic role of NT-proBNP for in-hospital and 1-year mortality in patients with acute exacerbations of COPD. Int J Chron Obstruct Pulmon Dis. 2020;15:57–67. doi:10.2147/COPD.S231808

6. Chen L, Chen L, Zheng H, et al. The association of blood urea nitrogen levels upon emergency admission with mortality in acute exacerbation of chronic obstructive pulmonary disease. Chron Respir Dis. 2021;18:14799731211060051. doi:10.1177/14799731211060051

7. Moll M, Qiao D, Regan EA, et al. Machine learning and prediction of all-cause mortality in COPD. Chest. 2020;158(3):952–964. doi:10.1016/j.chest.2020.02.079

8. Dong F, Ren X, Huang K, et al. Development and validation of risk prediction model for in-hospital mortality among patients hospitalized with acute exacerbation chronic obstructive pulmonary disease between 2015 and 2019. Front Med. 2021;8:630870. doi:10.3389/fmed.2021.630870

9. Chen L, Chen L, Zheng H, et al. Emergency admission parameters for predicting in-hospital mortality in patients with acute exacerbations of chronic obstructive pulmonary disease with hypercapnic respiratory failure. BMC Pulm Med. 2021;21(1):258. doi:10.1186/s12890-021-01624-1

10. Chen D, Chen C, Zhang P, et al. The arrival ward requiring help by wheelchair or medical cart, arterial oxygenation index, age, albumin and neutrophil count score: predicting in-hospital mortality in Chinese patients with acute exacerbations of chronic obstructive pulmonary disease. Chron Respir Dis. 2023;20:14799731231197226. doi:10.1177/14799731231197226

11. Agustí A, Celli BR, Criner GJ, et al. Global initiative for chronic obstructive lung disease 2023 report: GOLD executive summary. Eur Respir J. 2023;61(4):2300239. doi:10.1183/13993003.00239-2023

12. Stekhoven DJ, Bühlmann P. MissForest–non-parametric missing value imputation for mixed-type data. Bioinformatics. 2012;28(1):112–118. doi:10.1093/bioinformatics/btr597

13. Peduzzi P, Concato J, Kemper E, et al. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–1379. doi:10.1016/s0895-4356(96)00236-3

14. Guo D, Wang H, Lai X, et al. Development and validation of a nomogram for predicting acute kidney injury after orthotopic liver transplantation. Ren Fail. 2021;43(1):1588–1600. doi:10.1080/0886022X.2021.2009863

15. Yu X, Zhu GP, Cai TF, et al. Establishment of risk prediction model and risk score for in-hospital mortality in patients with AECOPD. Clin Respir J. 2020;14(11):1090–1098. doi:10.1111/crj.13246

16. Peng JC, Gong WW, Wu Y, et al. Development and validation of a prognostic nomogram among patients with acute exacerbation of chronic obstructive pulmonary disease in intensive care unit. BMC Pulm Med. 2022;22(1):306. doi:10.1186/s12890-022-02100-0

17. Chen S, Shi Y, Hu B, et al. A prediction model for in-hospital mortality of acute exacerbations of chronic obstructive pulmonary disease patients based on red cell distribution Width-to-Platelet ratio. Int J Chron Obstruct Pulmon Dis. 2023;18:2079–2091. doi:10.2147/COPD.S418162

18. Mekanimitdee P, Morasert T, Patumanond J, et al. The MAGENTA model for individual prediction of in-hospital mortality in chronic obstructive pulmonary disease with acute exacerbation in resource-limited countries: a development study. PLoS One. 2021;16(8):e0256866. doi:10.1371/journal.pone.0256866

19. Tabak YP, Sun X, Johannes RS, et al. Development and validation of a mortality risk-adjustment model for patients hospitalized for exacerbations of chronic obstructive pulmonary disease. Med Care. 2013;51(7):597–605. doi:10.1097/MLR.0b013e3182901982

20. Akbaş T, Güneş H. Characteristics and outcomes of patients with chronic obstructive pulmonary disease admitted to the intensive care unit due to acute hypercapnic respiratory failure. Acute Crit Care. 2023;38(1):49–56. doi:10.4266/acc.2022.01011

21. Sarkar M, Rajta PN, Khatana J. Anemia in Chronic obstructive pulmonary disease: prevalence, pathogenesis, and potential impact. Lung India. 2015;32(2):142–151. doi:10.4103/0970-2113.152626

22. Cireli E, Mertoğlu A. The impact of anemia on the mortality of COPD patients hospitalized for acute exacerbation resulting in respiratory failure. Monaldi Arch Chest Dis. 2022;93(2). doi:10.4081/monaldi.2022.2254

23. Zhang J, Qin Y, Zhou C, et al. Elevated BUN upon admission as a predictor of in-hospital mortality among patients with acute exacerbation of COPD: a secondary analysis of multicenter cohort study. Int J Chron Obstruct Pulmon Dis. 2023;18:1445–1455. doi:10.2147/COPD.S412106

24. Nozaki Y, Yoshihisa A, Sato Y, et al. Persistent hypochloremia is associated with adverse prognosis in patients repeatedly hospitalized for heart failure. J Clin Med. 2023;12(4):1257. doi:10.3390/jcm12041257

25. Prins KW, Kalra R, Rose L, et al. Hypochloremia is a noninvasive predictor of mortality in pulmonary arterial hypertension. J Am Heart Assoc. 2020;9(5):e015221. doi:10.1161/JAHA.119.015221

26. Byrd JB, Newby DE, Anderson JA, et al. Blood pressure, heart rate, and mortality in chronic obstructive pulmonary disease: the SUMMIT trial. Eur Heart J. 2018;39(33):3128–3134. doi:10.1093/eurheartj/ehy451