T he novel coronavirus disease 2019 (COVID-19) poses a major public health threat and is associated with an average mortality of 6.57%. Most patients with COVID-19 experience mild symptoms but others develop severe illness. Patients with mild symptoms present flu-like symptoms such as cough, fever, and sometimes fatigue. On the other hand, if patients fail to undergo timely treatment, they may progress to severe disease, which mostly presents with respiratory distress and/or hypoxemia a week after onset and may rapidly progress to acute respiratory distress syndrome, septic shock, uncorrectable metabolic acidosis, bleeding, coagulation dysfunction and multi-organ failure in severe cases. A recent large multi-center study showed a mortality rate as high as 49% in severe and critically ill patients^[1,2]. Compared to non-severe COVID-19, severe COVID-19 is characterized by a longer hospital stay and higher mortality. A number of studies have focused on the early identification of risk factors in patients who are likely to develop severe illness^[3,4], such as older age, neutrophilia, neutrophil-to-lymphocyte ratio (NLR), and elevated D-dimer levels. Nevertheless, there is a lack of validated indicators from a large number of variables screened to distinguish severe patients. Furthermore, studies from infectious disease hospitals with large numbers of severely and critically ill patients are better able to distinguish severe patients. Therefore, there is an urgent need to develop an accurate and easy scoring system, as well as define possible protective variables^[5-8].

In the present study, we developed a risk prediction score system that characterizes patients who are at risk of developing severe illness. This report was based on a cohort study in JinYinTan Hospital, Wuhan's designated hospital for the treatment of severe COVID-19 patients.

We conducted a retrospective cohort study involving COVID-19 patients, and characterized their demographic, clinical, and laboratory data between January 25, 2020, and March 31, 2020, at the JinYinTan Hospital. We included 188 moderate, 90 severe and 17 critical COVID-19 patients.

Nose and throat swab specimens or nasopharyngeal aspirates collected from the patients were tested for COVID-19 using positive high-throughput sequencing or real-time reverse-transcriptase polymerase chain reaction (RT‐PCR).

Demographic information, medical history, and clinical symptoms were recorded on admission. Imaging results, laboratory findings such as partial arterial oxygen pressure, oxygen saturation, complete blood counts (white blood cells, lymphocytes, monocytes, eosinophils, NLR, hemoglobin levels, and platelet counts), serum biochemical tests (liver and kidney function and lactate dehydrogenase) as well as coagulation indicators (D-dimer levels, prothrombin time, and activated partial thromboplastin time) were routinely conducted in all patients. Demographic characteristics such as age and sex were also recorded. In addition, medical history including cardiovascular disease, digestive system disease, endocrine system disease, malignant tumor, nervous system disease, or respiratory system disease was also recorded. Clinical parameters and symptoms measured included temperature, respiratory rate, heart rate, cardiac arrhythmia, blood pressure, fever, chest pain, palpitation, dyspnea, cough, and stridor, fatigue, xerostomia, nausea and vomiting, diarrhea, and anorexia.

The data were extracted from hospital information systems and laboratory information systems using a standardized data collection form. A group of experienced physicians determined the frequency of examinations, reviewed and cross-checked the data. The data were input double-blind by two physicians and were scrutinized by another 2 independent physicians.

The clinical spectrum of COVID-19 pneumonia ranged from the mild group, moderate group, severe group to the critical group, according to the Chinese management guideline for COVID-19; new coronavirus pneumonia diagnosis and treatment plan (trial version 7) developed by the National Health Committee of the People's Republic of China (http://www.nhc.gov.cn/). The clinical spectrum used was as follows: (1) Mild: mild clinical symptoms with imaging negative for pneumonia. (2) Moderate: respiratory symptoms and fever, with imaging showing pneumonia. (3) Severe: meeting any of the following: a) shortness of breath, respiratory rate ≥ 30 breaths/min; b) during resting-state, oxygen saturation without oxygen uptake ≤ 93%; c) arterial partial pressure of oxygen (PaO₂) / oxygen concentration ≤ 300 mmHg (1 mmHg = 0.133 kPa); d) progressive worsening of clinical symptoms, lung imaging findings showed rapid progression > 50% during the past 24–48 hours. 4) Critical: meeting any of the following: a) respiratory failure, and the need for mechanical ventilation; b) shock; c) multisystem organ failure, and urgent admission to the intensive care unit.

In our study, the severe group included severe and critical cases; in addition, given that no mild cases were enrolled, the non-severe group only included moderate cases.

Chest X-ray and chest computed tomography (CT) abnormalities were scored as follows: 0: Bilateral pneumonia with no abnormal lesions (unilateral or bilateral pneumonia); 1: Unilateral pneumonia with multiple mottling and ground-glass opacities; 2: Bilateral pneumonia with multiple mottling and ground-glass opacities.

We initially enrolled all 302 COVID-19 patients at the JinYinTan Hospital for the selection of variables and development of the risk score system. Categorical and continuous variables were represented by number (%) and by median (interquartile range), respectively. We applied the Chi-square test or Wilcoxon rank sum test to compare the differences between two groups.

Least absolute shrinkage and selection operator (LASSO) regression and 10-fold cross-validation were used to minimize the potential collinearity of variables while multiple logistic regression analysis was used to select the independent risk factors which influenced the results. The statistically significant variables were then used to establish a nomogram. In the case of P < 0.05 for non-linearity, the restricted cubic spline (RCS) was drawn to demonstrate the relationship between the variable and the risk.

A receiver operating characteristic curve (ROC) was drawn to measure the optimal clinical value of the prediction probability. The accuracy of COVID-19 risk factors was assessed by the area under the ROC (AUC), calibration, decision curve, net reclassification improvement and clinical impact curve. An AUC < 0.5 indicated meaningless prediction, an AUC of 0.5–0.7 indicated lower accuracy prediction, while an AUC > 0.7 indicated higher prediction accuracy. Integrated discrimination improvement was applied to evaluate the improvement of standard CURB-65 in our model. Analysis was performed with R software (version 3.6.2, the R Foundation for Statistical Computing, Vienna, Austria). SPSS (version 22.0, Statistical Product and Service Solution, IBM SPSS Inc., IL, USA), and a P-value < 0.05 was considered statistically significant.

For internal validation of the accuracy of the estimates, we used the training set in the 10-fold cross-validation, and used the validation set for external validation.

Out of 302 patients, 8 patients were excluded due to missing data, 90 patients (30.5%) were diagnosed with severe illness, and 17 (5.8%) patients with critical illness, while the remainder (188/295, 63.7%) had moderate illness based on the new coronavirus pneumonia diagnosis and treatment plan (trial version 7; National Health Committee of the People's Republic of China). Our data demonstrated an overall mortality rate of 5.08%. Overall, the median age of patients in the study was 60 (49−68) years, and the median age in the non-severe and severe group was 57.0 (46.7−67.0) years and 62.5 (55.7−70.0) years, respectively, and the difference was statistically significant (P = 0.002). One hundred and sixty-four (55.6%) patients were male and showed a significant difference between the moderate, severe and critical groups (P < 0.001). There was no significant difference in cardiovascular disease, digestive system disease, endocrine system disease, malignant tumor, nervous system disease, or respiratory system disease. Fever, cough, and tachypnea were the most common clinical symptoms. Laboratory findings are shown in Table 1.

A total of 74 variables were documented on hospital admission and were measured in the LASSO regression. Following the application of LASSO regression and L1-penalized least absolute shrinkage and selection regression for multivariable analyses, we used 10 fold-cross validation to obtain a training and validation set from the whole dataset. We then repeated the LASSO and logistic regression analyses which resulted in 3 variables which predicted the development of severe illness (Figure 1A, 1B), which were the NLR (OR, 1.27; 95% CI, 1.15−1.39), albumin (OR, 0.76; 95% CI, 0.70−0.83), chest CT abnormalities (OR, 2.01; 95% CI, 1.41−2.86) (Table 2).

Figure 1.  Predictive risk factors were selected by LASSO regression analysis. (A) LASSO coefficient profiles of the non-zero variables in COVID-19 patients. (B) The lowest point of the red curve indicates the mean−squared error plot, which contains thirteen variables. A more concise model including five variables is the dashed line on the right within one standard error. (C) Nomogram predicting the risk of developing severe COVID-19. LASSO, least absolute shrinkage and selection operator; NLR, neutrophil-to-lymphocyte ratio; CXR, chest CT abnormalities.

The nomogram was constructed based on the coefficients of the predictors (Figure 1C). This allowed clinicians to evaluate the possibility of developing severe COVID-19. The nomogram had a C-index of 0.822 (95% CI, 0.7667–0.8776) (Figure 2A, 2B). In the case of a P < 0.05 for non-linearity, the restricted cubic spline (RCS) was drawn to demonstrate the relationship between the NLR (P < 0.001) and albumin (P < 0.001) and the risk, which is non-linear (Figure 2C, 2D). The clinical impact and decision curve indicated that a combination of the risk factors had a good net benefit and influence on the patients (Figure 2E, 2F), (Supplementary Tables S1–S2 available in www.besjournal.com).

Figure 2.  Association of variables with the risk of severe illness and accuracy of the model. (A–B) The calibration curve and ROC curve of the new model. The C-index was 0.822, the specificity was 0.648 and the sensitivity was 0.814. (C–D) Association of albumin and NLR with the risk of severe illness in patients with COVID-19. (E–F) Decision curve and clinical impact curve of the new complex model in the nomogram for no severe illness survival in patients with COVID-19, compared with actual survival and CURB-65, in which the possibility of survival had a superior standardized net benefit.

During internal validation, the training set was applied and the calibration curve showed convergence of the predicted rates and actual results (Figure 3). In the external validation, the validation set was applied and the AUC of severe COVID-19 was 0.885 (95% CI, 0.7719–0.9666).

Figure 3.  External validation of the new complex prediction model. The ROC curve and calibration curve of internal validation, in which AUC was 86.9%, specificity was 0.576 and sensitivity was 1.00.

In this study, we developed a simple and reliable risk score calculator for clinicians that can predict the development of severe illness in patients with COVID-19. The risk score showed a satisfactory performance and an accuracy of 0.822 based on the training and testing set. Moreover, compared with the standard CURB-65, it not only had a superior sensitivity (34%), but also good specificity. Three indicators were obtained from the 49 variables using the scoring criteria, the NLR, albumin, and chest CT abnormalities, and were used to calculate the possibility of developing severe illness on hospital admission and biochemical test results were quickly obtained after initial diagnosis, which will save valuable time for clinicians.

Previous studies have demonstrated that some of these factors are closely related to the development of severe illness in COVID-19. Liu et al. found that a high NLR, as an inflammatory marker^[9-13], was associated with the diagnosis and poor clinical course of COVID-19. Researchers indicated that aggressive immune responses to COVID-19 initiated more severe symptoms and higher mortality among men than among women^[14,15]. CT abnormalities can reduce the overall survival of patients with COVID-19. Albumin is widely used in the intensive care unit (ICU) in patients with sepsis or critical pneumonia, especially during the COVID-19 pandemic^[16-18]. It is empirically used when patients with COVID-19 develop severe illness with limited proof. Our data indicated that albumin is a protective factor and the use of albumin is beneficial in severely ill patients. However, more clinical data are required to prove this finding.

Although it is critical to evaluate laboratory parameters, such as C-reactive protein, erythrocyte sedimentation rate, iron, CD4+, and CD8+, not all patients underwent all of these tests, especially in an emergency and when there was a shortage of medical resources. Thus, their laboratory variables were not included in the assessment. Therefore, a large sample multi-center study is required. Moreover, this study included 188 moderate, 90 severe and 17 critical COVID-19 patients. Therefore, these findings might not be generalized to those with mild COVID-19.

Taken together, our data demonstrated a predictive risk score to estimate the possibility of patients with COVID-19 developing severe illness. The system was based on 3 variables, NLR, albumin, and chest computed tomography abnormalities, which are easily measured on hospital admission, to help identify COVID-19 patients who have a high risk of progressing to severe illness, thereby requiring tailored treatments at an early stage to avoid moderate or mild cases developing into severe cases.

HUANG Lu Qi, LU Cheng, and XIONG Yi Bai conceived and designed the study, XIONG Yi Bai and MA Yan drafted the manuscript. TIAN Ya Xin and MA Yan performed the statistical analysis. LIU Bin inputted data. Patient were enrolled by RUAN Lian Guo. LU Cheng and HUANG Lu Qi revised the manuscript. All authors read and approved the final manuscript.

The authors thank all doctors in Wuhan JinYinTan Hospital for their help in treating patients with COVID-19.

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome. The authors have no ethical conflicts to disclose.