A nomogram based on a patient-reported outcomes measure: predicting the risk of readmission for patients with chronic heart failure
Health and Quality of Life Outcomes volume 18, Article number: 290 (2020)
Health-related quality of life, as evaluated by a patient-reported outcomes measure (PROM), is an important prognostic marker in patients with chronic heart failure. This study aimed to use PROM to establish an effective readmission nomogram for chronic heart failure.
Using a PROM as a measurement tool, we conducted a readmission nomogram for chronic heart failure on a prospective observational study comprising of 454 patients with chronic heart failure hospitalized between May 2017 to January 2020. A Concordance index and calibration curve were used to evaluate the discriminative ability and predictive accuracy of the nomogram. A bootstrap resampling method was used for internal validation of results.
The median follow-up period in the study was 372 days. After a final COX regression analysis, the gender, income, health care, appetite-sleep, anxiety, depression, paranoia, support, and independence were identified and included in the nomogram. The nomogram showed moderate discrimination, with a concordance index of 0.737 (95% CI 0.673–0.800). The calibration curves for the probability of readmission for patients with chronic heart failure showed high consistency between the probability, as predicted, and the actual probability.
This model offers a platform to assess the risk of readmission for different populations with CHF and can assist clinicians with personalized treatment recommendations.
Re-admission is the main adverse outcome for patients with heart failure (HF). Because it could be associated with a high mortality, lead to a decreased quality of life, and cause a significant financial burden [1,2,3]. Thus, it is important to assess the prognosis of HF, as patients at higher risk of poor outcomes could receive more intensive therapy and close monitoring . There are already efforts to develop novel prognostic models for HF . A number of clinical studies predicting hospitalization for the deterioration of HF have been summarized in detail by Rahimi et al. [6, 7] The majority of effective models were built based on clinical data collected from discharge records of patients ; however, research indicates that high health-related quality of life (HRQOL) predicts a more favorable prognosis in patients with cardiovascular disease . Thus, HRQOL should be used as a new prognostic indicator for HF patients [10, 11], and as an independent predictive factor of readmission with HF [12, 13].
The patient-reported outcomes measure (PROM) evaluates a patient’s quality of life by the way of scales to promote communication, inspect and identify HRQOL issues, enhance patient-centered treatment, and increase patient satisfaction . PROM is widely used in HRQOL and combines all aspects of emotional, psychological, physical, and social functions, and even includes an individual’s subjective perception of health [15, 16]. Over the years, there has been considerable progress in the measurement, analysis, and interpretation of PROM, which has been validated by multifarious studies . Thus, PROM is helpful in diagnosis and therapy, and is of significant importance in clinical practice [18, 19] and widely recognized by medical professionals.
Nomograms have been regarded as dependable instruments by creating simple intuitive diagrams of the predictive models that quantify the risk of clinical adverse events [20, 21]. In recent years, nomograms have also been used to predict the prognosis of HF [22,23,24]. This study, as far as we know, is the first attempt to construct a prognostic nomogram of chronic heart failure (CHF) based on the Chinese PROM data of 454 patients with CHF, to predict the possibility of readmission.
Study design and sample
This study is a prospective observational study, from May 2017 to January 2020, patients in the department of cardiology of the top three hospitals in the Shanxi Province (the first affiliated hospital of Shanxi Medical University, Shanxi Cardiovascular Hospital and Bethune Hospital) who were consecutively chosen for the study. We conducted a questionnaire survey of inpatients who met the inclusion criteria and screened the filled scale. Follow-up calls were made via phone at 1, 3, 6, 12, 18, and 24 months after discharge to record whether a patient was readmitted, as well as the readmission time.
Inclusion and exclusion criteria
Patients eligible for participation in the study had to meet the following criteria: (1) aged ≥18 years, (2) diagnosed with HF, according to the guideline for the diagnosis and treatment of HF in China (2018) , (3) fall under the New York Heart Association (NYHA) functional class II-IV, and (4) received HF treatment while in the hospital. Patients were excluded if they suffered from acute cardiovascular events within two months, or if they refused to participate in the project.
The study uses a Chinese questionnaire related to mainland China – CHF-PROM . The CHF-PROM is divided into four domains which are further divided into 12 subdomains, including a total of 57 items. The structural frame of CHF-PROM is shown in Table 1.
In this study, the reliability of 12 subdomains were considered acceptable: Cronbach’s alpha coefficient > 0.60, (i.e., SOM (0.691), APS (0.617), IND (0.846), ANX (0.750), DEP (0.823), FEA (0.787), PAR (0.884), SUP (0.726), UTI (0.809), COM (0.854), SAT (0.826), EOD(0.751)).
The CHF-PROM uses a 5-point Likert scale to rate the responses and the scores for each item range from 0 (never) to 4 (always). In the study, all responses were converted in the following way to ensure consistency between the PROM and answers to the 56 items: positively scoring items were recorded as the raw score, and negatively scoring items were recorded as 4 minus the raw score. This ensured that the score for each item still ranged from 0 to 4 and further, the correspondence of a higher score to a more positive PROM.
This study was approved by the ethics committee of Shanxi Medical University and we obtained the consent of all participants. We screened inpatients who met the diagnostic criteria from the hospital medical records system, and eligible patients were contacted and recruited through a face-to-face interview. Eligible patients who agreed to participate in the study filled out a questionnaire through which we obtained other clinical data, demographic data, as well as CHF-PROM data.
Categorical variables in demographic characteristics were described by frequency, and the Log-Rank test was performed for comparison between groups. The median follow-up time was calculated by the Reverse Kaplan-Meier method. The variables of P < 0.05 and subdomains of PRO (patient-reported outcomes) were initially included in the Cox regression analysis to identify the factors that increase the risk of CHF readmission. A subset of the predictors was then selected using a step-forward method to obtain the final model that was relatively streamlined with the maximum concordance index (C-index) and minimal Akaike information criterion (AIC). The factors of the final regression model were included in the construction of the nomograms to assess readmission probability.
Discrimination and calibration were generally used to evaluate the performance of the nomogram . The discrimination was evaluated by Harrell’s C-index, which is analogous to the area under curve (AUC), yet demonstrated to be more appropriate for censored data. The calibration curves can represent the calibration to appraise the consistency between predicted readmission and observed readmission. Moreover, 1000 bootstrap samples were used for internal validation to evaluate the performance of the prediction . The missForest method used for imputing for the missing values. Double entry of data using Epidata3.1 software was done to control data quality. The Reverse Kaplan-Meier method were performed using SPSS version 22.0. Cox regression and nomogram were implemented by the Survival and Hmisc package in R 3.6.1. In all analyses, P < 0.05 was regarded as statistically significant.
A total of 978 completed CHF-PROMs were collected in this study, among which 210 were not followed up, 224 were followed up once, 241 were followed up twice, 195 were followed up three times, and 108 were followed up four times. 90 patients with NYHA class I were excluded. Finally, 454 patients who met the inclusion criteria and were followed up more than once were entered to construct and verify our prediction model. The median follow-up time in this study was 372 days. The characteristics of demography and clinic for the patients in the study cohort are listed in Table 2.
Selected factors for the model
Demographic variables screened by univariate analysis (such as age, gender, smoking, health care, income) and PRO were selected into the multivariable analysis. The multivariable analysis analyses showed that the readmission of patients with CHF was significantly related with income (P = 0.049), appetite, sleep (P = 0.016), social support (P = 0.008), paranoia (P = 0.012) and independence in terms of self-care ability, as well as the performance of daily activities (P < 0.001). However, age, gender, health care, smoking, somatization, anxiety, depression, fear, satisfaction with hospital treatment services, and compliance were not significant. The final prediction model with maximal C-index and minimum AIC was obtained by the step-forward selection, which included gender, income, health care, appetite-sleep, anxiety, depression, paranoia, support, and independence (Table 3).
Predictive Nomogram for the probability of readmission
Based on the final Cox regression analysis, the nomogram that including nine predictors of appetite-sleep, independence, anxiety, depression, paranoia, support, health care, gender, and income was built to estimate probability of without readmission (Fig. 1). The score of each variable was revealed on the points axis and the total score with seven predictors was reached by adding each point. Further, by putting the total points projection on the probability axis, we could directly calculate the probability of without readmission (Probability of readmission = 1 - Probability of without readmission). For example, for middle income males, enrolled in the rural health care program, with an appetite-sleep score of 5, independence score of 6, anxiety score of 5, depression score of 8, paranoia score of 12 and support score of 6, the total score calculated would be the sum of: 23.5 + 45 + 21 + 10 + 27.5 + 15 + 12.5 + 32.5 + 20 = 207. This corresponded with a 6-month without readmission probability of 0.54 and 1-year without readmission probability 0.38. Therefore, this person was approximately at 46% risk of readmission at 6-months and 62% risk of readmission at 1 year.
Performance of the Nomogram
The discrimination of the nomogram was measured by the C-index and the calibration was assessed using a calibration curve comparing the predicted probabilities and observed probabilities. The C-index value of 0.5 indicates no predictive discrimination and a value of 1.0 indicates the perfect separation of patients with different outcomes . The C-index for a prognostic model is typically between about 0.6 and 0.85 (the larger the c-index, the more accurate the prognostic prediction) . Calibration can be investigated by plotting the observed proportions of events against the predicted risks for groups defined by ranges of individual predicted risks. Ideally, if the observed proportions of events and predicted probabilities agree over the whole range of probabilities, the plot shows a 45° line (that is, the slope is 1) . Bootstraps with 1000 resample were used for these activities. The nomogram displayed moderate discrimination with a C-index of 0.737 (95% CI 0.673–0.800) and splendid calibration. The calibration curve of the accuracy estimated by bootstrap (1000 resampling) was highly consistent with the diagonal, indicating that the predicted probability of ‘without readmission’ was in accordance with the actual probability (Fig. 2).
Many studies show that HRQOL is independent predictive factors of prognosis for inpatients with HF and that they have important predictive value [12, 13] and nomogram based on Cox regression has been widely used to predict the survival time of chronic diseases, especially cancer [30,31,32]. However, as far as we know, there have been no studies based on it to construct the quantitative model to predict the probability of readmission for patients with CHF. This study constructed a simple intuitive graph of the prediction model based on PROM to quantify the risk of readmission for CHF. This can be an important aid when doctors make treatment recommendations for patients with CHF.
There are some things that may be highlighted in this study. First, PROM used in our study is a questionnaire in Chinese based on the different cultural and societal value systems of mainland China as well as the medical and economic environments of the country. The reliability and validity of the scale have been verified by Tian et al. , and they were further verified and screened in this study. Second, only patients with CHF were selected in the study, regardless of etiology, LVEF, complications, etc. Thus, the database had covered, and was representative of, a wider population, further promoting the clinical application of the model. Third, internal validation through a bootstrap resampling method demonstrated moderate discrimination and excellent calibration, illustrating that the nomogram based on PROM may be valuable for patients with CHF.
This study, using data from strict screening and regular follow-up of CHF patients, confirms the significance of some demographic characteristics for prognosis; the results are consistent with those from other studies [33,34,35,36]. In our prediction model, anxiety showed the greatest effect on the risk of readmission, followed by paranoia, health care, independence, income, support and appetite-sleep, while the smallest contributors were gender and depression.
Recently, a prospective observational study provided evidence of physical weakness, independence, support from society and family, anxiety, and depression being likely predictors of 30-day prognosis after hospitalization for HF . Staniute et al. further also demonstrated that anxiety, depression and social support can indirectly affect the quality of life of patients with HF . Moreover, anxiety, appetite and sleep were confirmed to be predictors of readmission for CHF in retrospective studies, which may have been impacted by the fact that the patient’s status influences the risk of readmission [39,40,41,42]. Kitamura M not only confirmed that daily activities were independent predictors of readmission in heart failure patients within 90 days, but also calculated the cut-off value by ROC curve  and the study have also proved that self-care and daily activities are the mediating factors of readmission of heart failure . While Hochang Benjamin Lee et al. emphasized that personality disorders as predictors of incident cardiovascular disease increased risk disease , our results are quite the opposite, which may be because the subjects studied were different; Apart from the results on patient paranoia, these findings were similar to the results of our reports on the readmission risk factors for CHF. We found that gender, income, health care, appetite-sleep, anxiety, depression, paranoia, support, and independence were predictors of readmission for CHF.
Many clinicians are usually able to make a preliminary assessment of a patient’s prognosis through clinical data; however, combining it with PRO can provide a more comprehensive and accurate understanding of the real health status of the patient. In practice, the process has been streamlined as the simple nomogram can be incorporated in mobile applications.
It would also be important to note some limitations of the study. First, according to the analysis, patients who refuse to follow up may be worse off than those who cooperate with follow-up, so that excluding these patients will underestimate the rehospitalization rate of patients with chronic heart failure. In addition, because the scale was subjectively filled by the patient, part of the content in the scale may be missing. Although we have imputed the missing data, there may still be bias. Second, most critical patients were not included in the study due to their inability to complete the scale. Due to this selection bias, our model may underestimate patient readmission rates. If all patients who meet the inclusion criteria could be selected, the actual C-index might be higher. Third, though the internally validated model demonstrated moderate discrimination and splendid calibration, considering the epidemiological and clinical behavioral differences between regions, the universality of this nomogram still requires additional databases to be used for external validation, especially from other provinces.
In conclusion, by using PROM as a measurement tool, this study has constructed and internally validated a neoteric nomogram to predict the probability of readmission for CHF patients. The nomogram is moderate accurate, easy to use, and displays splendid calibration; so, it can assist clinicians in assessing the risk of readmission for CHF patients, and further, make personalized treatment recommendations for them.
Availability of data and materials
Please contact the corresponding author for the study data, which will be granted upon reasonable request.
Patient-reported outcomes measure for chronic heart failure
Health-related quality of life
Akaike information criterion
- C. I:
Yaku H, et al. Demographics, management, and in-hospital outcome of hospitalized acute heart failure syndrome patients in contemporary real clinical practice in Japan—observations from the prospective, multicenter Kyoto congestive heart failure (KCHF) registry—. Circ J. 2018;82(11):2811–9.
Yancy CW, et al. 2017 ACC/AHA/HFSA focused update of the 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology/American Heart Association task force on clinical practice guidelines and the Heart Failure Society of America. J Am Coll Cardiol. 2017;70(6):776–803.
Gheorghiade M, et al. Rehospitalization for heart failure: problems and perspectives. J Am Coll Cardiol. 2013;61(4):391-403.
Ponikowski P, et al. ESC guidelines for the diagnosis and treatment of acute and chronic heart failure. Eur J Heart Fail. 2016;18(8):891–975.
Pocock SJ, et al. Predicting survival in heart failure: a risk score based on 39 372 patients from 30 studies. Eur Heart J. 2013;34(19):1404–13.
Rahimi K, et al. Risk prediction in patients with heart failure: a systematic review and analysis. JACC: Heart Failure. 2014;2(5):440–6.
Ouwerkerk W, Voors AA, Zwinderman AH. Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure. JACC: Heart Failure. 2014;2(5):429–36.
Su A, Al'Aref SJ, Beecy AN, Min JK, Karas MG. Clinical and Socioeconomic Predictors of Heart Failure Readmissions: A Review of Contemporary Literature. Mayo Clin Proc. 2019;94(7):1304–20.
Fernandes CJ, et al. Quality of life as a prognostic marker in pulmonary arterial hypertension. Health Qual Life Outcomes. 2014;12(1):130.
Hoekstra T, Jaarsma T, Van Veldhuisen D J, et al. Quality of life and survival in patients with heart failure. Eur J Heart Fail. 2013;15(1):94–102.
Lupon J, Gastelurrutia P, De Antonio M, et al. Quality of life monitoring in ambulatory heart failure patients: temporal changes and prognostic value. Eur J Heart Fail. 2013;15(1):103–9.
Erceg P, et al. Prognostic value of health-related quality of life in elderly patients hospitalized with heart failure. Clin Interv Aging. 2019;14:935.
Rodríguez-Artalejo F, et al. Health-related quality of life as a predictor of hospital readmission and death among patients with heart failure. Arch Intern Med. 2005;165(11):1274–9.
Roller JM, Courtemanche RJ, Courtemanche DJ. A Review of Validated Quality-of-Life Patient-Reported Outcome Measures in Pediatric Plastic Surgery. Plastic Reconstructive Surg. 2018;142(5):694e–707e.
Fayers P. Outcomes Assessment in Cancer: Measures, Methods, and Applications. N Engl J Med. 2006;15(2):313–4.
Valderas JM, Alonso J. Patient reported outcome measures: a model-based classification system for research and clinical practice. Qual Life Res. 2008;17(9):1125.
McHorney CA. Health status assessment methods for adults: past accomplishments and future challenges. Annu Rev Public Health. 1999;20(1):309–35.
Emery M-P, Perrier L-L, Acquadro C. Patient-reported outcome and quality of life instruments database (PROQOLID): frequently asked questions. Health Qual Life Outcomes. 2005;3(1):12.
Bradley C. Feedback on the FDA's February 2006 draft guidance on patient reported outcome (PRO) measures from a developer of PRO measures. Health Qual Life Outcomes. 2006;4(1):78.
Zhang Y, et al. A clinicopathologic prediction model for postoperative recurrence in stage Ia non–small cell lung cancer. J Thorac Cardiovasc Surg. 2014;148(4):1193–9.
Wang Y, et al. Prognostic Nomogram for intrahepatic Cholangiocarcinoma after partial hepatectomy. J Clin Oncol. 2013;31(9):1188–95.
DiDomenico RJ, et al. Use of a simplified nomogram to individualize digoxin dosing versus standard dosing practices in patients with heart failure. Pharmacotherapy. 2014;34(11):1121–31.
Barlera S, et al. Predictors of mortality in 6975 patients with chronic heart failure in the Gruppo Italiano per lo studio della Streptochinasi nell'Infarto Miocardico-heart failure trial: proposal for a nomogram. Circ Heart Fail. 2013;6(1):31–9.
Fan Z, et al. Nomogram model to predict Cardiorenal syndrome type 1 in patients with acute heart failure. Kidney Blood Press Res. 2018;43(6):1832–41.
Heart Failure group, C.B., Chinese Medical Association. Chinese Guidelines for the Diagnosis and Treatment of Heart Failure 2018. Chinese J Cardiology. 2018;46(10):760.
Tian J, et al. CHF-PROM: validation of a patient-reported outcome measure for patients with chronic heart failure. Health Qual Life Outcomes. 2018;16(1):51.
Iasonos A, et al. How to build and interpret a Nomogram for Cancer prognosis. J Clin Oncol. 2008;26(8):1364–70.
Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87.
Royston P, et al. Prognosis and prognostic research: developing a prognostic model. BMJ. 2009;338(7707):1373–7.
Hoshino N, et al. Nomogram for predicting anastomotic leakage after low anterior resection for rectal cancer. Int J Color Dis. 2018;33(4):411–8.
Zhang J, et al. A nomogram for determining the disease-specific survival in Ewing sarcoma: a population study. BMC Cancer. 2019;19(1):667.
Rocco B, et al. A novel nomogram for predicting ECE of prostate cancer. BJUI. 2018;122(6):916–8.
White RS, et al. Readmission rates and diagnoses following total hip replacement in relation to insurance payer status, race and ethnicity, and income status. J Racial Ethn Health Disparities. 2018;5(6):1202–14.
Dewan P, et al. Income inequality and outcomes in heart failure: a global between-country analysis. JACC: Heart Failure. 2019;7(4):336–46.
Ahmed A, Aronow WS, Fleg JL. Higher New York heart association classes and increased mortality and hospitalization in patients with heart failure and preserved left ventricular function. Am Heart J. 2006;151(2):444–50.
Strom JB, et al. Short-term rehospitalization across the spectrum of age and insurance types in the United States. PLoS One. 2017;12(7):e0180767.
Sokoreli I, et al. Added value of frailty and social support in predicting risk of 30-day unplanned re-admission or death for patients with heart failure: an analysis from OPERA-HF. Int J Cardiol. 2019;278:167–72.
Staniute M, et al. Type D personality, mental distress, social support and health-related quality of life in coronary artery disease patients with heart failure: a longitudinal observational study. Health Qual Life Outcomes. 2015;13(1):1.
Suzuki T, et al. Impact of clustered depression and anxiety on mortality and rehospitalization in patients with heart failure. J Cardiol. 2014;64(6):456–62.
Selan S, Hellström A, Fagerström C. Impact of nutritional status and sleep quality on hospital utilisation in the oldest old with heart failure. J Nutr Health Aging. 2016;20(2):170–7.
Lee KS, et al. Prognostic importance of sleep quality in patients with heart failure. Am J Crit Care. 2016;25(6):516–25.
Taniguchi Y, Sakakura K, Yuri K, et al. Appetite Predicts Clinical Outcomes in High Risk Patients Undergoing Trans-Femoral TAVI. Int Heart J. 2019;60(6):1350–7.
Kitamura M, et al. Relationship between activities of daily living and readmission within 90 days in hospitalized elderly patients with heart failure. Biomed Res Int. 2017;2017:7420738.
Ma C, Zhou W. Predictors of rehospitalization for community‐dwelling older adults with chronic heart failure: A structural equation model. J Adv Nurs. 2020;76(6):1334–44.
Lee HB, et al. Personality disorders and traits as predictors of incident cardiovascular disease: findings from the 23-year follow-up of the Baltimore ECA study. Psychosomatics. 2010;51(4):289–96.
We would like to acknowledge data collectors and the hospital administrative staff for their permission and support to conduct this study. We would like to thank Editage (www.editage.cn) for English language editing.
This study was funded by the National Nature Science Foundation of China (Grant No. 81872714), the Shanxi Provincial Key Laboratory of Major Diseases Risk Assessment (Grant No. 201805D111006) and the General Program for Young Scholar of Shanxi Province (Grant No. 201801D221423).
Ethics approval and consent to participate
The research program received medical and ethical approval from Shanxi Medical University. All patients and their families were informed and agreed to the study.
Consent for publication
The author has declared that no competing interests exist.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Han, Q., Ren, J., Tian, J. et al. A nomogram based on a patient-reported outcomes measure: predicting the risk of readmission for patients with chronic heart failure. Health Qual Life Outcomes 18, 290 (2020). https://doi.org/10.1186/s12955-020-01534-6