- Open Access
The Center for Epidemiologic Studies Depression Scale is an adequate screening instrument for depression and anxiety disorder in adults with congential heart disease
Health and Quality of Life Outcomesvolume 15, Article number: 176 (2017)
The Center for Epidemiological Studies Depression Scale (CES-D) is an instrument that is commonly used to screen for depression in patients with chronic disease, but the characteristics of the CES-D in adults with congenital heart disease (CHD) have not yet been studied. The aim of this study was to investigate the criterion validities and the predictive powers of the CES-D for depression and anxiety disorders in adults with CHD.
Two hundred patients were screened with the CES-D and secondarily interviewed with a diagnostic instrument, i.e., the Mini International Neuropsychiatric Instrument. The sensitivity and specificity values of the CES-D were calculated by cross-tabulation at different cutoff scores. Receiver operating characteristic (ROC) curves were used to assess the optimal cutoff point for each disorder and to assess the predictive power of the instrument.
The CES-D exhibited satisfactory criterion validities for depression and for all combinations of depression and/or anxiety. With a desired sensitivity of at least 80%, the optimal cutoff scores were 18. The predictive power of the CES-D in the patients was best for major depression and dysthymia (area under the ROC curve: 0.92) followed by the score for any combination of depression and/or anxiety (0.88).
The use of CES-D to simultaneously screen for both depression and anxiety disorders may be useful in adults with CHD. Trial registration: CESDEP 212. Registered 2 March 2014 (retrospectively registered).
As a result of recent advancements in cardiac surgery, more than 85% of patients with congenital heart disease (CHD) reach adulthood, and 200,000 such patients are estimated to have reached adulthood in Korea . However, among this population, more than 55% have presented with medical problems such as arrhythmia, bacterial endocarditis, congestive heart failure, and pulmonary vascular diseases, and have needed subsequent operations even after receiving childhood surgery . Many adults with CHD also suffer from various psychological difficulties caused by the heart disease itself, e.g., fear of death, treatment decision-making, anxiety associated with the preparation for cardiac surgeries, maladjustment to implanted cardiac devices, and the transition from childhood to adulthood [3, 4]. With regards to their emotional functioning, there is an inconsistency in the literature. Some studies have reported that when compared with control groups [5, 6], adolescents and adults with CHD have poorer emotional functioning. However, other studies have reported that adolescents and adults with CHD fare well . The inconsistency in findings could be attributed to the use of different measurement tools. Therefore, evaluation using the same tool with CHD-specific should be performed, and development of an intervention to increase emotional health should be based on these results [7, 8].
In Korea, awareness of the psychological issues associated with CHD in adults has increased, and research in this field has recently been active [3, 9,10,11]. However, research on the development of psychological issues and related factors in adults with CHD in Korea, which differs from other countries in terms of social, economic, and cultural characteristics, is lacking. To be more specific, Westerners are reared to grow up as an independent and autonomous human being. However, Koreans are raised differently, affected by Confucianism and a collectivist culture centered around the family. The Korean culture emphasizes the absence of a self-concept, a social order that always prioritizes the group over the individual, the trait of saving one’s face as seen in shame culture, control based on excessive power and authority in patriarchal family systems, rigid gender roles, and a characteristic parent-child relationship . Therefore, the Western psychosocial evaluation criteria may not apply to Koreans, who have a different social and cultural environment.
Moreover, in Korea, where the degree of understanding and empathy for psychiatric illnesses is low and psychiatrists are not as accessible, it is critical that cardiologists who have already formed a rapport with patients to identify those with psychological issues. Such identification would improve the psychological care provided to these patients.
The screening tools used to investigate the psychological problems (i.e., depression and anxiety) that create great discomfort and difficulties for adults with CHD [7, 13] are, in fact, not used to definitively diagnose depression and anxiety. However, they can still improve diagnostic efficiency by helping to identify patients likely to have such psychological problems. Moreover, it is imperative that the psychological screening tools are well matched to those that are used officially by psychiatrists.
The Center for Epidemiological Studies Depression Scale (CES-D) is used clinically as a screening tool for various patient groups  because it has excellent sensitivity and reliability as a tool for diagnosing depression [15,16,17]. However, there are currently no reported cases of this scale being applied to adults with CHD. The CES-D cutoff scores differ depending on patient group and are higher for groups with more psychiatric risk factors than control groups . Thus, if the CES-D is to be applied to adults with CHD, it is necessary to investigate an optimal cutoff score.
Anxiety and depression are highly interrelated and share overlapping symptoms. Therefore, the CES-D can be used to evaluate anxiety as well as depression [18, 19]. Simultaneous screening for both depression and anxiety with a single tool represents advantages for both the examiners and patients because it is more convenient and time-efficient. However, the evaluation of anxiety in adults with CHD with the CES-D has not yet been reported. Further, the cutoff score used to diagnose anxiety may differ from that used for depression.
The purpose of this study was to evaluate the criterion validity of the CES-D as a screening test for the diagnosis of anxiety and depression in adults with CHD and to determine appropriate CES-D cutoff scores for depression and anxiety.
Materials and procedures
This was a prospective study of patients with CHD from the outpatients of the Grown-Up Congenital Heart (GUCH) Clinic, which is single tertiary center at the Samsung Medical Center. The criteria for selection were as follows: 1) age greater than 18 years, 2) absence of complications or syndromes associated with severe intellectual handicap(s) (e.g., trisomy 21), 3) the ability to understand and complete the questionnaires, and 4) Willingness to participate in the survey. The study period ranged from November 2013 to May 2014. In total, 212 patients visited the Samsung Medical GUCH clinic during this period. We excluded 12 patients; 6 had been diagnosed with Marfan syndrome and the remaining responded inadequately to the survey questions. Thus, the final analysis included 200 patients. The sample size satisfied the requirements for the test of the validity of the questionnaire [20, 21].
The survey was conducted after approval of the study protocol was obtained from the Samsung Medical Center Institutional Review Board. If the patients agreed to participate in the study, they were asked to sign a consent form and complete a questionnaire. One cardiovascular outpatient nurse collected the survey data (i.e., the CES-D, a questionnaire about functional status and demographic data) during one-to-one interviews during the patients’ visits to the outpatient clinic for check-ups or tests.
One-hundred twelve patients who scored higher than 9 for depression on the CES-D were recruited for a face-to-face interview, the Mini International Neuropsychiatric Instrument (MINI) by a single psychologist on the same day. To investigate the patients’ clinical characteristics, a researcher collected and reviewed the electronic medical records.
The symptoms of depression and anxiety were assessed with the CES-D, which was developed by Radolff  and translated into Korean by Nam and Lee . The CES-D was designed specifically to screen for depressive symptoms in the general population and in patients with chronic disease , but it has also been used to screen for anxiety symptoms [19, 24]. The CES-D is a self-reported tool consisting of 20 items including four statements that are rated on a scale of 0 to 3. The patients select a rating to describe how they felt during the previous week. The Korean version of the CES-D has adequate test/retest reliability (0.68 over several weeks), internal consistency (0.89–0.93) , and concurrent validity and requires approximately 4–5 min to complete. Scores range from 0 (lowest) to 60 (highest), and patients are categorized into one of the following four groups: a) not depressed (0–9 points), b) mildly depressed (10–15 points), c) moderately depressed (16–24 points), or d) severely depressed (more than 25 points). The standard cutoff point of 16 or more was used to classify patients with depressive symptoms . The internal consistency of the CES-D score in this study was 0.92.
Mini international neuropsychiatric interview
To test the criterion validity of the CES-D, the Mini International Neuropsychiatric Interview (MINI) was used. The MINI, which was developed by Sheehan et al. in 1997, is a short, structured diagnostic interview for major axis I diseases according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition and the International Statistical Classification of Disease and Related Health Problems, 10th revision . The MINI can diagnose major depression, dysthymia, panic disorder, social phobia, agoraphobia, general anxiety disorder (GAD), and other coexistent disorders and can be administered within 15 min. Structured psychiatric interviews, which take less time but produce a high level of accuracy, are required in epidemiological studies and multicenter clinical trials and to track outcomes in non-research clinical settings. In this regard, the MINI satisfies the conditions required for such structured psychiatric interviews. According to standardized data collected from 270 patients at 10 university hospitals and psychiatry clinics, the current internal consistencies of the questions for each psychiatric diagnosis, with the exception of drug abuse, are 0.60–0.84, and the test-retest reliability is excellent (above 0.75) .
Sociodemographic and clinical characteristics
The sociodemographic characteristics of the participants included age, gender, education level, occupation, marital status, and monthly average household income. Household income was classified into the following three categories: low class (under 15.00 million won), middle class (15.00 to 41.59 million won), and high class (over 41.60 million won) [28,29,30]. The clinical characteristics of CHD diagnosis and frequency of cardiac surgeries were investigated, and functional classes were investigated using the New York Heart Association (NYHA) functional classification  and percutaneous oxygen saturation (SaO2) level.
SPSS Statistics version 22.0 for Windows (SPSS Inc., Chicago, IL) was used for the data analysis. The general clinical characteristics were analyzed according to the real numbers, percentages, means, and standard deviations. The sensitivity, specificity, and positive predictive values for depression and anxiety of the CES-D were calculated by cross-tabulation and receiver operating characteristics (ROC) for multiple CES-D cutoff scores (i.e., 16, 18, 20, and 22). The specificities of the CES-D for the selected depressive and anxiety disorders, positive predictive value (PPV), and negative predictive value (NPV) were also analyzed by cross-tabulation and ROC. The associations between the CES-D scores and the diagnostic measurements were calculated using ROC curves. Finally, to investigate the possibility of a difference in the patterns of the two scores, we compared the mean scores for the 20 items of the CES-D for depression and for anxiety disorders.
Demographic and clinical characteristics of the participants
The demographic and clinical characteristics of the participants in this study are summarized in Table 1. The average age of the participants was 38.7 years, and 45% were female. Half (50%) lived alone, 53% had completed primary or secondary education, and 47% had completed higher or scientific education. Almost 70% (69.2%) had a job, and 49% were in the middle-income category (Table 1). In terms of CHD diagnoses, 36.5% of the participants had atrial septal defects, 13.5% of the subjects had ventricular septal defects, and 31.5% had a diagnosis of cyanotic heart disease. Over 45% of the participants were in each of the NYHA functional classes I and II, 38% had undergone one cardiac operation, 24.3% had undergone two cardiac operations, and 22.1% had undergone three or more cardiac operations. More than three-quarters (77.2%) of the participants had an oxygen saturation (SaO2) level greater than 95, and 11.3% had an SaO2 level less than 90% (Table 1).
The subjects’ depression scores ranged from 0 to 53 with a mean of 18.4 (SD 5.9). A total of 14% of the participants were suffering from severe depression, and 28.5% of the participants had depression and/or anxiety (Table 2).
Sensitivity, specificity, and positive predictive value of the CES-D for depressive and anxiety disorder
The sensitivities, specificities, and PPVs results from the MINI for different cutoff points are presented in Table 3. When the aims were to detect major depressive disorder (MDD) (sensitivity, 84.6%; specificity, 65.3%) and dysthymia (sensitivity, 73.2%; specificity, 60.3%) together, the best cutoff point was 18. When screening for anxiety disorders, the sensitivities for the detections of a GAD and for all anxiety disorders together were optimal at the cutoff point of 18 (GAD, 87.3%; all anxiety disorders, 84.7%). Additionally, the specificity levels for GAD and all anxiety disorders together at this cutoff point were 70.5 and 72.4%, respectively. When the CES-D was used to screen for any depressive and/or anxiety disorder using the cutoff point of 18, the PPV doubled from 28.5% at baseline to 53.6%. The NPV remained above 90% for all of the various disorders at different cutoff points with the exception of anxiety disorders at the cutoff point of 20 (NPV = 89%; NPV not shown).
The predictive power of the CES-D in these participants was best for MDD and for depressive disorders, i.e., the sum of the MDD and dysthymia scores. The AUCs as identified with ROC analysis were 0.92 for MDD (standard error (SE) = 0.029; 95% confidence interval (CI) 0.83–0.95; p < 0.001) and depressive disorder (SE = 0.031; 95% CI 0.82–0.92; p < 0.001). The second best predictive power was observed for depressive and/or anxiety disorder for which the AUC was 0.88 (SE = 0.031; 95% CI 0.83–0.91; p < 0.001). The predictive power for GAD was also very good at 0.85 (SE = 0.028; 95% CI 0.79–0.93; p < 0.001).
Differences between depression and anxiety scores
The patterns of scores for each of the 20 CES-D items individually did not differ between depressive and anxiety disorders (Table 4).
This study was conducted to examine the CES-D and its validity as a screening tool for psychological symptoms such as depression and anxiety in adults with CHD, as well as to determine its diagnostic cutoff points.
Approximately 28–35% of adults with CHD experience depression [12, 32]. The selection of patients who are at a high risk of psychological problems in the early stages and their referral to psychologists or psychiatrists reduces the incidence of more serious psychological disorders. This study verified that the CES-D is appropriate as a screening tool for the early stages of anxiety and depression in adults with CHD.
The CES-D was originally developed to evaluate depressive symptoms in general population epidemiological studies  and is currently widely used in various clinical settings as a case-finding measure for depression and as stand-alone diagnostic instrument . It is commonly used to screen and detect major depression. No significant differences have been found between different clinical settings .
In clinical settings, the Patient Health Questionnaire (PHQ-9) is also used to detect major depression. It is often used as an alternative to the CES-D and has been used commonly in many clinical settings. The PHQ-9 evaluates depression symptoms and severity . The PHQ-9 has excellent diagnostic accuracy, a sensitivity of 0.80 (95% CI: 0.71–0.87) and a specificity of 0.92 (95% CI: 0.88–0.95) . When the PHQ-9 and the CES-D were compared with respect to their positive likelihood ratios, 22–35% of non-patients were diagnosed with a mental disorder with the CES-D  compared to 53% with the PHQ-9 . Therefore, the CES-D is more suitable than the PHQ-9 for use as a tool for screening for depression.
It is best to attain both high sensitivity and specificity when determining a cutoff point. However, in the case that this may be impossible, it is more important to choose high sensitivity over specificity if the instrument is used for screening over diagnosis . With regard to sensitivity, the CES-D exhibited similarly excellent results for depressive disorders and anxiety disorders. These results are consistent with those obtained following the application of the CES-D to elderly patients living in residential homes . When investigating depression and anxiety together by setting the cutoff point at the score of 18, the predictive power of the tool was also satisfactory. In other words, the criterion validities were satisfactory and occurred in the following descending order: when only depression disorder was present, when both depression and anxiety disorders were present, and when only anxiety disorder was present.
In this study, the cutoff points were the same for depression and anxiety. The cutoff points for MDD and/or dysthymia were 18, whereas this value increased to 20 when only MDD was present. This value was higher than the cutoff point of 16 that was set at the time of the development of the CES-D  and is similar  to that recommended after a systematic meta-analytic review that was conducted to identify an appropriate cutoff point for the CES-D as a screening tool . Several questions in the CES-D address physical symptoms. Thus, the score can easily be higher for patients with poor functional class due to heart and lung function, as is often for adults with CHD . It is necessary to raise the cutoff point for the optimal screening of patients with heart or lung disease, including CHD. Overall, screening for depression and anxiety with the CES-D tool can be useful in the clinical setting as it is both convenient and efficient.
The CES-D can also be used practically during regular screenings because it has maximal sensitivity to identify depression and minimize missed cases . Minimizing the false-positive rate is vital, especially in Korea where psychiatric resources are limited and psychiatry is generally misunderstood. This can be achieved by increasing the cutoff point.
On the contrary, when the cutoff point is increased, a greater number of cases may be missed. When the CES-D is used for screening, the probability of correctly identifying depression and anxiety increases two-fold at the individual level. The probability of correctly identifying at the cutoff scores for all psychiatric disorders is greater than 90%, indicating satisfactory levels of identification [14, 24]. However, the CES-D is not recommended for use as an isolated diagnostic tool given the results that show low positive likelihood ratio. Therefore, further studies are needed to verify diagnoses made with the CES-D [14, 24].
Additionally, because the participants were not randomly selected, there could have been a referral bias. Therefore, participants who were concerned about their mental health or already had depression may have participated in our study. Such participants may have overstated their symptoms, and diagnosis of depression may therefore have been overestimated. Therefore, a well-structured randomized study will be needed in the future.
In conclusion, the CES-D was used as an initial screening tool to identify participants who need further in-depth assessment of their depressive symptoms. However, it is not recommended as a stand-alone diagnostic tool. Therefore, additional diagnostic evaluations are still required for all participants with scores greater than 18. Specifically, follow-up diagnostic evaluations are needed to confirm the diagnoses of disorders and also to distinguish depressive disorders and anxiety disorders.
The CES-D is highly accurate as a screening tool for adults with CHD in clinical settings; however, the interpretation of the results requires attention because there is the possibility that scores above 18 indicate depressive disorders as well as anxiety disorders. The CES-D can be used to screen for both depression and anxiety disorders.
Adults with congenital heart disease
Area under curve
Center for Epidemiological Studies Depression Scale
Congenital heart disease
General anxiety disorder
Grown-Up Congenital Heart
Major depressive disorder
Mini International Neuropsychiatric Instrument
Negative predictive value
New York Heart Association
Positive predictive value
Receiver operating characteristic
- SaO2 :
Lee HJ. Congenital heart disease after childhood; experience of grown up congenital heart clinic (GUCH clinic). Korean Circ J. 2001;31:537–41.
Bhatt AB, Foster E, Kuehl K, Alpert J, Brabeck S, Crumb S, Dabidson WR, et al. Congenital heart disease in the older adult. A scientific statement from the american heart association. Circulation. 2015;131:00. doi:10.1161/CIR.0000000000000204.
Kim GB. Psychosocial adjustment and quality of life of adolescents and adults with congenital heart disease. Korean J Pediatr. 2014;57:257–63.
Paulike LB. Depression in adults with congenital heart disease –public health challenge in a rapidly expanding new patient population. Wolrd J Cardiol. 2013;26:189–95.
Wang Q, Hay M, Clarke D, Menahem S. Association between knowledge of disease, depression and anxiety, social support sense of coherence and optimism with health-related quality of life of an ambulatory sample of adolescents with heart disease. Caridol Young. 2014;24:126–33.
Chen CA, Liao SC, Wang JK, et al. Quality of life in adults with congenital heart disease: biopsychosocial determinants and sex-related differences. Heart. 2011;97:38–43.
Jackson JL, Misiti B, Bridge JA, Daniels CJ, Bannatta K. Emotional functioning of adolescents and adults with congenital heart disease: a meta-analysis. Congenital Heart Dis. 2015;10:2–12.
Foster E, Graham TP Jr, Driscoll DJ, Reid GJ, Reiss JG, Russell IA, et al. Task force 2: special health care needs of adults with congenital heart disease. J Am Coll Cardiol. 2001;37:1176–83.
Moon JR, Huh J, Kang IS, Park SW, Jun TG, Lee HJ. Factors influencing depression in adolescents with congenital heart disease. Heart Lung. 2009;38:419–26.
Lee S, Kim S. The life of adolescent patients with complex congenital heart disease. J Korean Acad Nurs. 2010;40:411–22.
Lee S, Kim S, Choi Y. Coping and resilience of adolescents with congenital heart disease. J Cardiovasc Nurs. 2014;29:340–6.
Jung IS. Pastoral counseling study of healing methods for parent-child dependency in Korean culture and relationships [dissertation]. Kyung Ki: The graduate school, Asia United Theological University; 2008.
Bromberg JI, Beasley PJ, D'Angelo EJ, Landzberg M, DeMaso DR. Depression and anxiety in adults with congenital heart disease: a pilot study. Heart Lung. 2003;32:105–10.
Vilagut G, Forero CG, Barbaglia G, Alonso J. Screening for depression in the general population with the Center for Epidemiologic Studies Depression (CES-D): a systematic review with meta-analysis. Plos One. 2016;11:e0155431.
Blank K, Gruman C, Robison JT. Case-finding for depression in elderly people: balancing ease of administration with validity in varied treatment settings. J Gerontol A Biol Sci Med Sci. 2004;59:378–84.
Moullec G, Maiano C, Morin AJ, Monthuy-Blanc J, Rosello L, Ninot G. A very short visual analog form of the Center for Epidemiologic Studies Depression Scale (CES-D) for the idiographic measurement of depression. J Affect Disord. 2011;128:220–34.
Bradley KL, Bagnell AL, Brannen CL. Factorial validity of the Center for Epidemiological Studies Depression 10 in adolescents. Issues Ment Health Nurs. 2010;31:408–12.20.
Nishiyama T, Ozaki N, Iwata N. Practice-based depression screening for psychiatry outpatients: feasibility comparison of two-types of Center for Epidemiologic Studies Depression Scales. Psychiatry Clin Neurosci. 2009;63:632–8.
McQuaid JR, Stein MB, McCahill M, Laffaye C, Ramel W. Use of brief psychiatric screening measures in a primary care sample. Depress Anxiety. 2000;12:21–9.
Willan AR. Sample size determination for cost-effectiveness trials. PharmacoEconomics. 2011;29:933–49.
Tabachnick BG, Fidell LS. Using multivariate statistics. 3rd ed. New York: Harper Collins; 1996.
Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. App Psychol Meas. 1977;1:385–401.
Cho MJ, Kim KH. Diagnostic validity of the CED-D (Korean version) in the assessment of DSM-III-R major depression. J Korean Neuropsychiatr Assoc. 1993;32:381–99.
Dozeman E, van Schaik DJ, van Marwijk HW, Stek ML, van der Horst HE, Beekman AT. The center for epidemiological studies depression scale (CES-D) is an adequate screening instrument for depressive and anxiety disorders in a very old population living in residential homes. Int J Geriatr Psychiatry. 2011;26:239–46.
Weissman MM, Sholomskas D, Pottenger M, Prusoff BA, Locke BZ. Assessing depressive symptoms in five psychiatric populations: a validation study. Am J Epidemiol. 1977;106:203–14.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59 Suppl 20:22–33;quiz 4–57.
Yu SW, Kim YS, Noh JS, Oh KS, Kim CH, Namkoong K, et al. Validity of Korean version of the mini-international neuropsychiatric interview. Anxiety Mood. 2006;2:50–5.
Statistics Korea: Fourth Quarter 2013 and Annual Household Trends. [http://kostat.go.kr/portal/eng/pressReleases/6/2/index.board?bmode=read&bSeq=&aSeq=313133&pageNo=2&rowNum=10&navCount=10&currPg=&sTarget=title&sTxt=]. Accessed 10 Sept 2013.
Ministry for Health and Welfare: Minimum Cost of Living in 2013. [http://www.law.go.kr/행정규칙/2013년최저생계비]. Accessed 10 Sept 2013.
Goldberg DP, Gater R, Sartorius N, Ustun TB, Piccinelli M, Gureje O, et al. The validity of two versions of the GHQ in the WHO study of mental illness in general health care. Psychol Med. 1997;27(1):191–7.
The Criteria Committee of the New York Heart Association. Nomenclature and criteria for diagnosis of disease the heart and great vessel. 9th ed. Boston: Little Brown & Co; 1999. p. 113–26.
Pauliks LB. Depression in adults with congenital heart disease-public health challenge in a rapidly expanding new patient population. World J Cardiol. 2013;5:186–95.
Dptm. of Clinical Epidemiology and Biostatistics MU. Interpretation of diagnostic data: 5. How to do it with simple maths. CMAJ. 1983;129(9):947–54.
We would like to acknowledge and thank our study participants.
This study was supported by Samsung Biomedical Research Institute Grant, #SBRI C-B1–318-1. (South Korea).
Availability of data and materials
The dataset supporting the conclusions of this article is available by contacting the corresponding author.
Ethics approval and consent to participate
All of the participants gave their written informed consent, and the protocol of the study was approved by the Institutional Review Board at Samsung Medical Center.
Consent for publication
Not applicable. No individual patients’ data reported.
All authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.