- Open Access
International evaluation of the psychometrics of health-related quality of life questionnaires for use among long-term survivors of testicular and prostate cancer
Health and Quality of Life Outcomes volume 15, Article number: 97 (2017)
Understanding of the physical, functional and psychosocial health problems and needs of cancer survivors requires cross-national and cross-cultural standardization of health-related quality of life (HRQoL) questionnaires that capture the full range of issues relevant to cancer survivors. To our knowledge, only one study has investigated in a comprehensive way whether a questionnaire used to evaluate HRQoL in cancer patients under active treatment is also reliable and valid when used among (long-term) cancer survivors. In this study we evaluated, in an international context, the psychometrics of HRQoL questionnaires for use among long-term, disease-free, survivors of testicular and prostate cancer.
In this cross-sectional study, we recruited long-term survivors of testicular and prostate cancer from Northern and Southern Europe and from the United Kingdom who had participated in two phase III EORTC clinical trials. Participants completed the SF-36 Health Survey, the EORTC QLQ-C30 questionnaire, the QLQ-PR25 (for prostate cancer) or the QLQ-TC26 (for testicular cancer) questionnaires, and the Impact of Cancer questionnaire. Testicular cancer survivors also completed subscales from the Nordic Questionnaire for Monitoring the Age Diverse Workforce.
Two hundred forty-two men (66% response rate) were recruited into the study. The average time since treatment was more than 10 years. Overall, there were few missing questionnaire data, although scales related to sexuality, satisfaction with care and relationship concerns of men without partners were missing in more than 10% of cases. Debriefing showed that in general the questionnaires were accepted well. Many of the survivors scored at the upper extremes of the questionnaires, resulting in floor and ceiling effects in 64% of the scales. All of the questionnaires investigated met the threshold of 0.70 for group level reliability, with the exception of the QLQ-TC26 (mean reliability .64) and the QLQ-PR25 (mean reliability .69). The questionnaires were able to discriminate clearly between patients with and without comorbid conditions.
The currently available HRQoL questionnaires exhibit acceptable psychometric properties and were well received by patients, but additional efforts are needed to ensure that the full range of survivor-specific issues is assessed.
More than half of European patients diagnosed with cancer enjoy survival of five years or longer after primary diagnosis . However, a long period of survival is not synonymous with a life free of physical and psychosocial health problems related to the cancer and/or its treatment. Studies investigating health-related quality of life (HRQoL) in long-term cancer survivors have shown that cancer-related health concerns can persist for years after initial treatment [2,3,4].
Increasing attention is being paid to the HRQoL of long-term cancer survivors (i.e., 10 years or longer after diagnosis) [2, 5,6,7]. Understanding of the physical, functional and psychosocial health problems and needs of cancer survivors requires cross-national and cross-cultural standardization of HRQoL questionnaires that capture the full range of issues relevant to cancer survivors . To our knowledge, only one previous study has investigated in a comprehensive way whether questionnaires used to evaluate HRQoL in cancer patients under active treatment are also reliable and valid when used among (long-term) cancer survivors. This study addressed the psychometrics of the EORTC Prostate cancer module (QLQ-PR25) in a large population-based sample from Ireland .
Although our ultimate goal is to identify and/or develop HRQoL questionnaires relevant to a wide range of cancer survivor populations, for this study we focused on survivors of two important genitourinary cancers – testicular cancer (TCa) and prostate cancer (PCa) - treated in the context of two EORTC phase III clinical trials [10, 11]. TCa is a disease that affects young adults. At the turn of the current century, the cure rate for TCa was slightly greater than 90% , and in the ensuing years it has increased to 98% . Prostate cancer (PCa) is the most prevalent cancer among men in Western, industrialized countries . For men diagnosed with local or loco-regional PCa, the relative 10 year survival rate is 91% .
From previous studies we know that, although TCa survivors report a level of general HRQoL comparable to the general population, they are confronted with a number of specific health issues. A minority of TCa survivors experience long-term side-effects of their treatment, including fertility problems [16,17,18,19,20], peripheral neuropathy, ototoxicity, Raynaud phenomena, gastrointestinal symptoms, decreased pulmonary function, cardiovascular disease and secondary tumors [17, 19, 21,22,23]. Also, there is evidence of heightened levels of fatigue, anxiety, and cancer-related distress [24,25,26,27], practical problems related to obtaining insurance or bank loans [6, 12], and less satisfactory social contacts with friends and acquaintances . It should be noted, that TCa survivors also report positive consequences of having had cancer, including emotional growth, greater appreciation of life, and stronger relationships with family and friends .
As is the case for TCa survivors, PCa survivors report equal or better overall HRQoL as compared to healthy controls, and generally do well in terms of psychological well-being . However, almost 40% of PCa survivors expresses heightened fear of disease recurrence and psychological distress . Further, advanced age in (prostate) cancer survivors is associated with worse HRQoL outcomes [30,31,32], and comorbidity serves as an additional risk [33,34,35]. Moreover, treatment of PCa can have a profound impact on urinary, sexual and bowel function [36, 37]. These specific, long-term HRQoL problems are, in part, treatment dependent [31, 38,39,40,41]. Other long-term sequelae of PCa are hypertension, cerebrovascular episodes, osteoporosis, and neuropathy [3, 42].
The objectives of this study were twofold: 1) to determine the feasibility of conducting HRQoL research among long-term cancer survivors treated in EORTC phase III clinical trials; and 2) to evaluate the psychometrics of questionnaires for assessing the HRQoL of long-term cancer survivors (>10 years disease free). In a previous paper  we reported on the feasibility of and challenges associated with conducting HRQoL investigations among long-term cancer survivors. In the current paper, we focus on the second objective.
Participants were long-term survivors of two European Organisation for Research and Treatment of Cancer (EORTC) Genito-Urinary Cancers Group phase III clinical trials, with no evidence of active disease. The PCa survivors were recruited from Trial 22911  (a collaboration with the EORTC Radiation Oncology Group, inclusion 1992–2001), which investigated post-surgical (adjuvant) irradiation of the prostate surgical bed versus ‘wait and see’. We recruited survivors from the Netherlands, Belgium, France, and Italy. Since patients had entered this trial with a median age of 65 years, at the time of data collection at least 50% of patients were very elderly or had died. Therefore, we recruited four additional patients aged 65–75 years, who were not part of this trial but received the same treatment from the clinics that were included in this clinical trial.
The TCa survivors were recruited from Trial 30941/MRC TE20 , which investigated different regimen of bleomycin, etoposide, and cisplatin (BEP) (inclusion 1995–1998). We recruited survivors from the Netherlands, Italy, Norway, and the United Kingdom. Since this trial included only six Southern European survivors, 37 additional survivors, treated according to the same regimen (three cycles of BEP over 5 days), were recruited from Italy outside of the trial.
We believe that it was important to supplement our sample to compensate for the underrepresentation of certain subgroups in the available clinical trial samples (i.e., those aged 65–75 years in the case of PCa, and those from Southern Europe in the case of TCa). There is no reason to believe that the inclusion of these additional patients in the study sample would influence the psychometric properties of the questionnaires under investigation. They were all long-term survivors as well.
Eligible survivors received an invitation letter signed by their treating physician, an informed consent form and the questionnaire battery. A reminder was sent after three weeks (see  for detailed information on patient recruitment and procedures). The study was approved by the institutional review boards of the participating hospitals, and informed consent was obtained from all individual participants included in the study. Data were collected and processed at The Netherlands Cancer Institute.
We assessed HRQoL at 3 levels: (1) generic (the SF-36 Health Survey) ; (2) cancer-specific (the EORTC core questionnaire (QLQ-C30)  and the EORTC prostate or testicular cancer modules (QLQ-PR25 ; QLQ-TC26) ); and (3) cancer survivor-specific (the Impact of Cancer questionnaire, version 2 (IOCv2) [8, 48]). Table 1 displays the subscales of these questionnaires. Additional questions were posed regarding marital status, social economic status (SES), race, comorbidity (the Charlson Index and the International Prognostic Index (IPI)), work-related problems and problems related to obtaining health and life insurance, and a home mortgage loan . For the TCa survivors only, we also used four subscales of the Nordic Questionnaire for Monitoring the Age Diverse Workforce (QPSNordic) that assess work motivation, job and life satisfaction, health and well-being, and self-efficacy . Respondents were also asked to complete a 10-item debriefing questionnaire to identify any questions that were perceived as difficult to answer, confusing or upsetting, and to report important survivorship issues that were not (sufficiently) addressed by the questionnaires. All questionnaires had already been or were, for purposes of this study, translated using standard EORTC procedures .
We performed missing data analyses at both the item and scale level. A scale score was defined as missing if more than half of the items in that scale were missing. In all other cases, we generated a person-specific scale score based on the mean of the non-missing items [52, 53]. For scales for which scores could not be calculated for 5% or more of respondents, we conducted logistic regression analysis to determine if missingness was associated significantly with country/language, age, education or marital status.
Descriptive statistics were generated for all measures, including means, standard deviations and floor and ceiling effects (20% or more of the scores at the extremes of the scale ). We used Cronbach’s coefficient α  to estimate the reliability (internal consistency) of the questionnaire scales.
We employed analysis of variance to investigate known groups validity, i.e., the extent to which the questionnaire scores are able to discriminate between relevant predefined subgroups. Grouping variables included age, education, marital status, disease site (testicular versus prostate) and comorbidity. A sample size of 120 is sufficient to detect effect size differences of .25 in a 2-group design and .30 in a 3-group design with a power of .80 and an α .05. An effect size of .25 is the equivalent of a one-quarter standard deviation difference between groups in mean scores. An effect size of 0.2 is interpreted as a small difference, 0.5 as a moderate difference, and 0.8 as a large difference . The a priori hypotheses were that those who were younger, with a higher education, with a partner, and with fewer comorbid conditions (as assessed by self-report) would generally report better HRQoL. We also hypothesized that TCa survivors would report better HRQoL than PCa survivors, due to their younger age and to the relatively transient nature of some of the treatment effects.
Sample accrual and response rate
Data collection took place between February 2010 and June 2012. Of the 366 survivors invited to participate in the study, 242 (66%) agreed to do so. The response rate in the PCa survivor group was somewhat higher than in the TCa group (69% vs 64%). See van Leeuwen et al.  for more details on patients recruitment. Table 2 reports the patient background characteristics.
Completeness of the data
With a few exceptions, the percentage of missing item values was low, ranging from 0 to 7%. Although TCa survivors had to complete more items, items were missing less frequently in the TCa sample than in the PCa sample. The most important exceptions were the missing responses on the items of the Sexual Functioning scale of the QLQ-PR25. Among the PCa survivors who had indicated that they were sexually active during the past week, missing responses varied between 37 and 43%. Other QLQ-PR25 items with relatively high percentages of missing responses were one item from the Urinary Symptom scale (posed following an item which was only applicable to patients wearing incontinence material; 14%), the Sexual Activity scale (10%), and items from the Treatment-Related Symptom scale that were related to hormonal treatment (6 to 8%). For the IOCv2, the highest percentage of missing responses was for the Relationship Concerns (both partnered and not partnered) scales (7 to 23%).
There were four scales for which we could not calculate the scale scores for more than 5% of the PCa sample: the QLQ-PR25 Sexual Functioning and Bowel Symptom scales (37 and 7%, respectively); and the IOCv2 Relationship Concern scales (both partnered and not partnered) (20 and 8%, respectively). Missing responses on the QLQ-PR25 Sexual Functioning scale were associated significantly with (older) age. Belgian participants were significantly more likely to skip items of the Bowel Symptom scale. No other significant associations were observed in the PCa sample between item missingness and sociodemographic variables.
In the TCa sample, missing responses were primarily observed for those items that were associated with contingency questions (e.g., the IOCv2 items applicable only for sub-groups of respondents, such as those with a partner). With the exception of the item “I wonder how to tell a potential spouse, partner, boyfriend or girlfriend that I have had cancer” (skipped by 18% of the respondents), missing item responses for those scales varied between 4 and 8%. Items of the QLQ-TC26 that were related to satisfaction with care (14%) and to sexuality (6 to 10%) were also skipped more frequently.
There were four scales for which scores could not be calculated for more than 5% of the TCa sample: the IOCv2 Relationship Concerns scale (not partnered; 13%); and Satisfaction with Care scale (12%), the Sexual Enjoyment (5%), and Sexual Problems (6%) of the QLQ-TC26. The Italian patients were most likely to complete the Satisfaction with Care and the Sexual Problems scale. Items from the Sexual Enjoyment and Sexual Problems scales were most frequently left unanswered by respondents without a partner. Dutch respondents also skipped more items of the Sexual problems scale.
Sixty-five survivors (of whom 27 were from the TCa sample) reported an issue with one or more of the questions or left a general comment for the doctors or researchers on the debriefing questionnaire. Twelve survivors stated that they had difficulties distinguishing cancer-related problems from those due to aging and other conditions. Seven survivors reported they found the questions regarding sexual problems too personal, too confusing and/or too difficult to answer. Two survivors found the questions regarding incontinence too confusing or too difficult. Three survivors found that some questions were more related to the treatment and diagnosis phase. Further, 18 survivors reported missing issues, of which dealing with sexual problems (6) and long-term side effects of treatment (2) were the most often mentioned. Finally, four survivors over 80 years of age considered some questions redundant for survivors of their age (e.g. questions regarding work, feeling old, and future perspective).
Scale level descriptive statistics and reliability
Scale level descriptive statistics and reliability estimates are presented in Table 1.
Ceiling effects were observed for four of the eight SF-36 scales in the PCa sample, and for five scales in the TCa sample. The role functioning scales exhibited the most serious ceiling effects. Cronbach’s coefficient α varied between .68 and .93 (see Table 1).
The EORTC QLQ-C30
All scales of the EORTC QLQ-C30 exhibited ceiling or floor effects in both the PCa and the TCa, samples, with the exception of the Global quality of life scale in the PCa sample. Cronbach’s α coefficients varied between .70 and .96, with the exception of the Emesis scale, which was very low in the PCa sample (0.04) and could not be calculated in the TCa sample due to the extremely low prevalence rates for these symptoms in these survivor samples (see Table 1).
The Relation Concerns (partnered) and the Appearance Concerns scales exhibited floor effects in both samples. In the TCa sample, the Life Interferences, Employment Concerns and Relationship Concerns (not partnered) scales exhibited significant floor effects. Cronbach’s α coefficients ranged from .52 to .94. Three of the 11 IOCv2 scales had a Cronbach’s α lower than .70 in the PCa sample (Positive self-evaluation (.64), Employment concerns (.63) and Relationship partnered concerns (.52)), and two scales had a Cronbach’s α coefficient lower than .70 in the TCa sample (Positive self-evaluation (.69) and Relationship partnered concerns (.62); see Table 1).
Three of the six scales of the QLQ-PR25 exhibited a floor effect. Cronbach’s α coefficient varied between .41 and .89, with three scales below .70 (Sexual activity (.67), Treatment related symptoms (.45) and Sexual functioning (.41)) (see Table 1).
Except for Sexual Activity, all scales of the QLQ-TC26 showed substantial floor and ceiling effects. Cronbach’s α coefficient varied between .53 and .89, with four scales scoring below .70. The Cronbach’s α for Future perspective was .56, for Communication .53, for Treatment side effects .63, and for Sexual Problems .59 (see Table 1).
Only the Self-Efficacy scale of the QPSNordic showed a substantial ceiling effect. Cronbach’s α coefficient varied between .42 and .83, with only Work motivation being below .70 (see Table 1).
Tests of known-groups validity
In the PCa sample, (older) age was most strongly associated with physical health problems, although there was also a strong association with cognitive functioning and sexual activity level. Effects sizes ranged from 0.32 to 1.05. The association between age and HRQoL was much less pronounced in the TCa than in the PCa sample. However, the younger men in the TCa sample reported significantly more mental health problems (effect size (ES) .37–.58), were more sexually active (ES .17–.66) and worried significantly more about their fertility (ES.70–1.07) than older men (see Tables 3 and 4).
In the PCa sample, higher education was significantly associated with better physical and mental health, less appearance concerns and life interferences (effect sizes ranging from .25 to greater than 1). There was only a weak association between education level and HRQoL observed in the TCa sample (see Tables 3 and 4).
Marital status was not significantly associated with HRQoL in the PCa sample. In contrast, in the TCa sample, single men reported consistently and significantly worse HRQoL than partnered men across almost all scales of all questionnaires, with effect sizes ranging from .48 to .93 (see Tables 3 and 4).
In both the PCa and the TCa samples, men with two or more comorbid conditions (or one or more in the TCa sample) reported significantly and consistently poorer HRQoL, as measured by all questionnaires, than those without comorbid conditions. Effect sizes (ES) ranged from .36 to .86 (see Tables 3 and 4).
PCa versus TCa sample
As hypothesized, the TCa sample reported significantly better physical health, social and cognitive functioning, and fewer symptoms than the PCa sample, with effect sizes ranging from .29 to .82. No significant group differences were observed for fatigue, social functioning, mental health, or for most of the symptom scales of the QLQ-C30 (see Table 5).
In this study we have assessed the psychometric performance of a number of widely used HRQoL questionnaires for specific use in cancer survivorship research. The response rate was quite reasonable (66%), given that these were long-term survivors (on average, more than 10 years post-diagnosis and treatment). Our response rate is comparable to that of many mail surveys in health research [57, 58]. The questionnaires were well received by these survivors.
Psychometric performance of the questionnaires
Missing items were particularly an issue for the scales assessing employment and relationship concerns in the IOCv2, and the scales of the cancer-specific modules assessing sexuality, bowel functioning, complaints related to hormonal treatment (for PCa) and treatment satisfaction. This problem with missing item responses was often related to the sensitive nature of the questions or to branching and skip patterns in the questionnaire (i.e., use of contingency questions). The high level of missingness in the scales assessing sexual problems and functioning are particularly noteworthy, and this could lead to an underestimation of the prevalence of these problems in these survivor populations. Missing responses to sensitive questions (e.g. sexuality) are common and quite difficult to resolve, although there are studies suggesting that response rates could be improved by using online computer administration . Further, our findings suggest that branching questions should be used judiciously to minimize respondent confusion.
The questionnaires assessing symptoms and physical functioning all showed floor and ceiling effects, particularly in the younger TCa survivors. This can be expected in groups of survivors where the self-reported HRQoL is similar to that of the healthy, general population. Moreover, short-term, treatment-induced symptoms like nausea and vomiting or acute hair loss often are no longer relevant in these long-term survivors. These floor and ceiling effects are due, at least in part, to the fact that the SF-36 and the EORTC QLQ-C30 and its condition-specific modules were not developed for or intended to discriminate between individuals who are at the higher end of the health spectrum (e.g., those with moderate to excellent health). A number of studies have previously reported that the SF-36 and the EORTC QLQ-C30 exhibit significant ceiling and floor effects when used in general population surveys [60,61,62,63].
In general, the internal consistency reliability of the questionnaires was acceptable for group level comparisons. Problems were observed with several scales of the PR25 and of the TC26, two scales of the IOCv2, and one scale of the QPSNordic. In the case of the treatment-related symptom scales, this often reflected the low prevalence of certain symptoms among long-term survivors, as has also been reported by O’Leary in a population-based sample of prostate cancer survivors . However, suboptimal reliability has also been reported for some scales even during the period of active treatment . The problematic internal consistency of the positive self-evaluation scale of the IOCv2 was also observed in an Italian validation study  and in a French validation study the health awareness scale appeared problematic . However, in a Dutch validation sample none of these problems were observed .
The HRQoL questionnaires were able to discriminate well between patients with and without comorbid conditions. Comorbid disorders might affect self-reported functioning more strongly than any residual effect from cancer or its treatment . In the other known-group comparisons there was not always sufficient power to reach statistical significance. Nevertheless, the trends observed in the known groups were compatible with our hypotheses, except for the higher levels of mental health problems reported by the younger TCa survivors. This latter finding is similar to results reported in studies of younger breast cancer survivors [67,68,69,70]. The recent study of Drummond et al. shows that, with larger sample sizes, the EORTC QLQ-C30 and the QLQ-PR25 are able to discriminate between different initial treatment modalities in long-term prostate cancer survivors .
Recommendations for assessing HRQOL in cancer survivors
Despite their limitations, we consider both the SF-36 and the EORTC QLQ-C30 to be sufficiently robust psychometrically to be employed in cancer survivorship studies. Although they may not be able to differentiate clearly between survivors at higher levels of functioning and lower levels of symptom burden, they can identify functional health issues and relevant symptom levels that may require attention by health care providers.
The IOCv2 is a questionnaire which addresses the unique concerns related to the cancer experience that are not captured by the more generic HRQoL questionnaires . Most of the previous studies using the IOCv2 have been carried out in mixed samples and samples with women only e.g. [66, 72], which makes their results difficult to compare with ours. In particular, we know that women report more extreme impacts of cancer and its treatment (both positive and negative) on their lives than men [72, 73]. We can best compare our results with two previous studies of male hematological survivors and with the Norwegian validation study [66, 73, 74]. Compared to long-term male lymphoma survivors, survivors in our sample reported less extreme negative or positive impacts of having had cancer . Similar to what has been reported previously by Oerlemans et al. , the men in our sample reported few appearance concerns, and, like Dahl et al. , we found that TCa survivors were less negatively impacted by cancer than PCa survivors. These differences in outcomes on the IOCv2 underline the relevance of this questionnaire for detecting differences between survivor groups in how cancer and its treatment have affected their lives.
The selected scales from the QPSNordic can be particularly useful in assessing cancer survivors’ work experience. Ours as well as previous studies (e.g. [72, 75, 76]) indicate that a significant minority of cancer survivors experience work-related changes after being diagnosed with and treated for cancer. Particularly survivors with mental and physical health problems are vulnerable to work-related problems. Work ability levels observed in our sample of TCa survivors were comparable to that of mid-term Norwegian breast, TCa, and PCa cancer survivors. Gudbergsson et al. reported that the work ability in these cancer survivors was significantly worse than that of an age and gender matched control group .
The Patient-Reported Outcomes Measurement Information System (PROMIS) initiative  and the EORTC Quality of Life Group (EORTC QLG) Computer-Adaptive Testing project [79, 80] are currently developing HRQoL instruments based on item response theory which, among other things, will make it possible to discriminate better between individuals who score at the extremes of the health continuum. Also, the EORTC QLG is currently developing an HRQoL assessment strategy specifically for disease-free cancer survivors, focusing on longer-term physical effects of cancer and its treatment, psychological aspects of survivorship, and important issues such as fear of recurrence and health awareness, and excluding acute treatment related symptoms.
Generalizability of our findings
Even though our response rate is quite reasonable for this type of long-term follow-up study, we cannot rule out the possibility of some degree of selection bias in our sample. It could be that the non-respondents had fewer physical and psychosocial health issues (and thus were less motivated to participate in the study) than those who participated. It may also be that non-respondents did not want to be confronted with their earlier cancer experience. Unfortunately, we do not have any information about the actual reasons for not participating in the study, and thus the nature of any bias, if present, remains unclear.
As our study sample was restricted to male survivors only, we cannot generalize our findings to the larger population of cancer survivors. Additional studies are needed that include significant numbers of female cancer survivors. There is, for example, some evidence that women report higher levels of symptom burden and functional impairment than men. Problems observed in the current study with regard to floor and ceiling effects may be less pronounced in studies of female survivors or with mixed samples with regard to sex .
Additionally, our study sample was composed primarily of participants in randomized clinical trials. Patients treated in the context of clinical trials often are not entirely representative of the patient population encountered in clinical practice. Trial patients are often younger and have fewer comorbid conditions than non-trial patients [81, 82]. Also, patients treated in the context of a trial may receive more consistent and higher quality care than those treated outside of a trial context . While this may affect the actual prevalence of symptoms and functional limitations reported by long-term survivors, it is less relevant for evaluating the psychometric properties of HRQoL questionnaires, which was the focus of our study. Also, given the relatively long period of time since diagnosis and active treatment, the effect of having been treated in the context of a clinical trial probably has less impact on the generalizability of findings than might be the case in a study with a shorter follow-up [81, 82]. In any case, further studies are needed that investigate the performance of these and other HRQoL questionnaires when used with the broader clinic populations.
In general, the questionnaires evaluated in this study were found to be reliable and valid when used among male cancer survivors in various European settings. Several international initiatives are developing both computer-adaptive and survivorship-specific patient-reported outcome measures. These newer measures promise greater measurement precision and specificity. However, until they are sufficiently mature for general use, our results indicate that both the SF-36, the EORTC QLQ-C30 (and its modules), the IOCv2 and the QPSNordic are, within limits, useful tools for assessing the HRQoL of long term cancer survivors.
Bleomycin, etoposide, and cisplatin
The European Organisation for Research and Treatment of Cancer
Health-related quality of life
The Impact of Cancer questionnaire, version 2
Number of participants
Patient-Reported Outcomes Measurement Information System
Quality of Life Group
The EORTC core questionnaire
The EORTC prostate cancer module
The EORTC testicular cancer module
Nordic Questionnaire for Monitoring the Age Diverse Workforce
Social economic status
36-Item Short Form Survey
Berrino F, De Angelis R, Sant M, Rosso S, Bielska-Lasota M, Coebergh JW, Santaquilani M. Survival for eight major cancers and all cancers combined for European adults diagnosed in 1995–99: results of the EUROCARE-4 study. Lancet Oncol. 2007;8:773–83.
Bloom JR, Petersen DM, Kang SH. Multi-dimensional quality of life among long-term (5+ years) adult cancer survivors. Psychooncology. 2007;16:691–706.
Nord C, Mykletun A, Thorsen L, Bjorø T, Fosså SD. Self-reported health and use of health care services in long-term cancer survivors. Int J Cancer. 2005;114:307–16.
Bloom JR. Surviving and thriving? Psychooncology. 2002;11:89–92.
Deimling GT, Bowman KF, Sterns S, Wagner LJ, Kahana B. Cancer-related health worries and psychological distress among older adult, long-term cancer survivors. Psychooncology. 2006;15:306–20.
Joly F, Héron JF, Kalusinski L, Bottet P, Brune D, Allouache N, Macé-Lesec’h J, Couëtte JE, Pény J, Henry-Amar M. Quality of life in long-term survivors of testicular cancer: a population-based case-control study. J Clin Oncol. 2002;20:73–80.
Nout RA, van de Poll-Franse LV, Lybeert ML, Warlam-Rodenhuis CC, Jobsen JJ, Mens JW, Lutgens LC, Pras B, Van Putten WL, Creutzberg CL. Long-term outcome and quality of life of patients with endometrial carcinoma treated with or without pelvic radiotherapy in the post operative radiation therapy in endometrial carcinoma 1 (PORTEC-1) trial. J Clin Oncol. 2011;29:1692–700.
Zebrack BJ, Ganz PA, Bernaards CA, Petersen L, Abraham L. Assessing the impact of cancer: development of a new instrument for long-term survivors. Psychooncology. 2006;15:407–21.
O’Leary E, Drummond FJ, Gavin A, Kinnear H, Sharp L. Psychometric evaluation of the EORTC QLQ-PR25 questionnaire in assessing health-related quality of life in prostate cancer survivors: a curate’s egg. Qual Life Res. 2015;24:2219–30.
De Wit R, Roberts JT, Wilkinson PM, De Mulder PH, Mead GM, Foss SD, Cook P, De Prijck L, Stenning S, Collette L. Equivalence of three or four cycles of bleomycin, etoposide, and cisplatin chemotherapy and of a 3- or 5-day schedule in good-prognosis germ cell cancer: a randomized study of the European Organization for Research and Treatment of Cancer Genitourinary Tract Cancer Cooperative Group and the Medical Research Council. J Clin Oncol. 2001;19:1629–40.
Bolla M, Van Poppel H, Collette L, Van Cangh P, Vekemans K, Da Pozzo L, De Reijke TM, Verbaeys A, Bosset JF, Van Velthoven R, et al. Postoperative radiotherapy after radical prostatectomy: a randomised controlled trial (EORTC trial 22911). Lancet. 2005;366:572–8.
Fosså SD, Travis LB, Dahl AA. Medical and psychosocial issues in testicular cancer survivors. In: Ganz PA, editor. Cancer survivorship: today and tomorrow. New York: Springer; 2007. p. 101–13.
Quaresma M, Coleman MP, Rachet B. 40-year trends in an index of survival for all cancers combined and survival adjusted for age and sex for each cancer in England and Wales, 1971–2011: a population-based study. Lancet. 2015;385:1206–18.
Ferlay J, Soerjomataram I, Ervik M, Dikshit R, Eser S, Mathers C, Rebelo M, Parkin DM, Forman D, Bray F. GLOBOCAN 2012 v1.0, Cancer Incidence and Mortality Worldwide: IARC CancerBase No. 11 [internet]. Lion: International Agency for Research on Cancer; 2013.
Society AC. What are the key statistics about prostate cancer? 2008.
Brydøy M, Fosså SD, Dahl O, Bjorø T. Gonadal dysfunction and fertility problems in cancer survivors. Acta Oncol. 2007;46:480–9.
Dahl AA, Mykletun A, Fosså SD. Quality of life in survivors of testicular cancer. Urol Oncol. 2005;23:193–200.
Fegg MJ, Gerl A, Vollmer TC, Gruber U, Jost C, Meiler S, Hiddemann W. Subjective quality of life and sexual functioning after germ-cell tumour therapy. Br J Cancer. 2003;89:2202–6.
Grossfeld G, Small E. Long-term side effects of treatment for testis cancer. Urol Clin North Am. 1998;25:503–15.
Huddart RA, Norman A, Moynihan C, Horwich A, Parker C, Nicholls E, Dearnaley DP. Fertility, gonadal and sexual function in survivors of testicular cancer. Br J Cancer. 2005;93:200–7.
Blom JW, Vanderschoot JP, Oostindier MJ, Osanto S, Van der Meer FJ, Rosendaal FR. Incidence of venous thrombosis in a large cohort of 66,329 cancer patients: results of a record linkage study. J Thromb Haemost. 2006;4:529–35.
Stava C, Beck M, Schultz PN, Vassilopoulou-Sellin R. Hearing loss among cancer survivors. Oncol Rep. 2005;13:1193–9.
Haugnes HS, Aass N, Fossa SD, Dahl O, Brydoy M, Aasebo U, Wilsgaard T, Bremnes RM. Pulmonary function in long-term survivors of testicular cancer. J Clin Oncol. 2009;27:2779–86.
Fleer J, Sleijfer D, Hoekstra H, Tuinman M, Klip E, Hoekstra-Weebers J. Objective and subjective predictors of cancer-related stress symptoms in testicular cancer survivors. Patient Educ Couns. 2006;64:142–50.
Fosså SD, De Wit R, Roberts JT, Wilkinson PM, De Mulder PH, Mead GM, Cook P, De Prijck L, Stenning S, Aaronson NK, et al. Quality of life in good prognosis patients with metastatic germ cell cancer: a prospective study of the European Organization for Research and Treatment of Cancer Genitourinary Group/Medical Research Council Testicular Cancer Study Group (30941/TE20). J Clin Oncol. 2003;21:1107–18.
Fosså SD, Dahl AA, Loge JH. Fatigue, anxiety, and depression in long-term survivors of testicular cancer. J Clin Oncol. 2003;21:1249–54.
Skaali T, Fosså SD, Bremnes R, Dahl O, Haaland CF, Hauge ER, Klepp ON, Oldenburg J, Wist E, Dahl AA. Fear of recurrence in long-term testicular cancer survivors. Psychooncology. 2009;18:580–8.
Fleer J, Hoekstra HJ, Sleijfer DT, Hoekstra-Weebers JE. Quality of life of survivors of testicular germ cell cancer: a review of the literature. Support Care Cancer. 2004;12:476–86.
Litwin MS, Melmed GY, Nakazon T. Life after radical prostatectomy: a longitudinal study. J Urol. 2001;166:587–92.
Hewitt M, Rowland JH, Yancik R. Cancer survivors in the United States: age, health, and disability. J Gerontol A Biol Sci Med Sci. 2003;58:82–91.
Sanda MG, Dunn RL, Michalski J, Sandler HM, Northouse L, Hembroff L, Lin X, Greenfield TK, Litwin MS, Saigal CS, et al. Quality of life and satisfaction with outcome among prostate-cancer survivors. N Engl J Med. 2008;358:1250–61.
Baker F, Haffer SC, Denniston M. Health-related quality of life of cancer and noncancer patients in Medicare managed care. Cancer. 2003;97:674–81.
Arredondo SA, Elkin EP, Marr PL, Latini DM, DuChane J, Litwin MS, Carroll PR. Impact of comorbidity on health-related quality of life in men undergoing radical prostatectomy: data from CaPSURE. Urology. 2006;67:559–65.
Ramsey SD, Zeliadt SB, Hall IJ, Ekwueme DU, Penson DF. On the importance of race, socioeconomic status and comorbidity when evaluating quality of life in men with prostate cancer. J Urol. 2007;177:1992–9.
Van de Poll-Franse LV, Kwan L, Reiter RE, Lee SP, Litwin MS. The influence of cardiovascular disease on health related quality of life in men with prostate cancer: a 4-year followup study. J Urol. 2008;179:1362–7.
Mols F, Korfage IJ, Vingerhoets AJ, Kil PJ, Coebergh JW, Essink-Bot ML, Van de Poll-Franse LV. Bowel, urinary, and sexual problems among long-term prostate cancer survivors: a population-based study. Int J Radiat Oncol Biol Phys. 2009;73:30–8.
Penson DF, Litwin MS, Aaronson NK. Health related quality of life in men with prostate cancer. J Urol. 2003;169:1653–61.
Blank TO, Bellizzi KM. After prostate cancer: predictors of well-being among long-term prostate cancer survivors. Cancer. 2006;106:2128–35.
Fowler F, Barry M, Lu-yao G, Roman A, Wasson J, Wennberg J. Patient-re ported complications and follow-up treatment after radical prostatectomy : The national medicare experience: 1988–1990 (updated June 1993). Urology. 1993;42:622–8.
Galalae RM, Loch T, Riemer B, Rzehak P, Kuchler T, Kimmig B, Kovacs G. Health-related quality of life measurement in long-term survivors and outcome following radical radiotherapy for localized prostate cancer. Strahlenther Onkol. 2004;180:582–9.
Shrader-Bogen CL, Kjellberg JL, McPherson CP, Murray CL. Quality of life and treatment outcomes: prostate carcinoma patients’ perspectives after prostatectomy or radiation therapy. Cancer. 1997;79:1977–86.
Aksnes LH, Bruland ØS. Some musculo-skeletal sequelae in cancer survivors. Acta Oncol. 2007;46:490–6.
Van Leeuwen M, Efficace F, Fossa SD, Bolla M, De Giorgi U, de Wit R, Holzner B, van de Poll-Franse LV, van Poppel H, White J, et al. Recruiting long-term survivors of European Organisation for Research and Treatment of Cancer phase III clinical trials into quality of life studies: Challenges and opportunities. Eur J Cancer. 2014;50:1957–63.
McHorney CA, Ware Jr JE, Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 1993;31:247–63.
Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, Filiberti A, Flechtner H, Fleishman SB, De Haes JC. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85:365–76.
Van Andel G, Bottomley A, Fosså SD, Efficace F, Coens C, Guerif S, Kynaston H, Gontero P, Thalmann G, Akdas A, et al. An international field study of the EORTC QLQ-PR25: a questionnaire for assessing the health-related quality of life of patients with prostate cancer. Eur J Cancer. 2008;44:2418–24.
Holzner B, Efficace F, Basso U, Johnson CD, Aaronson NK, Arraras JI, Smith AB, Chow E, Oberguggenberger AS, Bottomley A, et al. Cross-cultural development of an EORTC questionnaire to assess health-related quality of life in patients with testicular cancer: the EORTC QLQ-TC26. Qual Life Res. 2013;22:369–78.
Crespi CM, Ganz PA, Petersen L, Castillo A, Caan B. Refinement and psychometric evaluation of the impact of cancer scale. J Natl Cancer Inst. 2008;100:1530–41.
Mols F, Helfenrath KA, Vingerhoets AJ, Coebergh JW, Van de Poll-Franse LV. Increased health care utilization among long-term cancer survivors compared to the average Dutch population: a population-based study. Int J Cancer. 2007;121:871–7.
Pahkin K, Björklund C, Mykletun RJ, Furunes T, Gard G, Lindström K. User’s guide for the QPSNordic-ADW Nordic questionnaire for monitoring the age diverse workforce. Copenhagen: Nordic Council of Ministers; 2008.
Dewolf L, Koller M, Velikova G, Johnson C, Scott N, Bottomley A. EORTC quality of life group translation procedure. Brussels: EORTC Quality of Life Group; 2009.
Fayers PM, Aaronson NK, Bjordal K, Groenvold M, Curran D, Bottomley A, on behalf of the EORTC QLG. EORTC QLQ-C30 Scoring Manual. 3rd ed. Brussels: EORTC; 2001.
Ware JE, Snow KK, Kosinski M, Gandek B. SF-36 health survey manual and interpretation guide. Boston: New England Medical Center, The Health Institute; 1994.
Andresen EM. Criteria for assessing the tools of disability outcomes research. Arch Phys Med Rehabil. 2000;81:S15–20.
Cronbach CJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297.
Cohen J. A power primer. Psychol Bull. 1992;112:155–9.
Sitzia J, Wood N. Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Health Care. 1998;10:311–7.
Koloski NA, Jones M, Eslick G, Talley NJ. Predictors of response rates to a long term follow-up mail out survey. PLoS One. 2013;8:e79179.
Wood E, Nosko A, Desmarais S, Ross C, Irvine C. Online and traditional paper-and-pencil survey administration: examining experimenter presence, sensitive material and long surveys. Can J Hum Sex. 2006;15:147–55.
McHorney CA, Ware Jr JE, Lu JF, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994;32:40–66.
Van de Poll-Franse LV, Mols F, Gundy CM, Creutzberg CL, Nout RA, Verdonck-de Leeuw IM, Taphoorn MJ, Aaronson NK. Normative data for the EORTC QLQ-C30 and EORTC-sexuality items in the general Dutch population. Eur J Cancer. 2011;47:667–75.
Schwarz R, Hinz A. Reference data for the quality of life questionnaire EORTC QLQ-C30 in the general German population. Eur J Cancer. 2001;37:1345–51.
Gandek B, Ware Jr JE, Aaronson NK, Alonso J, Apolone G, Bjorner J, Brazier J, Bullinger M, Fukuhara S, Kaasa S, et al. Tests of data quality, scaling assumptions, and reliability of the SF-36 in eleven countries: results from the IQOLA Project. International Quality of Life Assessment. J Clin Epidemiol. 1998;51:1149–58.
Muzzatti B, Flaiban C, Romito F, Cormio C, Annunziata MA. The Impact of Cancer Scale (IOC) in Italian long-term cancer survivors: adaptation and psychometric evaluation. Support Care Cancer. 2013;21:3355–62.
Blanchin M, Dauchy S, Cano A, Bredart A, Aaronson NK, Hardouin JB. Validation of the French translation-adaptation of the impact of cancer questionnaire version 2 (IOCv2) in a breast cancer survivor population. Health Qual Life Outcomes. 2015;13:110.
Oerlemans S, Smith SK, Crespi CM, Zimmerman S, van de Poll-Franse LV, Ganz PA. Assessing the impact of cancer among Dutch non-Hodgkin lymphoma survivors compared with their American counterparts: a cross-national study. Psychooncology. 2013;22:1258–65.
Knobf MT. Reproductive and hormonal sequelae of chemotherapy in women. Premature menopause and impaired fertility can result, effects that are especially disturbing to young women. Am J Nurs. 2006;106:60–5.
Kroenke CH, Rosner B, Chen WY, Kawachi I, Colditz GA, Holmes MD. Functional impact of breast cancer by age at diagnosis. J Clin Oncol. 2004;22:1849–56.
Wenzel LB, Fairclough DL, Brady MJ, Cella D, Garrett KM, Kluhsman BC, Crane LA, Marcus AC. Age-related differences in the quality of life of breast carcinoma patients after treatment. Cancer. 1999;86:1768–74.
Ganz PA, Greendale GA, Petersen L, Kahn B, Bower JE. Breast cancer in younger women: reproductive and late health effects of treatment. J Clin Oncol. 2003;21:4184–93.
Drummond FJ, Kinnear H, O’Leary E, Donnelly, Gavin A, Sharp L. Long-term health-related quality of life of prostate cancer survivors varies by primary treatment. Results from the PiCTure (Prostate Cancer Treatment, your experience) study. J Cancer Surviv. 2015;9:361–72.
Holterhues C, Cornish D, Van de Poll-Franse LV, Krekels G, Koedijk F, Kuijpers D, Coebergh JW, Nijsten T. Impact of melanoma on patients’ lives among 562 survivors: a Dutch population-based study. Arch Dermatol. 2011;147:177–85.
Dahl AA, Gudbergsson SB, Dorum A, Fossa SD, Liavaag AH, Sorebo O. “The impact of cancer scale” version 1: psychometric testing of the Norwegian translation in a heterogeneous sample of cancer survivors. Qual Life Res. 2012;21:1459–70.
Korszun A, Sarker SJ, Chowdhury K, Clark C, Greaves P, Johnson R, Kingston J, Levitt G, Matthews J, White P, et al. Psychosocial factors associated with impact of cancer in longterm haematological cancer survivors. Br J Haematol. 2014;164:790–803.
Gudbergsson SB, Fosså SD, Dahl AA. A study of work changes due to cancer in tumor-free primary-treated cancer patients. A NOCWO study. Support Care Cancer. 2008;16:1163–71.
Steiner JF, Cavender TA, Nowels CT, Beaty BL, Bradley CJ, Fairclough DL, Main DS. The impact of physical and psychosocial factors on work characteristics after cancer. Psychooncology. 2008;17:138–47.
Gudbergsson SB, Fosså SD, Dahl AA. Is cancer survivorship associated with reduced work engagement? A NOCWO Study. J Cancer Surviv. 2008;2:159–68.
Fries JF, Bruce B, Cella D. The promise of PROMIS: using item response theory to improve assessment of patient-reported outcomes. Clin Exp Rheumatol. 2005;23:S53–7.
Petersen MA, Groenvold M, Aaronson NK, Chie WC, Conroy T, Costantini A, Fayers P, Helbostad J, Holzner B, Kaasa S, et al. Development of computerized adaptive testing (CAT) for the EORTC QLQ-C30 physical functioning dimension. Qual Life Res. 2011;20:479–90.
Petersen MA, Groenvold M, Aaronson NK, Chie WC, Conroy T, Costantini A, Fayers P, Helbostad J, Holzner B, Kaasa S, et al. Development of computerised adaptive testing (CAT) for the EORTC QLQ-C30 dimensions - general approach and initial results for physical functioning. Eur J Cancer. 2010;46:1352–8.
Elting LS, Cooksley C, Bekele BN, Frumovitz M, Avritscher EBC, Sun C, Bodurka DC. Generalizability of cancer clinical trial results. Cancer. 2006;106:2452–8.
Unger JM, Barlow WE, Martin DP, Ramsey SD, Leblanc M, Etzioni R, Hershman DL. Comparison of survival outcomes among cancer patients treated in and out of clinical trials. J Natl Cancer Inst. 2014;106:dju002.
Braunholtz DA, Edwards SJL, Lilford RJ. Are randomized clinical trials good for us (in the short term)? Evidence for a “trial effect”. J Clin Epidemiol. 2001;54:217–24.
We thank Dr. Sally Stenning from the UK Medical Research Council Clinical Trials Unit for providing the participants list and clinical data for the UK and Norwegian patients in trial EORTC 30941/MRC TE20. We thank the survivors and all of the hospitals and their staff for their participation in this study. We also acknowledge the help of Dr. Manuela Tutolo and Prof. Alberto Briganti from the Department of Urology, San Raffaele Scientific Institute, Vita-Salute University, Milan, Italy and the Italian Germ Cell Cancer Group (IGG) for their contribution to the study. The following IGG centers participated in this project:
- Dr. Ugo De Giorgi, Istituto Scientifico Romagnolo per lo Studio e la Cura dei Tumori (IRST) – IRCCS, Meldola,
- Dr. Giorgio Papiani, S. Maria delle Croci Hospital, Ravenna,
- Dr. Umberto Basso, Istituto Oncologico Veneto, IOV - IRCCS, Padova,
- Dr. Giuseppe Fornarini, IRCCS Azienda Ospedaliera Universitaria San Martino - IST Istituto Nazionale per la Ricerca sul Cancro, Genova,
- Dr. Giovannella Palmieri, Federico II University, Napoli,
- Dr. Giuseppe Luigi Banna, Cannizzaro Hospital, Catania,
- Dr. Caterina Accettura, Vito Fazzi Hospital, Lecce
This study was funded by the European Organisation for Research and Treatment of Cancer (BE) Quality of Life Group (grant number Survivorship 001/ 2008). Dr. Collette’s contribution to this publication was supported by Fonds Cancer (FOCA) from Belgium.
Availability of data and materials
The data that support the findings of this study are available from the Division of Psychosocial Research and Epidemiology of the Netherlands Cancer Institute (contact person: N.K. Aaronson), but restrictions apply to the availability of these data due to an agreement between the Netherlands Cancer Institute and the European Organisation for Research and Treatment of Cancer (EORTC) Headquarters (Brussels), and so they are not publicly available. Data are available pending approval of both the Netherlands Cancer Institute and the EORTC.
Study conception and design: NA, SO, ML, FE, SD, MB, BH, LPF, HP, and RW. Acquisition of data: ML, FE, SF, MB, MC, UG, HP, JW, and SO. Analysis and interpretation of data: ML, KK, and NA. Drafting of manuscript: ML and NA. Critical revision: ML, JK, FE, SF, MB, LC, UG, LPF, HP, JW, SO, and NK. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. All participants signed informed consent for their participation and the use of their data. The study was approved by the Institutional Review Board of the Netherlands Cancer Institute (PTC10.0237), University Hospital, K.U. Leuven (S52511), West of Scotland Research Ethics Service (11/WS/0056), Romagna Ethical Committee (Prot. 2639/2010), and The Regional Committee for Medical Ethics (2010/390) (Ethical committee in the region of Oslo).
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
van Leeuwen, M., Kieffer, J.M., Efficace, F. et al. International evaluation of the psychometrics of health-related quality of life questionnaires for use among long-term survivors of testicular and prostate cancer. Health Qual Life Outcomes 15, 97 (2017) doi:10.1186/s12955-017-0670-4
- Prostatic neoplasms
- Testicular neoplasms