Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer
Health and Quality of Life Outcomes volume 5, Article number: 70 (2007)
Understanding what constitutes an important difference on a HRQL measure is critical to its interpretation. The aim of this study was to provide a range of estimates of minimally important differences (MIDs) in EQ-5D scores in cancer and to determine if estimates are comparable in lung cancer.
A retrospective analysis was conducted on cross-sectional data collected from 534 cancer patients, 50 of whom were lung cancer patients. A range of minimally important differences (MIDs) in EQ-5D index-based utility (UK and US) scores and VAS scores were estimated using both anchor-based and distribution-based (1/2 standard deviation and standard error of the measure) approaches. Groups were anchored using Eastern Cooperative Oncology Group performance status (PS) ratings and FACT-G total score-based quintiles.
For UK-utility scores, MID estimates based on PS ranged from 0.10 to 0.12 both for all cancers and for lung cancer subgroup. Using FACT-G quintiles, MIDs were 0.09 to 0.10 for all cancers, and 0.07 to 0.08 for lung cancer. For US-utility scores, MIDs ranged from 0.07 to 0.09 grouped by PS for all cancers and for lung cancer; when based on FACT-G quintiles, MIDs were 0.06 to 0.07 in all cancers and 0.05 to 0.06 in lung cancer. MIDs for VAS scores were similar for lung and all cancers, ranging from 8 to 12 (PS) and 7 to 10 (FACT-G quintiles).
Important differences in EQ-5D utility and VAS scores were similar for all cancers and lung cancer, with the lower end of the range of estimates closer to the MID, i.e. 0.08 for UK-index scores, 0.06 for US-index scores, and 0.07 for VAS scores.
It is common, if not usual practice, to include health-related quality of life (HRQL) measures in clinical trials in oncology. To justify the cost of new cancer drugs, decision-makers need to determine not only whether a drug has a statistically significant impact on survival and/or HRQL, but they also need to evaluate whether the improvement is meaningful. This is particularly important in lung cancer, where aggressive new therapies are being brought to market. In addition to the use of cancer-specific measures such as European Organization for Research and Treatment of Cancer-QLQ-C30 (EORTC QLQ-C30) [1, 2]and the Functional Assessment of Chronic Illness Therapy (FACIT) measurement system , clinical trials in oncology are increasingly incorporating generic preference-based measures such as EQ-5D. EQ-5D is an indirect measure of utility for health that generates an index-based summary score based upon societal preference weights . Utility scores enable comparisons of burden of disease across conditions and the calculation of quality-adjusted life-years (QALYs), an outcome used to compare the cost effectiveness of health care technologies.
A major challenge in HRQL measurement is the interpretation of scores, particularly with respect to defining what constitutes a minimally important difference (MID). The MID has been defined as the smallest change in a PRO measure that is perceived by patients as beneficial or that would result in a change in treatment . Approaches to estimation of MIDs have been classified as either distribution-based or anchor-based . Anchor-based approaches compare changes seen in an individual's HRQL to an external criterion, such as a clinical measure or using a patient rated global change question. Problematically, no single anchor represents a gold standard and no approach is ideal. Norman et al (1997) found that retrospective global ratings of change have questionable ability to yield information of treatment effects . Alternatively, distribution-based approaches rely on the distribution of scores and are computed using variations on effect size . The main disadvantage to distribution-based techniques is that they do not provide insight into the importance of the difference . Often both approaches are combined, with anchored-based HRQL changes initially framed in terms of the individual are then further analyzed as a group using distribution-based methods [10–15].
While MIDs have been estimated for EQ-5D index-based scores for some conditions , empiric work has not been performed in cancer. Additionally, it is not clear if lung cancer has a different range of MID estimates. Thus, the aim of this study was to provide a range of estimates for meaningful difference in EQ-5D scores in cancer and to determine if MIDs for lung cancer are different from all cancers.
A retrospective analysis was conducted on cross-sectional data collected from 534 cancer patients with eleven types of cancer who participated in a validation study of cancer symptoms scales . Participants had advanced (stage 3 or 4) cancer of the bladder, brain, breast (females patients only), colon/rectum, head/neck, liver/pancreas, kidney, lung, lymphoma, ovary (females patients only), and prostate (males patients only). All patients had received at least 2 cycles of chemotherapy, or if chemotherapy was non-cyclical, had been receiving it for at least 1 month. Efforts were made to recruit 50 patients for each type of cancer, with approximately equal proportions of male and female patients for the non-gender specific types of neoplasm. This dataset included 50 patients lung cancer patients, and between 50 and 52 patients with all other types of cancer except bladder cancer (n = 31).
The patients were recruited from six sites within the National Cancer Coalition Network (NCCN) and the Cancer Health Alliance of Metropolitan Chicago (CHAMC). The NCCN is a not for profit, tax-exempt corporation that is an alliance of National Cancer Institute (NCI) approved comprehensive cancer centers. The CHAMC organizations provide social, emotional and informational support services to cancer patients free of charge. These organizations are not affiliated with a medical center or university, and each CHAMC agency serves different geographical and socio-demographic cancer patient populations. All patients who completed the questionnaires consented to participate in the study. Institutional review board approval was obtained for secondary data analysis (University of Illinois at Chicago research protocol #2006-0891).
Patients completed several questionnaires, including the EQ-5D and the Functional Assessment of Cancer Therapy (FACT). The EQ-5D descriptive system consists of 5 dimensions: Mobility, Self-Care, Usual Activities, Pain/Discomfort, and Anxiety/Depression, each with 3 levels (e.g. no problems, moderate problems, extreme problems) . Index-based summary scores were calculated based on 2 different algorithms using societal preference developed from general population-based valuation studies in the United Kingdom  and the USA . The index-based score is typically interpreted along a continuum where 1 represents best possible health and 0 represents dead, with some health states being worse than dead (<0). In addition to the self-classifier, respondents rate their health today using a 20 centimeter visual analogue scale (VAS) that ranges from 0 (worst imaginable health state) to 100 (best imaginable health state).
Participants also completed the Functional Assessment of Cancer Therapy (FACT) quality of life questionnaire using a version specific to their tumor type. The general subscales common to all versions (FACT-G) include physical well-being (PWB), social/family well-being (SFWB), emotional well-being (EWB), and functional well-being (FWB). The FACT-G total score (FACT-G Total) is based on 26 summed items (responses 0 to 4) from the PWB (7 items), FWB (7 items), SFWB (6 items), and EWB (6 items). Higher scores represent better quality of life.
Performance status was evaluated using the Eastern Cancer Oncology Group (ECOG) classification system . ECOG grades range from 0, which is fully active, to 4, completely disabled, and 5 is dead. ECOG grades are used by physicians and researchers to assess progression of disease, impact of the disease on daily activities, and to guide appropriate treatment and prognosis.
Both anchor-based and distribution-based approaches were used to estimate MIDs for the EQ-5D in the overall cancer cohort, and in the subgroup of lung cancer patients, when possible. Distribution-based criteria included: 1/2 standard deviation (SD) and the standard error of the measure (SEM) . For consistency with past studies exploring MIDs, 1/3 SD was also reported, but it was not included in the summarized range of MIDs as there is less evidence to support that 1/3 SD represents an important difference. The SEM is calculated as where r is reliability of the measure. It is debatable which type of reliability, internal consistency or test-retest (TRT) reliability, is most appropriate. Very limited evidence of TRT reliability is available on the EQ-5D in cancer . Because the EQ-5D has single item dimensions, internal consistency reliability does not apply to each dimension. Although HRQL is considered a multi-dimensional construct, the aggregation of dimensional responses to create a single summary score is an implicit endorsement of HRQL as an overarching construct. However, item response theory-based analysis of the dimensional structure of the EQ-5D has indicated that the anxiety/depression dimension taps into a construct distinct from the other 4 items . Calculation of internal consistency reliability using Cronbach's alpha was 0.68, regardless of whether or not anxiety/depression was included. Thus, for the purposes of our analysis, a reliability of 0.68 was used in the calculation of the SEM.
Anchors can be constructed using clinically-based criteria, such as response to treatment, or more subjective criteria, e.g. health status. We used ECOG grades, assessed by physician, to group patients into categories of performance status, and determined mean difference scores between ECOG grades. Distribution-based criteria were then applied to the statistics associated with each anchor-based category. A second anchor-based approach used FACT-G scores. The cohort was stratified into quintiles based on FACT-G summary scores. Grouping the cohort into quintiles approximated an appropriate threshold for stratifying patients based on MID estimates for the FACT-G, have been identified as close to 6 in previous studies: 6–7 in hepatobiliary carcinoma , and 5–6 in breast cancer . Final results were summarized as a range of MID estimates and as an average MID across categories, weighted by the sample size within each category.
Similar demographic characteristics were observed in the overall cancer sample and the lung cancer subgroup (Table 1). A wide range of scores were observed in the overall cancer cohort, with UK-based scores ranging from worse than dead (-0.14) to full health (1.0). A smaller range was observed for the US-based scores (0.21 to 1.0). Compared to the mean (SD) scores for the overall cohort [UK 0.72 (SD 0.22); US 0.78 (SD 0.15)], the subgroup with lung cancer had lower mean utility scores but similar dispersion around the mean [UK 0.67 (SD 0.22); US 0.74 (SD 0.16)] (Table 2). Mean VAS scores for the lung cancer subgroup [68 (SD 18)] were the same as for the overall cancer cohort [68 (SD 20)].
For all cancer patients, mean difference scores anchored by ECOG status ranged from 0.09 to 0.16 for UK scores and from 0.07 to 0.11 for US scores (Table 3). Across ECOG-based strata, MIDs based on the SEM and 0.5 SD were similar, ranging from 0.08 to 0.16 for UK scores, and from 0.06 to 0.10 for US scores. For the lung cancer cohort (excluding the single patient with grade 3 PS), mean difference scores between ECOG levels ranged from 0.10 to 0.13 (UK scores), and from 0.07 to 0.09 (US scores). MIDs based on SEM and 0.5 SD ranged from 0.08 to 0.14 (UK scores), and from 0.07 to 0.12 (US scores).
Average mean estimates of MIDs across FACT-G based quintiles for the overall cancer cohort were 0.09 for UK scores, 0.06 for US scores (Table 4). Using distribution based criteria averaged across quintile-based groups, MIDs for the overall cohort were: SEMUK = 0.10, 1/2 SDUK = 0.09; SEMUS = 0.07, 1/2 SDUS = 0.06. For the lung cancer subgroup, average MIDs between quintiles were 0.10 (UK) and 0.07 (US), with SEMUK = 0.09, 1/2 SDUK = 0.08; SEMUS = 0.06, 1/2 SDUS = 0.06.
MID estimates for EQ-5D VAS scores based on FACT-G score quintiles were the same for both the overall cancer groups and the lung cancer subgroup (Table 5). MIDs for VAS scores ranged from 7 to 10 when MIDs were averaged across the anchor-based categories using FACT-G quintiles. Average mean difference was 7 between quintile categories; 10 according to the SEM; and 9 using 1/2 SD. MIDs for VAS scores tended to be slightly larger using ECOG grade to anchor difference scores compared to FACT-G score based quintiles, ranging from 8 to 11 (all cancers) and 7.5 to 11.5 (lung cancer).
Interpretation of scores is an important issue in the field of HRQL measurement, but there is no consensus on the most appropriate method for assessing the ability of an instrument to capture meaningful differences. In this study, we followed criteria established in previous investigations of MIDs [13–15]. We found that distribution and anchor-based estimates tended to converge, helping to triangulate support for the validity of the range of MID estimates. In addition, the MIDs for overall cancer and lung cancer cohorts were similar.
The issue of what constitutes an MID on a measure of HRQL is part of an ongoing dialogue about issues of interpretation. Developers of HRQL measures have not been forthcoming in the literature in explicitly attempting to establish MIDs. One reason to avoid this is because clinically important differences may vary with the target population. Limitations in the scaling properties of a measure can contribute to inconsistent MID estimates, as they may depend upon where a patient or group falls along the continuum of the measure. Distribution-based approaches for estimating important differences rely on the assumption of normality, and ceiling effects particularly in healthier patient populations produce skewed score distributions. Although ceiling effects have been associated with the use of EQ-5D , a ceiling effect was generally not observed in the cancer cohort, and standard deviations were relatively stable across the anchor-based strata.
MID estimates for EQ-5D in this study can be compared to other studies that have examined important differences using EQ-5D. A previous study by Walters and Brazier compared minimally important differences between SF-6D and EQ-5D, and reported a mean MID of 7.4 for the UK-based algorithm . Their estimate was at lower range of MIDs estimated in this study for cancer patients, which may imply that MIDs in cancer are slightly larger than for the conditions investigated, which included leg ulcer, back pain, early rheumatoid arthritis, limb reconstruction, osteoarthritis, irritable bowel syndrome, and chronic obstructive lung disease. An alternative explanation is that the anchors used in this study, particularly ECOG grade, provided benchmarks for meaningful differences that do not necessarily represent a minimally important difference.
MIDs are often estimated using longitudinal datasets, and difference scores based on changes over time were not available in this dataset, which was cross-sectional. However, the MIDs for EQ-5D UK-based utility scores reported using longitudinal data  were comparable to the estimates for UK scores generated in this study. Another limitation of our study was that sample size for lung cancer subgroups was small. When further stratified by ECOG grade, sub-sample sizes became extremely small and produced unreliable estimates in the lung cancer subgroup, although the average MID obtained in lung cancer tended to be similar to the overall cancer cohort. It is unclear if MIDs based on patients with advanced cancer in this study generalize to patients with less advanced stages of cancer.
An additional issue for users of EQ-5D is the selection of preference-based algorithm. As observed in this study, MIDs varied with the selection of the algorithm. MIDs for EQ-5D UK index-based utility scores ranged from 0.08 to 0.16 with a mode of 0.10. For US-based scores, a range of 0.06 to 0.12 was reported, with a mode of 0.07. This result was not unexpected, as the US preference-based algorithm produces scores with a smaller range than the UK scores, resulting in smaller difference scores and smaller standard deviations, thus smaller MIDs.
In summary, important differences in EQ-5D summary scores were similar for all cancers and lung cancer, with the lower bounds likely to represent a closer estimate of true MID, i.e. 0.08 for UK-based scores, 0.06 for US-based scores, and 0.07 for VAS scores. MIDs for EQ-5D UK-based utility scores in cancer were similar to estimated MIDs for other conditions in the published literature. To our knowledge, MIDs for EQ-5D VAS scores and US-based utility scores have not been previously reported. Across the different approaches, MIDs for US-based utility scores were consistently smaller than MIDs for UK-based utility scores.
– minimally important difference
– health-related quality of life
– Functional Assessment of Cancer Therapy – General
– standard error of the measure
– standard deviation
– physical well-being
– social/family wellbeing
– emotional well-being
– functional well-being
– Eastern Cancer Oncology Group
– performance status
Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, Filiberti A, Flechtner H, Fleishman SB, de Haes JC, et al.: The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993,85(5):365–376. 10.1093/jnci/85.5.365
de Haes J, Curran D, Young T, Bottomley A, Flechtner H, Aaronson N, Blazeby J, Bjordal K, Brandberg Y, Greimel E, Maher J, Sprangers M, Cull A: Quality of life evaluation in oncological clinical trials - the EORTC model. The EORTC Quality of Life Study Group. Eur J Cancer 2000,36(7):821–825. 10.1016/S0959-8049(00)00007-1
Webster K, Cella D, Yost K: The Functional Assessment of Chronic Illness Therapy (FACIT) Measurement System: properties, applications, and interpretation. Health Qual Life Outcomes 2003,1(1):79. 10.1186/1477-7525-1-79
Pickard AS, Wilke CT, Lin HW, Lloyd A: Health Utilities Using the EQ-5D in Studies of Cancer. PharmacoEconomics 2007,25(5):365–384. 10.2165/00019053-200725050-00002
Revicki DA, Cella D, Hays RD, Sloan JA, Lenderking WR, Aaronson NK: Responsiveness and minimal important differences for patient reported outcomes. Health Qual Life Outcomes 2006, 4: 70. 10.1186/1477-7525-4-70
Lydick E, Epstein RS: Interpretation of quality of life changes. Qual Life Res 1993,2(3):221–226. 10.1007/BF00435226
Norman GR, Stratford P, Regehr G: Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997,50(8):869–879. 10.1016/S0895-4356(97)00097-8
Norman GR, Sridhar FG, Guyatt GH, Walter SD: Relation of distribution- and anchor-based approaches in interpretation of changes in health-related quality of life. Med Care 2001,39(10):1039–1047. 10.1097/00005650-200110000-00002
de Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM: Minimal changes in health status questionnaires: distinction between minimally detectable change and minimally important change. Health Qual Life Outcomes 2006, 4: 54. 10.1186/1477-7525-4-54
Eton DT, Cella D, Yost KJ, Yount SE, Peterman AH, Neuberg DS, Sledge GW, Wood WC: A combination of distribution- and anchor-based approaches determined minimally important differences (MIDs) for four endpoints in a breast cancer scale. J Clin Epidemiol 2004,57(9):898–910. 10.1016/j.jclinepi.2004.01.012
Wyrwich KW, Tierney WM, Babu AN, Kroenke K, Wolinsky FD: A comparison of clinically important differences in health-related quality of life for patients with chronic lung disease, asthma, or heart disease. Health Serv Res 2005,40(2):577–591. 10.1111/j.1475-6773.2005.0l374.x
Metz SM, Wyrwich KW, Babu AN, Kroenke K, Tierney WM, Wolinsky FD: A comparison of traditional and Rasch cut points for assessing clinically important change in health-related quality of life among patients with asthma. Qual Life Res 2006,15(10):1639–1649. 10.1007/s11136-006-0036-6
Steel JL, Eton DT, Cella D, Olek MC, Carr BI: Clinically meaningful changes in health-related quality of life in patients diagnosed with hepatobiliary carcinoma. Ann Oncol 2006,17(2):304–312. 10.1093/annonc/mdj072
Cella D, Eton DT, Fairclough DL, Bonomi P, Heyes AE, Silberman C, Wolf MK, Johnson DH: What is a clinically meaningful change on the Functional Assessment of Cancer Therapy-Lung (FACT-L) Questionnaire? Results from Eastern Cooperative Oncology Group (ECOG) Study 5592. J Clin Epidemiol 2002,55(3):285–295. 10.1016/S0895-4356(01)00477-2
Cella D, Eton DT, Lai JS, Peterman AH, Merkel DE: Combining anchor and distribution-based methods to derive minimal clinically important differences on the Functional Assessment of Cancer Therapy (FACT) anemia and fatigue scales. J Pain Symptom Manage 2002,24(6):547–561. 10.1016/S0885-3924(02)00529-8
Walters SJ, Brazier JE: Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res 2005,14(6):1523–1532. 10.1007/s11136-004-7713-0
Pickard AS, De Leon MC, Kohlmann T, Cella D, Rosenbloom S: Psychometric Comparison of the Standard EQ-5D to a 5 Level Version in Cancer Patients. Med Care 2007,45(3):259–263. 10.1097/01.mlr.0000254515.63841.81
Rabin R, de Charro F: EQ-5D: a measure of health status from the EuroQol Group. Ann Med 2001,33(5):337–343.
Dolan P: Modeling valuations for EuroQol health states. Med Care 1997,35(11):1095–1108. 10.1097/00005650-199711000-00002
Shaw JW, Johnson JA, Coons SJ: US valuation of the EQ-5D health states: development and testing of the D1 valuation model. Med Care 2005,43(3):203–220. 10.1097/00005650-200503000-00003
Oken MM, Creech RH, Tormey DC, Horton J, Davis TE, McFadden ET, Carbone PP: Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol 1982,5(6):649–655. 10.1097/00000421-198212000-00014
Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky FD: Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med Care 1999,37(5):469–478. 10.1097/00005650-199905000-00006
Gunter OH, Matschinger H, Konig HH: An item response theory model analysis to evaluate the dimensionality of the EQ-5D across six countries (abstract #1656). Accessed October 30, 2006.[http://www.isoqol.org/2006AbstractsBook.pdf]
Johnson JA, Pickard AS: Comparison of the EQ-5D and SF-12 health surveys in a general population survey in Alberta, Canada. Med Care 2000,38(1):115–121. 10.1097/00005650-200001000-00013
National Comprehensive Cancer Network (Diane Paul, MS, RN), Dana Farber (Alice Kornblith, PhD), Duke University Medical Center (Amy Abernethy, MD), Fred Hutchinson Cancer Research Center (Karen Syrjala, PhD), H. Lee Moffitt Cancer Center (Paul B. Jacobsen, PhD), Robert H. Lurie Comprehensive Cancer Center of Northwestern University (Sarah Rosenbloom, PhD, Jamie Von Roenn, MD). Funding support for the data collection was provided by 11 pharmaceutical companies; support for this analysis was provided by GlaxoSmithKline.
A. Simon Pickard is a member of the executive committee of the EuroQol group, a not for profit group that developed and distributes the EQ-5D. David Cella is developer of the FACIT measurement system. Drs. Pickard and Cella have received consulting fees from GlaxoSmithKline, which financed this manuscript including the article-processing charge. They do not have any stocks or shares in an organization that may gain or lose financially from the publication of this manuscript.
ASP, MN and DC were responsible for the conception of the study. ASP and DC acquired the data. ASP performed the data analysis and drafted the manuscript. MN and DC revised it critically for intellectual content, and all authors approved of the final version.
About this article
Cite this article
Pickard, A.S., Neary, M.P. & Cella, D. Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer. Health Qual Life Outcomes 5, 70 (2007). https://doi.org/10.1186/1477-7525-5-70