- Open Access
A comparison of injured patient and general population valuations of EQ-5D health states for New Zealand
Health and Quality of Life Outcomesvolume 12, Article number: 21 (2014)
A near-universal finding internationally is that patient valuations of their own health, represented using the EQ-5D system, are mostly higher than general population valuations of the same EQ-5D states. This paper investigates whether this result also applies to New Zealand. Despite the EQ-5D’s widespread use for cost-utility analysis in New Zealand, in particular by the Pharmaceutical Management Agency (PHARMAC) for health technology assessments, no previous studies comparing patient and general population valuations have included data for New Zealand.
Valuations of 13 EQ-5D health states from a 1999 survey of the New Zealand general population (n = 396) are compared with injured New Zealanders’ (n = 2099) valuations of their own health (also represented on the EQ-5D) collected between 2007 and 2009 in the Prospective Outcomes of Injury Study. Which EQ-5D dimensions are most strongly associated with the population valuations is also investigated.
Injured population valuations are higher (better-rated health) than general population valuations for all 13 health states considered except 11111 (no problems on any EQ-5D dimension). This difference, which tends to be larger the ‘worse’ the state, is statistically significant at the 10% level for most of the states. State 11111 is rated significantly lower by the injured population than the general population. Pain/discomfort is more important in determining valuations for the general population than for injured people, whereas problems with self-care are more important for the injured population; anxiety/depression is important in both general and injured population valuations.
Consistent with the international literature, injured people’s valuations of their own health are mostly higher than the general population’s hypothetical valuations of the same EQ-5D states for New Zealand. These differences are practically significant in the sense that they are larger than minimally important differences for the EQ-5D from the literature, and they appear capable of significantly affecting CUA results.
Fundamental to measuring Quality-Adjusted Life Years (QALYs), as used for cost-utility analysis (CUA), are value sets for health states representing health-related quality of life (HRQoL). A value set comprises HRQoL values for all possible health states representable by the health state classification system concerned – of which well-known examples include the EQ-5D , HUI  and SF-6D . Most systems’ values range between unity for ‘perfect health’ and zero for ‘dead’ , with negative values for states considered worse than dead.
Value sets are created by asking interview or survey participants to value a subset of the system’s health states and then regression models are used to interpolate values for the full set of states. Standard practice is for participants to be drawn from the general population, in their dual roles as tax-payers and potential patients [4–6]. Thus value sets are derived from valuations mostly for hypothetical health states that participants are not personally familiar with and are required to imagine.
A near-universal finding internationally is that patients – i.e. people with direct personal experience of illness or injury – assign different health state valuations than the general population [7, 8]. A literature review by De Wit et al.  and a meta-analysis by Peeters et al.  concluded that patients’ valuations of their own (actual) health states tend to be higher than hypothetical valuations of descriptions of the same states by the general population. In contrast, however, a meta-analysis by Dolders et al. , despite finding some evidence for such relationships, found no statistically significant differences overall.
Most studies in the above-mentioned literature pertain to European and North American countries. To our knowledge, no studies comparing patient and general population valuations include data for New Zealand. And yet the EQ-5D system and Devlin et al.’s value set , which is based on valuations from a 1999 survey of the New Zealand general population, is widely used for CUA in New Zealand, in particular by the Pharmaceutical Management Agency (PHARMAC) for health technology assessments . It would be useful to know, therefore, whether the above-mentioned international finding that patient valuations tend to exceed general population valuations also applies to New Zealand.
In addition, most previous studies have sampled patients with illnesses such as cancer or stroke rather than injuries (as we do in this paper). It is possible that valuations may differ between people experiencing an illness (often accompanied by an ongoing decline in health prior to clinical diagnosis) and those experiencing an acute-onset change in health status, as caused by injury.
This paper compares Devlin et al.’s general population valuations from the 1999 survey with injured New Zealanders’ valuations of their own health (also represented on the EQ-5D). The latter valuations were collected between 2007 and 2009 in the Prospective Outcomes of Injury Study [13, 14]. We also investigate which of the EQ-5D’s dimensions are most strongly associated with the injured population valuations and compare the results with equivalent findings from the 1999 general population study.
Measurement of health state preferences
In both the injured population and general population studies (explained in turn below), health states were represented using the EQ-5D system and participants’ health state preferences were elicited using the visual analogue scale (VAS) method.
Developed by the EuroQol Group, the EQ-5D system represents HRQoL in terms of five dimensions – mobility, self-care, usual activities, pain/discomfort and anxiety/depression – where each dimension has three possible levels of experience: (1) no problems, (2) moderate problems and (3) extreme problems . The three levels combine across the five dimensions to define 243 health states, each representable by a five-digit combination relating to the relevant level within each dimension listed in the order above (e.g. 11111 represents no problems on any dimension). The EQ-5D has been widely used in studies of injury and illness [15, 16].
Participants in both studies were asked to indicate on a VAS depicted as a vertical line marked from 0 (“worst imaginable health state”) to 100 (“best imaginable health state”) where they considered the relevant states were for them. Despite lacking the theoretical underpinnings of the standard gamble or time trade-off methods, the VAS has been empirically verified as a valid method for eliciting health state valuations [17, 18].
Injured population study
Injured New Zealanders’ valuations of their own health on the EQ-5D were obtained from the Prospective Outcomes of Injury Study (POIS), a prospective cohort study of injured people aged between 18 and 64 years [13, 14]. POIS participants were recruited from entitlement claims registered between December 2007 and June 2009 with the Accident Compensation Corporation, New Zealand’s government-controlled, no-fault injury compensation insurance scheme. All injury types and a wide range of severities were eligible for recruitment to POIS, except for people whose injuries resulted from self-harm or sexual assault.
POIS participants (n = 2856) completed a first interview 3.2 months after injury, with follow-up interviews 4.6 and 12.3 months (medians) after injury. At each interview, participants were asked to represent their own health on the EQ-5D’s five dimensions and then to rate it on a VAS.
General population study
The general population valuations of EQ-5D health states come from a self-completed postal survey administered to 3000 adult New Zealanders randomly selected from the electoral roll in 1999  (also see  and ). Three versions of the survey questionnaire were administered, where each version comprised 13 or 14 different health states to be valued on two VAS, with some states repeated across two or more of the versions. Overall, valuations for 23 different health states were collected. For each state, when arriving at their valuation participants were instructed to imagine that it lasts for one year and to disregard what may happen afterwards.
Although 1360 responses were received (a 50% response rate of those who received the survey), 441 were unusable due to: too few health states being valued (because the valuation of so few states undermines their reliability), ‘dead’ being valued as better than full health, or all states being valued the same (considered to be implausible representations of a participant’s preferences). Also, in order to ensure that only data with the strongest claim to validity were analysed, following the approach recommended by Devlin et al. , we excluded a further 523 responses that had two or more logical inconsistencies in their valuations, where a ‘logical inconsistency’ occurs when a health state with a less severe problem on a particular dimension than another state, given its problems on the other dimensions are no more severe, is scored lower (e.g. state 21111 scored below 22111 would be logically inconsistent). The remaining 396 responses represent the highest quality data possible albeit one inconsistency is permitted in order to ensure a reasonable sample size (only 189 responses had no inconsistencies).
In order to calculate health utility values, Devlin et al. rescale each respondent’s valuations to lie on a scale from 0 (representing death) to 1 (representing full health). As hypothetical valuations for death and full health were not collected for POIS participants, we are unable to do a similar rescaling for these data. We therefore use the raw valuations (without rescaling) from the general population study to allow for direct comparisons between the general population and injury population.
Our choice of the appropriate samples from the injured population and general population studies to include in the present analysis was determined primarily by the availability of matching health states for which valuation data were collected – to allow comparison between samples – rather than matching the samples in terms of their socio-demographic characteristics. International evidence has shown that the usual socio-demographic and background variables have no, or at most weak, impact on individual’s health state valuations [11, 21, 22].
Thus, of the 23 health states for which valuations were collected in the general population study, our protocol was to include only those states which at least two people (in practice, four) from the injured population study reported as their own health state. For each of these health states, an independent samples t-test was used to determine whether the mean valuations from the two studies are statistically significantly different. Robust standard errors were used for each of these tests to allow for the possibility of clustering at the individual level (as each individual could value the same health state at multiple time periods), and critical values were adjusted, using the Holm-Bonferroni correction, to account for the large number of comparisons performed.
In addition, to investigate which of the five EQ-5D dimensions are most strongly associated with the injured population valuations, we estimated equations (1) and (2) below. These two equations are analogous to the equations used to derive Devlin et al.’s two value sets , both of which were selected after extensive testing of a wide range of linear and non-linear specifications, following the modelling approach used by Dolan .
where subscript i indexes individuals and t indexes time periods (interviews).
Both equations were estimated with individual fixed effects. Equation (1) is a linear main effects model in which the effect of changes between levels 1 and 2 and levels 2 and 3 (see Table 1 below) is constrained to being identical. Equation (2) is an extension of equation (1), in which an additional detriment to health is introduced if any of the EQ-5D dimensions are at level 3 (extreme problems). The dummy variables to represent each of these effects are described in Table 1.
As explained in Devlin et al. , because the EQ-5D system values health states other than full health as negative deviations from 11111 = 1, VAS it (as explained earlier, rendered on a 0-100 VAS) can be represented as 100 minus the combination of dummy variables and their coefficients (to be estimated). This allowed the dependent variable in equations (1) and (2) to be transformed to 100 - VAS it (so that higher values correspond to ‘worse’ health states).
Consistent with the explanations above, coefficients β1-β5 are interpretable as the decrements to a health state’s valuation of a move from level 1 to 2 or from level 2 to 3 on the respective EQ-5D dimensions, all else being equal. Similarly, coefficient β6 in equation (2) is interpretable as the extra decrement from any dimension being at level 3 (i.e. additional to the effects captured by β1-β5).
This study was approved by the New Zealand Health and Disability Multi-Region Ethics Committee (MEC/07/07/093).
The socio-demographic characteristics of participants in the injured population and general population studies are reported in Table 2. The first column for the injured population study refers to the smaller sample whose valuations were included in the direct comparison of health state valuations (i.e. people who, in at least one of the three interviews, reported a health state that was also included in the general population study), whereas the second column refers to those whose valuations were included in the regression analysis (i.e. those who completed the EQ-5D and VAS at least once in the three interviews).
Of the 2856 participants recruited to the injured population study, 2823 valued their own health on the EQ-5D at the first interview (3-months), 1457 at the second interview (5-months), and 2247 at the third interview (12-months). These three interviews resulted in a total of 6527 valuations, covering 115 of the 243 possible EQ-5D states. Of these 115 states, 18 matched the 23 states included in the general population study, of which 13 were valued by at least two participants, and so they could be included in the present study. Table 3 reports the mean valuations for these 13 states from the two studies.
As can be seen in Table 3, the mean health state valuations of the injured population are higher than the hypothetical valuations of the general population for all 13 health states considered except 11111. This difference, which tends to be larger the ‘worse’ the state, is statistically significant at the 10% level for all states except 11211, 12111, 22233 and 22323.
Table 4 presents the estimates of equations (1) and (2) for the injured population study and also, for comparison purposes, the general population study. For both equations, though the five dimensions all have statistically significant effects on participants’ valuations (of the expected positive sign), their relative magnitudes in the respective studies are very different.
Consistent with our earlier finding (Table 3) that the injured population own health valuations (means) are mostly higher than the general population hypothetical valuations, almost all of the estimated coefficients for both equations are larger for the general population. For injured people, the most important dimension in terms of negative effects on valuations is self-care (SC), followed by anxiety/depression (AD), usual activities (UA), mobility (MO) and pain/discomfort (PD). For the general population, AD is also an important dimension (most important); however, in contrast to the injured population, SC is the least or second-least important dimension (equation 1 or 2), and PD is the second most important dimension (both equations). The additional decrement to health valuations due to extreme problems on any dimension (N3) is much larger in the general population than in the injured population.
Our finding that injured New Zealanders’ valuations of their own health, represented using the EQ-5D system, are mostly higher than general population valuations of the same (hypothetical) states – with the notable exception of state 11111 (discussed below) – is consistent with the previous international research mentioned in the Introduction.
As discussed by Ubel et al. , possible reasons for this phenomenon include: (1) the two groups are, in effect, valuing different health states albeit they are represented identically on the EQ-5D; (2) patients adapt to their poor health states (and so value them relatively high); (3) the ‘focussing illusion’ , whereby members of the general population over-emphasise aspects of HRQoL most affected by illness; and (4) a shift of reference points as people assess health states with reference to their current health. See Ubel et al. for discussion of these and other explanations.
The one exception to the finding that injured population valuations exceed general population valuations is the valuation for state 11111: it is lower for the injured population than for the general population. Consistent with reason (1) above, perhaps this is because for injured people even when, as represented on the EQ-5D, they are at ‘full health’, to them there are other aspects of their health (not captured by the EQ-5D) that are sub-optimal.
Albeit the differences between the injured population and general population valuations – in the range 3.8 to 27.1 (Table 3) – are statistically significant, how practically significant are they? Minimally important differences for the EQ-5D (0-1 scale) have been estimated to be in the range of 0.04 to 0.08 [26–28] – equivalent to 4.0 to 8.0 on a 0-100 scale – which suggests that the above-mentioned differences are also practically significant.
The potential implications of our results for CUA can be illustrated via the example of a CUA of laparoscopic-assisted colectomy (LAC) for colon cancer relative to open colectomy (OC) undertaken by Hayes and Hansen . Applying Devlin et al.’s value set  – as explained earlier, derived from the general population study included in the present study – the authors reported two sets of results corresponding to data from two randomised control trials (RCTs): mean QALY gains of 0.018 and 0.049 per patient (depending on the RCT) and costs per QALY of $70,389 and $25,857 respectively.
These results were based on a simple model in which the authors of the CUA assumed that patients were in state 22222 when discharged from hospital and 11111 when they recovered from both LAC and OC (LAC has a quicker recovery time). State 22222 is worth 0.464 in Devlin et al.’s value set. For the demonstration here, it is sufficient to increase 22222’s value by 0.130, the difference between the injured population and general population means for 22222 (see Table 3) after normalising them to the conventional 0-1 scalea. The effect of 22222 being worth 0.594 instead of 0.464 is to lower the QALY gains from 0.018 and 0.049 (as above) to 0.013 and 0.037 and to raise the costs per QALY from $70,389 and $25,857 to $94,661 and $34,422 respectively. These revised cost-per-QALY estimates are approximately one-third larger than the original estimates. Although cost-effectiveness acceptability thresholds for New Zealand are not reported, such relatively large increases could, conceivably, result in LAC going from being cost-effective (if general population valuations were used) to being not cost-effective (if patient valuations were used). Note, though, the calculations above are intended only to illustrate the possible effects of differences between the injured population and general population valuations, not to serve as a bona fide CUA.
Strengths and weaknesses of the study
The main strength of this study is that, to the best of our knowledge, it is the first comparison of injured people’s valuations of their own health, represented using the EQ-5D system, vis-à-vis general population valuations of the same EQ-5D states for New Zealand.
This study has several potential weaknesses. As discussed in the Methods section, it was not possible to ensure that the samples employed from the general and injured populations were similar in terms of their socio-demographic characteristics; participants in the injured population study were younger, more likely to be male, employed and of Māori, Pacific Islands, Asian or ‘other’ ethnicity (not European or Pakeha). However, as also discussed earlier, international evidence suggests that this should not have a significant effect on health state valuations [11, 21, 22]. Another potential weakness is that the two sets of valuations are not from the same data-gathering exercise. Instead, they were run independently, separated in time by almost a decade (i.e. 1999 versus 2007-09). New Zealanders’ health state preferences could, potentially, have changed over this time period; however, the literature provides no indication as to whether this is likely to be a serious problem. Also, the general population study used a self-completed postal questionnaire whereas the injured population study, as part of the Prospective Outcomes of Injury Study, was mostly interviewer-administered. These differences between data collection methods have the potential to cause information bias; again, we cannot determine the nature or possible extent of this potential bias.
Consistent with the international literature, we find that injured people’s VAS valuations of their own health, represented using the EQ-5D system, are mostly higher than the general population’s hypothetical valuations of the same EQ-5D states for New Zealand. These differences are practically significant in the sense that they are larger than minimally important differences for the EQ-5D from the literature, and they appear capable of significantly affecting CUA results; hence they could potentially influence health technology assessments.
Our results are further evidence that it can make a difference whose health state valuations are included when value sets are being created. However, this does not mean, necessarily, that patients’ (or injured people’s) valuations should be used instead of the general population’s. That is a whole other matter that ultimately depends on the decision-making context and the purpose of the analysis . The consensus seems to be that for decisions at the individual-patient level, such as treatment options, the patient’s own valuations should be used. For decisions at the aggregate level, such as health technology assessments, value sets representing the general population should be used.
aNote that Devlin et al.’s value of 0.464 is not directly comparable with the general population value in Table 3 because Devlin et al.’s value set comes from estimating equation (2) above rather than it being a mean value, and also because, as is conventional when estimating values, the underlying survey valuation data were ‘rescaled’ relative to each participant’s hypothetical valuations of ‘dead’ and ‘perfect health’ (which were not collected in the injured population study).
Anxiety/depression dimension on the EQ-5D
Health-related quality of life
Mobility on the EQ-5D
Extreme problems on any EQ-5D dimension
Pain/discomfort on the EQ-5D
Pharmaceutical Management Agency
Prospective Outcomes of Injury Study
Quality-Adjusted Life Years
Randomised control trial
Usual activities on the EQ-5D
Visual analogue scale.
Brooks R: EuroQol: the current state of play. Health Pol 1996, 37: 53–72. 10.1016/0168-8510(96)00822-6
Feeny D, Furlong W, Boyle M, Torrance GW: Multi-attribute health status classification systems. Pharmacoeconomics 1995, 7: 490–502. 10.2165/00019053-199507060-00004
Brazier J, Roberts J, Deverill M: The estimation of a preference-based measure of health from the SF-36. J Health Econ 2002, 21: 271–292. 10.1016/S0167-6296(01)00130-8
Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB: Recommendations of the panel on cost-effectiveness in health and medicine. JAMA 1996, 276: 1253–1258. 10.1001/jama.1996.03540150055031
National Institute for Health and Clinical Excellence (NICE): Guide to the Methods of Technology Appraisal. NICE; 2008. [http://www.nice.org.uk/media/B52/A7/TAMethodsGuideUpdatedJune2008.pdf]
Pharmaceutical Management Agency (PHARMAC): Prescription for Pharmacoeconomic Analysis. Methods for Cost-Utility Analysis, Version 2.1. PHARMAC; 2012. http://www.pharmac.health.nz/ckeditor_assets/attachments/5/pfpafinal.pdf
Kostopoulou O: The transient nature of utilities and health preferences. Med Decis Making 2006, 26: 304. 10.1177/0272989X06290977
Stamuli E: Health outcomes in economic evaluation: who should value health? Brit Med Bull 2011, 97: 197–210. 10.1093/bmb/ldr001
De Wit GA, Busschbach JJ, De Charro FT: Sensitivity and perspective in the valuation of health status: whose values count? Health Econ 2000, 9: 109–126. 10.1002/(SICI)1099-1050(200003)9:2<109::AID-HEC503>3.0.CO;2-L
Peeters Y, Stiggelbout AM: Health state valuations of patients and the general public analytically compared: a meta-analytical comparison of patient and population health state utilities. Value Health 2010, 13: 306–309. 10.1111/j.1524-4733.2009.00610.x
Dolders MG, Zeegers M, Groot W, Ament A: A Meta-analysis demonstrates no significant differences between patient and population preferences. J Clin Epidemiol 2006, 59: 653–664. 10.1016/j.jclinepi.2005.07.020
Devlin N, Hansen P, Kind P, Williams A: Logical inconsistencies in survey respondents’ health state valuations – a methodological challenge for estimating social tariffs. Health Econ 2003, 12: 529–544. 10.1002/hec.741
Derrett S, Langley J, Hokowhitu B, Ameratunga S, Hansen P, Davie G, Wyeth E, Lilley R: Prospective outcomes of injury study. Inj Prev 2009, 15: e3-e3. 10.1136/ip.2009.022558
Derrett S, Davie G, Ameratunga S, Wyeth E, Colhoun S, Wilson S, Samaranayak A, Lilley R, Hokowhitu B, Hansen P, Langley J: Prospective outcomes of injury study: recruitment, and participant characteristics, health and disability status. Inj Prev 2011, 17: 415–418. 10.1136/injuryprev-2011-040044
Derrett S, Black J, Herbison GP: Outcome after injury-a systematic literature search of studies using the EQ-5D. J Trauma Acute Care Surg 2009, 67: 883–890. 10.1097/TA.0b013e3181ae6409
Polinder S, Haagsma J, Belt E, Lyons R, Erasmus V, Lund J, van Beeck E: A systematic review of studies measuring health-related quality of life of general injury populations. BMC Public Health 2010, 10: 783. 10.1186/1471-2458-10-783
Torrance GW, Boyle MH, Horwood SP: Application of multi-attribute utility theory to measure social preferences for health states. Oper Res 1982, 30: 1043–1069. 10.1287/opre.30.6.1043
Gudex C, Dolan P, Kind P, Williams A: Health state valuations from the general public using the visual analogue scale. Qual Life Res 1996, 5: 521–531. 10.1007/BF00439226
Devlin N, Hansen P, Kind P, Williams A: The health state preferences and logical inconsistencies of New Zealanders: a tale of two tariffs. CHE Discussion Paper 180 2000. The University of York: Centre for Health Economics
Perkins M, Devlin N, Hansen P: The validity and reliability of EQ-5D health state valuations in a survey of Māori. Qual Life Res 2004, 13: 271–274.
Dolan P, Gudex C, Kind P, Williams A: The Measurement and Valuation of Health. First Report on the Main Survey. The MVH Group: Centre for Health Economics, University of York; 1994.
Badia X, Fernandez E, Segura A: Influence of socio-demographic and health status variables on evaluation of health states in a Spanish population. Eur J Public Health 1995, 5: 87–93. 10.1093/eurpub/5.2.87
Dolan P: Modeling valuations for EuroQol health states. Med Care 1997, 35: 1095–1108. 10.1097/00005650-199711000-00002
Ubel PA, Loewenstein G, Jepson C: Whose quality of life? A commentary exploring discrepancies between health state evaluations of patients and the general public. Qual Life Res 2003, 12: 599–607. 10.1023/A:1025119931010
Schkade DA, Kahneman D: Does living in California make people happy? A focusing illusion in judgments of life satisfaction. Psychol Sci 1998, 9: 340–346. 10.1111/1467-9280.00066
Walters SJ, Brazier JE: Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res 2005, 14: 1523–1532. 10.1007/s11136-004-7713-0
Pickard AS, Neary MP, Cella D: Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer. Health Qual Life Out 2007, 5: 70. 10.1186/1477-7525-5-70
Luo N, Johnson JA, Coons SJ: Using instrument-defined health state transitions to estimate minimally important differences for four preference-based health-related quality of life instruments. Med Care 2010, 48: 365–371. 10.1097/MLR.0b013e3181c162a2
Hayes JL, Hansen P: Is laparoscopic colectomy for cancer cost-effective relative to open colectomy? ANZ J of Surg 2007, 77: 782–786. 10.1111/j.1445-2197.2007.04226.x
We are grateful to the participants in the 1999 New Zealand general population study and the 2007-09 Prospective Outcomes of Injury Study (POIS) for sharing their information with us. Thank you also to Susan McAllister for comments on an earlier draft of the paper, and to the journal’s three referees and editorial team. The 1999 general population study was funded by an Otago Research Grant (98/603). POIS, of which the injured population study included in the present study was a part, was funded by the Health Research Council of New Zealand (2007-13) and co-funded by the Accident Compensation Corporation, New Zealand (2007-10). The views and conclusions in this article are the authors’ and may not reflect those of the funders. POIS received ethical approval from the New Zealand Health and Disability Multi-region Ethics Committee (MEC/07/07/093).
All authors declare that they have no financial interests that may be relevant to the submitted work. SD is a member of the EuroQol Scientific Committee, which developed the EQ-5D instrument used in the study.
RW led the preparation of this article, with guidance from PH and SD, and conducted the statistical analysis. SD leads the Prospective Outcomes of Injury Study (POIS) research team. PH and JL are POIS co-investigators. All authors contributed to the writing of the final manuscript and approved it.
Ross Wilson, Paul Hansen, John Langley and Sarah Derrett contributed equally to this work.