Measures and questions used
EQ-5D-5L
EQ-5D-5L [11] was used as the key outcome measure in this study, with the administration of both the health descriptive system and visual analogue scale (VAS) compared across the modes. The EQ-5D-5L is an updated version of the three level EQ-5D (EQ-5D-3L) [12], and assesses health status across five dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression) each with five response levels (none, slight, moderate, severe and extreme/unable). The VAS assesses self-reported health status on a 0 to 100 scale where 0 is equivalent to worst imaginable health and 100 is equivalent to the best imaginable health.
The EQ-5D-3L descriptive system has an associated utility ‘value set’ based on the preferences of the general population that allows the measure to be used in the calculation of Quality Adjusted Life Years (QALYs) for use in the economic evaluation of health interventions, and is the measure accepted by reimbursement bodies such as the United Kingdom National Institute of Health and Care Excellence (NICE) and the Australian Pharmaceutical Benefits Advisory Committee (PBAC) for this purpose [13]. The UK value set was developed using the preference elicitation technique Time Trade Off (TTO), and ranges from 1 (for the ‘best’ health state described with no problems on each dimension (11111))–0.594 (for the worse health state with extreme problems (33333)). On this scale, 0 is equivalent to dead, and negative values given to states perceived as worse than being dead (following modelling of the TTO data) [14].
The EQ-5D-3L value set can be mapped onto EQ-5D-5L health states where the best and worst health states are anchored to the EQ-5D-3L utility scale [15]. This ‘crosswalk’ value set is used in the analysis carried out in this study and allows for an assessment of equivalence across modes using a value set directly linked to the EQ-5D-3L.
EQ-5D-5L was used in this study as it is a widely used, concise and validated generic COA that can be used both with the general population and across patient groups. Furthermore, the increasing use of EQ-5D-5L both in cost effectiveness analysis (as part of trials and other studies), population surveys and in routine healthcare settings mean that it is an instrument that can be widely administered using a range of administration modes. Therefore evidence regarding equivalence across modes is important, both for providing evidence for the use of the different modes and equivalence of responses, and also for the use of EQ-5D-5L.
The EQ-5D-5L and VAS were adapted for delivery via mobile technology by the project team (see online supplement 1), with the final version agreed by the EuroQol Group (the EQ-5D-5L copyright holders). The paper version was matched with that recommended for use by the EuroQol Group. In contrast to the paper version, it is recommended that each of the five dimensions appear on a separate page. Slight adaptations to the instructions were made to reflect the fact that a touchscreen was been used. The VAS was answered by pressing on the relevant number on the scale, and the number automatically appeared in the answer box, and could be changed before proceeding. The paper VAS requires respondents to fill in the relevant number in the available box. Each question had to be answered before proceeding, and respondents were able to scroll back to change their answers during the completion of the survey, but not once they had finished the questionnaire. Respondents could stop during the completion of the EQ-5D-5L by just exiting the application containing the measure on the phone, but we could not record this information. Due to the nature of the data collection process, questions on the paper version were not compulsory and changes could be made at any time during the completion of the overall questionnaire.
Demographic and wellbeing questions
A range of demographic and health questions were collected from all respondents using a paper questionnaire (that was appended to the EQ-5D-5L for the paper arm, and completed as a separate questionnaire following the EQ-5D-5L for the mobile arm). This included questions about age, gender, marital status, education, and long standing health conditions. The four subjective wellbeing and satisfaction questions used by the Office of National Statistics (ONS) as part of the household survey [16] were also collected, with the addition of an extra question. The questions examined health satisfaction, life satisfaction, happiness and anxiety yesterday, and whether life is perceived as worthwhile on a 10 point scale from ‘not at all’ (0) to ‘completely’ (10).
Usability questions
Questions investigating the usability and acceptability of the mobile phones were also included. These investigated whether respondents would use phones to complete COAs again, whether the questionnaire was easy to complete and read, whether the touchscreen could be used, and also the usability of the questionnaire (see online supplement 2). Paper arm respondents were asked whether they would complete COAs such as EQ-5D-5L using a mobile phone.
Study design
In this study a parallel groups design was used, with respondents randomly allocated to either the mobile (n = 100) or paper (n = 100) completion arm. The paper questionnaire and mobile device (if relevant) were then sent to the respondents. Detailed instructions about how to access the mobile questionnaire, and the order in which the questions should be completed, were included. After completion, respondents were asked to return the questionnaire and mobile device to the study team using a freepost envelope provided. If there was no response, or if the mobile questionnaire was completed but the device was not returned, then a follow up email (if possible) or letter was sent. This procedure was approved by the School of Health and Related Research, University of Sheffield, ethics committee.
Sampling and recruitment
Participants in this study were recruited from the Yorkshire Health Study (YHS) [17] between February and September 2014. The YHS is a large scale longitudinal study collecting information over two waves about the health of residents in Yorkshire and Humberside areas of the United Kingdom. The first wave contains records for 27,806 individuals (2010–12), aged between 16 and 85 from South Yorkshire. As part of the survey, respondents are asked to indicate whether they are willing to take part in further research projects, and respondents in this study were identified from those aged over 18 who agreed to be contacted. Eligible respondents were identified and stratified across five age groups (18–30; 31–40; 41–50; 51–60 and 60+) and gender. A random sample of 200 respondents across the age groups and who self-reported a range of health conditions was then selected and randomly allocated to each arm.
Analysis
The aim of the analysis was to assess the acceptability and usability of the mobile EQ-5D-5L, and compare the scores produced across different samples, in line with the recommendations of Coons and colleagues [10] for comparing COA equivalence. Proportion differences across the various demographic indicators and EQ-5D responses were tested using chi square analysis. ANOVA difference testing was used to assess the equivalence of EQ-5D-5L utility and VAS scores across the arms. The magnitude of the mean difference between the utility and VAS scores was also assessed in comparison to an estimated value for the minimally important difference (MID) of the EQ-5D-5L. The estimate was calculated from the study sample, as currently no estimate of the MID if EQ-5D-5L in the general population using the crosswalk tariff is available. The value range for the MID was calculated by multiplying the pooled standard deviation of the EQ-5D-5L utility and VAS scores by 0.2 (to estimate the lower bound) and 0.49 (to estimate the upper bound). This following Cohen’s effect size guidelines [18], where a small effect size between 0.2 SD and 0.49 SD may represent a MID range, and mean differences between arms at or below this range suggests equivalence.
Linear regression was used to assess whether the study arm and a range of background characteristics significantly impacted on EQ-5D-5L utility score (while holding the other indicators included in the model constant). This took the form:
$$ \mathrm{y}=\mathrm{X}\upbeta +\upvarepsilon $$
where y is the utility value, X represents the explanatory sociodemographic variables, and ϵ represents the error term capturing other factors. For each of the utility estimations and the VAS, three models were estimated: study arm and socio-demographic variables (models 1, 4 and 7); study arm and self-reported health variables (models 2, 5 and 8); study arm, demographics and self-reported health variables (models 3, 6 and 9).
EQ-5D-3L data from the respondent’s completion of the first wave YHS questionnaire (between June 2010 and April 2011) were used to compare health differences amongst those who did or did not respond. Differences in scores across the time points were also compared.