Skip to main content
Download PDF
Download ePub

A pilot study on the validity and psychometric properties of the electronic EQ-5D-5L in routine clinical practice

Health and Quality of Life Outcomes volume 19, Article number: 266 (2021) Cite this article

Abstract

Background

Electronic measurement of health-related quality of life (HRQOL) may facilitate timely and regular assessments in routine clinical practice. This study evaluated the validity and psychometric properties of an electronic version of the EQ-5D-5L (e-EQ-5D-5L) in Chinese patients with chronic knee and/or back problems.

Methods

151 Chinese subjects completed an electronic version of the Chinese (Hong Kong) EQ-5D-5L when they attended a primary care or orthopedics specialist out-patient clinic in Hong Kong. They also completed the Chinese Western Ontario and McMaster University Osteoarthritis Index (WOMAC), a Pain Rating Scale, and a structured questionnaire on socio-demographics, co-morbidities and health service utilization. 32 subjects repeated the e-EQ-5D-5L two weeks after the baseline. 102 subjects completed e-EQ-5D-5L and 99 completed the Global Rating on Change Scale at three-month clinic follow up. Construct validity was assessed by the association of EQ-5D-5L scores with external criterion of WOMAC scores. We tested mean differences of WOMAC scores between adjacent response levels of the EQ-5D-5L dimensions by one-way ANOVA, test–retest reliability by intra-class correlation, sensitivity by known group comparisons and responsiveness by changes in EQ-5D-5L scores over 3 months.

Results

There was an association between EQ-5D-5L and WOMAC scores. Mean WOMAC scores increased with the increase in adjacent response levels of EQ-5D-5L dimensions. Test–retest intraclass correlation coefficient (ICC) of EQ-5D-5L utility and EQ-VAS scores were 0.76 and 0.83, respectively, indicating good reliability. There were significant differences in the proportions reporting limitations in the EQ-5D-5L dimensions, the utility and VAS scores between the mild and severe pain groups (utility = 0.28, p = 0.001; VAS = 11.46, p < 0.001), and between primary care and specialist out-patient clinic patients (utility = 0.15, p = 0.001; VAS = 10.21, p < 0.001), supporting sensitivity. Among those reporting ‘better’ global health at three-months, their EQ-5D-5L utility and EQ-VAS scores were significantly increased from baseline (utility = 0.18, p < 0.001; VAS = 10.75, p = 0.005).

Conclusions

The electronic version of the EQ-5D-5L is valid, reliable, sensitive and responsive in the measurement of HRQOL in Chinese patients with chronic knee or back pain in routine clinical practice.

Background

Health-related quality of life (HRQOL) is the assessment of aspects of quality of life influenced by an individual's health [1]. HRQOL can be used as an outcome measure to assess the impact of illnesses and the effect of interventions on patients, monitor the health conditions of individual patients, and evaluation of quality of care [2,3,4]. The EQ-5D is a widely used HRQOL measure. It was first developed in Europe and later adapted to many other languages and cultures including Chinese in mainland China and Hong Kong [5,6,7,8]. It has been shown to be valid, reliable and responsive in general populations and specific patient groups in different cultures around the world [9,10,11,12,13].

The EQ-5D includes five items where respondents self-report any problems in relation to mobility, self-care, daily activities, pain/discomfort, and anxiety/depression. The original version, the EQ-5D three-level (EQ-5D-3L), contains three response options: ‘no problems’, ‘some/moderate problems’ and ‘extreme problems/unable to’ for each of the five items [14, 15]. In 2011, the EQ-5D-3L was updated to the EQ-5D five level (EQ-5D-5L), which increased the response options from three to five (‘no problems’, ‘slight problems’, ‘moderate problems’, ‘severe problems’, ‘extreme problems/unable to’) to enhance sensitivity [16, 17]. The EQ-5D has been shown to be a useful HRQOL measure in providing a more holistic picture of the health of patients [11, 18], for monitoring responses to treatment/surgery [19, 20], assessing the quality of care [21] and for health-economic evaluation [19]. In addition, the completion of HRQOL measures has been found to enable patients to be more aware of their health conditions and how the diseases affect them, which empower them to raise any issues or concerns with their clinicians [22, 23].

There have been increasing attempts to incorporate HRQOL measures in routine clinical care [18,19,20], however many barriers have been encountered, including high workload of staff [3, 4, 23] and a lack of time to collect, analyze and interpret the data [24]. Furthermore, some clinicians have questioned the validity and sensitivity of HRQOL data and are concerned that implementing HRQOL assessment may disrupt patient care [3] and increase patient burden [23]. Given these barriers, there have been calls for ways in which HRQOL data can be more effectively integrated into routine clinical practice [4, 25]. One such method is through electronic data collection and reporting [4]. Aside from the benefit of reducing workload and time burden on staff, it can also allow clinician’s immediate access to the results and tracking of changes [3, 18].

An electronic version of the EQ-5D-5L (e-EQ-5D-5L) has been available since 2014 [26] and many studies have applied e-EQ-5D-5L to measure HRQOL outcomes [27,28,29,30]. Although there is a large body of literature supporting the validity and psychometric properties of paper versions of EQ-5D [12, 13, 31,32,33], there are few such data on e-EQ-5D-5L. Our literature search found one recent study in English and French asthma patients reporting the validity and psychometric properties of e-EQ-5D-5L [34] but no study in the Chinese population. A change in the mode of administration can affect the validity, reliability and other psychometric properties of an instrument. Electronic administration can be challenging for many older Chinese patients in Hong Kong who have low education levels and are not familiar with computer technology. It is essential to confirm the validity and psychometric properties of e-EQ-5D-5L before it can be applied to clinical practice in Hong Kong and other Chinese populations especially in settings that have large elderly patient populations.

This pilot study aimed to test the validity and psychometric properties of e-EQ-5D-5L as a measurement of the HRQOL of patients with chronic knee and/or back problems in routine clinical practice. The objectives were to evaluate the construct validity, test–retest reliability, sensitivity and responsiveness of e-EQ-5D-5L among Chinese patients with chronic knee and/or back problems attending outpatient clinics in Hong Kong.

Methods

Study design, subject recruitment and data collection

This was a prospective longitudinal cohort study. We recruited patients with chronic knee and/or back problems by convenience sampling when they attended a public primary care general out-patient clinic (GOPC) and a public orthopedics specialist out-patient clinic (SOPC) in Hong Kong between August and November of 2018. Eligible patients were invited by either their doctors or trained research assistants to join the study. These public outpatient clinics were busy with an average workload of 6 to 8 patients per hour per doctor, therefore we could not invite all eligible patients who attended the clinics during the study period due to manpower constraints. The subject inclusion criteria were: 1) adults aged 18 years or above; 2) a doctor-diagnosed symptomatic chronic (≥ one month) knee and/or back problem; 3) attending the clinic for a doctor consultation and was scheduled for a follow-up visit to the clinic within 12 months; 4) able to communicate in Chinese; and 5) able to provide written consent to participate. Patients whose life expectancies were estimated to be less than 3 months by the attending doctors, or those who were too ill (either physically or cognitively) to complete the questionnaires, or those who were not willing or unable to give consent were excluded.

All subjects completed a written informed consent before participating in the study. Each subject was assigned a unique QR code for access to the e-EQ-5D-5L survey, and completed the electronic version of the Chinese (Hong Kong) EQ-5D-5L and EQ-VAS (e-EQ-5D-5L) online through an iPad that was connected to a central server via the clinic public Wi-Fi. One item was presented per screen and the subject could choose to move to the next item after completion or to skip the item. The original 200 mm EQ-VAS was modified to 100 mm to fit into the iPad screen. The detailed administration method of the e-EQ-5D-5L with screenshots is shown in the Additional file 1: Appendix 1. Trained research assistants (RA) were present on site to provide technical assistance and to read out the questions to respondents as required by some elderly subjects who had low literacy level or poor eyesight. Immediately after the subject had completed the e-EQ-5D-5L, the RA retrieved the report summarizing the EQ-5D dimension, utility and VAS scores from the server and printed a copy of the report for the consulting doctor’s information. Most subjects completed the e-EQ-5D-5L survey before seeing their doctors so that they could show the reports to their doctors during the consultations. A few subjects who were recruited by the doctors completed the survey after the doctor consultation. In addition to the e-EQ-5D-5L, subjects completed the paper-based WOMAC, Pain Rating Scale and a structured questionnaire on socio-demographics, co-morbidities (self-reported doctor-diagnosed chronic diseases with a duration of ≥ one month) and health services utilization.

We invited the first 51 subjects recruited from the GOPC to return to the clinic two weeks after their baseline visit to repeat the e-EQ-5D-5L to evaluate test–retest reliability. All subjects (including the first 51 subjects) who attended the clinics for follow up around 3 months post-baseline were asked to complete the e-EQ-5D-5L and Global Rating Scale on change in health (GRS). We took 3 months as the interval for reassessment as it is standard practice in the GOPC that patients with stable chronic problems are followed up every 3 months. On the other hand, the SOPC follow up interval would usually be longer for stable cases.

Research ethics approval was obtained from the institutional review board prior to subject recruitment. (HKU/Hospital Authority Hong Kong West IRB reference number: UW 18–270).

Study instruments

The Chinese (Hong Kong) EQ-5D-5L

The EQ-5D-5L comprises five items representing five HRQOL dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression) and a Visual Analogue Scale (EQ-VAS) on global health. The responses to the five EQ-5D-5L items have a combination of 3125 (55) health states [5, 16]. Each health state can be converted to a composite utility (preference) score from 0 (death) to 1 (perfect health), with a scoring algorithm derived from population-based valuation. The Chinese-Traditional (Hong Kong) translation of the EQ-5D-5L and the Hong Kong population specific EQ-5D-5L value set have been developed and normed on the local population [6, 8]. The EuroQol Group full version of EQ-5D-5L (Web version)—Chinese-Traditional (Hong Kong) was adapted for electronic administration. The EQ-VAS was modified from the original 200 mm to a 100 mm scale from 0 (the worst imaginable health state) to 100 (the best imaginable health state), in order to fit into the iPad screen.

Additional PROMs administered

  1. a.

    The Western Ontario and McMaster University Osteoarthritis Index (WOMAC) is a widely used condition-specific HRQOL measure to assess pain, stiffness and difficulty in physical functioning among patients with musculoskeletal conditions. It has been administered to patients with hip and/or knee osteoarthritis [35] and low back pain [36]. It consists of 24 items in 3 domains: pain (5 items), stiffness (2 items) and physical function (17 items). Each item is rated on a 5-point Likert scale, ranging from 0 to 4, with higher scores indicating more symptoms or greater impairment. The item scores in each domain are summated as the domain score. The total WOMAC score is the sum of the three domain scores [37]. A Chinese version of WOMAC is available and has been shown to be valid, reliable and sensitive in Chinese patients [37, 38].

  2. b.

    The Pain Rating Scale was administered to assess the severity of pain, scores range from 0 (no pain) to 10 (the worst pain).

  3. c.

    The Global Rating Scale on change in health (GRS) was used to assess the patient’s perception of any change in their overall health condition on a 7-point scale, ranging from much worse [1] to much better [7] at their 3-month follow up [39].

Statistical analysis

Data were analyzed using IBM SPSS version 26. Statistical significance was set at a p value of < 0.05. Construct validity of the e-EQ-5D-5L was assessed by its association with the external criterion of WOMAC, based on the hypothesis that subjects with a higher level of problem/impairment in the EQ-5D-5L dimensions would have higher WOMAC domain and total scores if the e-EQ-5D-5L is a valid measure of HRQOL. To assess correlations between e-EQ-5D-5L and WOMAC, one-way analysis of variance (ANOVA) along with post-hoc least significant difference was applied to compare the mean differences of WOMAC scores across levels of EQ-5D-5L dimensions, and between adjacent response levels (level 1 vs 2 vs 3 +).

Test–retest reliability of the e-EQ-5D-5L utility and EQ-VAS scores was assessed by intra-class correlation (ICC). A standard of ≥ 0.7 signifies good reliability [40]. Mean differences in EQ-5D-5L utility and EQ-VAS scores between baseline and 2-week re-test were also assessed by paired t tests. Test–retest reliability of the EQ-5D-5L dimension levels was assessed by examining the Gwet’s agreement coefficients (AC) and degree of agreement for five individual EQ-5D-5L dimension responses. A Gwet’s AC and degree of agreement of < 0.2 was interpreted as poor reliability between two assessments, 0.21–0.4 as fair, 0.41–0.6 as moderate, 0.61–0.8 as good and ≥ 0.8 as very good[41].

Sensitivity was measured by the ability of the e-EQ-5D-5L to detect a difference between groups (mild pain versus severe pain groups, GOPC versus SOPC groups, and knee pain versus back pain groups), tested by two-sample t tests. We also assessed the magnitude of the difference by Cohen's effect size [42], calculated as the difference between mean scores, divided by pooled standard deviations (SD).

We conducted trajectory analyses fitting censored normal mixture models to determine the changes in EQ-5D-5L utility and EQ-VAS scores at baseline and 3 months after baseline, and disaggregated subjects into trajectory classes. The best model with up to five classes was selected using the Bayesian Information Criteria (BIC) [43].

For assessing responsiveness, we hypothesized that participants with “better” Global Rating Scale scores would have an increase in the EQ-5D-5L utility and EQ-VAS scores and that those with “worsened” GRS scores would have reductions in these EQ-5D-5L scores. We categorized the responses of 1 (much worse), 2 (worse) and 3 (a little worse) as the “worsened”; 4 (No change) as the “same”; and 5 (a little better), 6 (better) and 7 (much better) as the “better” groups, respectively. Mean changes in EQ-5D-5L utility and EQ-VAS scores measured during follow-up visits at the clinics around 3 months after baseline in subjects with GRS ‘worsened’, ‘same’ and ‘better’ health were calculated and evaluated by paired t-tests and Cohen’s effect size. Chi squared tests were used to compare the difference in changes in the proportions of reported limitations in the EQ-5D-5L dimensions among the better, same and worsen groups in GRS.

Results

Subject characteristics

A total of 151 adult subjects with chronic knee and/or back problem (101 from GOPC and 50 from SOPC) participated in this study. Thirty-two subjects from the GOPC repeated the electronic EQ-5D-5L two weeks post-baseline. 104 subjects had attended follow-up consultations around 3 months at the clinics, while 47 subjects (14 GOPC patients, 33 SOPC patients) did not because they were not due for follow up or they defaulted their appointments. We missed the follow up of two GOPC subjects who attended follow up consultations during the weekends when our research assistants were off duty. 102 subjects (85 from GOPC and 17 from SOPC) completed the three-month follow-up assessment, but 3 subjects from the SOPC group did not respond to the GRS. Hence data from only 99 subjects were included in the analysis on responsiveness. The subject recruitment and follow-up flow diagram is shown in Fig. 1.

Fig. 1
figure 1

Study flow diagram

Full size image

Baseline socio-demographic and clinical characteristics of the subjects are presented in Table 1. Overall, the subjects were mostly older adults (mean age: 64.8 years ± 9.23, range 36 to 89 years old), and many (45%) had low education levels of primary school or less. In terms of diagnoses, 35% had chronic back problems, 58% had chronic knee problems, and 6.6% of subjects had both. Subjects from the SOPC were relatively younger (mean age: 61.5 years ± 9.73) with the female gender predominant (74%) when compared to subjects from the GOPC. SOPC patients had a higher mean WOMAC and pain rating scores indicating more severe diseases than those of GOPC patients. The number and percentage of subjects reporting at different levels of the EQ-5D-5L at baseline are also presented in Table 1.

Table 1 Baseline characteristics of study subjects in 2018 (N = 151)
Full size table

Validity and reliability

All subjects completed the e-EQ-5D-5L with no missing data at baseline. The mean completion time was 129.9 s (SD: 59.3; range 40 to 402 s). As presented in Table 2, the sign of mean differences in WOMAC scores between each EQ-5D-5L adjacent response levels were in the same direction. The differences in WOMAC scores between adjacent response levels of the EQ-5D dimensions were significant except the mean WOMAC Stiffness scores between level 1 and 2 of Mobility, level 2 and 3+ of Self-Care, level 2 and 3+ of Usual Activities and level 1 and 2 of Pain/Discomfort. There were significant correlations between the total WOMAC score and the EQ-5D-5L utility (-0.628) and EQ-VAS (-0.485) scores. We explored whether educational level had an effect on the e-EQ-5D-5L results, and found no statistically significant difference in the EQ-5D-5L, utility and EQ-VAS scores among three education level (primary or less, secondary and tertiary) groups.

Table 2 Comparison of WOMAC scores among EQ-5D-5L response levels at baseline in 2018 (N = 151)
Full size table

Table 3 showed the test–retest results of the Intraclass Correlation Coefficient (ICC) of EQ-5D-5L utility and EQ-VAS scores being 0.76 and 0.83 respectively, signifying good reliability. The proportions of level of agreement for each EQ-5D-5L dimension response between baseline and the 2-week re-test were as follows: 59% (mobility), 72% (self-care), 59% (usual activities), 59% (pain/discomfort), and 66% (depression/anxiety), indicating moderate to good reliability.

Table 3 Test–retest reliability of the electronic EQ-5D-5L in subjects from GOPC 2018 (N = 32)
Full size table

Sensitivity

Table 4 shows the sensitivity of the e-EQ-5D-5L dimensions, utility and EQ-VAS in detecting a difference between different known groups. The mild pain group had significantly lower proportions of subjects reporting limitation (moderate, severe and very severe responses) in all five EQ-5D dimensions and significantly higher utility scores and EQ-VAS scores, compared to the severe pain group. When compared to subjects from the SOPC, subjects from the GOPC had significantly lower proportions of limitation in the EQ-5D dimensions of usual activities, pain and anxiety/depression; and had significantly higher utility scores and EQ-VAS scores. Subjects with knee problems reported a significantly lower proportion of limitation in the EQ-5D dimensions of usual activities, pain and anxiety/depression and significantly higher utility scores and EQ-VAS scores when compared with subjects with back pain. The effect sizes of group differences in EQ-5D-5L utility and VAS scores were moderate to large (0.50 to 1.14), indicating high sensitivity.

Table 4 Sensitivity of electronic EQ-5D-5L by known group comparison at baseline in 2018 (N = 151)
Full size table

Responsiveness

A three-class trajectory model had the best fit for both the longitudinal data of the EQ-5D-5L utility and EQ-VAS scores of 102 subjects according to the BIC. In the EQ-5D-5L utility score model, the class 1 (from middle at baseline to low at follow-up), 2 (from low at baseline to middle at follow-up) and 3 (persistently high) included 10.0%, 8.7%, and 81.2% subjects, respectively. In the EQ-VAS score model, the class 1 (persistently low), 2 (persistently middle) and 3 (persistently high) included 28.6%, 69.9%, and 1.6% subjects, respectively. Plots of the EQ-5D-5L utility and EQ-VAS trajectories are shown in Fig. 2. There were differences in the baseline age, diagnosis and clinic setting among the three EQ-5D Utility classes, with class 3 (persistently high utility) subjects more likely to be younger, diagnosed to have knee problems and attending GOPC. The details are shown in Additional file 2: Tables S1a and S1b.

Fig. 2
figure 2

Trajectory analysis

Full size image

We evaluated the changes in the EQ-5D-5L utility and EQ-VAS scores and the EQ-5D response level proportions by GRS groups among 99 subjects who had completed both the e-EQ-5D-5L and GRS during their 3-month clinic follow up (Table 5). There were significant increases in mean EQ-5D utility and mean EQ-VAS scores from baseline to three-month follow-up among the GRS ‘better’ group. The effect sizes of change for this group were moderate (utility = 0.666 and VAS = 0.664). There were expected negative changes in both mean EQ-5D-5L Utility score and the VAS score in the GRS ‘worse’ group (effect sizes being 0.280 and 0.296 for utility score and EQ-VAS score, respectively) but the differences did not reach statistical significance. There was also a significant increase in the EQ-5D utility score at 3 months in the GRS “same” group. When looking into the changes in the EQ-5D-5L dimensions, as expected, the GRS ‘better’ group showed a decrease in the proportion of subjects who reported to have limitations/problems across all dimensions whereas an increase was noted amongst subjects who reported ‘worse’ on the GRS. The differences in changes in the proportions of limitations among the GRS groups were statistically significant.

Table 5 Change in electronic EQ-5D scores from baseline to 3 months follow up by GRS groups, Aug 2018 – Mar 2019 (N = 99)
Full size table

Discussion

To the best of our knowledge, this was the first study evaluating the validity and psychometric properties of an electronic version of the EQ-5D-5L in clinical practice in a Chinese population. Our study results demonstrated that the e-EQ-5D-5L was valid, reliable, sensitive and responsive among patients with chronic knee and/or back problems, many of whom were elderly with low education levels. It was reassuring to find that there was no significant difference in EQ-5D-5L scores among subjects with different education levels. The results support the application of the e-EQ-5D-5L in clinical practice, which has the potential to overcome many implementation barriers associated with data collection by paper-based EQ-5D-5L questionnaires, particularly the workload and time to collect, analyze and interpret the data [24]. Another advantage of e-EQ-5D-5L is instantaneous data analysis and generation of a report on the longitudinal data on the HRQOL dimension, utility and VAS outcomes, which can be available at the point of care to support clinical decisions.

As there is no gold standard measure of HRQOL, we could only infer validity of the e-EQ-5D-5L for musculoskeletal problems by comparing the results with those of a musculoskeletal disease specific HRQOL measure, namely WOMAC. The construct validity of the e-EQ-5D-5L was supported by its association with WOMAC scores. The sign of mean differences in WOMAC scores between each adjacent response level in EQ-5D-5L were in the same direction indicating that both measures were measuring the same construct, HRQOL. Due to the small number of respondents (n < 10) in EQ-5D-5L levels 4/5, we grouped the responses of 3/4/5 levels of the EQ-5D-5L dimensions as the ‘3+’ category to increase statistical power for the ANOVA analysis. Validity was further supported by a significant correlation between the EQ-5D Utility and EQ-VAS scores and the WOMAC total score. A study on the validity of the paper version of the EQ-5D-5L among UK patients with rheumatoid arthritis showed similar findings [44], which suggested that the electronic mode of administration did not affect the validity of the EQ-5D-5L in patients with musculoskeletal problems. We noted the difference in the WOMAC scores between subjects who reported level 1 and 2 in the EQ-5D Pain dimension was not significant, but that between levels 1 and 2 of most other EQ-5D dimensions were, which suggest non-linearity either associated with the "gap" between EQ-5D response levels, or to accommodation to pain. The other finding of interest was the Stiffness subscale in WOMAC did not "perform" well against the EQ-5D-5L. One possible explanation is that stiffness was not a significant problem among our subjects who mostly had non-inflammatory knee or back problems. The other explanation is that stiffness may be a subordinate dimension that is indirectly measured through pain/function.

As HRQOL measures are often used to monitor change over time, or with intervention, it is essential for the instrument to have inter-rater/test–retest reliability so that any difference on repeated measurements is a true change in the person’s HRQOL but not measurement variations [29]. Our test–retest ICC results were similar to findings on the paper versions of EQ-5D in Korea [45] and the UK [44]. There was also good test–retest agreement in the results for all five dimensions of the EQ-5D-5L. The findings assure the consistency of the subjects’ responses even when the EQ-5D-5L is presented in an electronic mode that they are not familiar with.

The e-EQ-5D-5L was able to detect significant differences between different known groups, as hypothesized. The effect sizes of the differences in the EQ-5D-5L utility and VAS scores between the known groups were moderate to large (0.50–1.14), suggesting they were likely to be clinically important [46]. It is expected that subjects with mild pain were less likely to report limitations or problems in the EQ-5D-5L dimensions. They also had higher EQ-5D-5L utility scores than those with severe pain. The e-EQ-5D-5L detected lower proportions with HRQOL limitations and higher utility and VAS scores in GOPC than SOPC subjects, which is consistent with the conventional practice that patients with milder problems are managed in primary care. The e-EQ-5D-5L utility and VAS scores showed statistically significant differences between patients with knee and back problems, with moderate effect sizes of 0.52 and 0.50, respectively. A higher proportion of the subjects with back problems were patients from the SOPC who tended to have more severe diseases. Apart from disease severity, other differences in the characteristics of subjects between primary and specialist care clinics could have affected the EQ-5D-5L results, but the small sample in this study did not have the power for clinic-diagnosis subgroup analysis. Further studies should be carried out to identify the other factors associated with HRQOL of chronic knee and/or back patients. The EQ-5D-5L specifically identified significantly more limitations in the pain, usual activities and anxiety/depression but not in mobility or self-care in the patient group with back problems than those with knee problems. The patient’s HRQOL profile can help clinicians identify specific areas of need so that the management can be more tailor-made. In addition to pain relief, strategies to enhance functioning in daily activities and to relieve psychological distress deserve more attention in the care of patients with chronic knee and back problems.

The responsiveness of e-EQ-5D-5L was established in the three-month follow-up measurement. The trend of change was consistent with those measured by the GRS, in that subjects who reported their global health had got better had an increase in EQ-5D-5L utility and EQ-VAS scores and a decrease in the proportions reporting limitations/problems in the EQ-5D-5L dimensions, and vice versa among subjects who reported their global health had got worse. The 3-month changes in the EQ-5D-5L utility and EQ-VAS scores were statistically significant and the effect sizes were moderate (0.666 and 0.664) in the GRS better group. The effect size changes in the EQ-5D-5L utility and EQ-VAS scores were smaller (0.28 to 0.30) in the worsen group and the difference did not reach statistical significance in this small sample. Our findings were consistent with those found in a systematic review by Payakachat et al. in that the EQ-5D was responsive to changes in musculoskeletal and pain conditions more consistently in detecting improvement than deterioration [47]. We noted a statistically significant increase in the EQ-5D-5L utility score in the group who reported the same global health at 3 months. The EQ-VAS score also showed an increase among the GRS same group although the difference was not statistically significant. One possible explanation is that the multi-dimensional EQ-5D-5L is more responsive than a transitional measure on change in global health in detecting a small HRQOL improvement. On the other hand, a small change in the EQ-5D-5L utility score could be “noise” that may not truly reflect a real change. Further studies using different external anchors are required to establish the minimal clinically important change of the EQ-5D-5L.

Strengths and limitations

To the best of our knowledge, this study was the first to establish the validity and psychometric properties of an electronic version of the EQ-5D-5L as a HRQOL measurement in clinical practice amongst Chinese patients. Knee and back problems are the most common musculoskeletal problems and we included subjects from primary and specialist care who had a broad spectrum of disease severity. We were able to demonstrate the applicability and validity of e-EQ-5D-5L in elderly patients with low education levels who are not as familiar with computer technology. We therefore believe that the e-EQ-5D-5L is likely to be valid in other Chinese patients with musculoskeletal problems.

Our study had some limitations in that the sample size was small, the follow-up period was short (3 months only) and subjects from only two public outpatient clinics were included. We did not use the paper EQ-5D-5L as a ‘gold standard’ criterion to test the validity and concordance of e-EQ-5D-5L, as to do so would require a larger randomized controlled study. Results on validity and psychometric properties do not necessarily imply the e-EQ-5D-5L data are clinically useful. Further studies with longer follow-up period and larger samples from different clinical settings should be carried out to establish the usefulness and acceptability of e-EQ-5D-5L in measuring HRQOL in routine clinical practice. Specifically, we need to determine whether the HRQOL data measured by e-EQ-5D-5L is useful in improving the health outcomes of patients and quality of care. An evaluation on the acceptability to patients and staff, feasibility and resource implication of routine electronic measurement of EQ-5D-5L in clinical practice should also be carried out before implementation.

Conclusions

Electronic administration of the Chinese (Hong Kong) EQ-5D-5L was found to be valid, reliable, sensitive and responsive for the measurement of HRQOL of Chinese patients with chronic knee and/or back problems in routine clinical practice. We are now ready to proceed to the next research study to determine the clinical usefulness of the e-EQ-5D-5L data in improving health outcomes. If proven to be useful, the e-EQ-5D-5L can be incorporated into electronic medical record systems to facilitate the evaluation and monitoring of HRQOL as part of routine clinical care for patients with chronic musculoskeletal problems.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

EQ-5D-5L:

EuroQol-5 dimensions-5 levels

HRQOL:

Health-related quality of life

WOMAC:

Western Ontario and McMaster University Osteoarthritis Index

ANOVA:

Analysis of variance

VAS:

Visual analogue scale

ICC:

Intraclass correlation coefficient

EQ-5D-3L:

EuroQol-5 dimensions-3 levels

GOPC:

General out-patient clinic

GRS:

Global Rating Scale

IBM:

International Business Machines Corporation

SPSS:

Statistical Product and Service Solutions

AC:

Agreement coefficients

SOPC:

Specialist out-patient clinic

SD:

Standard deviations

References

  1. National Center for Chronic Disease Prevention and Health Promotion, Division of Population Health. Health-Related Quality of Life (HRQOL): HRQOL Concepts 2018 [2021 15 Sep]. Available from: https://www.cdc.gov/hrqol/concept.htm.

  2. Romero M, Vivas-Consuelo D, Alvis-Guzman N. Is Health Related Quality of Life (HRQoL) a valid indicator for health systems evaluation? Springerplus. 2013;2(1):1–7.

    Article  Google Scholar 

  3. Boyce MB, Browne JP, Greenhalgh J. The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: a systematic review of qualitative research. BMJ Qual Saf. 2014;23(6):508–18.

    Article  PubMed  Google Scholar 

  4. Van Der Wees PJ, Nijhuis‐Van Der Sanden MW, Ayanian JZ, Black N, Westert GP, Schneider EC. Integrating the use of patient‐reported outcomes for both clinical practice and performance measurement: views of experts from 3 countries. Milbank Q. 2014;92(4):754–75.

  5. Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med. 2001;33(5):337–43.

    Article  PubMed  CAS  Google Scholar 

  6. Wong EL, Yeoh EK, Slaap B, Tam WW, Cheung AW, Wong AY, et al. Validation and valuation of the preference-based healthindex using Eq-5d-5l in the Hong Kong Population. Value Health. 2015;18(3):A27-A.

    Article  Google Scholar 

  7. Yang Z, Busschbach J, Liu G, Luo N. EQ-5D-5L norms for the urban Chinese population in China. Health Qual Life Outcomes. 2018;16(1):210.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Wong EL, Cheung AW, Wong AY, Xu RH, Ramos-Goñi JM, Rivero-Arias O. Normative profile of health-related quality of life for Hong Kong general population using preference-based instrument EQ-5D-5L. Value Health. 2019;22(8):916–24.

    Article  PubMed  Google Scholar 

  9. Sullivan PW, Ghushchyan VH. EQ-5D scores for diabetes-related comorbidities. Value health. 2016;19(8):1002–8.

    Article  PubMed  Google Scholar 

  10. Liang Z, Zhang T, Lin T, Liu L, Wang B, Fu AZ, et al. Health-related quality of life among rural men and women with hypertension: assessment by the EQ-5D-5L in Jiangsu, China. Qual Life Res. 2019;28(8):2069–80.

    Article  PubMed  Google Scholar 

  11. Wong ELY, Xu RH, Cheung AWL. Health-related quality of life among patients with hypertension: population-based survey using EQ-5D-5L in Hong Kong SAR, China. BMJ Open. 2019;9(9):e032544.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Bilbao A, García-Pérez L, Arenaza JC, García I, Ariza-Cardiel G, Trujillo-Martín E, et al. Psychometric properties of the EQ-5D-5L in patients with hip or knee osteoarthritis: reliability, validity and responsiveness. Qual Life Res. 2018;27(11):2897–908.

    Article  PubMed  Google Scholar 

  13. Cheung PWH, Wong CKH, Samartzis D, Luk KDK, Lam CLK, Cheung KMC, et al. Psychometric validation of the EuroQoL 5-Dimension 5-Level (EQ-5D-5L) in Chinese patients with adolescent idiopathic scoliosis. Scoliosis Spinal Disord. 2016;11(1):19.

    Article  PubMed  PubMed Central  Google Scholar 

  14. The EuroQol Group. EuroQol-a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208.

    Article  Google Scholar 

  15. Dolan P. Modeling Valuations for EuroQol Health States. Med Care. 1997;35(11):1095–108.

    Article  PubMed  CAS  Google Scholar 

  16. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20(10):1727–36.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  17. Janssen MF, Bonsel GJ, Luo N. Is EQ-5D-5L better than EQ-5D-3L? A head-to-head comparison of descriptive systems and value sets from seven countries. Pharmacoeconomics. 2018;36(6):675–97.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Crane HM, Lober W, Webster E, Harrington RD, Crane PK, Davis TE, et al. Routine collection of patient-reported outcomes in an HIV clinic setting: the first 100 patients. Curr HIV Res. 2007;5(1):109–18.

    Article  PubMed  CAS  Google Scholar 

  19. Rolfson O, Kärrholm J, Dahlberg L, Garellick G. Patient-reported outcomes in the Swedish Hip Arthroplasty Register: results of a nationwide prospective observational study. J Bone Joint Surg Br. 2011;93(7):867–75.

    Article  PubMed  CAS  Google Scholar 

  20. Forsberg HH, Nelson EC, Reid R, Grossman D, Mastanduno MP, Weiss LT, et al. Using patient-reported outcomes in routine practice: three novel use cases and implications. J Ambul Care Manag. 2015;38(2):188–95.

    Article  Google Scholar 

  21. Department of Health. Guidance on the routine collection of patient reported outcome measures (PROMs). Department of Health London; 2008.

  22. Greenhalgh J, Gooding K, Gibbons E, Dalkin S, Wright J, Valderas J, et al. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J Patient Rep Outcomes. 2018;2(1):42.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Lavallee DC, Chenok KE, Love RM, Petersen C, Holve E, Segal CD, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff. 2016;35(4):575–82.

    Article  Google Scholar 

  24. Turner GM, Litchfield I, Finnikin S, Aiyegbusi OL, Calvert M. General practitioners’ views on use of patient reported outcome measures in primary care: a cross-sectional survey and qualitative study. BMC Fam Pract. 2020;21(1):14.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Greenhalgh J. The applications of PROs in clinical practice: what are they, do they work, and why? Qual Life Res. 2009;18(1):115–23.

    Article  PubMed  Google Scholar 

  26. EuroQol Research Foundation. EQ-5D-5L | Self-complete version on Tablets 2020 [cited 2020 10 Dec]. Available from: https://euroqol.org/eq-5d-instruments/eq-5d-5l-available-modes-of-administration/self-complete-on-tablets/.

  27. Ping W, Zheng J, Niu X, Guo C, Zhang J, Yang H, et al. Evaluation of health-related quality of life using EQ-5D in China during the COVID-19 pandemic. PLoS ONE. 2020;15(6):e0234850.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  28. Luo N, Liu G, Li M, Guan H, Jin X, Rand-Hendriksen K. Estimating an EQ-5D-5L value set for China. Value Health. 2017;20(4):662–9.

    Article  PubMed  Google Scholar 

  29. Bagattini ÂM, Camey SA, Miguel SR, Andrade MV, de Souza Noronha KVM, Teixeira MAdC, et al. Electronic version of the EQ-5D quality-of-life questionnaire: Adaptation to a Brazilian population sample. Value Health Reg Issues. 2018;17:88–93.

  30. Azzolina D, Minto C, Boschetto S, Martinato M, Bauce B, Iliceto S, et al. Anchoring vignettes in EQ-5D-5L questionnaire: validation of a new instrument. Open Nurs J. 2017;11(1):144–56.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Golicki D, Niewada M, Buczek J, Karlińska A, Kobayashi A, Janssen M, et al. Validity of EQ-5D-5L in stroke. Qual Life Res. 2015;24(4):845–50.

    Article  PubMed  Google Scholar 

  32. Nolan CM, Longworth L, Lord J, Canavan JL, Jones SE, Kon SS, et al. The EQ-5D-5L health status questionnaire in COPD: validity, responsiveness and minimum important difference. Thorax. 2016;71(6):493–500.

    Article  PubMed  Google Scholar 

  33. McCaffrey N, Kaambwa B, Currow DC, Ratcliffe J. Health-related quality of life measured using the EQ-5D–5L: South Australian population norms. Health Qual Life Outcomes. 2016;14(1):133.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Hernandez G, Garin O, Dima AL, Pont A, Pastor MM, Alonso J, et al. EuroQol (EQ-5D-5L) validity in assessing the quality of life in adults with asthma: cross-sectional study. J Med Internet Res. 2019;21(1):e10178.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Bellamy N, Buchanan WW, Goldsmith CH, Campbell J, Stitt LW. Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee. J Rheumatol. 1988;15(12):1833–40.

    PubMed  CAS  Google Scholar 

  36. Wolfe F. Determinants of WOMAC function, pain and stiffness scores: evidence for the role of low back pain, symptom counts, fatigue and depression in osteoarthritis, rheumatoid arthritis and fibromyalgia. Rheumatology (Oxford). 1999;38(4):355–61.

    Article  CAS  Google Scholar 

  37. Woo J, Lau E, Lee P, Kwok T, Lau WC, Chan C, et al. Impact of osteoarthritis on quality of life in a Hong Kong Chinese population. J Rheumatol. 2004;31(12):2433–8.

    PubMed  Google Scholar 

  38. Symonds T, Hughes B, Liao S, Ang Q, Bellamy N. Validation of the Chinese Western Ontario and McMaster Universities Osteoarthritis Index in Patients From Mainland China With Osteoarthritis of the Knee. Arthritis Care Res (Hoboken). 2015;67(11):1553–60.

    Article  Google Scholar 

  39. Jaeschke R, Singer J, Guyatt GH. Measurement of health status Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407–15.

    Article  PubMed  CAS  Google Scholar 

  40. Nunnally JC, Bernstein IH. Psychometric theory. New York: McGraw-Hill; 1994.

    Google Scholar 

  41. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.

    Article  CAS  PubMed  Google Scholar 

  42. Cohen J. Statistical power analysis for the behavioral sciences. New York: Academic Press; 2013.

    Book  Google Scholar 

  43. Klotsche J, Reese JP, Winter Y, Oertel W, Irving H, Wittchen H-U, et al. Trajectory classes of decline in health-related quality of life in Parkinson’s disease: a pilot study. Value Health. 2011;14(2):329–38.

    Article  PubMed  Google Scholar 

  44. Hurst N, Kind P, Ruta D, Hunter M, Stubbings A. Measuring health-related quality of life in rheumatoid arthritis: validity, responsiveness and reliability of EuroQol (EQ-5D). Br J Rheumatol. 1997;36(5):551–9.

    Article  PubMed  CAS  Google Scholar 

  45. Kim MH, Cho YS, Uhm WS, Kim S, Bae SC. Cross-cultural adaptation and validation of the Korean version of the EQ-5D in patients with rheumatic diseases. Qual Life Res. 2005;14(5):1401–6.

    Article  PubMed  Google Scholar 

  46. Norman GR, Sloan JA, Wyrwich KW. Interpretation of Changes in Health-Related Quality of Life: The Remarkable Universality of Half a Standard Deviation. MED CARE. 2003;41(5):582–92.

    Article  PubMed  Google Scholar 

  47. Payakachat N, Ali MM, Tilford JM. Can the EQ-5D detect meaningful change? A systematic review. Pharmacoeconomics. 2015;33(11):1137–54.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank EuroQol Group for the permission to use the full version of the EQ-5D-5L Web version for this designated research purpose. We are grateful to the doctors and nurses of the study clinics for their help with subject recruitment. Thanks also go to our research staff, Will Cheng and Eric Tang for assistance in data analysis and preparation of the manuscript.

Funding

Financial support for this study was provided by the General Research Fund (Ref No. 17100119), Hong Kong Research Grant Council. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Author information

Authors and Affiliations

  1. Department of Family Medicine, The University of Hong Kong Shenzhen Hospital, Shenzhen, China

    Cindy Lo Kuen Lam & Emily Tsui Yee Tse

  2. Department of Family Medicine and Primary Care, Li Ka Shing Faculty of Medicine, The University of Hong Kong, 3/F, Ap Lei Chau Clinic, 161 Main Street, Ap Lei Chau, Hong Kong, SAR, China

    Cindy Lo Kuen Lam, Emily Tsui Yee Tse, Carlos King Ho Wong, Joyce Sau Mei Lam, Sikky Shiqi Chen & Laura Elizabeth Bedford

  3. Department of Pharmacology and Pharmacy, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, SAR, China

    Carlos King Ho Wong

  4. Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, SAR, China

    Jason Pui Yin Cheung

  5. Industrial and Manufacturing Systems Engineering, Faculty of Engineering, The University of Hong Kong, Hong Kong, SAR, China

    Calvin Kalun Or

  6. Department of Applied Health Research, Institute of Epidemiology and Health Care, University College London, London, UK

    Paul Kind

Authors
  1. Cindy Lo Kuen Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emily Tsui Yee Tse
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carlos King Ho Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joyce Sau Mei Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sikky Shiqi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Laura Elizabeth Bedford
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jason Pui Yin Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Calvin Kalun Or
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Paul Kind
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CLKL contributed to study design, acquisition of data and writing of the manuscript. JSML contributed to the statistical analysis, interpretation of results and writing of the manuscript. SSC contributed to data collection, statistical analysis, interpretation of results and writing the manuscript. ETYT, LEB, CKHW, JPYC, CKO and PK contributed to interpretation of results and reviewing and editing of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Emily Tsui Yee Tse.

Ethics declarations

Ethics approval and consent to participate

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Research ethics approval was obtained from the institutional review board prior to subject recruitment (HKU/Hospital Authority Hong Kong West IRB reference number: UW 18–270). Informed consent was obtained from all participants included in the study.

Consent for publication

Not applicable.

Competing interests

All authors declare that he/she has no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Appendix 1

. Electronic EQ-5D-5L and EQ-VAS Completion Procedure with Screenshots

Additional file 2: Table S1a

. Baseline characteristics of subjects by three trajectory classes of EQ-5D-5L utility scores. Table S1b. Baseline characteristics of subjects by three trajectory classes of EQ-VAS scores

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lam, C.L.K., Tse, E.T.Y., Wong, C.K.H. et al. A pilot study on the validity and psychometric properties of the electronic EQ-5D-5L in routine clinical practice. Health Qual Life Outcomes 19, 266 (2021). https://doi.org/10.1186/s12955-021-01898-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12955-021-01898-3

Keywords

Health and Quality of Life Outcomes

ISSN: 1477-7525

Contact us