Cross-cultural adaptation and content validation of the Singapore English version of EQ-5D-Y: a qualitative study

Abstract

Background

The EQ-5D-Y is a generic preference-weighted measure for children and adolescents which was developed within Europe. Two versions exist, the EQ-5D-Y-3L (Y-3L) and EQ-5D-Y-5L (Y-5L). This study aimed to cross-culturally adapt the Y-3L and Y-5L for use in Singapore and to assess the content validity, specifically, the relevance and comprehensiveness of the EQ-5D-Y descriptive system (DS) in Asia.

Methods

To culturally adapt the instruments, an expert panel consisting of paediatricians and primary school educators were consulted. Modifications suggested by the expert panel were tested via cognitive debriefing interviews with children aged 8–12 in Singapore. To assess the content validity of the EQ-5D-Y DS, interviews were conducted with both healthy (n = 8) and ill children (n = 6) aged 8–15. In the interviews, children discussed their experience with poor health and commented on the comprehensiveness and relevance of the EQ-5D-Y DS.

Results

The cross-cultural adaptation process led to minor modifications to the UK English Y-3L and Y-5L versions, including using phrases familiar to the local children and adding examples to facilitate understanding. The five health dimensions in the EQ-5D-Y DS were spontaneously elicited when children discussed their experience with poor health. All health dimensions related to poor health elicited from the interviews fell into three broad categories: physical health (e.g. Appetite, Mobility, and Sleep), mental well-being (e.g. Annoyed/Frustrated and Scared/Worried), and social relationships (e.g. Family and Friends). The EQ-5D-Y DS was generally found to be relevant and comprehensive, although some health dimensions that may be relevant to the local population (Social relationship and Appetite) were not covered.

Conclusions

The UK English EQ-5D-Y instruments were adapted to produce the Singapore English EQ-5D-Y instrument that were comprehensible to local children as young as 8 years old. The EQ-5D-Y DS was generally relevant and comprehensive to measure poor health of local children. Future studies should ascertain the benefits of adding bolt-on items related to social relationships and appetite to the EQ-5D-Y DS.

Background

The EQ-5D-Y-3L (Y-3L) is a generic, preference-weighted, patient-reported outcome measure (PROM) designed to be suitable for self-completion by children and adolescents. An international collaborative effort by researchers within Europe produced the Y-3L, by adapting a pre-existing instrument, the EQ-5D-3L, which was designed for use in adults. Differences between EQ-5D-3L and Y-3L include simplified language and modified wording of some of the dimensions to be more relevant to the child and adolescent population [1]. The descriptive system (DS) of the Y-3L contains five health dimensions: mobility, looking after myself, doing usual activities, experiencing pain or discomfort, and feeling worried, sad, or unhappy, with response options covering three levels of severity in each dimension (no problems, some problems/a bit, and a lot of problems/ very). The instrument also contains a visual analogue scale (EQ VAS), ranging from 0 (the worst health you can imagine) to 100 (the best health you can imagine). More recently, a version of the same DS but with five levels of severity, EQ-5D-Y-5L (Y-5L) has been developed [2].

Since its development, the UK English Y-3L and Y-5L have been adapted/translated for use in different local contexts and languages. To date, several adaptations and translations of the Y-3L and Y-5L exist in languages such as Australian English, Brazilian Portuguese, Canadian English, mainland Chinese, and French. Given the different cultural contexts and language nuances, these adaptations/translations are necessary to ensure linguistic equivalence [3]. It is important to note that there is currently no adapted version for use in Singapore. While English is the primary language spoken and written by children and adolescents in Singapore, the local variant differs from UK English due to influences from native languages such as Chinese, Malay, and Tamil. Furthermore, Singapore’s unique cultural norms and etiquette, possibly stemming from its multicultural environment and roots, may affect how respondents understand and interpret the questionnaire, as well as how they respond to it. Cognitive debriefing can help to delineate these aspects.

With their worldwide application, the EQ-5D-Y instruments have also been widely studied, with numerous studies demonstrating their construct validity [4,5,6]. However, little has been done to investigate the content validity of the EQ-5D-Y DS. Unlike construct validity which refers to the degree to which an instrument measures the theoretical construct it is intended to measure [7], content validity refers to the extent to which the content of an instrument adequately covers and measures the construct(s) it aims to measure [8, 9]. It concerns the comprehensibility, comprehensiveness, and relevance of a PROM in its context of use and is considered one of the most important measurement properties of a PROM by researchers [9]. Some efforts have been made to study the content validity of the EQ-5D-Y DS [10, 11], however, these studies did not directly ascertain the comprehensiveness and relevance of the DS with the instrument’s intended users, children and adolescents, but with proxies instead. To our knowledge, only one study [11] directly interviewed children and adolescents on the content validity of the EQ-5D-Y DS. However, the findings were limited to paediatric patients who needed psychiatric inpatient care. Further, no studies have been conducted to assess the content validity of the EQ-5D-Y DS in Asian settings.

Therefore, the aims of this study are to (i) report on the cross-cultural adaption process of the Y-3L and Y-5L to produce a Singapore English version that is linguistically equivalent to the UK English version and (ii) assess the content validity (comprehensiveness and relevance) of the EQ-5D-Y DS in children and adolescents in Singapore.

Methods

This study followed the standards for reporting qualitative research studies [12]. The procedure and materials used in this study were approved by the National Healthcare Group Domain Specific Review Board (Ref: 2020/00282) and the National University of Singapore- Institutional Review Board (Ref: NUS-IRB-2021-423). Children and their legal guardians provided assent and informed consent, respectively, prior to completing the study procedures.

All interviews were conducted by RLYT and LAC, who had no pre-existing relationships with any of the study participants. As researchers, they made conscious efforts not to accept potentially common assumptions at face value. All interviews were conducted in a private area convenient for the children and were audio-recorded. Data collected from the adaptation process were entered into a data collection form while data collected during interviews to assess content validity of the DS were voice recorded and transcribed verbatim. All interviews were anonymized, and each interviewee was assigned a code number.

Cross cultural adaptation process

The study team followed the cultural adaptation guidelines recommended by the EuroQol Group [13, 14] with slight modification. On top of the recommended process in the guidelines, the study team invited a group of local experts (including paediatricians and primary school teachers who have frequent communication with local children) to review and assess the suitability of the language used in the UK English version for Singapore. The focus was on the comprehensibility of the questionnaire in the local context, for children as young as 8 years old. Feedback and suggestions from the expert team, together with input from the questionnaire developer, the EuroQol Research Foundation were used to develop the draft Singapore English Y-3L and Y-5L. Following EuroQol Group’s translation guidelines, cognitive debriefing interviews focusing on comprehensibility of the instruments were conducted between June 2020- August 2020, with 5 Singaporean children per instrument. Children included were of varying ages and the study team sought to understand if they understood the instructions, interpreted the items as intended, and understood and appropriately selected from the response options. Methods such as verbal probes and having participants paraphrase were employed. In the case of the Y-5L, a ranking exercise [15] was also conducted to determine whether the children ordered the severity labels (presented in separate showcards and in random order) in the intended order.

Content validation

Content validation of EQ-5D-Y DS adhered closely to the COSMIN guidelines [8, 9]. Qualitative methods involving semi-structured, one-on-one in-person interviews with Singaporean children were conducted. Children who participated in the cognitive debriefing interviews for cross cultural adaption were not eligible to take part in the content validity testing. All interviews were conducted in English.

Study participants

This study recruited both healthy children and children with chronic condition. Healthy children from the general public were recruited using social media via their parents while children with chronic diseases were referred by a team member who was a paediatrician from a local hospital. Purposive sampling was conducted to ensure maximum variation in age, gender, ethnicity, and experience with chronic disease(s). Sample size was determined by data saturation. Data saturation was deemed achieved when no new themes and information emerged from the last three transcripts.

Procedures

Semi-structured interviews were conducted between December 2021- January 2023 using an interview guide developed by the study team, designed to evaluate the content validity (comprehensiveness and relevance) of the EQ-5D-Y DS. The semi-structured one-on-one interview consisted of two parts: In the first part of the interviews, children were asked to discuss their direct and indirect experience with poor health. Open-ended questions (e.g. Can you describe a time when you felt very ill/ someone you know who felt very ill? How did you feel/ how do you think he/she felt? How did that [health problem] affect you/him/her?) were used to elicit health concepts that children considered relevant components of poor health without suggestions from the interviewer or a health questionnaire. In the second part of the interviews, children were asked to complete the Singaporean English Y-3L independently (the Y-5L was not presented as the focus of the cognitive interview was the EQ-5D-Y DS and not the response levels). After that, cognitive debriefing was used to consult participants on the comprehensiveness of the EQ-5D-Y DS (e.g., Do you think we should add any more questions to improve this health questionnaire for children like yourself? Are there any important questions you think we missed out to measure health of children like yourself?), and relevance (e.g., Is this question on walking suitable to measure health of children like yourself? Why?). As the cognitive debriefing interviews during cross-cultural adaptation showed that local children had no problems understanding and differentiating the severity levels, we decided they should not be a focus of content validity testing. In this way, we hoped to reduce respondent burden on the child participants. The parent/caregiver of the recruited children were not invited to participate in the interview; however, they were allowed to sit in the interview, away from sight of the children to minimize distraction.

Data analysis

Framework analysis [16, 17] was used to analyse data from the first part of the semi-structured interviews where dimensions of poor health were spontaneously elicited. The framework used to analyse data from this study was developed by a local study team which did similar work on the general adult population in Singapore [18]. It organized themes and subthemes related to being in poor health and consists of five broad domains: Physical health, mental well-being, social relationship, medical conditions & treatment, and health promotion knowledge and behaviours. Health dimensions mentioned by respondents that were not already defined in the existing framework were considered unique and were discussed and reviewed by the study team to determine their relevance. Once deemed relevant, these dimensions were defined and added to the framework. The codebook previously developed by the local study team was used as a guide during the coding process to maintain consistency. Each transcript was coded line by line by 2 independent coders. The principal investigator was consulted whenever the two coders did not reach consensus on any discrepancy. Comprehensiveness and relevance of the EQ-5D-Y DS can be deduced by comparing the framework of spontaneously elicited health dimensions to the health dimensions included in the EQ-5D-Y DS. The definition of the health dimensions included in the EQ-5D-Y [13] were compared with the definition of the health dimensions in the framework to ensure thorough and unbiased comparison. Content analysis [19] was used to analyse data from the second part of the interviews which provides further evidence on the comprehensiveness, and relevance of the EQ-5D-Y DS.

Results

The Singapore english Y-3L and Y-5L

Modifications with justification proposed by the expert panel (6 paediatricians and 2 primary school teachers) are shown in Table 1. For example, ‘washing and dressing myself’ was revised for clarification to ‘showering or wearing clothes myself’ to better fit the phrases local children are familiar with. These modifications were included in the draft Singapore English version tested with children during the cognitive debriefing interviews as part of the cross cultural adaptation process. In total 11 children completed the cognitive debriefing (6 and 5 children interviewed to test Y-3L and Y-5L, respectively). Due to external interference (environmental noise and interruption by other people) during one interview, one additional child was recruited to test the Y-3L to ensure data quality. The mean age of the recruited sample was 9.8 years old (SD = 1.7) (Table 2). All children appropriately ranked the severity levels of the Y-5L without difficulty in all dimensions during the ranking exercise. Based on findings from the cognitive interviews, slight modifications were further proposed by the study team (Table 1) to produce the final Singapore English Y-3L and Y-5L. For example, examples of discomfort (aches, breathlessness, itching, and feeling like vomiting) were added in parentheses to facilitate children’s understanding of ‘discomfort’. These added examples of discomfort were drawn from a standard list of examples which the EuroQol group uses as needed to support understanding of the term ‘discomfort’ during translation processes. They were tested during cognitive debriefing to assess if local children understood them and only examples from the standard list which were understood by all local children interviewed were used in the final Singapore English versions. All modifications have been endorsed by the EuroQol Research Foundation.

Table 1 Modifications made to develop the Singapore English EQ-5D-Y-3L and EQ-5D-Y-5L

Table 2 Characteristics of participants who took part in the cognitive interviews for cross-cultural adaptation

Content validation

A total of 14 children aged between 8 and 15 years old (mean: 10.4; SD: 2.1) completed the cognitive interviews to assess the content validity of the EQ-5D-Y DS. Half of the recruited sample were female and half were Chinese; eight children were healthy, four had a chronic condition, one had an acute condition, and one had both a chronic and acute condition. Chronic conditions experienced by the recruited sample included alopecia, eczema, haemophilia, immune thrombocytopenic purpura (ITP), and sleep apnea. Acute conditions included a leg fracture and severe hand burns. Table 3 illustrates the full demographics of the recruited sample.

Table 3 Demographics of the recruited children who took part in the interviews to assess content validity

All themes and subthemes identified from part 1 of the interviews fit into the three overarching domains of Physical Health, Mental Well-being, and Social Relationship of the framework. The operational definition of the domains and themes are described in Table 4. Most of the themes and subthemes of poor health identified in the interviews fell under the domain of physical health. Four of the physical health dimensions (mobility, doing usual activities, taking care of myself, and having pain or discomfort) included in the Y-3L descriptive system were spontaneously elicited from part 1 of the interviews. The health dimension ‘feeling worried, sad, or unhappy’ was identified under the mental well-being domain. ‘having pain or discomfort’ and ‘doing usual activities’ were the two health dimensions most discussed by children in the interviews. The full framework is illustrated in Fig. 1. Exemplar quotes for each theme and subthemes are provided in Table 4.

Table 4 Operation definition of themes and exemplars

Fig. 1

Framework of poor health

Findings from part 2 of the cognitive interviews focusing on the comprehensiveness and relevance of the five health dimensions included in the descriptive system of the EQ-5D-Y are reported below:

Mobility

Generally, the children shared that mobility was an important health dimension as it could be compromised due to an injury, which is a reflection of their health status. This is shown in the excerpt below:

Boy, 12 years-old: The time in which the problem walking comes is when, is (when I have an) injury. But other than that, normally, I don’t have any problems walking.

Taking care of myself

All children agreed that assessing children on their ability to shower and wear clothes themselves was important to measure the health of children. Excerpt below shows how poor health can affect self-care:

Girl, 8 years-old (with severe hand burns): Yes, this is important… I cannot touch water (to shower).

Doing usual activities

Children generally defined usual activities as going to school or engaging in play or sports. All children agreed that having poor health will affect their ability to conduct their usual activities and was therefore an important health dimension to be included in the health questionnaire. It is, however, worth noting that one child answered this item using a non-health lens (the child rated that she had problems doing her usual activities, which she defined as not being able to rollerblade as her roller blades were not with her.) One child shared that having problems with usual activities can also reflect one’s mental health:

Girl, 15 years-old: Some people… have an interest in… playing sports, and then suddenly… they don’t feel like doing the things they love or like… like those people who have depression. I think by asking this kind of question… it’s suitable for the younger generation… (to) know how they’re feeling.

Having pain or discomfort

Except for one child who interpreted this item as both mental and physical pain, all other children interpreted it as only physical. All children agreed that pain and discomfort was an important health dimension to be included. The children shared that pain or discomfort can manifest as symptoms of a certain illness and can be an indicator of poor health. Excerpt below shows how pain and discomfort reflect poor health:

Boy, 12 years-old: Having pain or discomfort is even more important…because it could be life threatening.

Feeling worried, sad, or unhappy

All children agreed that this was an important health dimension. Reasons provided include the item’s ability to capture information about a child’s mental health, as shown in the excerpt below:

Boy, 12 years-old: Feeling worried, said or unhappy is also important because it describes their mental health.

Though most children indicated that the EQ-5D-Y DS was sufficient to capture the health of children, a few children suggested adding more items to increase comprehensiveness. Items suggested by children include more items relating to mental health on top of the existing single item measuring “worried, sad, or anxiety” (Boy, 13 years old), an item on having problems sleeping (Boy, 9 years old), an item on appetite (Girl, 12 years old) and an item on quality of relationship with friends and family (Boy, 12 years old). Examples of items related to quality of relationship with friends and family are highlighted in the excerpt as follows:

Boy, 12 years-old: I would say (add these items)… How are your relationships? Do you have any problem with relationships like friendship?…Any conflicts with your friends? Do you have some friends who don’t like you/avoid you/do not want to talk to you…And how is your family…like any bad relationship with family members?

Discussion

This study reported on the adaptation of the UK English Y-3L and Y-5L to Singapore English and assessed for content validity of the EQ-5D-Y DS in Singaporean children. To our knowledge, this is the first study to assess content validity of the EQ-5D-Y DS in Asian settings. This study found that the dimensions included in the EQ-5D-Y DS were generally considered to be relevant aspects of poor health by the local children, but some areas of poor health that may be relevant to the local population including social relationship and appetite were not covered. Though with the addition of more health concepts could improve the comprehensiveness of the EQ-5D-Y DS, it is important to note that instead of comprehensiveness, the EQ-5D-Y instruments, like their predecessors (EQ-5D-3L and EQ-5D-5L), were designed with brevity as priority. It was intended to be a short generic HRQoL measure that could capture fundamental aspects of health deemed important to both healthy and paediatric disease populations. To improve the comprehensives of the EQ-5D descriptive system for specific populations, bolt-on research has been undertaken [20,21,22]. This research involves the addition of supplementary health dimensions to the five existing ones. For example, a cognition bolt-on was found to improve the discriminatory power of EQ-5D-Y for children in Germany [20].

Interestingly, like the study conducted to assess the content validity of the adult version EQ-5D DS in Singapore [18], ‘appetite’ was raised as one of the items that could be added as a bolt-on to improve comprehensiveness. In addition, a previous study conducted to evaluate participants’ attitudes towards health and pleasure with regard to eating found that “eating pleasure” was important for a large majority of Singaporeans [23]. Future studies can consider testing ‘appetite’ as a bolt-on for the Singaporean EQ-5D versions due to its cultural relevance in Singapore to assess if any improvements in psychometric properties is worth the additional respondent burden of the added bolt-on.

Although this study established the content validity of Y-3L for use in Singapore, it is important that researchers thoroughly assess the instrument’s psychometric performance prior to administering it in Singapore. The instrument was originally developed in the UK [1] and has been psychometrically validated in various counties and populations [24], however, due to cultural, context, and language differences between those countries and Singapore, findings from those studies might not be generalizable. Psychometric properties important to such HRQoL instruments includes construct validity, test-retest reliability, and responsiveness [7].

Study limitation

There are some limitations to this study. Firstly, children as young as 8 years-old were recruited for this study to assess content validity. Young children might not have enough experience with poor health to comment on the comprehensiveness and relevance of a health questionnaire. This is mitigated by the study team asking children about their indirect experience with poor health and recruiting children with chronic conditions. As the instrument was designed to be suitable for children aged 8 and older, recruitment of children as young as 8 is crucial to understand the instrument’s performance within the lower age bound group. Secondly, while health dimensions associated with poor health were elicited from children, the relative importance of these health dimensions, including health dimensions in the EQ-5D-Y DS, were not explored in this study. Lastly, this study only assessed the content validity of the EQ-5D-Y DS and not the EQ VAS. Although part of the same instrument (Y-3L and Y-5L), the instructions, labels, and response options are different. Hence, results from this study cannot be applied to the EQ VAS. Future research can assess the content validity of the EQ VAS, including the understandability of the instructions and interpretation of labels ‘worst imaginable health’ and ‘best imaginable health’ by the younger population.

Conclusion

The Singapore English version of the Y-3L and Y-5L questionnaires were adapted from their original UK English versions. Content validity was also established for the EQ-5D-Y DS. Specifically, its DS was found to be comprehensive and relevant to measure the health of Singaporean children and adolescents.