The CHU9D

The CHU9D [17-20] was designed for use in children aged 7–11 years, but with interviewer assistance can be used in children as young as 6 [21], and research has demonstrated its validity in adolescents up to age 17 [22]. It consists of 9 items, each with 5 response categories (scored 1–5) that assess the child/adolescent’s functioning “today” across domains of worry, sadness, pain, tiredness, annoyance, school, sleep, daily routine and activities. The instrument is available in both self-report (completed by the child) and proxy-report (caregiver completed) forms.

The CHU9D was developed in response to a perceived paucity of paediatric preference-based measures for use in health care resource-allocation decision making [17]. The 9 domains of the questionnaire were identified from qualitative interviews with children aged 7 to 11 years, who described the areas of their life affected by their health conditions [18]. As with other PBHRQOL instruments, the CHU9D has undergone ‘valuation’ where the various health states described by the instrument (i.e. potential combination of scores across the different items) have been valued by the general public generating tariffs for calculating utility weights from an individual’s score [9]. In fact, 2 sets of preference weights (tariffs) are available. The first (UK Adult Tariff) was generated from health state valuation interviews with 300 members of the UK adult general population [9] using the standard gamble method. This tariff generates utility weights between .33 and 1. The second set (Australian Adolescent Tariff) was developed by Ratcliffe and colleagues [10], based on interviews with 590 Australian adolescents using profile case best-worst scaling (BWS) discrete-choice experiment (DCE) methods. It similarly generates values between .33 and 1, although demonstrates some significant differences in the valuation of some health states, particularly related to mental health attributes [22].

There are a number of features of the CHU9D that make it a potential candidate as a routine outcome measure in child and adolescent mental health. It was developed using research with children, is brief and simply worded, has a low response burden, is available in proxy and self-report forms, has been used in children and adolescents from 7–18 years old, uses a shortened reference time frame (“today”) suitable for repeated measurements, has a good representation of mental health related items (sad, worried, annoyed), and is impact rather than symptom focused, complementing existing measures. Previous validation studies with adolescents from the community have found the instrument to be well understood, to discriminate between individuals based on their self-reported health status and show expected correlations with other generic quality of life instruments [22,24]. A validation study with children aged 6–7 [21] showed they appeared to comprehend the questions when asked by an interviewer, but there was some doubt as to the reliability of their answers, given relatively low test-retest reliability. Validation studies with clinical populations have not been carried out to our knowledge.

Our own unpublished pilot testing of the CHU9D with children, caregivers and CAMHS providers indicated that children as young as 6 could complete the instrument with assistance, that caregivers found the instrument brief and simple to use, and providers felt the instrument provided a reasonable overview of the child’s functioning. There was also correlational evidence that a young person’s score on the CHU9D (either self-report or proxy) corresponded with their clinical severity as indicated by the service provider.

The purpose of this paper is to report on the findings of using the proxy version of the CHU9D alongside the widely-used Strengths and Difficulties Questionnaire (SDQ) with 200 caregivers of children receiving mental health services. As our aim was to determine whether the CHU9D would make a suitable instrument for use in CAMHS, we explored multiple aspects of its performance: face validity, practicality, internal consistency, and convergent validity. We also compared the utility weights obtained in this child and adolescent mental health population, with utility weights from other studies of child and adolescent mental health populations.

Design

The study employed a cross-sectional telephone survey design, in which caregivers of children receiving services from a local child and adolescent mental health service were asked to complete the CHU9D and the SDQ in a single sitting. The study was approved by both Health Service (#384.11) and University ethics committees (#25739).

Participants

Participants were parents or other adult relatives of children aged 5–17 years (inclusive), who were registered as ‘current clients’ of a regional child and adolescent mental health service. ‘Current client’ status was defined as having an open episode of care and a recorded contact within the last 6 weeks. Excluded were caregivers who had no recorded telephone number, had specific “no contact” instructions in the electronic clinical record, were foster carers, or whose child was the subject of current guardianship or family court orders.

Procedure

Potential participants were identified from the electronic clinical record of the CAMH service and placed on a list. The order of participants on the list was randomised before being provided to telephone interviewers. All listed participants were sent out introductory letters at least one week prior to being contacted by phone by interviewers. Where a participant was identified as having more than one child receiving CAMH services, a coin toss method was used to identify which child the participant would be asked to rate.

Measures

A telephone survey was developed that consisted of the CHU9D, the SDQ and additional demographic, presenting issue, and service satisfaction questions. The order of presentation was the same for all participants. Child health status and emotional and behavioural health were assessed by proxy (i.e. by the child’s caregiver). Proxy outcome measurement is common practice both in CAMH services and quality of life studies where seeking self-report from children can be compromised by age and comprehension issues.

Child Health Utility – 9D

The CHU9D, described previously, consists of 9 items each with a 5-level response category. Each item taps into a different domain (worry, sadness, pain, tiredness, annoyance, school, sleep, daily routine and activities). The time frame for the questions is “today”. Because of this, we asked a sub-sample of participants an additional question of whether “today” was a typical day for the child, to determine the representativeness of the child’s functioning on that day, of their general functioning. In cases where participants struggled to rate their child’s behaviour on that day, we asked them to rate their child’s behaviour on an average day. In examining the performance of the CHU9D, we present utility weights using both available tariffs, the UK Adult Tariff and the Australian Adolescent Tariff. Completed CHU9D questionnaires were scored using SPSS syntax provided by the authors of the tariffs [10,17].

Strengths and Difficulties Questionnaire

The Strengths and Difficulties Questionnaire (SDQ) [6] was first developed as a shorter alternative to behavioural screening questionnaires such as the Rutter [26] and Child Behaviour Checklist [27,28] but with an additional focus on young people’s “strengths”. The SDQ has repeatedly demonstrated equivalence to these longer measures in terms of factor structure, reliability, sensitivity to detecting psychiatric diagnoses, and sensitivity to change [29-31]. The instrument is now a widely-used mental health screening measure in children and adolescents aged 4–17 years. In fact the SDQ is now a mandated consumer self-report routine outcome measure in Australian CAMHS [32], and a standard measure in UK routine outcome collections [33].

The SDQ comprises 25 items, each describing a psychological or behavioural attribute (some positive, some negative) which the responder indicates as being “very true”, “somewhat true” or “not true” of the child/adolescent in question over the last 6 months. The instrument generates both a total score and scores for 5 subscales including emotional, conduct, hyperactivity-inattention, peer problems and prosocial behaviour. The total score ranges from 0–40 with higher values indicating greater behavioural and emotional pathology. Individual sub-scales are scored from 0–10 with higher scores indicating poorer functioning for four of the subscales (emotional, conduct, hyperactivity-inattention and peer problems), and better functioning for one of the subscales (prosocial). There are also cutoff scores available for the sub-scales and the total score that define the following clinical bands: normal, borderline and abnormal. These are based on a population-based UK survey in which cutoffs were chosen such that 80% of children scored normal, 10% scored borderline and 10% scored abnormal. The SDQ is available in 3 forms - adolescent self-report, caregiver-administered and teacher-administered. We used the caregiver-administered form in this study and utilised SPSS syntax from the SDQ website [34] to score the instrument. The impact supplement was not used in this study.

Crossover between measures

Analysis

Question 1 – face validity and practicality

Interviewers reported that the CHU9D proxy was simple and quick to administer, typically taking less than 2 minutes to complete. There was only one missing CHU9D data point across the 200 participants, indicating the CHU9D was well suited to interviewer administration to proxies.

Ninety (90) participants were asked if “today” (the reference time frame for the CHU9D) represented a typical day in terms of the child’s behaviour. This question was added to the survey after some parents reported the child’s behaviour during the survey period to not be representative of their general behaviour. Twenty-nine (32%) reported ‘no’ suggesting 1/3 of CHU9D ratings might not accurately capture the child’s average level of functioning. Of these 29, 18 (62%) indicated that today was ‘better than usual’ indicating a very subtle bias at the group level for the caregiver-completed CHU9D to underestimate dysfunction in some children.

Another issue encountered by interviewers was that 3 parents struggled to answer the typical day question because of limited exposure to their child that day and hence lack of knowledge about their mood, sleep, school and daily routine.

Question 2 – internal consistency

The Cronbach alpha for the CHU9D was .781, indicating an acceptable level of internal consistency.

Questions 3 and 4 – convergent validity

Figure 2 shows the scatterplot of CHU9D utility values (UK Adult and Australian Adolescent tariffs) and SDQ Total scores. Utility weights were moderately correlated with the SDQ total score, for both the UK Adult Tariff [r(199) = −.487 (p < .001)] and Australian Adolescent Tariff [r(199) = −.494 (p < .001)].

Question 5 – comparisons of utility weights

Mean utility weights were lower in this study than for 2 Australian community samples (aged 11–17) tested using the CHU9D self-report version [22,24]. Furthermore, utility weights in this sample were lower than the only published utility norms for this age range (0.90 to .92 — Canadian norms using Health Utilities Index [39]) consistent with the sample being taken from a clinical population.

Limitations

Due to the nature of the study we were unable to test a number of useful metrics. For example, we were not able to calculate test-retest reliability or the sensitivity of the instrument to change in mental health symptoms over time, having only one measurement point. These are particularly important metrics, as the value of the CHU9D for economic evaluation will depend on its capacity to reliably detect change due to intervention, rather than large natural fluctuations in an individual’s responses. We were also unable to explore concordance between caregiver and child self-completed versions of CHU9D as the telephone survey methodology was not well suited to collection of responses from children and adolescents. As a result, we recommend further psychometric validation of the CHU9D with a focus on repeated measures and multiple raters (e.g. child, caregiver, therapist).

In terms of the study sample, although it was drawn randomly from a list of active CAMHS clients, difficulties in contacting people from the list possibly led to the sample being higher functioning than a true random sample. In essence, families who were not contactable were assumed to have greater dysfunction, although we did not have data to test this hypothesis. There is also the question of whether the study sample is representative of CAMHS clients elsewhere. Comparisons not reported in this paper however show SDQ scores in our sample compare very similarly to SDQ scores collected routinely from other Australian CAMHS, hence we have reasonable confidence that the study sample is at least fairly representative of clients of CAMHS in Australia.

It should also be noted that there is a broader debate relating to the use of preference-based quality of life instruments in children and adolescents [55]. Concerns include that the valuation procedures used in defining the tariffs may not translate well from adult measures to children and adolescents, the capacity of children to understand and complete instruments, the accuracy of proxy raters, the need to consider family interactions in children’s measures, and the wide variation in utility weights noted for different childhood disorders from existing instruments. We also note (and this applies to adult instruments as well) that tariffs generated in one population or age group may not be comparable to those in other populations, further hampering efforts to use preference-based HRQOL instruments to facilitate cross comparisons. Whilst it is beyond the scope of this paper to address these issues in any depth, it should be noted that there are valid arguments that preference-based instruments (as they currently stand) might not be the best fit for child and adolescent populations, and therefore alternative metrics of outcomes with economic relevance (e.g. school attendance) should be similarly explored.

Future work

The telephone survey method used in this study proved to be a viable and efficient way of communicating with caregivers about the mental health and quality of life of their children receiving mental health services. The process, which was separate from their clinical care, caused minimal disruption to clients and therapists. As such, this method might be suitable for exploring test-retest, sensitivity to change, and comparisons between PBHRQOL instruments. Telephone or web-based survey methods may also be suitable for tracking adolescents receiving services. For example, Ratcliffe and colleagues were able to obtain consent and collect data from adolescents using a web-based survey [24]. Eliciting answers to the CHU9D from very young children has required direct contact [21], thus studies involving children as young as 5 might need to be situated within clinics.

Future studies could thus employ telephone and web-based survey methods, but with a larger range of utility measures, as well as a follow-up assessment and sub-samples with repeated measures. Studies looking to obtain scores from children may need to supplement telephone and web-based methods with assessments conducted within the clinical services themselves. Templates for such studies include the Multi Instrument Comparison Project [56]. The outcomes of comparing the performance of different utility instruments in children and adolescents would provide much clearer guidance on whether preference-based instruments are a suitable addition to mental health services, and if so, which ones are superior.