Comparison of the ECOHIS and short-form P-CPQ and FIS scales

Background The development of short-form versions of child oral–health-related quality of life (OHRQoL) scales has resulted in two closely related sets of measures. We set out to compare the properties and responsiveness of the Early Childhood Oral Health Impact Scale (ECOHIS – both “child” and “family” versions) and short-form Parental-Caregiver Perceptions Questionnaire (P-CPQ) and the Family Impact Scale (FIS) measures among New Zealand children with early childhood caries who underwent treatment under general anaesthesia (GA). Methods Secondary analysis of data from pretest/post-test clinical studies of consecutive clinical convenience samples undertaken in Wellington in 2005 and Auckland in 2010/11, with cross-sectional analyses using the former, and longitudinal analyses using the latter. Results Cronbach’s α values for the ECOHIS-Child, P-CPQ-16 and P-CPQ-8 were 0.80, 0.88 and 0.80 respectively, and 0.83 and 0.68 (respectively) for the FIS-8 and the ECOHIS-Family. All scales showed acceptable cross-sectional construct validity, although that of the ECOHIS-Family was not as marked as that observed with the FIS-8. Responsiveness was acceptable, with the three child-focused measures showing similar effect sizes. The two family-focused measures were also similar. Conclusions The ECOHIS-Child and the P-CPQ scales are very similar in their properties, but the ECOHIS-Family falls short of the FIS-8 in some important ways. The ECOHIS scales may be better deployed in epidemiological survey work rather than in health services research, whereas the P-CPQ-8, P-CPQ-16 and the FIS-8 seem to be well suited for the latter (particularly with children suffering from severe caries), but their epidemiological utility remains to be demonstrated.


Background
The last decade or so has seen considerable progress in the development, testing and validation of child measures of oral-health-related quality of life (OHRQoL). A number of competing measures now exist [1][2][3], with most of those intended for use with children who are old enough to give valid and reliable responses. Measuring oral health in children younger than six years of age has necessitated the use of scales which use adult informants (such as parents) who know the child well. The Parental-Caregiver Perceptions Questionnaire (P-CPQ) and the Family Impact Scale (FIS) are part of the Child Oral-Health-Related Quality of Life (COHQoL) suite of instruments developed over a decade ago [4], and are intended for use with those younger children and their families. The 33-item P-CPQ has four subscales (oral symptoms, functional limitation, emotional well-being and social well-being), and the 14-item FIS has three (parental emotions, parental/family activity, and family conflict). The validity and responsiveness of the P-CPQ and the FIS have been demonstrated recently with children undergoing dental treatment under general anaesthesia in New Zealand [5,6].
OHRQoL measures tend to be long, and short-form versions are more desirable because respondent burden is minimised and the chance of the measures being used routinely in day-to-day practice is greater. Consistent with the usual pattern with such measures, the development of short-form versions has taken two different paths and resulted in two closely related sets of measures: the 13-item Early Childhood Oral Health Impact Scale, or ECOHIS [7], and the short-form P-CPQ (both 8-and 16-item versions) and FIS (with 8 items) [8]. The ECOHIS was developed using the original 45-item pool used by Jokovic and Locker in developing the P-CPQ scales. The ECOHIS team obtained ratings of those items from health professionals (and associated staff and researchers) who were experienced in dealing with young children. The 36 items remaining from that process then underwent item reduction with a convenience sample of 30 parents of 3-5-year-old children with a range of dental care needs. This resulted in 13 items (9 child-related and 4 on family impact) which were then field-tested with a convenience sample of parents/caregivers of 5-year-olds selected for participation in a larger epidemiological study. Those children had not been selected specifically because they had early childhood caries (ECC). By contrast, the short-form P-CPQ and FIS measures [8] were developed and tested by item impact analysis of data collected in two New Zealand studies (in Wellington and Auckland) of changes in OHRQoL in ECC-affected children undergoing dental treatment under general anaesthesia [5,6]. The item impact analyses were undertaken with the Wellington study data-set, and the examination of validity and responsiveness was done with the Auckland one.
Cross-sectional construct validity, reliability and responsiveness have been shown to be acceptable for both the ECOHIS [7,9,10] and the short-form P-CPQ and FIS measures [8], but there has been no direct comparison of their properties to date. It could be argued that the similarity in item content of the ECOHIS's child section and the 8-item short-form P-CPQ (whereby the latter contains 5 of the 9 items in the former) means that there would be little difference between them in performance. By contrast, the family section of the ECOHIS and the 8-item FIS differ considerably; the former comprises the two domains of parent distress and family function, and the latter includes the three domains of parental emotions, parental/family activity and family conflict, all of which were in the original FIS. Only 3 items are common to both scales, and the ECOHIS omits the item pertaining to disrupted sleep, an impact which most parents of ECC-affected children would rate as being important. Its omission most likely results from the use of an epidemiological sample (rather than a clinical one) for field-testing the ECOHIS. It might therefore be expected that the real interest in making a direct comparison of the two instruments lies in the relative performance of the components which measure the impact of a child's condition upon the family and household. Accordingly, the aim of this study was to compare the psychometric properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales among New Zealand children with early childhood caries who underwent treatment under general anaesthesia (GA).

Methods
This secondary data analysis used data from pretest/posttest clinical studies of consecutive clinical convenience samples undertaken in Wellington in 2005 [5] and Auckland in 2010/11 [6]. Each study obtained prior ethical approval, and written informed consent was obtained from participants before data collection. Parents of children with ECC were asked to complete questionnaires before and after dental care provided to the child under general anaesthetic (GA). Those contained the full P-CPQ and FIS instruments (and therefore also the ECOHIS) and the global rating question "How much is your child's overall wellbeing affected by the condition of his/her teeth, lips, jaws or mouth?", scored on a 5-point scale ranging from 'Excellent' to 'Poor' (and asked before the other scales). The reference period for the baseline questionnaire was 3 months. Follow-up evaluations took place 1-3 weeks after the child's procedure, and the follow-up questionnaire asked about the period since the child's operation. Full details of the methods used in each study are in those earlier reports [5,6]. The IRBs were the Central Regional Ethics Committee for the Wellington sample, and the Northern X Regional Ethics Committee for the Auckland sample.

Data analysis
Cross-sectional analyses were undertaken with the Wellington data-set, while the Auckland one was used for the longitudinal analyses, as in the previouslyreported development of the P-CPQ-8, P-CPQ-16 and FIS-8 short forms [8]. Scale and subscale scores for those and the ECOHIS were computed after their internal consistency reliability was determined using Cronbach's alpha. Cross-sectional construct validity was determined by scrutinising the gradient in means for pre-treatment scores across the global item categories of how much the child's oral condition affected his/her overall well-being. The responsiveness of the various short forms was determined by computing change scores (through subtracting post-treatment scores from pretreatment scores, where a positive change score represented improvement in OHRQoL), and testing the significance of the observed changes using Wilcoxon paired tests. The magnitude of change was represented by effect sizes, calculated by dividing a mean change score by the standard deviation of the pre-treatment score. Effect size statistics of less than 0.2 indicate a small clinically meaningful magnitude of change, 0.2 to 0.7 a moderate change, and greater than 0.7 a large change.

Results
Descriptive data on the participants at baseline and followup are presented by sample in Table 1. There were fewer Europeans and more Pacific Island children in the Auckland sample, and it was slightly younger, on average. The follow-up rate in the Auckland sample was higher than that in the Wellington one.
Data on the child-focused scales' item content and the outcome of the item impact analyses are presented in Table 2. The three scales differ in their sampling of the original four domains: the ECOHIS-Child includes one oral symptoms item, four functional limitations items, and two each from the emotional well-being and social well-being domains; by contrast, the P-CPQ-16 and P-CPQ-8 respectively sample four and two items from each of those. The ECOHIS-Child scale includes a number of items which scored relatively low in the item impact analysis.
Data on the family-focused scales' item content and the outcome of the item impact analyses are presented in Table 3. The two scales differ in their sampling of the original three domains: the ECOHIS-Family includes the two items with the greatest impact in the parental emotions domain, the third highest-impact item from the parental/family activity domain, and no item from the family conflict domain. It also includes the financial difficulties item which was not included in the original Family Impact Scale. The FIS-8 includes two parental emotions items, four parental/family items, and two items from the family conflict domain.
Data depicting the scales' cross-sectional construct validity are presented in Figure 1. All scales showed ascending gradients in their mean scores across the response categories of the global item. The gradient for the ECOHIS-Child scores was as steep as that observed with the P-CPQ-16 (and greater than that seen with the P-CPQ-8), whereas the gradient seen for the ECOHIS-Family was not as marked as that observed with the FIS-8.

Examination of responsivenessthe Auckland sample
Data on changes associated with treatment are presented in Table 4. The three child-focused measures showed similar effect sizes. The two family-focused measures were also similar.

Discussion
This study set out to compare the properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales, using data obtained from the parents of New Zealand children with early childhood caries who underwent treatment under GA. It has found that the ECOHIS-Child and the P-CPQ scales are very similar in their internal consistency reliability, cross-sectional construct validity and responsiveness, but that the ECOHIS-Family and the FIS-8 differ in some important ways, despite being similar in their responsiveness.
Before discussing the findings, the limitations of the study must be considered. First, it was a secondary analysis of data collected using the long-form (original) versions of the instruments. The extracted data were used not only in the current comparison, but also in the development of the short-form P-CPQ and FIS measures. Second, ethical concerns meant that we did not investigate test-retest reliability: it would have been an imposition on parents who had already been through a stressful time. Third, we did not investigate family structure and functioning, perhaps compromising our investigation of family impact and how it changed.
Turning to the findings, it appears that, at least for determining changes in OHRQoL associated with treatment for early childhood caries, the ECOHIS-Child and the short-form P-CPQ scales are comparable. This means that investigators planning to use a parentreported measure of child OHRQoL in monitoring the outcomes of treatment could use either measure, whether the ECOHIS or the 8-or 16-item version of the P-CPQ. However, it could be argued that, other factors being equal, it is preferable to use a measure which adequately covers all four domains. In this respect, the ECOHIS-Child's relative oversampling of the functional limitations domain and under-sampling of the oral symptoms one may be problematic. As discussed earlier, that most likely reflects the development process for that measure, where field-testing used an epidemiological sample (rather than a clinical one), and there may not have been the same prevalence or impact of symptoms in that sample. Thus, either of the two short-form P- There were no significant sociodemographic differences between those who were followed up and those who were not.
CPQ scales would be preferable for work in clinical samples where disease levels are high, and the ECOHIS-Child might be better deployed in epidemiological survey work.
Turning to the family impact measures, the ECOHIS-Child fell short of the FIS-8 on all aspects except responsiveness (where it was equivalent). Its internal consistency reliability was short of the 0.80 which is deemed to be acceptable [11] (although that is partly a consequence of the lower number of items). The data presented in Figure 1 indicate that its cross-sectional construct validity was inferior (if, indeed, the gradient in mean scores across the response categories for the global question can be taken to be an adequate representation  of that). However, its face validity is arguably its greatest weakness, with one of the three family impact domains not sampled at all, and only one item representing another. To be fair, it does use the two parental emotions items which had the greatest impact, but the undersampling of the parental/family activity and family conflict domains is problematic, and the omission of the disrupted sleep item is particularly so, given its high impact in the New Zealand data. This difference in item content clearly reflects the scales' different provenance: that of the FIS-8 arises from its testing in children with severe caries, whereas the epidemiological origins of the ECOHIS-Family mean that the more severe effects on families were not sufficiently prevalent to make an impact. A noteworthy inclusion in the ECOHIS-Family is the item pertaining to the child's condition causing financial difficulty for the household. That particular item is not included in the FIS-8 because of its low impact. It placed 11 th out of the 14 original items, meaning that it would not have been included in the FIS-8 even if it had been seriously considered (its rank in the Auckland sample was no different from that in the Wellington sample, either). It is also worth considering that the relevance of such an item would differ according to the health system in which the instrument was used, and that would unnecessarily complicate international comparisons.

Conclusions
This investigation of the properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales in New Zealand children undergoing dental treatment under GA for early childhood caries has found that the ECOHIS scales have some weaknesses which undermine their suitability for use with children with that condition. They may be better deployed in epidemiological survey  work rather than in health services research with highcaries-experience samples. By contrast, the P-CPQ-8, P-CPQ-16 and the FIS-8 seem to be well suited for the latter, but their epidemiological utility remains to be demonstrated.