Open Access

Comparison of the ECOHIS and short-form P-CPQ and FIS scales

  • William M Thomson1Email author,
  • Lyndie A Foster Page1,
  • Penelope E Malden2,
  • Wanda N Gaynor3 and
  • Norhasnida Nordin1
Health and Quality of Life Outcomes201412:36

DOI: 10.1186/1477-7525-12-36

Received: 24 October 2013

Accepted: 5 March 2014

Published: 11 March 2014

Abstract

Background

The development of short-form versions of child oral–health-related quality of life (OHRQoL) scales has resulted in two closely related sets of measures. We set out to compare the properties and responsiveness of the Early Childhood Oral Health Impact Scale (ECOHIS – both “child” and “family” versions) and short-form Parental-Caregiver Perceptions Questionnaire (P-CPQ) and the Family Impact Scale (FIS) measures among New Zealand children with early childhood caries who underwent treatment under general anaesthesia (GA).

Methods

Secondary analysis of data from pretest/post-test clinical studies of consecutive clinical convenience samples undertaken in Wellington in 2005 and Auckland in 2010/11, with cross-sectional analyses using the former, and longitudinal analyses using the latter.

Results

Cronbach’s α values for the ECOHIS-Child, P-CPQ-16 and P-CPQ-8 were 0.80, 0.88 and 0.80 respectively, and 0.83 and 0.68 (respectively) for the FIS-8 and the ECOHIS-Family. All scales showed acceptable cross-sectional construct validity, although that of the ECOHIS-Family was not as marked as that observed with the FIS-8. Responsiveness was acceptable, with the three child-focused measures showing similar effect sizes. The two family-focused measures were also similar.

Conclusions

The ECOHIS-Child and the P-CPQ scales are very similar in their properties, but the ECOHIS-Family falls short of the FIS-8 in some important ways. The ECOHIS scales may be better deployed in epidemiological survey work rather than in health services research, whereas the P-CPQ-8, P-CPQ-16 and the FIS-8 seem to be well suited for the latter (particularly with children suffering from severe caries), but their epidemiological utility remains to be demonstrated.

Keywords

Oral-health-related quality of life Children Short-form scales Measurement scales

Background

The last decade or so has seen considerable progress in the development, testing and validation of child measures of oral-health-related quality of life (OHRQoL). A number of competing measures now exist [13], with most of those intended for use with children who are old enough to give valid and reliable responses. Measuring oral health in children younger than six years of age has necessitated the use of scales which use adult informants (such as parents) who know the child well. The Parental-Caregiver Perceptions Questionnaire (P-CPQ) and the Family Impact Scale (FIS) are part of the Child Oral-Health-Related Quality of Life (COHQoL) suite of instruments developed over a decade ago [4], and are intended for use with those younger children and their families. The 33-item P-CPQ has four subscales (oral symptoms, functional limitation, emotional well-being and social well-being), and the 14-item FIS has three (parental emotions, parental/family activity, and family conflict). The validity and responsiveness of the P-CPQ and the FIS have been demonstrated recently with children undergoing dental treatment under general anaesthesia in New Zealand [5, 6].

OHRQoL measures tend to be long, and short-form versions are more desirable because respondent burden is minimised and the chance of the measures being used routinely in day-to-day practice is greater. Consistent with the usual pattern with such measures, the development of short-form versions has taken two different paths and resulted in two closely related sets of measures: the 13-item Early Childhood Oral Health Impact Scale, or ECOHIS [7], and the short-form P-CPQ (both 8- and 16-item versions) and FIS (with 8 items) [8]. The ECOHIS was developed using the original 45-item pool used by Jokovic and Locker in developing the P-CPQ scales. The ECOHIS team obtained ratings of those items from health professionals (and associated staff and researchers) who were experienced in dealing with young children. The 36 items remaining from that process then underwent item reduction with a convenience sample of 30 parents of 3-5-year-old children with a range of dental care needs. This resulted in 13 items (9 child-related and 4 on family impact) which were then field-tested with a convenience sample of parents/caregivers of 5-year-olds selected for participation in a larger epidemiological study. Those children had not been selected specifically because they had early childhood caries (ECC). By contrast, the short-form P-CPQ and FIS measures [8] were developed and tested by item impact analysis of data collected in two New Zealand studies (in Wellington and Auckland) of changes in OHRQoL in ECC-affected children undergoing dental treatment under general anaesthesia [5, 6]. The item impact analyses were undertaken with the Wellington study data-set, and the examination of validity and responsiveness was done with the Auckland one.

Cross-sectional construct validity, reliability and responsiveness have been shown to be acceptable for both the ECOHIS [7, 9, 10] and the short-form P-CPQ and FIS measures [8], but there has been no direct comparison of their properties to date. It could be argued that the similarity in item content of the ECOHIS’s child section and the 8-item short-form P-CPQ (whereby the latter contains 5 of the 9 items in the former) means that there would be little difference between them in performance. By contrast, the family section of the ECOHIS and the 8-item FIS differ considerably; the former comprises the two domains of parent distress and family function, and the latter includes the three domains of parental emotions, parental/family activity and family conflict, all of which were in the original FIS. Only 3 items are common to both scales, and the ECOHIS omits the item pertaining to disrupted sleep, an impact which most parents of ECC-affected children would rate as being important. Its omission most likely results from the use of an epidemiological sample (rather than a clinical one) for field-testing the ECOHIS. It might therefore be expected that the real interest in making a direct comparison of the two instruments lies in the relative performance of the components which measure the impact of a child’s condition upon the family and household. Accordingly, the aim of this study was to compare the psychometric properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales among New Zealand children with early childhood caries who underwent treatment under general anaesthesia (GA).

Methods

This secondary data analysis used data from pretest/post-test clinical studies of consecutive clinical convenience samples undertaken in Wellington in 2005 [5] and Auckland in 2010/11 [6]. Each study obtained prior ethical approval, and written informed consent was obtained from participants before data collection. Parents of children with ECC were asked to complete questionnaires before and after dental care provided to the child under general anaesthetic (GA). Those contained the full P-CPQ and FIS instruments (and therefore also the ECOHIS) and the global rating question “How much is your child’s overall well-being affected by the condition of his/her teeth, lips, jaws or mouth?”, scored on a 5-point scale ranging from ‘Excellent’ to ‘Poor’ (and asked before the other scales). The reference period for the baseline questionnaire was 3 months. Follow-up evaluations took place 1-3 weeks after the child’s procedure, and the follow-up questionnaire asked about the period since the child’s operation. Full details of the methods used in each study are in those earlier reports [5, 6]. The IRBs were the Central Regional Ethics Committee for the Wellington sample, and the Northern X Regional Ethics Committee for the Auckland sample.

Data analysis

Cross-sectional analyses were undertaken with the Wellington data-set, while the Auckland one was used for the longitudinal analyses, as in the previously-reported development of the P-CPQ-8, P-CPQ-16 and FIS-8 short forms [8]. Scale and subscale scores for those and the ECOHIS were computed after their internal consistency reliability was determined using Cronbach’s alpha. Cross-sectional construct validity was determined by scrutinising the gradient in means for pre-treatment scores across the global item categories of how much the child’s oral condition affected his/her overall well-being. The responsiveness of the various short forms was determined by computing change scores (through subtracting post-treatment scores from pre-treatment scores, where a positive change score represented improvement in OHRQoL), and testing the significance of the observed changes using Wilcoxon paired tests. The magnitude of change was represented by effect sizes, calculated by dividing a mean change score by the standard deviation of the pre-treatment score. Effect size statistics of less than 0.2 indicate a small clinically meaningful magnitude of change, 0.2 to 0.7 a moderate change, and greater than 0.7 a large change.

Results

Descriptive data on the participants at baseline and follow-up are presented by sample in Table 1. There were fewer Europeans and more Pacific Island children in the Auckland sample, and it was slightly younger, on average. The follow-up rate in the Auckland sample was higher than that in the Wellington one.
Table 1

Number of participants in each sample at baseline and follow-up, by sociodemographic characteristics (brackets contain column percentages unless otherwise indicated)

 

Wellington

Auckland

Baseline characteristics

  

Number in sample

195

157

Number of females

95 (48.7)

68 (43.3)

Ethnic group

  

European

70 (35.9)

28 (17.8)a

Māori

54 (27.7)

31 (19.7)

Pacific Island

53 (27.2)

58 (36.9)

Other

18 (9.2)

40 (25.5)

Mean age of sample (sd)

5.5 (1.5)

4.8 (1.7)b

Number of preschoolers

82 (42.1)

74 (47.1)

Number assessed at follow-up

124 (63.6)

144 (91.7)c

aP < 0.05; Chi-square test.

bP < 0.05; independent samples t-test.

cThere were no significant sociodemographic differences between those who were followed up and those who were not.

Cross-sectional analyses – the Wellington sample

For internal consistency reliability, Cronbach’s α values for the ECOHIS-Child, P-CPQ-16 and P-CPQ-8 were 0.80, 0.88 and 0.80 respectively. For the FIS-8 and the ECOHIS-Family, they were 0.83 and 0.68 respectively.

Data on the child-focused scales’ item content and the outcome of the item impact analyses are presented in Table 2. The three scales differ in their sampling of the original four domains: the ECOHIS-Child includes one oral symptoms item, four functional limitations items, and two each from the emotional well-being and social well-being domains; by contrast, the P-CPQ-16 and P-CPQ-8 respectively sample four and two items from each of those. The ECOHIS-Child scale includes a number of items which scored relatively low in the item impact analysis.
Table 2

Comparison of item content and item impact of the ECOHIS and the 16- and 8-item versions of the P-CPQ scales (Wellington sample only)

  

Short-form scale

   

Item

Domaina

ECOHIS-child

P-CPQ-16

P-CPQ-8

Prevalenceb

Meanc

Impactd

Pain in the teeth, lips, jaws or mouth

OS

Included

Included

Included

66.2

2.5

166

Food caught in or between the teeth

OS

 

Included

Included

61.0

2.4

146

Been upset

EW

Included

Included

Included

56.9

2.5

142

Bad breath

OS

 

Included

 

53.3

2.5

133

Been irritable or frustrated

EW

Included

Included

Included

49.2

2.6

128

Difficulty biting or chewing firm foods

FL

Included

Included

Included

45.6

2.6

119

Taken longer than others to eat a meal

FL

 

Included

Included

44.1

2.7

119

Had trouble sleeping

FL

Included

Included

 

45.6

2.4

109

Breathed through the mouth

FL

 

Included

 

42.6

2.4

102

Had difficulty drinking or eating hot or cold foods

FL

Included

  

40.0

2.4

96

Been anxious or fearful

EW

 

Included

 

37.9

2.5

95

Acted shy or embarrassed

EW

 

Included

 

25.1

2.3

58

Missed school or preschool

SW

Included

Included

Included

23.1

2.4

55

Food stuck in the roof of the mouth

OS

 

Included

 

19.0

2.5

48

Had difficulty saying any words

FL

Included

  

19.0

2.4

46

Not wanted to talk to other children

SW

 

Included

Included

16.9

2.2

37

Had a hard time paying attention in school

SW

 

Included

 

12.8

2.6

33

Avoided smiling or laughing when around other children

SW

Included

Included

 

12.8

2.2

28

aOS = Oral Symptoms; FL = Functional Limitations; EW = Emotional Well-being; SW = Social Well-being.

bPercentage reporting it ‘Sometimes’, ‘Often’, or ‘Every day or almost every day’.

cMean item score among those reporting it ‘Sometimes’, ‘Often’, or ‘Every day or almost every day’.

dThe product of the prevalence multiplied by the mean score.

Data on the family-focused scales’ item content and the outcome of the item impact analyses are presented in Table 3. The two scales differ in their sampling of the original three domains: the ECOHIS-Family includes the two items with the greatest impact in the parental emotions domain, the third highest-impact item from the parental/family activity domain, and no item from the family conflict domain. It also includes the financial difficulties item which was not included in the original Family Impact Scale. The FIS-8 includes two parental emotions items, four parental/family items, and two items from the family conflict domain.
Table 3

Comparison of item content and item impact of the ECOHIS and the 8-item FIS, showing items ranked by impact (Wellington sample only)

Item

Domaina

ECOHIS-family

FIS-8

Prevalenceb

Meanc

Impactd

Felt guilty

PE

Included

Included

53.8

2.7

145

Been upset

PE

Included

Included

44.6

2.5

112

Had sleep disrupted

PF

 

Included

41.0

2.5

103

Required more attention from you or others in the family

PF

 

Included

29.2

2.5

73

Taken time off work

PF

Included

Included

25.6

2.4

61

Had less time for yourself or the family

PF

 

Included

22.1

2.5

55

Blamed you or another person in the family

FC

 

Included

17.9

2.3

41

Argued with you or others in the family

FC

 

Included

15.9

2.5

40

Caused financial difficulties for your family

Included

 

5.1

2.2

11

aPE = Parental Emotions; PF = Parental/Family activity; FC = Family Conflict.

bPercentage reporting it ‘Sometimes’, ‘Often’, or ‘Every day or almost every day’.

cMean item score among those reporting it ‘Sometimes’, ‘Often’, or ‘Every day or almost every day’.

dThe product of the prevalence and the mean score.

Data depicting the scales’ cross-sectional construct validity are presented in Figure 1. All scales showed ascending gradients in their mean scores across the response categories of the global item. The gradient for the ECOHIS-Child scores was as steep as that observed with the P-CPQ-16 (and greater than that seen with the P-CPQ-8), whereas the gradient seen for the ECOHIS-Family was not as marked as that observed with the FIS-8.
https://static-content.springer.com/image/art%3A10.1186%2F1477-7525-12-36/MediaObjects/12955_2013_Article_1421_Fig1_HTML.jpg
Figure 1

Mean scale scores by global rating of the child’s OHRQoL (Welllington sample; all score gradients were statistically significant at the P < 0.001 level).

Examination of responsiveness – the Auckland sample

Data on changes associated with treatment are presented in Table 4. The three child-focused measures showed similar effect sizes. The two family-focused measures were also similar.
Table 4

Scale responsiveness: comparison of mean scores in the ECOHIS-Child, P-CPQ-16, P-CPQ-8, ECOHIS-Family and FIS-8 at baseline and follow-up in the Auckland sample, with effect sizes (N = 157)

 

Pre-treatment

Post-treatment

   
 

Mean score (sd)

Range (number scoring 0)

Mean score (sd)

Range (number scoring 0)

Mean change score (sd)

Effect size

Effect size description

Child measures

       

ECOHIS-child

7.7 (5.6)

0-28 (11)

2.6 (3.2)a

0-16 (52)

5.2 (5.7)

0.9

Large

P-CPQ-16

15.7 (9.9)

0-50 (3)

4.4 (5.6)a

0-30 (43)

11.4 (9.9)

1.2

Large

P-CPQ-8

9.0 (5.5)

0-26 (3)

2.9 (3.2)a

0-12 (47)

6.1 (5.5)

1.1

Large

Family measures

       

ECOHIS-family

3.8 (3.2)

0-14 (26)

1.8 (2.1)a

0-10 (61)

2.0 (3.2)

0.6

Moderate

FIS-8

6.6 (5.6)

0-25 (18)

3.0 (3.7)a

0-17 (88)

3.6 (5.4)

0.6

Moderate

aDifference between baseline and follow-up scores significant at P < 0.0001; Wilcoxon paired test.

Discussion

This study set out to compare the properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales, using data obtained from the parents of New Zealand children with early childhood caries who underwent treatment under GA. It has found that the ECOHIS-Child and the P-CPQ scales are very similar in their internal consistency reliability, cross-sectional construct validity and responsiveness, but that the ECOHIS-Family and the FIS-8 differ in some important ways, despite being similar in their responsiveness.

Before discussing the findings, the limitations of the study must be considered. First, it was a secondary analysis of data collected using the long-form (original) versions of the instruments. The extracted data were used not only in the current comparison, but also in the development of the short-form P-CPQ and FIS measures. Second, ethical concerns meant that we did not investigate test-retest reliability: it would have been an imposition on parents who had already been through a stressful time. Third, we did not investigate family structure and functioning, perhaps compromising our investigation of family impact and how it changed.

Turning to the findings, it appears that, at least for determining changes in OHRQoL associated with treatment for early childhood caries, the ECOHIS-Child and the short-form P-CPQ scales are comparable. This means that investigators planning to use a parent-reported measure of child OHRQoL in monitoring the outcomes of treatment could use either measure, whether the ECOHIS or the 8- or 16-item version of the P-CPQ. However, it could be argued that, other factors being equal, it is preferable to use a measure which adequately covers all four domains. In this respect, the ECOHIS-Child’s relative oversampling of the functional limitations domain and under-sampling of the oral symptoms one may be problematic. As discussed earlier, that most likely reflects the development process for that measure, where field-testing used an epidemiological sample (rather than a clinical one), and there may not have been the same prevalence or impact of symptoms in that sample. Thus, either of the two short-form P-CPQ scales would be preferable for work in clinical samples where disease levels are high, and the ECOHIS-Child might be better deployed in epidemiological survey work.

Turning to the family impact measures, the ECOHIS-Child fell short of the FIS-8 on all aspects except responsiveness (where it was equivalent). Its internal consistency reliability was short of the 0.80 which is deemed to be acceptable [11] (although that is partly a consequence of the lower number of items). The data presented in Figure 1 indicate that its cross-sectional construct validity was inferior (if, indeed, the gradient in mean scores across the response categories for the global question can be taken to be an adequate representation of that). However, its face validity is arguably its greatest weakness, with one of the three family impact domains not sampled at all, and only one item representing another. To be fair, it does use the two parental emotions items which had the greatest impact, but the under-sampling of the parental/family activity and family conflict domains is problematic, and the omission of the disrupted sleep item is particularly so, given its high impact in the New Zealand data. This difference in item content clearly reflects the scales’ different provenance: that of the FIS-8 arises from its testing in children with severe caries, whereas the epidemiological origins of the ECOHIS-Family mean that the more severe effects on families were not sufficiently prevalent to make an impact.

A noteworthy inclusion in the ECOHIS-Family is the item pertaining to the child’s condition causing financial difficulty for the household. That particular item is not included in the FIS-8 because of its low impact. It placed 11th out of the 14 original items, meaning that it would not have been included in the FIS-8 even if it had been seriously considered (its rank in the Auckland sample was no different from that in the Wellington sample, either). It is also worth considering that the relevance of such an item would differ according to the health system in which the instrument was used, and that would unnecessarily complicate international comparisons.

Conclusions

This investigation of the properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales in New Zealand children undergoing dental treatment under GA for early childhood caries has found that the ECOHIS scales have some weaknesses which undermine their suitability for use with children with that condition. They may be better deployed in epidemiological survey work rather than in health services research with high-caries-experience samples. By contrast, the P-CPQ-8, P-CPQ-16 and the FIS-8 seem to be well suited for the latter, but their epidemiological utility remains to be demonstrated.

Declarations

Acknowledgments

We thank the New Zealand Dental Association Research Foundation for funding the Wellington study, and we acknowledge the staff of the Capital and Coast District Health Board Oral Health Service for their invaluable assistance. The New Zealand Ministry of Health is thanked for funding the Auckland study. We are grateful to the participants and their families.

Authors’ Affiliations

(1)
Sir John Walsh Research Institute, School of Dentistry, The University of Otago
(2)
Capital and Coast District Health Board
(3)
Oral Health Department, Auckland District Health Board, Greenlane Clinical Centre

References

  1. Jokovic A, Locker D, Stephens M, Kenny D, Tompson B, Guyatt G: Validity and reliability of a questionnaire for measuring child oral-health-related quality of life. J Dent Res 2002, 81: 459–463. 10.1177/154405910208100705View ArticlePubMed
  2. Broder HL, McGrath C, Cisneros GJ: Questionaire development: face validity and item impact testing of the Child Oral Health Impact Profile. Community Dent Oral Epidemiol 2007, 35(Suppl 1):8–19.View ArticlePubMed
  3. Abanto J, Tsakos G, Ardenghi TM, Paiva SM, Raggio DP, Sheiham A, Bonecker M: Responsiveness to change for the Brazilian Scale of Oral Health Outcomes for 5-year-old children (SOHO-5). Health Qual Life Outcomes 2013, 11: 137. 10.1186/1477-7525-11-137PubMed CentralView ArticlePubMed
  4. Locker D, Jokovic A, Stephens M, Kenny D, Thomson B, Guyatt G: Family impact of child oral and oro-facial conditions. Community Dent Oral Epidemiol 2002, 30: 438–448. 10.1034/j.1600-0528.2002.00015.xView ArticlePubMed
  5. Malden PE, Thomson WM, Jokovic A, Locker D: Changes in parent-assessed oral health-related quality of life among young children following dental treatment under general anaesthetic. Community Dent Oral Epidemiol 2008, 36: 108–117. 10.1111/j.1600-0528.2007.00374.xView ArticlePubMed
  6. Gaynor WN, Thomson WM: Changes in young children’s OHRQoL after dental treatment under general anaesthesia. Int J Paed Dent 2012, 22: 258–264. 10.1111/j.1365-263X.2011.01190.xView Article
  7. Patel BT, Rozier RG, Slade GD: Parental perceptions of children’s oral health: the Early Childhood Oral Health Impact Scale (ECOHIS). Health Quality Life Outcomes 2007, 5: 6. 10.1186/1477-7525-5-6View Article
  8. Thomson WM, Foster Page LA, Gaynor WN, Malden PE: Short-form versions of the Parental-Caregivers Perceptions Questionnaire (P-CPQ) and the Family Impact Scale (FIS). Community Dent Oral Epidemiol 2013, 41: 441–450.PubMed
  9. Li S, Malkinson S, Verroneau J, Allison PJ: Testing responsiveness to change for the early childhood oral health impact scale (ECOHIS). Community Dent Oral Epidemiol 2008, 36: 542–548. 10.1111/j.1600-0528.2008.00434.xView ArticlePubMed
  10. Lee GHM, McGrath C, Yiu CKY, King NM: Sensitivity and responsiveness of the Chinese ECOHIS to dental treatment under general anaesthesia. Community Dent Oral Epidemiol 2011, 39: 372–377. 10.1111/j.1600-0528.2010.00604.xView ArticlePubMed
  11. Fayers PM, Machin D: Quality of Life: Assessment, Analysis and Interpretation. Chichester: John Wiley and Sons; 2000:85–87.View Article

Copyright

© Thomson et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://​creativecommons.​org/​licenses/​by/​2.​0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://​creativecommons.​org/​publicdomain/​zero/​1.​0/​) applies to the data made available in this article, unless otherwise stated.

Advertisement