Research | Open | Published:
Comparison of the self-administered and interviewer-administered modes of the child-OIDP
Health and Quality of Life Outcomesvolume 6, Article number: 40 (2008)
The mode of questionnaire administration may affect the estimates and applicability of oral health-related quality of life indicators. The aim of this study was to compare psychometrically the self-administered Child-OIDP index with the original interviewer-administered instrument.
This was a cross-sectional study of 144 consecutive children aged 9–16 years referred to orthodontic clinics in Bedfordshire. To compare the two administration modes of the Child-OIDP, the sample was randomly split in two groups. The two groups were analysed in terms of baseline characteristics, self-perceived measures (self-rated oral health, self-perceived need for braces, happiness with dental appearance, frequency of thinking about dental appearance), Child-OIDP performance scores and overall score and psychometric properties (criterion validity and internal reliability).
No significant difference between the two groups was found in relation to their sociodemographic profile and self-perceived measures. The self- and interviewer-administered Child-OIDP had identical mean scores and did not differ in recording any of the eight performances (p ≥ 0.206). For criterion validity, the correlation coefficients of the Child-OIDP with self-perceived measures were not different between the two modes of administration (p ≥ 0.118). Furthermore, the Cronbach's alpha values of the two groups were similar (p = 0.466).
This study demonstrated that the self-administered Child-OIDP performed the same as the original interviewer-administered mode, while at the same time reducing administration burden. This provides support for the use of the self-administered Child-OIDP. Further studies should focus on a more comprehensive psychometric evaluation.
This study assesses differences between two different administration modes of an oral health-related quality of life (OHRQoL) measure for children. The Child-OIDP  is an interviewer-administered OHRQoL measure that assesses the frequency and severity of oral impacts on eight daily life performances. Through its condition-specific feature, where the oral impacts are attributed to specific oral conditions according to the respondent's perceptions, the Child-OIDP can be used in needs assessment and for planning services . Indeed, its usefulness has been demonstrated in assessing general paedodontic treatment needs , as well as orthodontic treatment needs .
The effect of the mode of questionnaire administration on the estimates of health-related indicators is important [5, 6]. In general, interviewer-administered questionnaires are associated with higher response rates compared to self-administered instruments. On the other hand, they are also characterised by higher administration costs, hence limiting their practical applicability. Furthermore, interaction between respondent and interviewer may introduce bias in the estimates, while self-administered questionnaires may suffer from respondent bias, through for example exclusion of participants with reading difficulties. However, the most important conceptual issue relates to the comparability between data collected with interviewer- and self-administered questionnaires [5–7].
Studies indicate that in general self-administered questionnaires and face-to-face interviews provide similar estimates of self-assessed status [7–10], but others have expressed a preference for face-to-face interviews . Previous studies comparing self- and interviewer-administered modes of health-related quality of life questionnaires showed that both performed successfully in terms of psychometric properties [12, 13]. However, studies have also shown differences between interviews and self-administrations, with the former providing a more favourable picture of quality of life [6, 12–15]. A study on OHRQoL that used interviews and self-administrations of two measures (the OIDP and the OHIP-14) in a primary care department of a dental hospital showed that OIDP overall scores were unrelated to the administration mode ; however, no comparison of psychometric properties was carried out.
In child populations in particular, the assessment of OHRQoL should be considered in the light of the cognitive development of children , especially as complex language or conditional sentences do not become common until the age of 11–12 years . Therefore, child-specific measures should avoid using complex constructs if they are to be applied to younger age groups. Indeed, quality of life measures in relation to both general and oral health have performed satisfactorily as self-administered instruments in even younger populations [19, 20]. The administration of the Child-OIDP involves an individual face-to-face interview with each child. In order to reduce interview time and respondent burden, the use of pictures have been used in the Thai version of Child-OIDP [1, 21] while a shorter version of the OIDP based only on the assessment of frequency of oral impacts has been used in other settings [22, 23]. A self-administered Child-OIDP questionnaire would be more practical and cost effective than the current face-to-face interview and would further facilitate the applicability of the instrument in both clinical practice and population epidemiological survey settings. Therefore, the aim of this study was to compare psychometrically the self-administered Child-OIDP with the original interviewer-administered instrument.
This was a cross-sectional study of 144 consecutive children aged 9–16 years referred to orthodontic clinics in the Bedfordshire Personal Dental Service (PDS) for orthodontic diagnosis and treatment. By consecutive, we mean that all children referred to orthodontic clinics were included in the study. The Orthodontic Personal Dental Services (PDS) Pilot Scheme in Bedfordshire Heartlands Primary Care Trust (PCT) involves independent orthodontic practitioners with a contract with the local PCT. Its aim was to prioritise and provide orthodontic services to children with the greatest oral health needs . The study focussed on a sample of orthodontic patients, as this is an important oral health issue at this age group. Furthermore, a patient rather than a school-based sample would be expected to have higher prevalence of oral impacts and would also allow for the assessment of practicality of using the self-administered Child-OIDP in a clinical setting.
Ethical approval was obtained from the Brent Medical Ethics Committee, the Research and Development Panel of the Bedfordshire Heartlands Primary Care Trust and the Research and Development Directorate of the University College London Hospitals National Health Service Trust.
For practical reasons, both versions of the Child-OIDP questionnaire were administered on the same visit and with the same order of administration (self-administered first, interviewer-administered second). This may introduce bias in the responses of the participants and result in extensive agreement between the two versions. Consequently, in order to address this potential bias and assess whether there are differences between the two modes of the Child-OIDP administration, namely the self- versus the interviewer-administered questionnaire, the sample was randomly split into two groups of 72 children each and one Child-OIDP version was used per child. The self-administered version from one group, hereafter named the Self-administered Questionnaire (SAQ) group was compared with the interviewer-administered version from the other group, hereafter named Face-To-Face Interview (FTFI) group.
For the Child-OIDP interview [1, 25], children were first given a list of common oral problems and were asked whether they had experienced any of them within the last 3 months. Then, the single interviewer asked about difficulties in daily life caused by the problems that they marked on the list. The impacts of oral problems on daily life were quantified by using frequency and severity scores (scales from 1 to 3) for difficulty in carrying out 8 daily life performances (eating, speaking, cleaning mouth, sleeping, emotion, smiling, study, and social contact). If no impact was reported, then a zero score was assigned. Performance scores were calculated by multiplying the frequency and severity scores, while the overall Child-OIDP score is the sum of the 8 performance scores (ranging from 0 to 72) multiplied by 100 and divided by 72. To facilitate its appropriateness for self-completion, the self-administered Child-OIDP used a different, more user-friendly layout, with clear guiding instructions throughout the questionnaire, than the interviewer-administered Child-OIDP. In addition, the content and language were slightly simplified by avoiding some technical terms (e.g. in the common oral problems list, "erupting permanent tooth" was changed into "a new tooth pushing through") and using a single question for the assessment of frequency of oral impacts*. A researcher was available to identify potential difficulties in completing the self-administered questionnaire and address queries by the children.
The self-administered questionnaire also contained socio-demographic information; age, sex and postcode. The postcode data was provided by the parent and was used to calculate the level of social deprivation using the Index of Multiple Deprivation (IMD)  that combines indicators across seven domains (income deprivation, employment deprivation, health deprivation and disability, education, skills and training deprivation, barriers to housing and services, living environment deprivation and crime) into a single deprivation rank. Based on the aforementioned characteristics, IMD scores are available for every postcode in England. According to the IMD distribution of the study sample, participants were categorised into high (two highest IMD quintiles) and low (three lowest IMD quintiles) deprivation groups. In addition, the self-administered questionnaire for the children contained questions about self-rated oral health status (5-point scale ranging from "very poor" to "very good"), self-perceived need for braces (4-point scale from "not at all" to "a lot"), frequency of thinking about dental appearance (5-point scale from "never" to "almost all the time") and satisfaction with dental appearance (5-point scale from "very unhappy" to "very happy"). Based on their distribution in the sample, they were all further categorised into 3-point scales for the analysis.
The analysis started with a baseline comparison of socio-demographic characteristics and self-perceived measures between the SAQ and FTFI groups. This was done using Chi-square test or Mann-Whitney test. In addition, performance and overall Child-OIDP scores for both groups were calculated and statistically compared through the Mann-Whitney test.
Thereafter, the psychometric properties for the self-administered and interviewer-administered Child-OIDP questionnaire were first assessed individually, and then, compared with each other. In this study, the psychometric testing refers to criterion validity and internal reliability (consistency).
Criterion validity of each mode of administration was assessed against 4 proxy measures because of the lack of a gold standard to measure oral health-related quality of life. First, the correlation of the Child-OIDP score with four subjective oral health measures (self-rated oral health status, self-perceived need for braces, frequency of thinking about dental appearance and satisfaction with dental appearance) was estimated for each group by means of the Spearman's rho coefficient. Then, each one of these four correlations was compared between the FTFI and SAQ groups using Fisher's Z-transformation; that is, changing correlation values to Z-scores, and then using Fisher's Z test for the statistical comparison [27, 28].
To assess the internal reliability of each mode of administration, inter-item correlations among the 8 performances scores were calculated as correlation matrices. Then, the comparison of the 28 inter-item correlations between the FTFI and SAQ groups was carried out in 2 stages: correlations were first compared as matrices using an asymptotic Chi-squared test , and if a difference was found at that level, individual comparisons were subsequently performed, using Fisher's transformation, to identify which ones of the 28 inter-item correlations differed between groups.
Finally, corrected item-total correlations and Cronbach's alpha coefficient were also calculated for each group. The item-total correlations were then compared between groups using the Fisher's Z transformation whereas Cronbach's alphas were compared by means of the Feldt's W test .
Baseline comparison between FTFI and SAQ groups is shown in Table 1. There was no statistically significant difference between the two groups in any of the socio-demographic characteristics (sex, age, social deprivation), as well as in the four self-perceived variables evaluated (p ≥ 0.614 and p ≥ 0.422 in all cases respectively). Similarly, there were no statistically significant differences when the different performances and overall scores were compared between the FTFI and SAQ groups (p ≥ 0.206 in all cases). And the same was the case for the comparison between the two groups in relation to the prevalence of the different performances and the overall Child-OIDP (p ≥ 0.165 in all cases). Furthermore, most performances had quite similar scores between the two groups. The mean overall Child-OIDP scores for the FTFI and SAQ groups were identical (3.16) (Table 2).
In relation to differences in the criterion validity testing between the two modes of administration, the Child-OIDP score was significantly correlated in the FTFI group to frequency of thinking about dental appearance, happiness with dental appearance, as well as self-perceived need for braces (p = 0.001, 0.015 and 0.027 respectively), but not to the self-rated oral health status (p = 0.747). On the other hand, in the SAQ group the Child-OIDP score was correlated significantly only to the question on happiness with dental appearance (p = 0.039), but not to the frequency of thinking about dental appearance, self-perceived need for braces or self-rated oral health status (p = 0.297, 0.325 and 0.346 respectively). Nevertheless, there was no statistically significant difference when each correlation coefficient was compared between the two modes of Child-OIDP administration. In addition, irrespective of their statistical significance, the direction of all examined associations followed the expected pattern, depending on the wording of the correlated variables, and it was similar for the FTFI and SAQ groups; namely, negative for self-rated oral health and happiness with dental appearance and positive for self-perceived need for braces and frequency of thinking about dental appearance (Table 3).
The inter-item correlations for the FTFI and SAQ groups were estimated as correlation matrices for the internal reliability analysis of each mode of administration. Two out of 28 inter-item correlations in the FTFI group and 8 out of 28 inter-item correlations in the SAQ group were negative. However, none of them was statistically different from zero (p > 0.05 in all cases). A statistically significant difference was found when the correlation matrices were compared between both groups using an asymptotic Chi-squared test (p < 0.001). But during the subsequent individual comparisons, only 6 out of 28 inter-item correlations differed between the FTFI and SAQ groups (p < 0.05), with 3 of these 6 correlations including the performance on social contact.
The corrected item-total correlations ranged between 0.13 and 0.52 for the FTFI group and between -0.04 and 0.48 for the SAQ group (Table 4). Cronbach's alpha coefficient was 0.54 and 0.55 for the FTFI and SAQ groups respectively. The alpha coefficient decreased when any performance was deleted from the Child-OIDP, with the exception of sleeping and speaking in the SAQ group. None of the item-total correlation coefficients was statistically different between the FTFI and SAQ groups (p > 0.086 in all cases). The same was the case for Cronbach's alpha (p = 0.466).
This is the first study that assessed potential differences between the original interviewer-administered and the self-administered Child-OIDP. The self-administered version of the Child-OIDP performed similarly to the interviewer-administered original version of the instrument. Despite the demonstrated appropriateness of the interviewer-administered Child-OIDP in different cultural settings [1, 25, 31, 32], an equivalent self-administered instrument would be more brief and cost-effective than face-to-face interviews, as it would not require an interviewer for administering the questionnaire. This would make the index more applicable in clinical settings, as it would be less disruptive to clinic routines. Furthermore, it would also favour its wider use in epidemiological studies of child populations, usually carried out in school settings where brevity and supervision have a considerable impact on resources needed.
This study shows the potential of the self-administered Child-OIDP to give comparable results to the interviewer-administered version, while at the same time reducing administrative burden. Children took less than 5 minutes to complete the self-administered Child-OIDP, without needing any help or clarification. This highlights its brevity and ease of understanding. Furthermore, unlike studies on self-administered questionnaires [11, 33] and in contrast to a previous study using the OIDP which indicated lower response rates for a self-administered version , all children in this study fully completed the self-administered questionnaire, without requiring any clarifications or having queries about it, hence showing that the self-administration of the Child-OIDP was not associated with lower response rates or missing responses. This is encouraging and indicates that the concepts and wording of the Child-OIDP questions are appropriate for this age group, especially taking into consideration that a few children were quite young (9–10 year-olds), when comprehensiveness of complex questions goes beyond their capability and cognitive development .
In order to avoid any potential for bias in the comparison between the two modes, the sample was randomly divided into two equal groups and the comparison between the self-administered and the original interviewer-administered Child-OIDP was based on comparisons between the two groups. This approach has also been used in previous studies on modes of questionnaire administration for the assessment of sexual behaviour , smoking behaviour  and mental health . In addition to their random selection, the comparability between the two groups was also established by showing no differences in their sociodemographic and subjective oral health backgrounds.
The similarities in the results between the self-administered and the original interviewer-administered Child-OIDP covered a variety of different aspects. First, there were no differences between the two groups in any of the eight performances or the overall Child-OIDP score. Moreover, most performances had similar mean scores for the two administration modes, while the overall Child-OIDP score was identical. More importantly, there were no differences between the self- and the interviewer-administered Child-OIDP in their associations with a number of different subjective measures of oral health that ranged from the broader measure of self-rated oral health status to questions more closely related to appearance and orthodontic treatment need, such as self-perceived need for braces. In terms of internal consistency, there was no significant difference between the FTFI and the SAQ groups in relation to any of the 8 item-total correlations as well as the overall alpha, while differences between the two groups were located in only 6 of the 28 examined inter-item correlations.
The results of this study are in accordance with previous studies assessing different questionnaire administration modes. They showed that there were no differences in health status and behaviours [7–10], and no differences in the psychometric performances related to health-related quality of life [12, 13]. In line with a previous study using the OIDP  and one on EuroQoL , but in contrast to other relevant studies [6, 12–15], we found no effect of the administration mode on the prevalence estimates of the outcome measure, as shown by the identical Child-OIDP scores between the two administration modes. However, the different settings and disease profiles of the samples limit comparability of our results with the aforementioned studies.
The nature of this study was to focus on differences between the two administration modes, not on the actual evaluation of the psychometric properties of the self-administered Child-OIDP. Indeed, the Cronbach's alpha was relatively low compared to respective figures from most other studies on Child-OIDP and some of the associations between the Child-OIDP and proxy measures were not significant. While the low value of alpha is affected by the nature of the Child-OIDP index and the fact that it contains only 8 items, the lack of statistical significance of the validity testing associations was mainly due to the limited sample size (n = 72 in each group), as we purposefully divided the sample into two groups in order to assess differences between the two modes of administration. Future studies with a different design, such as using the two administration modes per child to address the potential order effect and test-retest evaluation, are required to corroborate our results. The comprehensive psychometric evaluation of the self-administered Child-OIDP should be undertaken using a larger sample. Furthermore, this sample consisted of children referred for orthodontic treatment. A future study should also extend these results by using a general child population.
The self-administered and the original interviewer-administered Child-OIDP performed similarly, thus providing support for the self-administration of the index. Further studies are needed on the comprehensive psychometric evaluation of the self-administered Child-OIDP.
Gherunpong S, Tsakos G, Sheiham A: Developing and evaluating an oral health-related quality of life index for children; the CHILD-OIDP. Community Dent Health 2004, 21: 161–169.
Gherunpong S, Sheiham A, Tsakos G: A sociodental approach to assessing children's oral health needs: integrating an oral health-related quality of life (OHRQoL) measure into oral health service planning. Bull World Health Organ 2006, 84: 36–42. 10.2471/BLT.05.022517
Gherunpong S, Tsakos G, Sheiham A: A sociodental approach to assessing dental needs of children: concept and models. Int J Paediatr Dent 2006, 16: 81–88. 10.1111/j.1365-263X.2006.00701.x
Gherunpong S, Tsakos G, Sheiham A: A socio-dental approach to assessing children's orthodontic needs. Eur J Orthod 2006, 28: 393–399. 10.1093/ejo/cji114
Guyatt GH, Feeny DH, Patrick DL: Measuring health-related quality of life. Ann Intern Med 1993, 118: 622–629.
McHorney CA, Kosinski M, Ware JE Jr.: Comparisons of the costs and quality of norms for the SF-36 health survey collected by mail versus telephone interview: results from a national survey. Med Care 1994, 32: 551–567. 10.1097/00005650-199406000-00002
Siemiatycki J: A comparison of mail, telephone, and home interview strategies for household health surveys. Am J Public Health 1979, 69: 238–245.
Durant LE, Carey MP: Self-administered questionnaires versus face-to-face interviews in assessing sexual behavior in young women. Arch Sex Behav 2000, 29: 309–322. 10.1023/A:1001930202526
Kaplan CP, Hilton JF, Park-Tanjasiri S, Perez-Stable EJ: The effect of data collection mode on smoking attitudes and behavior in young African American and Latina women. Face-to-face interview versus self-administered questionnaires. Eval Rev 2001, 25: 454–473.
Vuillemin A, Oppert JM, Guillemin F, Essermeant L, Fontvieille AM, Galan P, Kriska AM, Hercberg S: Self-administered questionnaire compared with interview to assess past-year physical activity. Med Sci Sports Exerc 2000, 32: 1119–1124. 10.1097/00005768-200006000-00013
O'Toole BI, Battistutta D, Long A, Crouch K: A comparison of costs and data quality of three health survey methods: mail, telephone and personal home interview. Am J Epidemiol 1986, 124: 317–328.
Cook DJ, Guyatt GH, Juniper E, Griffith L, McIlroy W, Willan A, Jaeschke R, Epstein R: Interviewer versus self-administered questionnaires in developing a disease-specific, health-related quality of life instrument for asthma. J Clin Epidemiol 1993, 46: 529–534. 10.1016/0895-4356(93)90125-K
Weinberger M, Oddone EZ, Samsa GP, Landsman PB: Are health-related quality-of-life measures affected by the mode of administration? J Clin Epidemiol 1996, 49: 135–140. 10.1016/0895-4356(95)00556-0
Cheung YB, Goh C, Thumboo J, Khoo KS, Wee J: Quality of life scores differed according to mode of administration in a review of three major oncology questionnaires. J Clin Epidemiol 2006, 59: 185–191. 10.1016/j.jclinepi.2005.06.011
Lyons RA, Wareham K, Lucas M, Price D, Williams J, Hutchings HA: SF-36 scores vary by method of administration: implications for study design. J Public Health Med 1999, 21: 41–45. 10.1093/pubmed/21.1.41
Robinson PG, Gibson B, Khan FA, Birnbaum W: A comparison of OHIP 14 and OIDP as interviews and questionnaires. Community Dent Health 2001, 18: 144–149.
Eiser C, Morse R: The measurement of quality of life in children: past and future perspectives. J Dev Behav Pediatr 2001, 22: 248–256.
Owen P: Language development. 4th edition. Edited by: Bacon A. Boston; 1996.
Humphris G, Freeman R, Gibson B, Simpson K, Whelton H: Oral health-related quality of life for 8–10-year-old children: an assessment of a new measure. Community Dent Oral Epidemiol 2005, 33: 326–332. 10.1111/j.1600-0528.2005.00220.x
Meuleners LB, Lee AH, Binns CW, Lower A: Quality of life for adolescents: assessing measurement properties using structural equation modelling. Qual Life Res 2003, 12: 283–290. 10.1023/A:1023221913292
Gherunpong S, Tsakos G, Sheiham A: The prevalence and severity of oral impacts on daily performances in Thai primary school children. Health Qual Life Outcomes 2004, 2: 57. 10.1186/1477-7525-2-57
Astrom AN, Okullo I: Validity and reliability of the Oral Impacts on Daily Performance (OIDP) frequency scale: a cross-sectional study of adolescents in Uganda. BMC Oral Health 2003, 3: 5. 10.1186/1472-6831-3-5
Masalu JR, Astrom AN: Applicability of an abbreviated version of the oral impacts on daily performances (OIDP) scale for use among Tanzanian students. Community Dent Oral Epidemiol 2003, 31: 7–14. 10.1034/j.1600-0528.2003.00011.x
Kettler CJ: The bedfordshire PDS orthodontic pilot. Br J Orthod 1999, 26: 342–345. 10.1093/ortho/26.4.342
Yusuf H, Gherunpong S, Sheiham A, Tsakos G: Validation of an English version of the Child-OIDP index, an oral health-related quality of life measure for children. Health Qual Life Outcomes 2006, 4: 38. 10.1186/1477-7525-4-38
The Office of the Deputy Prime Minister: The English Indices of Deprivation 2004 - Summary (revised). 2004.
Dawson-Saunders B, Trapp RG: Basic and clinical biostatistics. 2nd edition. New Jersey, Prentice-Hall International; 1994:162–187.
Fisher RA: On the "probable error" of a coefficient of correlation deduced from a small sample. Metron 1921, 1: 3–32.
Jennrich RI: An asymptotic Chi2 test for the equality of two correlation matrices. J Am Stat Assoc 1970, 65: 904–912. 10.2307/2284596
Feldt LS, Kim S: Testing the difference between two alpha coefficients with small samples of subjects and raters. Educ Psychol Mes 2006, 66: 589–600. 10.1177/0013164405282488
Tubert-Jeannin S, Pegon-Machat E, Gremeau-Richard C, Lecuyer MM, Tsakos G: Validation of a French version of the Child-OIDP index. Eur J Oral Sci 2005, 113: 355–362. 10.1111/j.1600-0722.2005.00230.x
Bernabe E, Sheiham A, Tsakos G: A comprehensive evaluation of the validity of Child-OIDP: further evidence from Peru. Community Dent Oral Epidemiol 2007., (in press):
Polednak AP, Lane DS, Burg MA: Mail versus telephone surveys on mammography utilization among women 50–75 years old. Med Care 1991, 29: 243–250. 10.1097/00005650-199103000-00006
Fournier L, Kovess V: A comparison of mail and telephone interview strategies for mental health surveys. Can J Psychiatry 1993, 38: 525–533.
Wu AW, Jacobson DL, Berzon RA, Revicki DA, van der HC, Fichtenbaum CJ, Saag MS, Lynn L, Hardy D, Feinberg J: The effect of mode of administration on medical outcomes study health ratings and EuroQol scores in AIDS. Qual Life Res 1997, 6: 3–10.
The study would not have been carried out without the considerable contribution of Ms S. Gregory, Consultant in Dental Public Health, Bedfordshire Primary Care Trust, and of the orthodontists participating in the Orthodontic PDS Pilot Scheme (C. Kettler, H. Turner, S. Hindle, J. Evans, D. Chappell, M. Wolkenstein, R. Darbar, Y. Mohamed).
EB is supported by the Programme Alâan, the European Union Programme of High Level Scholarships for Latin America, Scholarship N° E06D1000352PE.
*The self-administered Child-OIDP can be obtained from the authors on request.
Declaration of competing interests
The authors declare that they have no competing interests.
GT conceived of the study, participated in the study design and led the writing of the manuscript, EB carried out the data analysis and participated in writing the manuscript, KOB participated in the study design and critically reviewed the manuscript, AS conceived of the study and critically reviewed the manuscript, CO participated in the study design, carried out the data collection and participated in writing the manuscript. All authors read and approved the final manuscript.