Skip to main content

Advertisement

Clinimetric evaluation and clinical outcomes of the Dutch version of the Chronic Ear Survey

Abstract

Background

To validate and evaluate the reliability of the Dutch version of the Chronic Ear Survey (CES) in patients suffering from Chronic Suppurative Otitis Media (CSOM) and to evaluate clinical outcomes of surgery using this questionnaire.

Methods

We developed the Dutch version of the CES (D-CES) using forward-backward translation of the original CES into the Dutch language. Next, patients with CSOM and controls completed the D-CES pre- and postoperatively. Internal consistency, test-retest reliability, known-group validity and convergent validity were evaluated. In addition to the D-CES, the Short Form 36 (SF-36) was administered to all participants to correlate D-CES data to quality of life.

Results

A total of 29 patients with CSOM scheduled for ear surgery were included. Our control group consisted of 26 patients scheduled for eye surgery, all without signs and symptoms of CSOM. Cronbachs’ α of the complete questionnaire was 0.69. The Intraclass Correlation Coefficients (ICCs), reflecting test-retest reliability, ranged between 0.69 and 0.82. Scores differed significantly between CSOM patients and controls with substantial lower (more impaired) D-CES scores in the CSOM group. Duration of complaints preoperatively and the presence of a dry ear and/or improvement of hearing postoperatively all had a significant impact on D-CES improvement scores. Small to moderate correlations were found between D-CES subscales and matching subscales of the SF-36.

Conclusion

The D-CES is an appropriate disease specific questionnaire to assess a patient’s perceived functional health in CSOM.

Background

Patient reported outcome measures (PROM’s) and impact of medical and/or surgical treatment on quality of life are of increasing importance in current medical practice, including the field of otology [1,2,3,4]. In addition to objective measures such as pure tone audiometry and Computer Tomography (CT), PROM’s provide supplemental information regarding the impact of symptoms on patient burden and patient’s perception of treatment efficacy [5]. Therefore the need for clinimetrically solid patient reported outcome measures is paramount. Koenraads et al. showed that especially the field of otology lacks validated questionnaires [6]. For the Dutch population, only a few questionnaires addressing Chronic Suppurative Otitis Media (CSOM) are available [1, 4]. As CSOM effects approximately 2% of the population we felt the urge to develop a Dutch PROM for this disease [7]. The version of the COMQ-12 by Van Dinther et al. has already been published as a ‘Dutch’ version [4]. However, there are substantial differences between Flemish and Dutch language. Therefore, in our opinion, the Flemish version is not suitable for the Dutch population. Currently, Bruinewoud et al. are developing a Dutch generic patient related outcome measure consisting of 34 items addressing the severity of ear complaints and their impact on quality of life [1]. According to our opinion, a generic questionnaire is not as useful when studying a well-defined subpopulation, for example a population of patients suffering from CSOM. The to be developed generic ear questionnaire is quite long and therefore time consuming to administer. Both arguments favoured the need for a short disease-specific functional health questionnaire for patients suffering from CSOM with which we are able to detect pre- and post-operative differences in health. With these demands in mind, the Chronic Ear Survey (CES) was potentially very appropriate. The CES is a validated outcomes measure for evaluating the impact of CSOM on adults [8]. This questionnaire was originally validated for the English language and later for the Chinese and Italian language [8, 9]. Our objective was to validate and evaluate the reliability of the Dutch version of the Chronic Ear Survey (CES) in patients suffering from CSOM and to evaluate clinical outcomes of surgery using this questionnaire.

Methods

Questionnaire

The CES is a 13 item disease-specific questionnaire, aiming to measure health impact and treatment effectiveness in patients with CSOM (Appendix 1). The questionnaire consists of three domains: activity restriction (AR; 3 items), symptoms (ST; 7 items) and medical resource (MR; 3 items). The answers are ordered as a Likert-type scale varying from 4 to 6 answers per item. The most positive answer is always positioned on the right. A total score is the sum of all 13 items and domain scores are calculated as the sum of concerning domain items. Total score of the CES ranges from 13 to 71, with higher scores indicating better functional health.

Translation

Permission to translate and validate the original CES was obtained from the Clinical Outcome Research Unit of the Massachusetts Eye and Ear Infirmary [10]. The original CES was translated into the Dutch language independently by the first and the last author. This translation was send to an official translator and native English speaker who retranslated the Dutch version back to the English language. The Dutch and English version were compared by the first, third and last author. A few corrections were made to obtain fluent Dutch sentences without changing the original meaning of the questions. Appendix 2 depicts the Dutch CES (D-CES) after translation.

Patients and control group

A prospective monocenter observational cohort study was performed at the Amsterdam University Medical Centers (Amsterdam UMC) location AMC between May 2015 and October 2017. Adult patients with CSOM scheduled for ear surgery were asked to complete the D-CES pre- and postoperatively. CSOM was defined as a condition of the middle ear of at least 3 months’ duration and characterised by irreversible pathological changes in the mucosa of the middle ear and/or mastoid and associated with constant or intermittent discharge of bacterial origin [7]. Patient characteristics in terms of age, sex, duration of complaints, number of previous operations, presence of a dry ear postoperatively and results upon pure tone audiometry pre- and postoperatively were collected. Improved postoperative hearing was defined as ≥10 dB improvement of high Fletcher Index (average of 1, 2 and 4 kHz). A decrease of ≥10 dB was used as definition of deterioration of postoperative hearing. Control subjects were also asked to participate in this study. The control group consisted of patients with an eye condition and indication for eye surgery. The study was approved by the medical ethics committee of the Amsterdam UMC and written informed consent was obtained from all patients. Patients in the CSOM group were asked to complete the questionnaire two times preoperatively with at least a two-week-interval in order to investigate test-retest reliability. Approximately 6 months postoperatively all patients completed the questionnaire again. The control group was asked to complete the questionnaire one time preoperatively and once 6 months postoperatively. The D-CES was self-administered, either in the hospital or at home, and if necessary returned by post.

Internal consistency and test-retest reliability

With regard to the reliability of the D-CES we focused on internal consistency and test-retest reliability of D-CES scores. Internal consistency or homogeneity reflects the statistical coherence of scale items and can be measured using Cronbach’s α coefficient. Cronbach’s α coefficient is based on the weighted average correlation of items within the scale. We also calculated the corrected item-total correlations, which represent the correlation of the single item and the respective total subscale score, excluding that item. Correlations ≥0.30 are considered to be sufficient [11]. Homogeneity and test-retest reliability were investigated preoperatively.

Known group validity and convergent validity

We investigated two aspects of validity: known-group validity (or clinical validity) and convergent validity. A scale demonstrates known-group validity if it discriminates between (sub) groups of patients with known differences in clinical status. Convergent validity is demonstrated by significant correlations between scale scores and another instrument(s) measuring the same or closely associated heath domains. The known-group validity was assessed by comparing (a) the D-CES baseline scores with baseline scores of our control group and (b) relating the D-CES change scores between baseline and 6 months post-operatively with the duration of preoperative otorrhea, primary or revision surgery, amount of previous operations, presence of a postoperative dry ear, post-operative improved hearing, and postoperative deterioration in hearing. Concerning convergent validity, we compared the pre- and postoperative D-CES scores with correlated subscales of the Dutch version of the Medical Outcome Study 36-Item Short-Form Health Survey (SF-36) which was also administered to all participants [12]. This Quality of Life (QoL) questionnaire was administered twice, once preoperatively and once 6 months postoperatively. The SF-36 consist of eight subscales representing physical (role) functioning, social functioning, emotional (role) functioning, vitality, body pain, and general health perceptions. The scale scores are transformed to a scale of 0 to 100, with a higher score indicating a better QoL. The physical and mental components of the eight subscales can also be combined into a physical and mental component summary score [13]. For this study the social functioning subscale of the SF-36 was compared to the activity subscale of the D-CES, bodily pain to symptoms and general health perceptions to the total questionnaire. No sufficient subscale was present in the SF-36 for comparison with the medical resources subscale of the D-CES.

Statistics

Patient characteristics were analysed using simple descriptive statistics. Internal consistency was expressed using Cronbach’s α correlation coefficient [14]. Item-total correlations and test-retest reliability were analysed using a Pearson’s correlation coefficient and Intraclass Correlation Coefficient (ICC). ICC values < 0.40 can be considered as poor; between 0.40 and 0.59 as fair; between 0.60 and 0.74 as good; and between 0.75 and 1.00 as excellent [15]. Statistical uncertainty of ICC values was expressed using a 95% confidence interval (CI). Differences between mean (change) scores were analysed using a two-group t test. Convergent correlation patterns were expressed in Pearsons’ rank correlation coefficients. All analyses were performed in Statistical Package for the Social Sciences (SPSS), version 24.

Results

Patients and control group

Between March 2015 and October 2016 forty-seven patients with CSOM were included. Twenty-nine out of these 47 patients (62%) fully completed all questionnaires: both the two preoperative and the postoperative questionnaire. In the same period 35 patients in the control group were included. Twenty-six out of 35 control patients (74%) responded to the questionnaire pre- and postoperatively. Table 1 summarizes the patient characteristics. The median age of patients in the CSOM group was 54 compared to 64 in the control group. Median duration of otorrhea was 33 months (range 3 to 380 months).

Table 1 Patient characteristics

Internal consistency and test-retest reliability

Internal consistency calculated with Cronbach’s α correlation coefficient of the total scale was 0.69. Homogeneity analyses of the separate domains resulted in α = 0.47 for the AR subscale, α = 0.72 for the ST subscale and α = 0.68 for the MR subscale (Table 2). In general, the items contributed to the reliability of their respective domains. However three relative weak items could be identified; item a3 in the activity restriction domain showed poor correlation with the domain score (item-total correlation is r = 0.10). Removing this item improved the homogeneity of the domain to α = 0.74. Two other weak items were found in the symptom domain (s3, r = 0.11 and s6, r = 0.12). Deletion of these items improved the homogeneity of this subscale to α = 0.76 and α = 0.74, respectively. The Intraclass correlation coefficient (ICC) reflecting test-retest reliability for the total questionnaire was 0.71 (95% CI: 0.44 to 0.86). The ICC for subscale AR was 0.82 (95% CI: 0.65 to 0.92), for subscale ST the ICC was 0.69 (95% CI: 0.42 to 0.85) and for subscale MR it was 0.64 (95% CI: 0.36 to 0.82).

Table 2 Homogeneity of the three domains of the D-CES

Known group validity and convergent validity

Significant differences were found between the subscale scores and total scores pre- and postoperatively of the CSOM group and the control group, with substantial lower (more impaired) D-CES scores in our target population (Table 3). Also patients with a relative shorter duration of preoperative otorrhea (median < 33 months) had higher mean change of D-CES scores (improved more) compared to patients with longer duration of disease (Table 4). Difference between mean change scores were significant on the subscale AR (p = 0.02) and on the total questionnaire (p = 0.03). Twenty-four ears (83%) remained dry after surgery objectively. Compared to the patients with persistent discharging ears these patients improved significantly more on the subscales ST and MT, and on the complete questionnaire (p values < 0.01). The otorrhea-specific items S2, S6 and M3 all showed significant improvement (p <  0.05) postoperatively in patients with dry ears. Twenty-one patients (72%) showed improved hearing postoperatively which resulted in significantly higher change score on each subscale. No significant impact on the D-CES change scores was seen for type of surgery, amount of previous operation and postoperative decreased hearing. Convergent validity of the D-CES demonstrated a moderate correlation between the AR subscale of the D-CES and the social function subscale of the SF-36 in the preoperative situation (r = 0.59, Table 5). This correlation was weaker in the postoperative phase (r = 0.29). Weaker correlations were found for ST and the pain subscale of the SF-36 (r = 0.29), and for the total questionnaire and the general health perception subscale of the SF-36 (r = 0.17 and r = 0.37).

Table 3 Known-group-validity: D-CES scores in the CSOM group and control group
Table 4 Known-group-validity: clinical characteristics in relation to mean change scores from baseline to follow-up
Table 5 Convergent validity of the D-CES and its domains

Discussion

The D-CES is a suitable questionnaire to obtain subjective data of patients with CSOM. The Dutch otologist is now able to use this questionnaire as a disease specific tool to evaluate pre- and postoperative satisfaction. As the CES is now available in four languages it facilitates easier comparison of different study populations [8,9,10]. Forward-backward translation of the original CES into the Dutch language was performed similar to other studies [2, 4]. A few items of the CES have a 6-point Likert-type response scale. This poses a limitation as it denies the option ‘no change’ within the answers. Questionnaires like the Zurich chronic middle ear inventory (ZCMEI-21) specifically chose a 5-point response option to address this problem [3] compared to the original Chronic Otitis Media Questionnaire-12 (COMQ-12) which comprises of a 6-point Likert scale [16]. At this moment the COMQ-12 is translated into Flemish, Russian and Serbian [4, 17, 18]. Surprisingly it appears that the Russian version is based on a 5-point Likert scale. In that case we believe that these questionnaires are not comparable anymore. Recently Phillips et al. developed the Chronic Otitis Media Benefit Inventory (COMBI) [19]. This is a mixed generic and specific dynamic patient-reported outcome measure. It is derived from the COMQ-12 and asks patients to report any perceived changes in symptoms so it can only been used following an intervention. We believe that it is important to start with baseline subjective data before surgery to enable comparison of disease specific patient reported outcomes. As the original CES was already validated in a large population we chose a relatively small population for the Dutch validation [10]. This is in concordance with the translation and validation as performed by Van Dinther et al. [4]. In our opinion patients with eye disease are very similar to patients with ear disease and the postoperative burden is also comparable. Therefore these patients were suitable to serve as control subjects. Internal consistency of the D-CES was not as good as the original English version [10]. Especially the alpha coefficient of the AR subscale was relative low. As the Cronbach’s α values partly depends on the number of items on a scale, the internal consistency of the questionnaire was additionally described in terms of corrected item-total correlations. The low item-total correlation of item A3 can probably explain that this item reflects social function instead of physical activities. Still the homogeneity coefficient of the total questionnaire is acceptable. The point estimates of the test-retest reliability (of the subscales) of the D-CES turned out to be good. Joseph et al. did not find significant associations between clinical parameters and functional outcomes, using the Glasgow Benefit Inventory (GBI) [5]. Using the disease-specific D-CES we found that improvement scores were significantly influenced by preoperative duration of complaints, postoperative dry ears and postoperative improved hearing. These findings contribute to the clinical validity of the D-CES. Correlation between D-CES and SF-36 subscales were only moderate to small, showing no substantial correlation between a disease-specific functional measure and a generic measure quality of life, as suspected. A limitation of this study is that a number of clinimetric properties was not evaluated and are thus unsettled. This include the assessment of convergent validity with a disease-specific measure, a more in-depth investigation of the responsiveness of the D-CES to measure health changes over time, and – related to this issue - the determination of the smallest D-CES change score (after treatment) that is considered clinically important for the patient.

Conclusions

The Dutch version of the Chronic Ear Survey provides an appropriate short, easy to administer, disease specific questionnaire to assess patient’s perceived functional health in CSOM. The CES is now validated in four languages with good potential of population comparison. The D-CES showed acceptable homogeneity and good test-retest reliability. Known-group (or clinical) validity demonstrated a clear association with preoperative duration of complaints, postoperative dry ears and postoperative improved hearing. These findings suggest that the D-CES can be used to evaluate surgical outcome as well.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Amsterdam UMC:

Amsterdam University Medical Centers

AR:

Activity Restriction

CES:

Chronic Ear Survey

CI:

Confidence Interval

COMBI:

Chronic Otitis Media Benefit Inventory

COMQ-12:

Chronic Otitis Media Questionnaire-12

CSOM:

Chronic Suppurative Otitis Media

CT:

Computer Tomography

D-CES:

Dutch version of the Chronic Ear Survey

GBI:

Glasgow Benefit Inventory

ICC:

Intraclass Correlation Coefficient

MR:

Medical Resources

NA:

Not applicable

PROM:

Patient Reported Outcome Measure

QoL:

Quality of Life

SD:

Standard Deviation

SF-36:

Short Form 36

SPSS:

Statistical Package for the Social Sciences

ST:

Symptoms

ZCMEI-21:

Zurich chronic middle ear inventory

References

  1. 1.

    Bruinewoud EM, Kraak JT, Leeuwen van LM, Kramer SE, Merkus P. The otology questionnaire Amsterdam: a generic patient reported outcome measure about the severity and impact of ear complaints. A cross-sectional study on the development of this questionnaire. Clin Otolaryngol. 2018;43:240–8.

  2. 2.

    Oorts E, Phillips J, Van de Heyning P, Yung M, Van Rompaey V. Dutch health-related quality of life measure for chronic otitis media. B-ent. 2015;11(4):291–5.

  3. 3.

    Bachinger D, Roosli C, Ditzen B, Huber AM. Development and validation of the Zurich chronic middle ear inventory (ZCMEI-21): an electronic questionnaire for assessing quality of life in patients with chronic otitis media. Eur Arch Otorhinolaryngol. 2016;273(10):3073–81.

  4. 4.

    van Dinther J, Droessaert V, Camp S, Vanspauwen R, Maryn Y, Zarowski A, et al. Validity and test-retest reliability of the Dutch version of the chronic otitis media questionnaire 12 (COMQ-12). J Int Adv Otol. 2015;11(3):248–52.

  5. 5.

    Joseph J, Miles A, Ifeacho S, Patel N, Shaida A, Gatland D, et al. Comparison of quality of life outcomes following different mastoid surgery techniques. J Laryngol Otol. 2015;129(9):835–9.

  6. 6.

    Koenraads SPC, Aarts MCJ, Veen van der EL, Grolman W, Stegeman I. Quality of life questionnaires in otorhinolaryngology: a systematic overview. Clin Otolaryngol. 2016;41(6):681–68.

  7. 7.

    Nadol JB, Staecker H, Gliklich RE. Outcomes assessment for chronic otitis media: the chronic ear survey. Laryngoscope. 2000;110(94):32–5.

  8. 8.

    Wang PC, Chu CC, Liang SC, Tai CJ, Gliklich RE. Validation assessment of the Chinese-version chronic ear survey: a comparison between data from English and Chinese versions. Ann Otol Rhinol Laryngol. 2003;112(1):85–90.

  9. 9.

    Ralli G, Milella C, Ralli M, Fusconi M, Torre GL. Quality of life measurements for patients with chronic suppurative otitis media: Italian adaptation of “chronic ear survey”. Acta Otorhinolaryngolica Italica. 2017;37:51–7.

  10. 10.

    Wang PC, Nadol JB, Austin E, Merchant S, Gliklich RE. Validation of outcomes survey for adults with chronic suppurative otitis media. Ann Otol Rhinol Laryngol. 2000;109:249–54.

  11. 11.

    Field A. Discovering statistics using SPSS. 3rd ed. London: Sage Publications Ltd; 2009.

  12. 12.

    Aaronson NK, Muller M, Cohen PD, Essink-Bot ML, Fekkes M, Sanderman R, et al. Translation, validation, and norming of the Dutch language version of the SF-36 health survey in community and chronic disease populations. J Clin Epidemiol. 1998;51(11):1055–68.

  13. 13.

    Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30:473–83.

  14. 14.

    Nunnaly JC, Bernstein IH. Psychometric theory. 3rd ed. New York: McGraw-Hill; 1994.

  15. 15.

    Cicchetti DV. Guidelines, criteria and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6(4):284–90.

  16. 16.

    Phillips JS, Haggard M, Yung M. A new health-related quality of life measure for active chronic otitis media (COMQ-12): development and initial validation. Otol Neurotol. 2014;35(3):454–8.

  17. 17.

    Kosyakov SI, Minavnina JV, Phillips JS, Yung MW. International recognition of the chronic otitis media questionnaire 12. J Laryngol Otol. 2017;131:514–7.

  18. 18.

    Bukurov B, Arsovic N, Gruijcic SS, Haggard M, Spencer H, Marinkovic JE. Psychometric chracteristics of the chronic otitis media questionnaire 12 (COMQ-12): stability of factor structure and replicability shown by the Serbian version. Healt Qual Life Outcomes. 2017;15(207):1–11.

  19. 19.

    Phillips JS, Haggard M, Spencer H, Yung M. The chronic otitis media benefit inventory (COMBI): development and validation of a dynamic quality of life questionnaire for chronic ear disease. Otol Neurotol. 2017;38:701–7.

Download references

Acknowledgements

Not applicable.

Funding

The authors declare that they have not received any funding.

Author information

SG has made substantial contributions to the conception and design of the work, translation of the questionnaire, acquisition and analysis of the data, interpretation of data and has drafted the work. RJH has made substantial contributions to analysis and interpretation of data and has substantially revised the work. MJFW has made substantial contributions to interpretation of data and has substantially revised the work. FAE has made substantial contributions to translation of the questionnaire, interpretation of data and has substantially revised the work. ES has made substantial contributions to the conception and design of the work, translation of the questionnaire, interpretation of data and has substantially revised the work. AND has approved the submitted version; AND has agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. All authors read and approved the final manuscript.

Correspondence to S. Geerse.

Ethics declarations

Ethics approval and consent to participate

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was approved by the medical ethics committee of the Amsterdam UMC.

Informed consent was obtained from all individual participants included in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

Table 6 The original Chronic Ear Survey (CES) [8]

Appendix 2

Table 7 The Dutch Chronic Ear Survey (D-CES)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Keywords

  • Surveys and questionnaires
  • Translations
  • Outcome assessment
  • Quality of life
  • Otitis media, Suppurative