Skip to main content

Interviewer versus self-administered health-related quality of life questionnaires - Does it matter?



Patient-reported outcomes are measured in many epidemiologic studies using self- or interviewer-administered questionnaires. While in some studies differences between these administration formats were observed, other studies did not show statistically significant differences important to patients. Since the evidence about the effect of administration format is inconsistent and mainly available from cross-sectional studies our aim was to assess the effects of different administration formats on repeated measurements of patient-reported outcomes in participants with AIDS enrolled in the Longitudinal Study of Ocular Complications of AIDS.


We included participants enrolled in the Longitudinal Study of Ocular Complications in AIDS (LSOCA) who completed the Medical Outcome Study [MOS] -HIV questionnaire, the EuroQol, the Feeling Thermometer and the Visual Function Questionnaire (VFQ) 25 every six months thereafter using self- or interviewer-administration. A large print questionnaire was available for participants with visual impairment. Considering all measurements over time and adjusting for patient and study site characteristics we used linear models to compare HRQL scores (all scores from 0-100) between administration formats. We defined adjusted differences of ≥0.2 standard deviations [SD]) to be quantitatively meaningful.


We included 2,261 participants (80.6% males) with a median of 43.1 years of age at enrolment who provided data on 23,420 study visits. The self-administered MOS-HIV, Feeling Thermometer and EuroQol were used in 70% of all visits and the VFQ-25 in 80%. For eight domains of the MOS-HIV differences between the interviewer- and self- administered format were < 0.1 SD. Differences in scores were highest for the social and role function domains but the adjusted differences were still < 0.2 SD. There was no quantitatively meaningful difference between administration formats for EuroQol, Feeling Thermometer and VFQ-25 domain scores. For ocular pain (VFQ-25), we found a statistically significant difference of 3.5 (95% CI 0.2, 6.8), which did, however, not exceed 0.2 SD. For all instruments scores were similar for the large and standard print formats with all adjusted differences < 0.2 SD.


Our large study provides evidence that administration formats do not have a meaningful effect on repeated measurements of patient-reported outcomes. As a consequence, longitudinal studies may not need to consider the effect of different administration formats in their analyses.


Patient-reported outcomes (PRO) are measured in studies using information that is provided directly by study participants. Probably most commonly, PROs are used as outcome measures in epidemiologic studies and clinical trials [15]. But PROs also contribute importantly to the study participants' profile and are often associated with future health outcomes. For example, health-related quality of life (HRQL) or symptoms such as dyspnea can be strong prognostic indicators [68].

PRO instruments are either completed by study participants' themselves (self-administered) or administered by an interviewer. Self-administered PRO questionnaires offer the advantage of not requiring research staff as interviewers and participants to complete the questionnaire at their own pace. It may be offered as a paper- and pencil method both at the study site or at home (mail) or through web-based applications. Interviewer-administered PRO questionnaires are more resource intensive but offer additional control over the quality of the measurement. Interviewers may administer the questionnaires face-to-face or over the telephone. In many epidemiologic studies, both self- and interviewer-administered questionnaires are available to accommodate preferences, physical impairment or literacy of participants [9, 10].

In a study, the format of questionnaire administration often varies between participants but it may also vary within participants from one follow-up to another. The evidence on the effects of different administration formats on PRO scores is inconsistent. A number of studies (randomized trials or observational studies) found that the administration format had an effect on PRO scores for some or all of the domains [9, 1119]. In some studies scores indicated less health impairment when PRO instruments were administered by an interviewer. A common interpretation of this phenomenon, which is not entirely understood, is that participants may indicate less impairment when interviewed by research staff as compared to self-administered questionnaires. Some refer to this phenomenon as a social desirability bias [20]. Other studies did not find meaningful differences between administration formats [10, 2123]. If effects of different administration formats exist in epidemiological studies or clinical trials estimates of associations or treatment effects may be affected.

Most studies comparing different administration formats were relatively small and considered only one or two measurements [9, 1119]. The results of these studies are inconsistent and it is uncertain whether such unwarranted effects detected in some methodological studies are also present in a particular epidemiologic study where PRO instruments are administered repeatedly over time. Therefore, our aim was to assess the effects of different administration formats on repeated measurements of patient-reported outcomes in a large cohort of persons with AIDS that completed PRO instruments repeatedly over a long period of time.


Study design and participants

We included all participants enrolled in the Longitudinal Study of Ocular Complications of AIDS (LSOCA). Enrollment started in September 1998 and the data included here were collected through December 31st 2009. LSOCA is one of the largest prospective observational studies of persons with AIDS. Study participants have AIDS diagnoses according to the 1993 Centers for Disease Control and Prevention case surveillance definition of AIDS. Over the course of the study, recruitment has been performed at 19 clinical centers across the United States, located in urban areas with sizable HIV-infected populations. The current number of active study sites is 13 [24, 25]. In this analysis, we included participants with both incident and prevalent AIDS at the time of enrollment.

The study protocol was reviewed and approved by institutional review boards at each of the participating clinics and the coordinating center. Adult participants have given written informed consent. For adolescents, a Consent Statement was signed by parents or guardians and an Assent Statement signed by adolescents and their parents or guardians. More detailed information about the study protocol, data forms and the study handbook is available on

PRO instruments

At enrollment and every six months thereafter, study participants completed the Medical Outcome Study (MOS)-HIV Health Survey, the EuroQol, the Feeling Thermometer and the Visual Function Questionnaire 25 (VFQ-25). Between 1998 and 2008, the subset of participants with major ocular complications (ocular opportunistic infections and major retinal vessel occlusions) had study visits every three months where they completed the questionnaires. The MOS-HIV has 35 items and scores range from 0 (lowest score) to 100 (highest score) [26, 27]. Its development was based on the Short-Form 20 of the Medical Outcomes Study and HIV/AIDS-specific domains were added (energy, cognitive functioning, health distress, health transition and quality of life) to the existing domains (general health perceptions, physical function, role function, role function, social functioning, pain and mental health). One item was added to the pain domain. The MOS-HIV has been used extensively in clinical trials and cohorts studies of patients with HIV/AIDS.

The EuroQol consists of five questions about anxiety/depression, mobility, usual activities, pain/discomfort and self-care [28]. Different combinations of responses (on a 5-point Likert-type scale) for the five dimensions are weighted using preferences identified by the US general population [29] The lowest possible score is -0.594 and the highest is 100. The Feeling Thermometer complements the five questions of the EuroQol and asks participants to rate their health status from 0 (equivalent to the worst imaginable health state) to 100 (equivalent to the best imaginable health state). The Feeling Thermometer has been shown to be a reliable, valid and responsive utility measure for various diseases.

The National Eye Institute VFQ-25 was developed to measure vision-specific HRQL in patients with varying eye conditions such as cataract, glaucoma, diabetic retinopathy, cytomegalic virus retinitis and corneal diseases [30, 31]. The VFQ-25 measures the influence of visual ability and visual symptoms on health domains and on task-oriented domains. There are domain scores for social functioning, role limitations, dependency on others, mental health, future expectations on vision, near vision activities, distance vision activities, driving difficulties, pain and discomfort in or around the eyes, limitations with peripheral vision and color vision. The VFQ-25 provides reliable and valid scores that are responsive to change. Scores range from 0 (lowest score) to 100 (highest score). In LSOCA, the VFQ-25 was introduced in September 2008.

Administration formats

The most common format used to complete the HRQL instruments in LSOCA is the self-administered format. This means that participants complete the questionnaires themselves using paper and pencil. Reasons to switch to interviewer-administered questionnaires include inability to read because of sight limitations, dilated pupils for eye examination, illiteracy or for logistical reasons to save time. Thus the choice of administration format depends on characteristics of participants and the study site. The wording and layout of self- and interviewer-administered questionnaires was identical.

In addition, a large print version for all questionnaires was added in May 2008. Participants can complete the large print version if they desire. The font size of the large print version is 14 points compared to 10 points in the standard version. The reasons to switch to a large print format usually relate to the participant's visual impairment or failure to bring reading glasses to a visit. The choice of administration format is made at every visit. Theoretically, the administration format may change from visit to visit although this is rarely the case. The questionnaire administration format is recorded for every visit. For the current analyses, only data from in-person visits were included whereas data from telephone interviews were not considered.

Statistical analysis

We first determined the number and proportion of interviewer- versus self-administered and large-versus small-print questionnaires, respectively, at baseline and follow-up visits and assessed how these numbers changed as a function of time from enrollment. We also determined the number of participants who switched from the standard self- to an interviewer-administered questionnaire. We calculated mean scores for all HRQL domains stratified by administration ("Proc Univariate" command). We then compared the HRQL scores between administration formats (interviewer- versus self-administered and large- versus small-print) to assess whether they differed, which we defined as ≥0.2 standard deviations from the baseline assessment. The standard deviations for the different instruments and their domains at baseline as well as our thresholds for a quantitatively meaningful difference are shown in Table 1. For each patient, we considered all measurements and administration formats used over time and employed linear regression models ("regress" command of Stata) while accounting for within subject correlation ("cluster" option) and calculating robust standard errors using the Huber-White sandwich estimators ("robust" option). Since the choice of administration format is not random as explained above, patient and study site characteristics are likely to be associated with differences between HRQL scores of different administration formats. Therefore, we adjusted the comparison for study site and the participants' sex and for the time-varying variables age, CD4+ T cells, HIV viral load and visual acuity. We also checked for the potential influence of sex, age and disease severity (CD4+ T cell count) on the effect of administration format and included interaction terms into the regression models to test for effect modification. In a sensitivity analysis, we assessed a cross-sectional sample of participants who switched administration formats from self to interview for the first time. We compared the differences in their mean scores on the two administration formats using the Wilcoxon signed rank test We used SAS (version 9.2, SAS Institute, Cary, NC) for data management and for computing descriptive statistics and Stata for the regression analyses (version 10.1, Stata Corp; College Station, TX).

Table 1 Standard deviations for generic and vision specific health-related quality of life scores as obtained from baseline assessment of 2,261 participants enrolled in the Longitudinal Study of Ocular Complications in AIDS (LSOCA)


We included 2,261 participants in the analysis. The patient population was predominantly male (81%) with a median age of 43 years (interquartile range [IQR] 38-49), of non-hispanic white (46%) or black ethnicity (36%). At enrollment, 409 participants (18%) had been diagnosed with AIDS for one year or less (incident AIDS) and 1,852 participants (82%) for more than a year. Median CD4+ T cell count at enrollment was 174 cells/μL (IQR 61-339), median nadir CD4+ T cell count was 31 cells/μL (IQR 10-91) and median HIV RNA (viral load) level was 2.9 (log10[copies/mL], IQR 1.9-4.7). Overall, 83.0% of participants received HAART at enrollment.

Administration formats

The majority of visits involved self-administered PRO questionnaires (70% of a total of 23,420 study visits). Of the 2,261 patients, 929 (41%) completed their first (baseline) questionnaires via interview and 1,332 (59%) completed it via self-administration. In 6,910 (30%) visits the HRQL questionnaires were interviewer-administered and in 224 (1%) visits participants used the self-administered version with large print letters. These percentages changed with follow-up (Figure 1). The percentages of self-administered questionnaires (standard and large print formats) increased from 63% in the first year of enrollment to 77% beyond five years of enrollment. Of a total of 2,336 visits where the VFQ-25 was completed, participants used the self-administered format in 1,878 (80%) visits (standard print in 1,708 [91%] visits and large print in 170 [9%] visits) and had it interviewer- administered in 458 (20%) visits.

Figure 1
figure 1

Study participants and administration formats. The graph shows the percentage of study participants and the different administration formats they chose since time of enrolment. All study visits (n = 23,420) of all participants (n = 2,261) contributed to the analyses. The percentage of self administration with the standard print increased from 62% in the first year of enrolment to 75% if participants were enrolled six years or more. Interviewer administration with standard print decreased from 37% to 23% and self administration with large print increased from 1% to 2%.

Out of the 2,261 participants, 1,730 (77%) started with the self-administered MOS-HIV, EuroQol and Feeling Thermometer whereas 531 participants (23%) started with the interviewer-administered format. 1,265 (56%) never switched the administration format of the MOS-HIV, EuroQol and Feeling Thermometer, 335 (15%) switched permanently and 661 (29%) switched intermittently. Of the 1,096 participants who completed the VFQ-25 989 (90%) never switched administration format, 93 (9%) switched permanently and 14 (1%) switched intermittently.

Interviewer- versus self-administered questionnaires

For eight domains of the MOS-HIV, we did not find statistically significant differences between the interviewer- and self- administered formats (Table 2). For the general health perceptions, role function and social function domains, scores were higher for the self- administered format but adjusted differences were < 0.2 SD. The difference between self- and interviewer-administered questionnaires was statistically significant for the Feeling Thermometer but also < 0.2 SD. For the VFQ-25, there was no significant (adjusted) difference for eleven of the twelve domains (Table 3). For ocular pain, we found a significant difference of 3.5 (95% CI 0.2, 6.8) but this difference was below the threshold of 0.2 SD that we defined to be quantitatively meaningful. Unadjusted results were similar to adjusted results for the MOS-HIV, Feeling Thermometer and EuroQol with all differences between administration formats < 0.2 SD. For the VFQ-25, we found differences ≥0.2 SD for six out of twelve domains. We did not find any evidence for an interaction of sex, age and disease severity (CD4+ T cell count) with administration format (none of the interaction terms with p ≤ 0.05).

Table 2 Generic health-related quality of life scores: Interviewer- versus Self-administration and Large- versus Small print
Table 3 Vision-related health-related quality of life scores: Interviewer- versus Self-administration and Large- versus Small print

465 participants who started with self-administration and switched at least once to interviewer-administered questionnaires were available for the sensitivity analysis. We did not find any statistically significant differences between scores of the MOS-HIV, Feeling Thermometer, EuroQol and VFQ-25 from the last study visit before the switch (self-administered) to scores obtained at the first visit where interviewer administration was chosen. All differences were below the thresholds for a meaningful difference.

Large- versus standard print format

For all domains of the MOS-HIV, the Feeling Thermometer and EuroQol the scores were similar for the large and standard print formats and we did not find statistically significant differences (Tables 2 and 3). All differences were below 0.2 SD. Also, we did not find any significant differences for the VFQ-25. Unadjusted differences were also all < 0.2 SD.


In our analysis of more than 23,000 clinic visits of participants with AIDS, different administration formats of generic or disease-specific PRO instruments did not have a meaningful effect on HRQL scores measured repeatedly over time. Differences between all scores of the interviewer- and self-administered questionnaires were below our predefined threshold for a quantitatively meaningful difference. Also, the use of the large print format did not have an impact on HRQL scores.

We defined a meaningful difference between administration formats to be ≥0.2 SD, which corresponds to a small but potentially important difference as first defined by Cohen [32]. Other studies used similar criteria for defining a threshold for meaningful differences between PRO scores [9]. Adjusted differences were all below 0.2 SD, but it should be noted that the estimates were precise for the comparison of the interviewer- and self-administered HIV-MOS with confidence intervals that were mostly within ± 0.2 SD. In contrast, since the VFQ-25 and the large print format were introduced more recently, sample size was considerably smaller for these comparisons and some 95% confidence intervals overlapped ± 0.2 SD. Hence, although mean differences were small for most comparisons of the VFQ-25 and large print format, we cannot claim equivalence of scores measured by interviewer- and self-administered questionnaires. We did not calculate sample size requirements for our study. But if a randomized trial was planned to compare administration formats and the "General Health" domain of the HIV-MOS was the outcome of interest, 526 patients would be needed per trial arm to detect a difference of at least 0.2 SD (4.5 points, assuming a pooled SD of 22.5 points) and a standard 5% chance of two-sided type I (false positive) error and 90% power. Our study sample far exceeded that sample size.

The results of studies comparing different administration formats, including ours, are heterogeneous. Some studies found scores indicating less impairment with interviewer- than with self-administered questionnaires [9, 1119], which was more pronounced for mental health domains in some studies [9, 13]. In other studies, investigators did not observe such differences [10, 2123]. To our knowledge, our analysis is the only one embedding the comparison of administration formats in a large cohort study with repeated measurements over time. This allows us to estimate differences between administration formats with greater precision, in a cohort study setting and to do a sensitivity analysis that compared PRO scores within participants. A possible explanation for the absence of differences between administration formats could be that a social desirability bias, that is commonly proposed to explain why interviewer administration leads to higher scores [20], may wash out over time. It seems unlikely, that participants, who come repeatedly for study visits, would consistently overestimate their health. In most studies that compared administration formats, there was only one (cross-sectional) administration where patients are likely to be unfamiliar with the study or clinic setting and where a social desirability bias may be more likely to be present than in a study with follow-up. However, the hypothesis that the social desirability bias washes out over time would require further testing in a randomized trial comparing administration formats where repeated measurements are available.

If an effect of administration format is present investigators should be concerned with a potential effect that may alter inferences in two ways. First, different administration formats may introduce additional measurement variability, which makes the detection of small but important associations or effects more difficult. Sample size requirements to detect a certain difference in PRO could be larger if different administration formats are used because of greater standard deviations and because effect estimates are likely to be attenuated by additional (non-differential) measurement error [9, 10]. Second, if different administration formats influence scores, effect estimates could be affected.

For example, one could be interested in comparing HRQL between HIV-infected persons with and without AIDS (Figure 2). Let us assume that HRQL is measured by a HRQL instruments with scores from 0 to 100. In the absence of effects from administration format (scenario 1 in Figure 2), we could, for example, expect a mean score of 50 for persons with AIDS and of 70 for HIV-infected persons without AIDS resulting in a mean difference of 20 units. If interviewer-administered questionnaires are offered it can be expected that more persons with AIDS will choose this format (for example 30%) because they have, on average, more visual (for example because of cytomegalovirus retinitis) and more cognitive impairment (for example because of brain toxoplasmosis) than HIV-infected persons without AIDS (scenarios 2 and 3 in Figure 2). If interviewer-administered questionnaires lead to different scores compared to self-administered questionnaires the difference in HRQL between AIDS and non-AIDS persons would be affected.

Figure 2
figure 2

Relationship of administration format with exposure, outcome and other variables. A hypothetical scenario is represented by a causal diagram. Investigators may be interested in comparing health-related quality of life (HRQL) between HIV-infected patients with and without AIDS. Both interviewer and self-administration are available. Patients with the AIDS-defining illnesses cytomegalovirus (CMV) retinitis or brain toxoplasmosis are more likely to require interviewer administration because of visual or cognitive impairment, respectively. Administration format is not a confounder since it is on the causal pathway from exposure to outcome and does not cause CMV retinitis nor brain toxoplasmosis. The table shows three scenarios. In the first scenario the administration format is restricted to self-administration and the difference in HRQL is 20 units. In the second and third scenario, both interviewer and self-administration are available and it is assumed that patients with AIDS are more likely to require interviewer administration because of CMV retinitis or brain toxoplasmosis. The effect of interviewer-administration is ± 10 units in the second and third scenario, respectively, which has an effect of ± 2 units on the between-group comparisons.

We see two solutions to address this issue. One solution would be to restrict the administration format strictly to one mode of administration. Since this may be unrealistic in many studies a second solution would be to record the administration format at each study visits and check for an independent effect of administration format on PRO scores as we did in this study. Intuitively, investigators may think that one should adjust the effect estimate for administration format, which has also been proposed in the literature [9]. However, the causal diagram in Figure 2 shows that administration format does not act as a confounder since it is affected by AIDS status, but as an intermediate. In fact, adjusting for administration format would attenuate or increase the association of AIDS status and HRQL and lead to potential under- or over-estimation. Instead, we propose that the effect caused by administration format in some studies should be corrected by the use of methods to account for measurement error such as regression calibration or multiple imputation [3335]. The idea of regression calibration is to correct the observed value, which is known or suspected not to represent the true value, using information from repeated measurements or from substudies that yield the true values for some patients (e.g. by sing a reference standard measurement method or some instrumental variable). With the multiple imputation approach, the true values are regarded to be missing and can be imputed using information similar to the information used in the regression calibration approach (repeated measurements or true values from substudy). We would like to point out though that we would not consider the effect of administration format to be a measurement error and its effect on estimation an information bias (bias in effect estimation caused by measurement error) since neither of the different methods of measurement are superior and since no reference standard for PRO exists.

Strengths of our analysis include the large sample size and the repeated administrations of PRO instruments over time. Thereby, our analysis represents the typical cohort study settings for which there is little evidence on the effects of administration formats and we have little reason to assume that the results of our study are specific to patients with AIDS only. Another strength is the adjustment for patient and study site characteristics that could have confounded the comparisons. However, a limitation of our study is the lack of randomization for administration format so that some residual confounding might still be present. Also, since the VFQ-25 and the large print format were introduced rather recently the sample size was smaller for investigating the effects of administration format on VFQ-25 scores or of the large versus standard print.

Our large study provides evidence that administration formats do not have a meaningful effect on repeated measurements of PRO. As a consequence, longitudinal studies may not need to consider different administration formats in their analyses. However, if investigators find an effect of administration format they should not adjust for the administration format but consider using one of the methods available for correcting systematic measurement error.


  1. Goldsmith KA, Dyer MT, Schofield PM, Buxton MJ, Sharples LD: Relationship between the EQ-5D index and measures of clinical outcomes in selected studies of cardiovascular interventions. Health Qual Life Outcomes 2009, 7: 96. 10.1186/1477-7525-7-96

    Article  PubMed Central  PubMed  Google Scholar 

  2. Schulz R, Beach SR, Ives DG, Martire LM, Ariyo AA, Kop WJ: Association between depression and mortality in older adults: the Cardiovascular Health Study. Arch Intern Med 2000,160(12):1761–1768. 10.1001/archinte.160.12.1761

    Article  CAS  PubMed  Google Scholar 

  3. Puhan MA, Gaspoz JM, Bridevaux PO, Schindler C, Ackermann-Liebrich U, Rochat T, Gerbase MW: Comparing a disease-specific and a generic health-related quality of life instrument in subjects with asthma from the general population. Health Qual Life Outcomes 2008, 6: 15. 10.1186/1477-7525-6-15

    Article  PubMed Central  PubMed  Google Scholar 

  4. Bing EG, Hays RD, Jacobson LP, Chen B, Gange SJ, Kass NE, Chmiel JS, Zucconi SL: Health-related quality of life among people with HIV disease: results from the Multicenter AIDS Cohort Study. Qual Life Res 2000,9(1):55–63. 10.1023/A:1008919227665

    Article  CAS  PubMed  Google Scholar 

  5. Spilker B: Quality of Life and Pharmacoeconomics in Clinical Trials. 2nd edition. Philadelphia: Lippincott Williams & Wilkins; 1996.

    Google Scholar 

  6. Rodriguez-Artalejo F, Guallar-Castillon P, Pascual CR, Otero CM, Montes AO, Garcia AN, Conthe P, Chiva MO, Banegas JR, Herrera MC: Health-related quality of life as a predictor of hospital readmission and death among patients with heart failure. Arch Intern Med 2005,165(11):1274–1279. 10.1001/archinte.165.11.1274

    Article  PubMed  Google Scholar 

  7. Montazeri A: Quality of life data as prognostic indicators of survival in cancer patients: an overview of the literature from 1982 to 2008. Health Qual Life Outcomes 2009, 7: 102. 10.1186/1477-7525-7-102

    Article  PubMed Central  PubMed  Google Scholar 

  8. Puhan MA, Garcia-Aymerich J, Frey M, ter Riet G, Anto JM, Agusti AG, Gomez FP, Rodriguez-Roisin R, Moons KG, Kessels AG, et al.: Expansion of the prognostic assessment of patients with chronic obstructive pulmonary disease: the updated BODE index and the ADO index. Lancet 2009,374(9691):704–711. 10.1016/S0140-6736(09)61301-5

    Article  PubMed  Google Scholar 

  9. Cheung YB, Goh C, Thumboo J, Khoo KS, Wee J: Quality of life scores differed according to mode of administration in a review of three major oncology questionnaires. J Clin Epidemiol 2006,59(2):185–191. 10.1016/j.jclinepi.2005.06.011

    Article  PubMed  Google Scholar 

  10. Weinberger M, Oddone EZ, Samsa GP, Landsman PB: Are health-related quality-of-life measures affected by the mode of administration? J Clin Epidemiol 1996,49(2):135–140. 10.1016/0895-4356(95)00556-0

    Article  CAS  PubMed  Google Scholar 

  11. Feveile H, Olsen O, Hogh A: A randomized trial of mailed questionnaires versus telephone interviews: response patterns in a survey. BMC Med Res Methodol 2007, 7: 27. 10.1186/1471-2288-7-27

    Article  PubMed Central  PubMed  Google Scholar 

  12. Lopes AD, Vilar e Furtado R, Silva CA, Yi LC, Malfatti CA, Araujo SA: Comparison of self-report and interview administration methods based on the Brazilian versions of the Western Ontario Rotator Cuff Index and Disabilities of the Arm, Shoulder and Hand Questionnaire in patients with rotator cuff disorders. Clinics (Sao Paulo) 2009,64(2):121–125.

    Article  Google Scholar 

  13. Lungenhausen M, Lange S, Maier C, Schaub C, Trampisch HJ, Endres HG: Randomised controlled comparison of the Health Survey Short Form (SF-12) and the Graded Chronic Pain Scale (GCPS) in telephone interviews versus self-administered questionnaires. Are the results equivalent? BMC Med Res Methodol 2007, 7: 50. 10.1186/1471-2288-7-50

    Article  PubMed Central  PubMed  Google Scholar 

  14. Okamoto K, Ohsuka K, Shiraishi T, Hukazawa E, Wakasugi S, Furuta K: Comparability of epidemiological information between self- and interviewer-administered questionnaires. J Clin Epidemiol 2002,55(5):505–511. 10.1016/S0895-4356(01)00515-7

    Article  PubMed  Google Scholar 

  15. Perkins JJ, Sanson-Fisher RW: An examination of self- and telephone-administered modes of administration for the Australian SF-36. J Clin Epidemiol 1998,51(11):969–973. 10.1016/S0895-4356(98)00088-2

    Article  CAS  PubMed  Google Scholar 

  16. Puhan MA, Behnke M, Frey M, Grueter T, Brandli O, Lichtenschopf A, Guyatt GH, Schunemann HJ: Self-administration and interviewer-administration of the German Chronic Respiratory Questionnaire: instrument development and assessment of validity and reliability in two randomised studies. Health Qual Life Outcomes 2004, 2: 1. 10.1186/1477-7525-2-1

    Article  PubMed Central  PubMed  Google Scholar 

  17. Schunemann HJ, Goldstein R, Mador MJ, McKim D, Stahl E, Puhan M, Griffith LE, Grant B, Austin P, Collins R, et al.: A randomised trial to evaluate the self-administered standardised chronic respiratory questionnaire. Eur Respir J 2005,25(1):31–40. 10.1183/09031936.04.00029704

    Article  CAS  PubMed  Google Scholar 

  18. Jorngarden A, Wettergen L, von Essen L: Measuring health-related quality of life in adolescents and young adults: Swedish normative data for the SF-36 and the HADS, and the influence of age, gender, and method of administration. Health Qual Life Outcomes 2006, 4: 91. 10.1186/1477-7525-4-91

    Article  PubMed Central  PubMed  Google Scholar 

  19. Rhodes T, Girman CJ, Jacobsen SJ, Guess HA, Hanson KA, Oesterling JE, Lieber MM: Does the mode of questionnaire administration affect the reporting of urinary symptoms? Urology 1995,46(3):341–345. 10.1016/S0090-4295(99)80217-9

    Article  CAS  PubMed  Google Scholar 

  20. Bowling A: Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf) 2005,27(3):281–291. 10.1093/pubmed/fdi031

    Article  Google Scholar 

  21. Gundy CM, Aaronson NK: Effects of mode of administration (MOA) on the measurement properties of the EORTC QLQ-C30: a randomized study. Health Qual Life Outcomes 8: 35.

  22. Irvine EJ, Feagan BG, Wong CJ: Does self-administration of a quality of life index for inflammatory bowel disease change the results? J Clin Epidemiol 1996,49(10):1177–1185. 10.1016/0895-4356(96)00136-9

    Article  CAS  PubMed  Google Scholar 

  23. Wu AW, Jacobson DL, Berzon RA, Revicki DA, van der Horst C, Fichtenbaum CJ, Saag MS, Lynn L, Hardy D, Feinberg J: The effect of mode of administration on medical outcomes study health ratings and EuroQol scores in AIDS. Qual Life Res 1997,6(1):3–10.

    Article  CAS  PubMed  Google Scholar 

  24. Jabs DA, Van Natta ML, Holbrook JT, Kempen JH, Meinert CL, Davis MD: Longitudinal study of the ocular complications of AIDS: 1. Ocular diagnoses at enrollment. Ophthalmology 2007,114(4):780–786. 10.1016/j.ophtha.2006.11.008

    Article  PubMed  Google Scholar 

  25. Jabs DA, Van Natta ML, Holbrook JT, Kempen JH, Meinert CL, Davis MD: Longitudinal study of the ocular complications of AIDS: 2. Ocular examination results at enrollment. Ophthalmology 2007,114(4):787–793. 10.1016/j.ophtha.2006.07.065

    Article  PubMed  Google Scholar 

  26. Wu AW, Hays RD, Kelly S, Malitz F, Bozzette SA: Applications of the Medical Outcomes Study health-related quality of life measures in HIV/AIDS. Qual Life Res 1997,6(6):531–554. 10.1023/A:1018460132567

    Article  CAS  PubMed  Google Scholar 

  27. Wu AW, Revicki DA, Jacobson D, Malitz FE: Evidence for reliability, validity and usefulness of the Medical Outcomes Study HIV Health Survey (MOS-HIV). Qual Life Res 1997,6(6):481–493. 10.1023/A:1018451930750

    Article  CAS  PubMed  Google Scholar 

  28. Kind P: The EuroQol instrument: An index of health-related quality of life. In Quality of life and Pharmacoeconomics in clinical trials. 2nd edition. Edited by: B. Spilker. Philadelphia, Lippincott-Raven; 1996.

    Google Scholar 

  29. Agency for Health Research and Quality. U.S. Valuation of the EuroQol EQ-5D Health States 2005. []

  30. Mangione CM, Lee PP, Pitts J, Gutierrez P, Berry S, Hays RD: Psychometric properties of the National Eye Institute Visual Function Questionnaire (NEI-VFQ). NEI-VFQ Field Test Investigators. Arch Ophthalmol 1998,116(11):1496–1504.

    Article  CAS  PubMed  Google Scholar 

  31. Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD: Development of the 25-item National Eye Institute Visual Function Questionnaire. Arch Ophthalmol 2001,119(7):1050–1058.

    Article  CAS  PubMed  Google Scholar 

  32. Cohen J: Statistical power analysis for the behavioral sciences. 2nd edition. Hillsdale, NJ: Lawrence Erlbaum; 1988.

    Google Scholar 

  33. Cole SR, Chu H, Greenland S: Multiple-imputation for measurement-error correction. Int J Epidemiol 2006,35(4):1074–1081. 10.1093/ije/dyl097

    Article  PubMed  Google Scholar 

  34. Thurigen D, Spiegelman D, Blettner M, Heuer C, Brenner H: Measurement error correction using validation data: a review of methods and their applicability in case-control studies. Stat Methods Med Res 2000,9(5):447–474. 10.1191/096228000701555253

    Article  CAS  PubMed  Google Scholar 

  35. Freedman LS, Midthune D, Carroll RJ, Kipnis V: A comparison of regression calibration, moment reconstruction and imputation for adjusting for covariate measurement error in regression. Stat Med 2008,27(25):5195–5216. 10.1002/sim.3361

    Article  PubMed Central  PubMed  Google Scholar 

Download references


Studies of Ocular Complications of AIDS Research Group

Participating Clinical Centers

Baylor College of Medicine, Cullen Eye Institute, Houston, TX: Richard Alan Lewis, MD, MS (Director); John Michael Bourg; Victor Fainstein, MD; Zbigniew Krason, CRA; Joseph F. Morales, CRA; Silvia Orengo-Nania, MD; Tobias C. Samo, MD; Steven Spencer, BA, COMT; Mitchell P. Weikert, MD. Former Members: Richard C. Allen, MD; Pamela Frady, COMT; Ronald Gross, MD; Allison Schmidt, CRA; Laura Shawver, COT/CCRP; James Shigley, CRA; Benita Slight, COT; Rachel Sotuyo, COT; Stephen Travers, CRA.

Emory University Eye Center, Atlanta, GA: Sunil K. Srivastava, MD (Director); Allison Gibbs, BS; Deborah Gibbs, COMT; Debora Jordan, CRA; Bob Myles, CRA; Janna Rutter, CRA. Former Members: Antonio Capone, Jr. MD; David Furukuwa, PA; Baker Hubbard, MD; Daniel F. Martin, MD.

Indiana University, Indianapolis, IN: Former Members: Mitchell Goldman, MD (Director); Janice Brown; Thomas Ciulla, MD; Jean Craft, RN, CS; Ronald Danis, MD; Paul Fry; Hua Gao, MD; Samir Gupta, MD; Janet Hernandez, RN; Debra Poe; Linda Pratt, RN; James D. Richardson, MD; Tim Steffens, CRA; L. Joseph Wheat, MD; Beth Zwickl, RN, CS, MSN.

Johns Hopkins University School of Medicine, Baltimore, MD: J.P. Dunn, MD (Director); Diane M. Brown, RN; Dennis Cain; David Emmert; Mark Herring; Adam Jacobowitz, MD; Henry A. Leder, MD; Alison G. Livingston, RN, BSN; Yavette Morton; Kisten D. Nolan, RN, BSN, MPH; Richard D. Semba, MD, MPH; Priscilla Soto; Jennifer E. Thorne, MD, PhD. Former Members: Patricia Barditch-Crovo, MD; Marie-Lyne Bélair, MD; Stephen G. Bolton, CRNP; Joseph B. Brodine; Lisa M. Brune, RN, BSN; Anat Galor, MD; Douglas A. Jabs, MD, MBA; Meera Kapoor; Sanjay R. Kedhar, MD; John H. Kempen, MD, PhD; Stephen J. Kim, MD; Armando L. Oliver, MD; George B. Peters, III, MD; Ricardo Stevenson, MD; Michelle Tarver-Carr, MD, PhD; Susan Wittenberg, MD; Michelle Yue Wang, MD.

Louisiana State University Health Sciences Center, New Orleans, LA: Donald Bergsma, MD (Director); Rebecca Clark, MD; Robin Cooper, COMT; Jasmine Elison, MD; Butler Fuller, MD; Christine Jarrott, RN, ACRN; Lynn Otillio, COT; Maria Reinoso, MD; Christine Romero, COT, ROUB. Former Members : Bruce Barron, MD; Robin Bye, RN; Mandi Conway, MD; Larry Dillon, COT/CRA; Audrey Lombard, RN; Gholman Peyman, MD.

New Jersey Medical School, Newark, NJ: Former Members: Ronald Rescigno, MD (Director); Neelakshi Bhagat, MD; Rosa Paez-Boham, COMT; Marta Paez-Quinde.

New York Hospital - Cornell Medical Center, New York, NY: Murk-Hein Heinemann, MD (Director); Susana Coleman; Sara Daniel; Roberta Janis, RN, BSN; Aziz Khanifer, MD; Andrzej Kozbial; Diane Iglesias Rivera, COA; Kent Sepkowitz, MD. Former Members: Kenneth Boyd; Robinson V.P. Chan, MD; Cynthia Chiu, MD; Charles Cole, MD; Charles Doering, MD; Jasmine Elison, MD; Sangwoo Lee, MD; Fang Lu; Joseph Murphy; Sophia Pachydaki, MD; Christina Peroni, MD; Firas M. Rahhal, MD; Ashok Reddy, MD; Scott Warden, MD.

New York University Medical Center, New York, NY: Dorothy N. Friedberg, MD, PhD (Director); Adrienne Addessi, MA, RN; Douglas Dieterich, MD; Monica Lorenzo-Latkany, MD; Maria Pei, COA. Former Member: Alex McMeeking, MD.

Northwestern University, Chicago, IL: Alice T. Lyon, MD (Director); Lori Ackatz, RN, MPH; Manjot Gill, MD; Lori Kaminski, RN, MS; Rukshana Mirza, MD; Robert Murphy, MD; Frank Palella, MD; Carmen Ramirez; Zuzanna Rozenbajgier; Dawn Ryan; Evica Simjanoski; Former Members: Alexander Habib; Jill Koecher; Jeevan Mathura, MD; Annmarie Muñana, RN; Jonathan Shankle; David V. Weinberg, MD; James Yuhr.

Rush University, Chicago, IL: Former Members: Mathew W. MacCumber, MD, PhD (Director); Bruce Gaynes, OD, PharmD; Christina Giannoulis; Pamela Hulvey; Harold Kessler, MD; Heena S. Khan; Andrea Kopp; Pauline Merrill, MD; Frank Morini; Nada Smith; Allen Tenorio, MD; Denise Voskuil-Marre; Kisung Woo.

University of California, Irvine: Former Members: Baruch D. Kuppermann, MD, PhD (Director); Bogdan Alexandiescu, MD; Donald N. Forthal, MD; Jeff Grijalva, COT; Faisal Jehan, MD; Karen Lopez; Rosie Magallon, BA; Nader Moinfar, MD; Bret Trump; Melody Vega, COA; Randy Williams.

University of California, Los Angeles: Gary N. Holland, MD (Director); Robert D. Almanzor, COA; Margrit E. Carlson, MD; Jose T. Castellanos, COT; Jeffrey A. Craddock, COT; Serina Gonzales; Ann K. Johiro, MN, RN, BC, FNP-C, AACRN, AAHIVS; Partho S. Kalyani, MD; Michael A. Kapamajian, MD; David L. LeBeck; Kristin M. Lipka; Susan S. Ransome, MD. Former Members: Suzette A. Chafey, RN, NP; Alexander C. Charonis, MD; Peter J. Kappel, MD; Ardis A. Moe, MD; Germán Piñón; Angela Sanderson; Kayur H. Shah, MD; Robert Stalling, COA; Dennis Thayer, CRA; Jean D. Vaudaux, MD.

University of California, San Diego: William R. Freeman, MD (Director); Denise Cochran; Igor Kozak, MD; Megan Loughran; Luzandra Magana; Victoria Morrison, MD; Vivian Nguyen; Stephen Oster, MD. Former Members: Sunan Chaidhawanqual, MD; Lingyun Cheng, MD; Tom Clark; Mark Cleveland; Randall L. Gannon; Claudio Garcia, MD; Daniel Goldberg, MD; Joshua Hedaya, MD; Marietta Karavellas, MD; Tiara Kemper; Brian Kosobucki; Alona Mask; Nicole Reagan MD; Mi-Kyoung Song, MD; Francesca Torriani, MD; Dorothy Wong; Tekeena Young.

University of California, San Francisco: Jacque Duncan, MD (Director); Fermin Ballesteros, Jr.; Robert Bhisitkul, MD, PhD; Debra Brown; David Clay; Michael Deiner; Donald Eubank; Mark Jacobson, MD; Mary Lew, COT; Todd Margolis, MD, PhD. Former Members: Judith Aberg, MD; Jacqueline Hoffman; Alexander Irvine, MD; James Larson; Jody Lawrence, MD; Michael Narahara; Monique Trinidad.

University of North Carolina, Chapel Hill: Travis A. Meredith, MD (Director); Sandy Barnhart; Debra Cantrell; Seema Garg, MD, PhD; Elizabeth Hartnett, MD; Maurice B. Landers, MD; Sarah Moyer; David Wohl, MD;. Former Members : Stephanie Betran; Kelly DeBoer; David Eifrig, MD; John Foley, MD; Angela Jeffries; Jan Kylstra, MD; Barbara Longmire; Sharon Myers; Fatima N'Dure, COA; Kean T. Oh, MD; Jeremy Pantell; Susan Pedersen, RN; Cadmus Rich, MD; Cecilia A. Sotelo, RN; Charles van der Horst, MD; Samir Wadhvania.

University of Pennsylvania Medical Center, Philadelphia, PA: Charles W. Nichols, MD (Director); Mark Bardsley, BSN; Cheryl C. Devine; Jay Kostman, MD; Albert Maguire, MD; William Nyberg; Leslie Smith, RN. Former Members: Chris Helker, RN; RobRoy MacGregor, MD; Karen McGibney, RN; Keith Mickelberg, RN.

University of Southern California, Los Angeles, CA: Former Members: Jennifer I. Lim, MD (Director); Rizwan Bhatti, MD; John Canzano, MD; Thomas S. Chang, MD; Alexander Charonis, MD; Lawrence Chong, MD; Robert Equi, MD; Amani Fawzi, MD; Christina Flaxel, MD; Jesus Garcia; Todd Klesert, MD; Francoise Kramer, MD; Lori Levin, MPH; Tracy Nichols, COA, CRA; Christopher Pelzek, MD; Margaret Podilla, BS; Len Richine; Danny Romo, COA; Srinivas Sadda, MD; Richard Scartozzi, MD; Robert See, MD; Kevin Shiramizu, MD; Mark Thomas; A. Frances Walonker, CO, MPH; Alexander Walsh, MD; Ziquiang Wu, MD.

University of South Florida, Tampa, FL: Peter Reed Pavan, MD (Director); JoAnn Leto, COT; Brian Madow, MD; Richard Oehler, MD; Nandesh Patel, MD; Wyatt Saxon; Susan Sherouse, COT. Former Members: Andrew Burrows, MD; Steve Carlton; Burton Goldstein, MD; Sandra Gompf, MD; Bonnie Hernandez, COT; Mohan Iyer, MD; Patrick Kelty, MD; Amy Kramer, COT; Sharon Millard, RN, COT; Jeffrey Nadler, MD; Scott E. Paulter, MD; Jennifer Tordilla-Wadia, MD; Nancy Walker, COA.

University of Texas Medical Branch, Galveston, TX: Former Members : Garvin Davis, MD (Director); Robert Blem, MD; J. Mike Bourg, VA; John Horna, BS; Craig Kelso; Zbigniew Krason, BS; Helen K. Li, MD; Lan-Chi Nguyen, COMT; Rhonda Nolen, BS, CRC; Michelle Onarato, MD; David Paar, MD; Steven Rivas; Vicky Seitz, COT; Happy Spillar; Sami Uwaydat, MD.

Chairman's Office, Mount Sinai School of Medicine, New York, New York: Douglas A. Jabs, MD, MBA (Study Chairman); Yasmin Hilal, MHS; Melissa Nieves, BA; Karen Pascual, BBA; Jill Slutsky, MPA; Maria Stevens, CM. Former member: Judith C. Southall.

Coordinating Center, The Johns Hopkins University Bloomberg School and Public Health: Curtis L. Meinert, PhD (Director); Alka Ahuja, MS; Debra A. Amend-Libercci; Karen L. Collins; Betty J. Collison; Ryan Colvin; John Dodge; Michele Donithan, MHS; Cathleen Ewing; Kevin Frick, PhD; Janet T. Holbrook, MS, MPH, PhD; Milana R. Isaacson, BS; Rosetta M. Jackson; Hope Livingston; Lee McCaffrey, MA; Milo Puhan, MD, PhD; Girlie Reyes; Jacki Smith; Michael Smith; Elizabeth Sugar, PhD; Jennifer E. Thorne, MD, PhD; James A. Tonascia, PhD; Mark L. Van Natta, MHS; Annette Wagoner. Former members: Carley Benham; Gregory Foster; Judith Harle; Adele M. Kaplan Gilpin, JD, PhD; John H. Kempen, MD, PhD; Barbara K. Martin, PhD; Nancy Min, MPH, PhD; Laurel Murrow, MS; Maria J. Oziemkowska, MS, MPH; Wai Ping Ng, BS; Pamela E. Scott, MA; Erica Smothers; Emily West; Claudine Woo, MPH; Albert Wu, MD, MPH; Alice Zong.

Fundus Photograph Reading Center, University of Wisconsin: Ronald Danis, MD (Director); Charles Chandler; Sapna Gangaputra, MD, MPH; Gregory Guilfoil; Larry Hubbard, MAT; Jeffrey Joyce; Thomas Pauli; Nancy Robinson; Dennis Thayer; Jeong Won Pak; Grace Zhang. Former members: Michael Altaweel, MD; Jane Armstrong; Matthew D. Davis, MD; Sheri Glaeser; Katrina Hughes; Dolores Hurlburt; Linda Kastorff; Michael Neider, BA; Therese Traut; Marilyn Vanderhoof-Young; Hugh Wabers;.

National Eye Institute, Bethesda, MD: Natalie Kurinij, PhD.

Officers of the Study: Douglas A. Jabs, MD, MBA (Chair); Ronald Danis, MD; Natalie Kurinij, PhD; Curtis L. Meinert, PhD; Jennifer E. Thorne, MD, PhD. Former Members: Matthew D. Davis, MD; Janet T. Holbrook, MS, MPH, PhD.

Steering Committee: Douglas A. Jabs, MD, MBA (Chair); Ronald Danis, MD; James P. Dunn, MD; Gary N. Holland, MD; Milana R. Isaccson, BS; Mark Jacobson, MD; Natalie Kurinij, PhD; Richard Lewis, MD, MS; Kisten D. Nolan, RN, BSN, MPH; Curtis L. Meinert, PhD; William Nyberg; Frank Palella, MD; Jennifer E. Thorne, MD, PhD. Former Members : Adrienne Addessi, MA, RN; Lisa Brune, RN, BSN; Rebecca Clark, MD; Tom Clark, CRA; Janet Davis, MD; Matthew D. Davis, MD; William R. Freeman, MD; Dorothy Friedberg, MD; James Gilman; Janet T. Holbrook, MS, MPH, PhD; John Horna; Larry Hubbard, MAT; Mark Jacobson, MD; Daniel F. Martin, MD; Travis A. Meredith, MD; Annmarie Muñana, RN; Robert Murphy, MD; P. Reed Pavan, MD; Steven Spencer, BA, COMT; Tim Steffens, CRA; Dennis Thayer; Charles van der Horst, MD; Fran Wallach.

Policy and Data Monitoring Board: John P. Phair, MD (Chair); Brian P. Conway, MD; Barry R. Davis, MD, PhD; Douglas A. Jabs, MD, MBA; Natalie Kurinij, PhD; Curtis L. Meinert, PhD; David Musch, PhD; Robert B. Nussenblatt, MD; Jennifer E. Thorne, MD, PhD; Richard Whitley, MD. Former Members: B. William Brown, Jr., PhD; Matthew D. Davis, MD; James Grizzle, PhD; Argye Hillis, PhD; Janet T. Holbrook, MS, MPH, PhD; Harmon Smith, PhD; James A. Tonascia, PhD.

Visual Function Quality Assurance Committee: Steven Spencer, BA, COMT (Chair); Robert D. Almanzor; Deborah Gibbs, COMT; Milana Isaacson, BS; Mary Lew, COT; Richard Alan Lewis, MD, MS (Advisor);. Former Members: Ferman Ballesteros; Jeff Grijalva, COT; Karen Lopez; Laura G. Neisser, COT; Rosa Paez-Boham, COST.

Financial support:

LSOCA is supported by cooperative agreements from the National Eye Institute, Bethesda, Maryland, to Mount Sinai School of Medicine (grant no. U10 EY 08052), Johns Hopkins University Bloomberg School of Public Health (grant no. U10 EY 08057), and University of Wisconsin, Madison (grant no. U10 EY 08067). Additional support was provided by the National Center for Research Resources, Bethesda, Maryland, through General Clinical Research Center grants 5MO1 RR 00188 (Baylor College of Medicine), MO1 RR 00052 (Johns Hopkins University School of Medicine), M01 RR00096 (NYU School of Medicine), 5MO1 RR 05096 (Louisiana State University, Tulane, Charity Hospital), 5MO1 RR 00865 (University of California, Los Angeles), 5MO1 RR 05280 (University of Miami), 5M01 RR00046 (University of North Carolina, Chapel Hill), 5MO1 RR 00043 (University of Southern California), and 5MO1 RR 00047 (Weill Medical College of Cornell University). Support also is provided through cooperative agreements U01 AI 27674 (Louisiana State University, Tulane), U01 AI 27660 (University of California, Los Angeles), U01 AI 276670 (University of California, San Diego), U01 AI 27663 (University of California, San Francisco), U01 AI 25858 (University of North Carolina, Chapel Hill), U01 AI 25903 (Washington University at St. Louis), and U01 AI 32783 (University of Pennsylvania) from the National Institutes of Health.

Author information

Authors and Affiliations



Corresponding author

Correspondence to Milo A Puhan.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

MP, AA, MVN, CM designed the study. MP, AA, MVN undertook the statistical analyses. MP wrote the first draft of the manuscript. All authors contributed to and have approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Puhan, M.A., Ahuja, A., Van Natta, M.L. et al. Interviewer versus self-administered health-related quality of life questionnaires - Does it matter?. Health Qual Life Outcomes 9, 30 (2011).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: