As HRQoL measures are increasingly used in the general population, it is important to consider various forms of, and reasons for, non-optimal assessment and the extent of potential resulting biases. This study is the first to consider, comprehensively and simultaneously, non response, and incomplete and inconsistent responses to a widely used HRQoL, the SF-36, and their consequences in terms of the validity of estimates, in a general population setting.
Only a half of eligible subjects were found to provide an optimal (complete and consistent) measurement of HRQoL. This proportion could be increased to 66% by accepting sub-optimal (slightly incomplete and inconsistent) questionnaires, specifically those questionnaires that could be reasonably and easily handled using standard rules for managing missing data i.e. the “half item” rule and personal mean score [11, 20] after identifying inconsistencies. Note, however, that such procedures are not widely used in practice despite the fact that they are straightforward and simple to implement. Therefore, for one third of the general adult population that could be assessed for common health questions by face-to-face interview, self assessment of HRQoL using a standardized questionnaire was unsatisfactory. The three groups of subjects we identified with inadequate measurement were of different sizes: 25% of all eligible subjects were non-responders, 6% were poor or very partial responders and 2% inconsistent responders. However, these three groups shared similar socio-demographic determinants. Indeed, several common factors were found associated with both non- and partial response to the SF-36, the strongest being age and educational level. HRQoL is predictive of mortality  and validly reflects the cumulative burden of chronic diseases and disabilities. Clearly, aging populations have been, and will be, targeted for HRQoL studies . This study confirms problems of measurement of HRQoL in the elderly population, with an increased risk of all of non-, partial and inconsistent response after age 50 years. Among men and women aged 75 to 79 years, the proportions of inadequate measurement were about 50% and 55%, respectively, raising serious concerns about the use of a standard “generic” instruments (as is the SF-36) in such (healthy) older populations. Higher rates of missing items in HRQoL questionnaires have already been reported in elderly populations [7, 23–28], but this problem has generally been minimized or resolved by minor adaptations of questionnaires or by interviewer administration [5, 24, 29]. Non responses (missing forms) for HRQoL measures has been less specifically investigated in relation to age, although this issue introduces a major selection bias . However, the findings we report are supported by previous studies showing high non-response rates for elderly subjects to mailed surveys [30–33]. Educational level, marital status and other socio-economic characteristics have less often been considered than age in relation to missing items or non-participation in previous studies of HRQoL instruments. Nevertheless, the evidence available is consistent with the result that subjects with low educational level, foreign origin, low economic status and who are isolated (especially divorced and widowed) are at increased risk of having missing items in HRQoL questionnaires [7, 27] and of non participation in mailed surveys [32, 33]. In the same way as for HRQoL measurement in elderly populations, precautions may be required when measuring HRQoL in groups of subjects less well-educated and well-integrated into western societies.
The relationships between morbid conditions and non- and partial responses observed appeared more complex than expected: some conditions were associated with increased, and others with decreased, partial and non-response rates. Despite the low power of this study for some important but uncommon conditions, as shown in the wide confidence intervals around odds ratios (Table 3), and possible type I error due to testing almost 30 such conditions, a consistent pattern emerged from the data: this pattern suggests that subjects with minor somatic and psychological disorders (e.g. hypertension, anxiety and migraine) are more likely to accept HRQoL measurement than both “healthy” and more seriously affected subjects. Possibly, these subjects whose condition is closely related to impaired HRQoL (i.e. whose expression is mostly decreased HRQoL) find its assessment particularly relevant and are therefore more likely to respond and to do so more meticulously than “average” subjects. However, this behavior, which has not been previously reported, requires further confirmation and also more rigorous analysis in terms of its potential contribution to bias in HRQoL measurements.
Using a multiple imputation method to provide the best corrected estimates of HRQoL in the sample studied, it was possible to assess and quantify the impact of non- and partial responses on the validity of HRQoL estimates. The magnitude of the biases was large in several groups of partial responders and especially non-responders. This confirms the “missing not random” process of missing information in HRQoL, to use the terminology coined by Little and Rubin . These biases, including selection biases  but also non-differential information biases , should be carefully considered. Non-responders in epidemiological studies have long been recognized to have an impact on the validity of the results. Our study evidenced several groups of non-responders to HRQoL questionnaires having different and sometimes opposite impacts on the estimates. This argues for a differentiated approach taking their different causes and/or mechanisms into account. Similarly, non-differential information biases, resulting from partial or inconsistent responses to HRQoL questionnaires, did not appear to be negligible. These biases were especially large for the subgroup of subjects with inconsistent responses, which are seldom examined in standard practice. Although in this study we observed that biases may run in opposite directions and partially neutralize each other, this may of course not be always the case and therefore a careful analysis of the impact of each is required. This issue is particularly pertinent for HRQoL investigations in certain populations: the elderly, and deprived or frail groups. No simple general rule can be given to predict the impact on HRQoL estimation of missing data associated with the various different processes. We therefore strongly recommend using missing value methods such as multiple imputation to evaluate the consequences systematically [4–9, 11].
In conclusion, this empirical study confirms serious problems with HRQoL measurement in the general population due to missing data (both items and forms), especially in elderly, educationally and socio-economically deprived, foreign and isolated groups. Missing data methods and imputation techniques, which are increasingly implemented in standard software packages (SAS, SPSS, etc.), appear to be useful for quantifying potential biases and are therefore recommended to evaluate HRQoL estimates systematically and, if necessary, correct for the resulting biases.