Main findings
Patients who did not respond to the post-operative questionnaire tended to have more severe primary conditions and a poorer pre-operative quality of life. In addition, response was lower in patients who were men, younger, non-white, deprived, living alone, and having undergone previous similar surgery. Orthopaedic patients were less likely to respond if they had been treated in an NHS rather than independent provider. The provider's pre-operative recruitment rate and the timing of pre-operative questionnaire administration were not associated with the post-operative response rate.
Comparison with previous research on non-response to post-operative questionnaires
We have studied many more determinants of the response rate to PROM questionnaires mailed to patients after a hospital episode than have been considered previously in published studies. Only the impact of age, sex and health status have been studied before (apart from in our own small study [4] in which we found more deprived people were less likely to respond). Our finding in this study that younger patients (under 55 years in particular) are less likely to respond is consistent with our previous study [4] and a study in Norway of spinal surgery patients, [12] whereas two other studies have found the opposite [10, 11]. The reason for this lack of consistency is unclear. It may reflect the influence of the medical condition and the clinical treatment: studies finding older people less likely to respond were based on investigations of men undergoing prostate surgery [10] and patients with acute coronary syndrome [11].
Our finding that men were less likely to respond is at odds with the literature, which has reported no association with sex for knee replacement [14] or for a mixed population of hospital patients [13]. Again, the reason for this difference in observed association is unclear.
In contrast, the association we observed of non-response with poorer health has been consistently reported in a wide variety of patients: prostate surgery [10], acute coronary syndrome [11], shoulder surgery [15], all hospital admissions [16], and our previous study of elective surgery [4].
Limitations of the study
The first limitation is that date of surgery was not available for 25% of the patients because their pre-operative questionnaire could not be linked to their HES record. As a result, these patients were not included in the analysis considering the timing of administration of pre-operative questionnaires. It is reassuring that in those patients whose PROMs could be linked to HES, we did not find evidence that the timing of the administration of the pre-operative questionnaires influenced response rates.
Second, it is important to take into account the volume of the data and the fact that they are observational. The large volume allows effects to be estimated with great precision, i.e. narrow confidence intervals, so even small effects are regarded as highly statistically significant. However, there is scope for selection biases and residual confounding such that even modest confounding could alter the apparent significance of a small effect.
Implications of findings
The implications of non-response for the use of PROMs data will depend on the extent to which non-response is associated with outcomes. Previous research has shown that those who do not respond have worse outcomes [10, 11, 13–20]. If this were to prove true for elective surgery and non-response is not taken into account, outcomes will be over-estimated, particularly among providers with high non-response rates.
A number of approaches have been suggested to adjust for bias resulting from non-response [2, 3, 11, 26]. Both weighting and multiple imputation are options for including the outcomes of patients who do not respond, based on the outcomes of patients for whom complete data are available [26, 27]. If this approach is adopted, this study has identified the patient and organisational factors that should be included in such models. However, the validity of these techniques is dependent on the assumption that the probability that data are missing does not depend on the value of the missing item having adjusted for observed characteristics (ie those that have been measured). This, however, may be problematic for PROMs data if non-response bias associated with the outcome is not predicted by observed characteristics (ie characteristics for which measurements do not exist). In such circumstances, index function models, including the Heckman method, would need to be employed [10, 11, 13–20].