Measuring the impact of caregiving on informal carers: a construct validation study of the CarerQol instrument

The attention for informal care appears to be increasing, given the inherent and increasingly noticed scarcity of formal health care resources in many Western countries [1]. Informal care is an important part of total care, especially in the context of chronic illness and frailty due to ageing, and is often provided voluntarily by family, friends or acquaintances. Informal care may reduce the pressure on the capacity and budget of formal health care [2–4]. Moreover, it may be preferred by both patient and informal caregiver over formal care [5].

Notwithstanding the fact that providing informal care can be rewarding [5, 6], caring can have considerable negative effects on the health and general well-being of informal caregivers [7–13]. Therefore, the impact of providing informal care on carers should be recognised by policy makers when making decisions concerning the structure and provision of health care services. Moreover, information on the impact of informal care is valuable input for policy decisions regarding arrangements facilitating and supporting informal caregiving in health care.

Economic evaluations aim to support optimal allocation of scarce health care resources. Although inclusion of informal care in economic evaluations is highly desirable [14–17], at present informal care commonly is ignored in economic evaluations. Thus, policymakers remain ignorant of the impact of interventions in health care on informal caregivers and risk making non-optimal decisions. Moreover, in the few instances that informal care is included in economic evaluations, the comparability of results is hampered by differences in measuring and valuing informal care [18–20]. This is, for example, reflected in different approaches to measure and value caregivers’ time input [19–22], health [11, 23], and well-being [19, 24].

Common approaches to value informal care, such as the willingness to pay method or the proxy good method, typically provide limited information regarding the underlying informal care situation and its potentially diverse impact [20, 25–27]. Subjective burden measures for informal care focus more on this latter issue. Several generic and disease-specific subjective burden instruments are available describing the negative impacts of caring, such as problems experienced with mental health, physical health, or social and financial aspects [21, 28–32]. Some instruments aim to capture the positive impacts of caring as well [25, 29, 33–35].

While many of these subjective burden instruments provide a detailed description of caregiving burden, they do not value the impact of caregiving in economic terms, making them unsuitable for economic evaluations. At this time, only two instruments combine an economic valuation of informal care with the informational density of burden instruments: The Caregiver Experience Scale (CES) [35, 36] and Care-related Quality of Life instrument (CarerQol) [34]. Both instruments describe the care situation in terms of the negative and positive impact of caregiving, and value the overall impact of informal care. The CarerQol instrument values this impact in two ways: general well-being and care-related quality of life. The (latter) utility scores for the CarerQol are based on preference information from the general public in the Netherlands [37]. The CES instrument values the impact of caregiving with care-related quality of life scores, based on preference information from caregivers of elderly persons in the UK for the CES [35, 36].

When patient interventions are compared in economic evaluations, the CES or CarerQol can be used as an additional source of information in cost-effectiveness analyses using conventional outcome measures, such as patient Quality Adjusted Life Years (QALYs), or as one of the principal outcome measures in cost-consequence or multi-criteria analyses. Furthermore, cost-effectiveness analyses of interventions or support programmes targeted directly at informal caregivers can apply the CES or CarerQol as main outcome.

The focus in this paper is on the CarerQol (see Figure 1). This instrument was developed in 2006, in a similar way as the EuroQol instrument [38], and it has been applied in several studies since [26, 39–45].

Four validation studies of the CarerQol have been conducted previously. Brouwer et al. [34] & Hoefman et al. [46] performed tests of the feasibility of the CarerQol instrument. Construct validity of the CarerQol instrument was studied in different caregiver samples: two heterogeneous groups of caregivers that were members of regional caregiver support centres in the Netherlands (n = 175 in Brouwer et al. [34] and n = 230 in Hoefman et al. [47]), caregivers of persons permanently living in or receiving day care from a nursing home in the Netherlands (n = 108) [46] and a sample of children with craniofacial malformations living in the US (n = 65) [44]. Test-retest reliability of the CarerQol was also investigated in the sample of caregivers of nursing home care patients [46]. These various tests of the psychometric properties of the CarerQol showed favourable results concerning its feasibility [34, 46], construct validity [34, 44, 46, 47], and test-retest reliability [46].

The results of these four studies require further confirmation for several reasons. First, the validation studies so far used caregiver samples that were relatively small: the number of respondents ranged between 65 and 230. Second, these samples were either overrepresented by relatively strained caregivers [34, 47] or by caregivers in a specific informal care situation (e.g., caring for young disabled children [44] or institutionalized elderly [46]). Third, the range of tests used for construct validation was limited. Most tests concerned whole sample analyses and did not investigate possible heterogeneity among caregivers (because of study sample size). Moreover, few other subjective burden instruments were available from these studies to compare the CarerQol with (e.g. [44, 46]).

The study presented in this paper aimed to overcome most of these shortcomings by using a much larger, heterogeneous sample of informal caregivers (n = 1,244), representing a broad range of informal care situations and levels of caregiving burden. This sample size also allows for the construct validation of the CarerQol to be tested in a more elaborate way, which is important given the lack of a gold standard for the impact of caregiving. These tests will be conducted by comparing the performance of the CarerQol with a number of other subjective burden instruments and a range of subgroup analyses comparing between groups of carers characterised by differences in caregiving strain, health and socio-economic characteristics.

This study was specifically designed to validate the CarerQol and to demonstrate its ability to assess the overall impact of caregiving in diverse types of caregiving situations. The availability of a validated instrument to measure and value the impact of caregiving will support its application in informal care research. Moreover, it facilitates the inclusion of informal care impacts in economic evaluations of diverse patients and caregiver interventions, and better evidence-based decision making in health care.

This study investigated whether the CarerQol validly assessed the overall impact of caregiving in a large, heterogeneous sample of caregivers from the Netherlands. Results of clinical, convergent and discriminative validity tests confirmed the favourable results from previous studies. Both the subjective burden (CarerQol-7D) and the well-being (CarerQol-VAS) component of the CarerQol were significantly associated in the expected direction with other measures of the impact of caring. Additionally, well-being was related to important caregiver, care recipient and care situation characteristics, in expected directions. Hence, this study adds to previous evidence suggesting that the CarerQol may be a useful measure of the impact of caregiving in research in a wide variety of informal care contexts. Moreover, it facilitates inclusion of informal care in economic evaluations of health care interventions.

The CarerQol instrument measures care-related quality of life of caregivers. This concept is broader than the generally used outcome measure in economic evaluations, as for instance health-related quality of life in cost-utility analyses. Therefore, the results of the CarerQol cannot be easily combined with patient outcomes cost-effectiveness or -utility analyses. Nevertheless, as stated in the Introduction, the results of the CarerQol can be presented alongside the results of an economic evaluation and so inform decision makers more completely about the total impact of an intervention on society. The CarerQol can also be used with other types of economic evaluation, such as multi-criteria analyses, which allow considering multiple outcome measures. Finally, the CarerQol can very well serve as the main outcome measure in economic evaluations of programmes for caregivers (e.g., support programmes, respite care).

Before we discuss the implications of our results in more detail, it is important to note some limitations of our study. First, although we used a representative sample of the adult Dutch population in terms of age and gender, our sample may be somewhat selective due to the use of an online panel. Internet was considered to be a suitable medium to gather data, because more than 90 per cent of the Dutch population uses internet. In addition, while elderly may be a typical group expected to be underrepresented in internet surveys, it needs noting that in recent years elderly became more active on the internet with six out of ten elderly of 65 to 75 years currently being internet users [59]. Furthermore, selection bias could have occurred in using an online panel to select caregivers. We did not have a priori reasons to suspect that caregivers in general would be less likely to subscribe to online panels. This may be the case for caregivers experiencing high levels of strain, but this group is generally difficult to approach and less likely to participate in any type of survey. Hence, we expect that the subgroup of caregivers experiencing severe strain may be underrepresented in our sample, but not more or less than in other studies among caregivers. On the other hand, in previous validation studies [34, 44, 46, 47] caregivers were recruited via organisations providing information and support services for caregivers. It is likely that people who come to such organisations to ask for assistance see themselves as caregivers and concern a selection of caregivers experiencing relatively higher strain in their caregiving situation. Through the online panel and the selection questions used in this study it is likely that we have reached persons lending care who would normally not define themselves as caregivers, for instance because their burden is low, and therefore would not be represented in these previous studies. All in all, we expect our sample to be more representative of the caregiver population, in particular at the low burden end.

Second, in our multivariate models, we used the multiple imputation method (MI) to handle missing values. An assumption of this method is that these values are, at least, missing at random [56, 60]. While income is a classic example of missing not at random [61], we nevertheless considered MI as the best alternative to other strategies, such as excluding respondents with missing values from our model, or mean imputation. In addition, income correlated with other background characteristics in our data, such as gender of the caregiver, which gives some support to our imputation method.

Third, validation is an on-going process, and therefore, testing psychometric properties among caregivers in other settings, such as caregivers in other countries than the Netherlands, as for instance recently in the US [44], remains desirable. In addition, other psychometric properties of the CarerQol could be further tested, such as reliability [46] and sensitivity to change.

The overall well-being in this sample was relatively high compared to caregiver samples of previous validation studies. Possibly this is due to the recruitment procedure in this study that may have attracted significantly more caregivers in low burden care situations, as discussed before. Higher well-being scores were found among caregivers with positive care experiences in terms of fulfillment from caregiving and assistance from others in lending care, which many caregivers reported to have. Furthermore, as previous CarerQol validation and informal care studies underlined, well-being of caregivers was positively related to health of both the caregiver and care recipient [23, 34, 46, 47, 62, 63]. Moreover, as other studies on informal care also suggest (e.g. [23, 62], well-being of carers was positively influenced by more general aspects of life, not necessarily (directly) related to caregiving [64], such as having a full-time paid work position or a high income. Besides these positive influences, caregivers also experienced negative consequences of lending care. Important to note here, is that we used cross-sectional data and hence were not able to determine the causality of the established associations. Caregivers often faced problems with their health and also reported difficulties combining care with other daily activities. Having these problems, negatively affected their well-being. Further, especially those caring for their parents (in-law) reported lower happiness scores.

Our study clearly indicates that the diverse problems associated with informal care are not equally important for all caregivers. For example, financial problems were negatively associated with happiness of caregivers with a relatively low income particularly. In addition, mental health problems and problems with daily activities were associated with well-being among caregivers with a relatively low health status, while physical health problems and financial problems were among relatively healthy caregivers. Differences were also observed for receiving support with care tasks. The positive influence of support was especially prevalent among highly burdened caregivers.

Furthermore, a note should be made on the CarerQol-7D dimension relational problems, which showed overall satisfactory convergent and discriminative validity results, but did not impact well-being when considered in combination with other burden dimensions. This seems to contradict some previous validity results [34, 46]. However, additional tests in subgroups, more closely resembling the samples used in earlier studies, confirmed that relational problems did affect well-being among carers of recipients with a persistent care need often due to chronic or age-related health problems.

This diversity in importance of burden dimensions among subgroups indicates that although some aspects of caregiving burden may not seem relevant in some group of caregivers, they may matter to caregivers in another context, as was described above for the dimension relational problems. Additionally, given that not all problems are equally important for caregivers, it is recommendable that support programmes target the problems that are relevant to specific (groups of) caregivers, such as relieving financial problems of caregivers on low income.

Convergent and discriminative validity tests using the ASIS, SRB, PU, CSI and Pt instruments, which aim to measure a similar construct as the CarerQol, showed that the CarerQol instrument performs well. That is, when the ASIS, SRB, PU, CSI and Pt reported higher caregiving burden, the CarerQol also indicated a higher negative impact of caregiving. All these associations between the CarerQol and the overall scores of the ASIS, SRB, PU, CSI and Pt were as expected and the strength of correlations was small to high. The support dimension of the CarerQol-7D showed the weakest correlation with these other measures.

The associations between single CarerQol-7D dimensions and similar individual items of the CSI confirmed our hypothesized relationships as well. The only hypothesis that was rejected was that of the CarerQol-7D dimension daily activities and the CSI item 'other demands on time’. This counterintuitive result may be explained by different content of both items, because the CSI item merely registers whether caregivers perform other activities, while the CarerQol-7D focuses on difficulties with combining these with caregiving.

Discriminative validity tests showed that the CarerQol-7D discriminated well between caregivers with low or high burden. These results were less stable for the dimension support. Although in general support is an important issue for caregivers [35], some of our validity results were less satisfactory for the support dimension of the CarerQol. This could be explained by the fact that receiving support could have both a positive and a negative effect on caregiving strain. For example, it has been shown that sharing tasks with other informal caregivers tends to decrease burden, but that cooperation with others may also increase burden in case of disagreements between caregivers [12]. Given that support from family or friends with caregiving can both relieve and intensify strain among caregivers, the overall effect of support can level out when studying a large, diverse group of caregivers. It would be interesting to investigate this in more detail in future studies using the CarerQol, for instance by adding a few supplementary questions about the amount, type and perceived quality of support.

This study largely confirmed previous findings on construct validation of the CarerQol and added new, strong arguments that this instrument is a valid measure of the overall impact of informal care. Therefore, the CarerQol can be applied in both informal care research and in economic evaluations of diverse patients or caregiver interventions to reveal the important, but often hidden, impact of informal caregiving for well-informed health care policy.