Skip to main content
Download PDF
Download ePub

Measuring the outcomes of long-term care for unpaid carers: comparing the ASCOT-Carer, Carer Experience Scale and EQ-5D-3 L

Health and Quality of Life Outcomes volume 17, Article number: 184 (2019) Cite this article

Abstract

Background

The ASCOT-Carer and Carer Experience Scale are instruments designed to capture aspects of quality of life ‘beyond health’ for family carers. The aim of this study was to compare and validate these two carer care-related measures, with a secondary aim to compare both instruments to the three-level EQ-5D (EQ-5D-3 L) measure of health-related quality of life.

Methods

An interview survey was conducted with 387 carers of adults who used long-term care (also known as social care) support in England. Construct validity by hypothesis testing was assessed using Pearson correlation coefficient. Exploratory factor analysis was also applied to investigate the dimensionality of the combined items from the ASCOT-Carer and CES (as measures of carer quality of life ‘beyond health’) and the EQ-5D (as a measure of health-related quality of life).

Results

In the construct validity analysis, hypothesised differences in correlations were observed with two exceptions. The exploratory factor analysis indicated that the ASCOT-Carer, CES and EQ-5D-3 L items loaded onto three separate factors. The first factor comprised the seven ASCOT-Carer items plus two CES items (activities outside caring, support from friends and family). The second factor comprised three of the six CES items (fulfilment from caring, control over caring and getting on with the person you care for). The third factor included four of the five EQ-5D-3 L items.

Conclusion

The findings indicate that the ASCOT-Carer, CES and EQ-5D-3 L capture separate constructs of social care-related quality of life (ASCOT-Carer) and carer experience (CES), which partially overlap in relation to activities outside caring and social support, and health-related quality of life (EQ-5D-3 L). The ASCOT-Carer and CES are both promising measures for the evaluation of social care support for carers that capture aspects of quality of life ‘beyond health’. The choice of whether to use the ASCOT-Carer or CES depends on the study objectives.

Introduction

Social care (or long-term care) refers to services that seek to maintain or improve quality of life of people who experience difficulties with everyday activities due to long-term health conditions, disability, or age-related impairments (for example, home care) [1, 2]. More recently, the scope of social care policy and interventions have broadened to consider family or friends who support adults with care needs (for example, carers’ support groups or support to stay in employment) [3]. These developments recognise the significant contribution of informal care to long-term care systems [4, 5] and also the needs of carers in terms of the impact of caregiving on their own health, wellbeing and ability to remain in employment or education [6,7,8,9].

In England, there has been a trend towards recognising carers as users of social care services in their own right [8, 9]. The Care Act (2014) places responsibility on local authorities to identify and address the needs of carers alongside those of the adults they support. Therefore, especially in the context of outcomes-based management and administration in public services [10], there is policy-driven interest in considering the quality of life outcomes of informal carers alongside adults with care needs [11, 12]. There are also other arguments for the measurement of carers’ outcomes alongside those of the patient or service user. If the aim of publicly-funded health and social care services is to maximise outcomes through resource allocation on a societal-level, for example, it has also been argued that the impact on carers should also be considered [13,14,15,16,17]. In addition, the benefits of an intervention may be over- or underestimated if its effects on informal carers’ quality of life are not also considered.

In the area of healthcare, the benefits of policy or interventions may be calculated using quality-adjusted life years (QALY), which is the product of life expectancy and health-status of an individual during those years. In calculating QALYs, the EuroQol five-dimension questionnaire (EQ-5D), a measure of health-related quality of life, is widely used [18]. The EQ-5D-3 L captures individual functioning in five health-related domains: pain, mobility, usual activities, anxiety/depression, and self-care [19, 20]. The responses to the five questions, from no problems (1) to severe problems (3), are combined into an EQ-5D health state, for example, 1–1–2-1-3. Using weights developed from preference studies, these states may be converted into a single summary index value, from 0 (being dead) to 1 (in perfect health) [21]. In the evaluation of the (cost-)effectiveness of interventions designed to support carers, however, the attributes captured by the EQ-5D or other health-related quality of life (QoL) instruments may not relate to carers’ concerns or the broader aspects of QoL that may be affected by health and social care interventions [16]. Furthermore, as EQ-5D has been found to lack specificity and sensitivity in the evaluation of social care interventions for service users [22], the impact of social care interventions or policy for carers may likewise be captured more effectively by measures of broader QoL (‘beyond health’): for example, social relationships, support and connectedness, autonomy and control or feeling supported and encouraged in the caring role [16, 23,24,25,26,27,28].

The ASCOT-Carer four-level interview (INT4) [25, 26] and Carer Experience Scale (CES) [27,28,29] are instruments designed for economic evaluation of the impact of services on carers beyond health. The ASCOT-Carer is part of the Adult Social Care Outcomes Toolkit (ASCOT), which is a suite of measures suitable for the economic evaluation of social care interventions or policy [1, 30,31,32,33,34,35]. The measures capture the social care-related quality of life (SCRQoL) of carers and users of adult social care services [25, 26], which relates to aspects of quality of life that may be influenced by social care services [22, 30]. In the UK, social care refers to a range of long-term care services from residential care through to community-based services for adults with care needs that may indirectly also support carers (e.g. domiciliary care, equipment and home adaptations), as well as services directed at the specific needs of carers (e.g. support groups, information and advice) [24, 36].

The seven ASCOT-Carer social care-related quality of life attributes (see Table 1) were identified through focus groups with care managers and carers [37], one-to-one cognitive interviews with carers [38], and a pilot survey of carers in England [39]. Based on this preliminary work, a three-level response version with seven items was developed, the Carer SCRQoL [38, 39]. The Adult Social Care Outcomes Framework (ASCOF) in England includes an abbreviated version of this instrument with only six attributes. (Time and space to be myself is omitted.) This is used as an overarching indicator of carer-reported QoL (ASCOF 1D) [12]. Further research refined the questionnaire to expand the number of response options from three to four-levels [26], which correspond to the ASCOT outcomes states of ideal state, no needs, some needs and high-level needs [30]. The questionnaire was also harmonised with the user version of ASCOT and adapted to incorporate feedback from carers in cognitive debriefing interviews [30].

Table 1 Description of ASCOT-Carer and Carer Experience Scale attributes
Full size table

The Carer Experience Scale is designed to capture the effect of health and social care interventions on aspects of carers’ experience ‘beyond health’ in economic evaluation [27,28,29]. It is a measure of carers’ experiences related to the process of providing care [27, 29]. The six conceptual attributes considered in the scale were developed through a meta-ethnography to synthesise the findings of qualitative research on caregiving experience followed by semi-structured interviews with carers to establish the content validity of the identified attributes [27]. The final set of six attributes are activities outside of caring, support from friends and family, assistance from government and organisations, fulfilment from caring, control over caring and getting on with the care recipient (see Table 1). The three levels of response to each attribute either relate to amount (‘a lot’, ‘some’, ‘a little’) or frequency (‘rarely’, ‘sometimes’, ‘mostly’) [27]. Unlike the CES, the ASCOT-Carer quality of life attributes relate to social care outcomes rather than caregiving experience: for example, the ASCOT-Carer construct of control captures whether the carer feels that s/he has sufficient control over their daily life and activities. By contrast, the CES item on control relates more narrowly to the carer’s control over aspects of caregiving only (see Table 1).

This article presents a comparison of the ASCOT-Carer and CES, as two measures of carer outcomes ‘beyond health’. This is to understand the overlap and distinctiveness of the constructs captured by the two measures, so they may be used appropriately. Both measures were also compared to the three-level EQ-5D (EQ-5D-3 L), to establish whether they measure the construct of carer outcomes ‘beyond health’. A secondary aim was to contribute to the evidence of the construct validity of the ASCOT-Carer [25] and CES [29] by hypothesis testing.

Method

Design and participants

The study sample comprises carers who participated in the Identifying the Impact of Adult Social Care study, which is described in more detail elsewhere [23, 25]. Carers were recruited in 22 English local authorities. All carers supported someone who used publicly-funded adult social care services, who received support due to physical or intellectual disabilities or mental health conditions, and had taken part in an interview for the Identifying the Impact of Adult Social Care study.

In the interview with social care service recipients, participants were asked whether they had been helped by family and friends with regard to activities of daily living (ADLs) and instrumental activities of daily living (IADLs), as well as the number of hours of help received in the last week. At the end of the interview, all participants who had reported that they had received help was asked whether they agreed for their primary carer (defined as the person who had provided the most hours of care in the past week) to be invited to also take part in an interview. A total of 990 interviews with social care recipients were conducted. From these, 739 carers were identified; 510 (69%) were invited to take part in an interview. A total of 387 interviews with carers were completed, either face-to-face (n = 336) or telephone (n = 51), between June 2013 and March 2014.

Ethical approval for the study was obtained from the national social care research ethics committee in England (12/IEC08/0049).

Quality of life measures

Three quality of life measures are compared in this study: a measure of health-related quality of life (the EQ-5D-3 L) and two measures of the QoL ‘beyond health’ (the ASCOT-Carer and CES).

ASCOT-Carer

The Adult Social Care Outcomes Toolkit for Carers (ASCOT-Carer) is a measure of social care-related quality of life [25, 26]. The ASCOT-Carer interview (INT4) used in this study captures three measures of SCRQoL: (1) current SCRQoL; (2) expected SCRQoL; and (3) SCRQoL ‘gain’, an estimate of the impact of social care on QoL using counterfactual self-estimation methodology developed as part of the ASCOT for service users [30, 33] and applied to the carer version of the instrument [26]. In this paper, we consider only current social care-related quality of life. An index score of zero (worst possible SCRQoL) to one (best possible SCRQoL) is calculated by the sum of preference-weights for each selected outcome level, which were developed through best-worst scaling with a general population sample in England [40].

Carer experience scale

The Carer Experience Scale is a measure of caregiving experience [27,28,29]. A total index score is calculated by summing the preference-weights that correspond to the level selected for each domain. The preference weights were developed through best-worst scaling in a sample of carers in the United Kingdom [29]. The index score ranges from 0 to 100, where zero represents the lowest caregiving experience state and 100 the highest caregiving experience state.

EQ-5D-3 l

The EQ-5D-3 L is a five-item instrument that measures health-related quality of life (HRQoL) [19, 20]. The items capture five HRQoL attributes: mobility; self-care; usual activities; pain and discomfort; and anxiety/depression. Each item has three levels of response: no, some or extreme problems. Health states are converted into an index score by calculating the sum of preference-based weights, which correspond to the selected level for each item. The index score range from − 0.594 to 1, where − 0.594 represents extreme problems in all five attributes and 1 represents full health. The preference-weights used to determine the index score were calculated from a study of adults in the United Kingdom [21].

Other measures

The questionnaire also included items to capture the sample characteristics, including the sex, age and employment status of the carers, as well as the context of care (i.e. whether the carer and care-recipient live together, the duration and intensity of caregiving) using items adapted from the household survey of carers in England 2009/10 [41]. The functional ability of the care-recipient was collected through the care-recipient interview, as a self-report of whether s/he found it difficult to complete a list of eight activities of daily living or instrumental activities of daily living (I/ADLs), e.g. getting washed, dressed, in/out of bed. These ratings were combined into a score from none (0) to all eight I/ADLs (8).

The carer questionnaire also included items or scales for construct validity testing by hypothesis testings (see Table 4). These include the perceived choice subscale of the self-determination scale (SDS) [42], which reflects the extent to which individuals feel that they have a choice with respect to their behaviour. The subscale captures the subjective degree of perceived autonomy in everyday activities. Five items are rated from 1 to 5, where a higher rating represents a higher sense of autonomy. The overall subscale score is the average score across the five items.

Positive aspects of the relationship between the carer and care-recipient was measured using the relationship rewards scale, which includes four items: feeling happy with the relationship; the relationship making the carer feel good about themselves; feeling emotionally close to the care recipient; and feeling bored in the relationship [43]. The frequency of these experiences was rated from never (0) to always (3). The overall score was the sum of the four item scores, with the fourth item reverse scored, to form a scale from 0 (lowest relationship reward) to 12 (highest relationship reward). Social loneliness was measured by the three-item UCLA loneliness subscale [44]. The sum score of the items forms a scale from 3 to 9. Higher scores represent a higher degree of perceived loneliness. Frequency of contact with friends and family by telephone or face-to-face was rated on five-point scales from ‘never’ (1) to ‘on most days’ (5).

The questionnaire also included a self-rated health question to rate current health on a five-point scale from very bad (1) to very good (5). Overall quality of life was rated on a seven-point scale from ‘so bad it couldn’t be worse’ (1) to ‘so good it couldn’t be better’ (7). The carers’ satisfaction with social care support (i.e. all social care services used by the carer and/or care recipient) was rated on the seven-point scale from extremely dissatisfied (1) to extremely satisfied (7).

Analysis

Analyses were conducted in Stata version 13 [45].

Construct validity

The construct validity of the EQ-5D-3 L, CES index, ASCOT-Carer SCRQoL index scores was assessed by testing a priori hypotheses of the relationship between measures and other items or scales, which were developed through literature review, previous research and discussion within the research team. The hypothesised correlations were evaluated by Pearson correlation coefficients. The expected associations were considered to be significant where the difference between the Pearson correlation coefficients were greater than 0.1, which has been applied as the smallest expected difference between correlation coefficients in other construct validity studies [46, 47].

The hypothesised associations are outlined in Table 4. First, we anticipated that ASCOT-Carer and CES would be more strongly correlated to each other than either scale with EQ-5D-3 L because the ASCOT-Carer and CES intend to measure broader aspects of QoL than health-related QoL (1). We also expected that the ASCOT-Carer would be less strongly correlated than the EQ-5D-3 L to self-rated health (2). Likewise, we expected both the ASCOT-Carer SCRQoL and CES to be more strongly correlated to each other than the EQ-5D-3 L on these indicators of health (14).

It was also anticipated that the ASCOT-Carer and CES would be more strongly correlated to overall quality of life rated on a 7-point Likert scale than the EQ-5D-3 L, which focuses more narrowly on health-related aspects of QoL, and the CES (3, 14). The ASCOT-Carer was expected to be more strongly correlated to the perception of ability to make choices on the SDS choice subscale than the EQ-5D-3 L, which does not include choice or control as a domain, or the CES, which only captures choice in relation to the caregiving role rather than more broadly (4, 10). Relationship rewards, or positive aspects of the relationship between the carer and care recipient, were expected to be less strongly associated with EQ-5D-3 L than ASCOT-Carer (5), but more strongly associated to the CES than ASCOT-Carer since one of the five CES domains captures how well the carer gets on with the care recipient (11).

It was expected that the UCLA loneliness subscale and frequency of contact with friends or family would be more strongly related to ASCOT-Carer than EQ-5D-3 L since social loneliness forms part of the SCRQoL construct; however, it is not included in the EQ-5D-3 L construct of HRQoL (6, 7, 8). By contrast, it was anticipated that the ASCOT-Carer would be more strongly correlated to UCLA loneliness score than the CES (12) because the CES captures perceived social support, rather than social loneliness. As such, since frequency of social contact may be similarly associated with the constructs of social loneliness and social support, the correlation between ASCOT-Carer or CES and frequency of contact with friends/family was anticipated to be similar (15, 16).

Finally, it was anticipated that the ASCOT-Carer would be more strongly related than the EQ-5D-3 L to satisfaction with social care services (9), while there would be correlation of approximately equal strength for the ASCOT-Carer and CES (17) since both instruments capture care-related aspects of quality of life and experience respectively.

Exploratory factor analysis

Exploratory factor analysis (EFA) is used to identify the underlying dimension(s) within a measurement instrument that may form subscales [48]. In this study, EFA was used to investigate the structural validity (dimensionality) of the ASCOT-Carer, CES and EQ-5D-3 L. An EFA was applied to all items from each of the three measures. A similar method has been used to explore the dimensionality of the corresponding measures of ICEpop CAPability measure for Older people (ICECAP-O) [49, 50] and ASCOT SCRQoL for users of social care services (the ASCOT) [30] in a study of older social care users in the United Kingdom [51]. The ASCOT-Carer and CES are both measures of carer care-related quality of life, albeit with differences in the measurement constructs (i.e. social care-related quality of life (ASCOT-Carer) and carer experience (CES)). By combining the CES and ASCOT-Carer items in the analysis presented here, we sought to determine whether the items could be reduced to the same underlying constructs. The EQ-5D-3 L items were also included in the EFA. It was expected that these items would not load onto the factor(s) as the ASCOT-Carer and/or CES, as the EQ-5D-3 L is a measure of a different construct that is distinct to care-related quality of life (i.e. health-related quality of life).

As the CES and EQ-5D-3 L items are scored on three-levels and the ASCOT-Carer on four-levels of response, the factor analysis was run on the correlations between variables. Since Pearson correlation coefficients can lead to incorrect conclusions with ordinal variables [52, 53], polychoric correlations were calculated and applied to the EFA. Bartlett’s test of sphericity [54] and the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy [55] were used to test whether EFA was appropriate. Mardia’s test for skewness was used to evaluate whether the data were multivariate normal [56]. A combination of visual inspection of the Scree plot, parallel analysis and consideration of the Kaiser criterion (eigenvalue> 1) was used to determine the number of factors [57]. Oblique oblimin rotation of the factors, which allows for correlated factors unlike orthogonal rotation, was applied to support the interpretation of the model [57]. Factor loadings of ≥0.40 are regarded as reliable for interpretation [58]; therefore, we only report factor loadings ≥0.40.

Results

The characteristics of the sample are shown in Table 2. The majority of the sample were female (58.9%) and aged over 65 years (42.9%). Of those aged 64 or younger, the majority were aged 45–64 years (40.8% of the sample). This is comparable with the national estimate that 61% of carers in England are female and 42% are aged 45–64 years; however, the study sample has an older profile than the national estimate of 25% of carers aged over 65 years [59]. Correspondingly, the sample had a higher proportion of carers retired from paid employment (46.2%) compared with the estimate of the population of carers in England (27%) [59].

Table 2 Descriptive statistics of sample (n = 387)
Full size table

The distribution of responses to the items in the carer care-related quality of life measures, the ASCOT-Carer and CES, are shown in Figs. 1 and 2.

Fig. 1
figure 1

Distribution of responses to ASCOT-Carer items

Full size image
Fig. 2
figure 2

Distribution of responses to CES items

Full size image

Pearson correlation coefficients are shown in Table 3. All hypothesised differences in correlations were observed, with two exceptions (see Table 4). These results provide evidence for the construct validity of the instruments as three measures of the distinct, but related, constructs. The EQ-5D-3 L is a measure of health-related quality of life. By contrast, the ASCOT-Carer and CES are measures of broader QoL, beyond health. The ASCOT-Carer captures aspects of carers’ quality of life that may be influenced by social care services and support (i.e. social care-related quality of life). The CES is a measure that captures carer experience more broadly.

Table 3 Pearson correlation coefficients
Full size table
Table 4 Hypothesised associations for construct validity of ASCOT-Carer, CES and EQ-5D-3 L
Full size table

Two hypothesized differences were not observed related to the comparison of the ASCOT-Carer and CES (see Table 4). First, it was hypothesised that relationship rewards, or positive aspects of the relationship between the carer and care recipient, would more strongly associated to the CES than ASCOT-Carer. This is because one of the five CES domains captures how well the carer gets on with the care recipient (Table 4 (11)), whereas the ASCOT-Carer does not explicitly capture the carer/care-recipient relationship quality. As expected, the CES was more strongly correlated to relationship rewards than the ASCOT-Carer; however, this did not reach the 0.1 difference criterion (CES, r = .45; ASCOT-Carer, r = .38). Second, it was anticipated that the ASCOT-Carer would be more strongly correlated to UCLA loneliness score than the CES (Table 4 (12)) because the CES captures perceived social support, rather than social loneliness, whereas the ASCOT-Carer captures social relationships and the perception of social isolation. Again, the ASCOT-Carer was more strongly correlated to loneliness than the CES (CES, r = −.50; ASCOT-Carer, r = −.58), but the difference was less than the 0.1 difference criterion.

Table 5 presents the exploratory factor analysis of the ASCOT-Carer, CES and EQ-5D-3 L items. Bartlett’s test of sphericity indicated that the correlation between items is sufficient for a factor analysis to be conducted (Χ2(153) = 2300.32, p < 0.001). The KMO statistic of sampling adequacy was 0.88, which is well-above the minimum acceptable value of 0.50 for EFA to be applied [60]. Mardia’s test for skewness indicated that the data were not multivariate normal [56]; therefore, principal axis factoring was used for the factor extraction [57]. The Eigenvalues for the first four extracted factors were 6.51, 2.28, 1.18 and 0.68 respectively. This indicated a three-factor solution by the application of the Kaiser criterion (Eigenvalue< 1.00), which was also confirmed by visual inspection of the Scree plot for the point of inflection [57]. Parallel analysis using Horn’s test of factors [61] also indicated a three-factor solution.

Table 5 Oblimin-rotated factor loadings for the ASCOT-Carer, CES and EQ-5D-3 L items (n = 387)
Full size table

The first factor included all seven of the ASCOT-Carer items and two of the CES items (activities outside caring, support from friends and family). These items all relate to the construct of social care-related quality of life (i.e. aspects of quality of life that may be influenced by social care support). All of these items had low uniqueness (≤0.60) with the exception of CES support from friends and family. Together these items explained 59.3% of the variance in the items. The second factor comprised the EQ-5D-3 L items, except for anxiety/depression, and accounted for 10.0% of the variance. These items relate to physical aspects of health-related quality of life. The third factor comprised three of the six CES items: fulfilment from caring, control over caring and getting on with the person you care for. These relate to the carer experience (i.e. the subjective experience of caring from the perspective of the carer). This second factor accounted for 20.8% of the variance. Only one of these three items had high uniqueness (fulfilment from caring). Two items did not show factor loadings of greater than the threshold for reliability set at 0.40 [58]. These were the CES item support from government and organisations (Factor 1 = .18; Factor 2 = .03; Factor 3 = −.0.2) and the EQ-5D-3 L item anxiety / depression (Factor 1 = .36; Factor 2 = .19, Factor 3 = .22), both of which also had high uniqueness (≥0.60).

Discussion

This study tested the construct validity of the ASCOT-Carer, CES and EQ-5D-3 L index scores and the dimensionality of the items using data collected from a survey of carers in England. The findings of hypothesis testing broadly support that the three measures capture different constructs. Specifically, the EQ-5D-3 L is a measure of carer outcomes in terms of the carer’s health, whereas the CES and ASCOT-Carer measures capture broader aspects of QoL ‘beyond health’. This is important because it is known that the experience of caregiving may affect these broader aspects of QoL, like the ability to stay in work, or social relationships. Correspondingly, social care interventions via services or policy are often designed to address these broader aspects of quality of life, rather than health (e.g. support to stay in employment) [6,7,8,9]. Therefore, in order to ensure that the effects of health and social care interventions are adequately captured, it is important to use outcome measures that are able to capture these broader aspects of QoL. These findings add to earlier studies of the construct validity of the CES and ASCOT-Carer measures [25, 29] and also directly compares these measures with each other, as well as with the EQ-5D-3 L, as a measure of health-related QoL.

While the CES and ASCOT-Carer are both measures of carers’ QoL ‘beyond health’ that are distinct from the concept of health-related QoL measured by the EQ-5D-3 L, the construct validity analysis provides evidence that the CES and ASCOT-Carer also capture distinct constructs. The ASCOT-Carer captures aspects of QoL that are typically targeted by social care services in England (i.e. social care related quality of life) and was more closely related to carers’ perceived satisfaction with social care support than the CES. By contrast, the CES captures more general aspects of carers’ experience, including caring-related aspects of QoL that may be beyond the scope of social care interventions.

In the construct validity analysis, however, two of the hypothesised differences in correlations were not observed. It had been expected that the CES would be more strongly related to relationship reward (quality) between the carer and care-recipient than the ASCOT-Carer because the CES includes an item that seeks to capture relational quality (getting on with the person you care for). The ASCOT-Carer was anticipated to be more strongly related to loneliness, due to the related concept of ‘feeling isolated’ captured by the social participation and involvement item. However, the observed differences in correlation were less than the criterion applied for this study. This indicates an area of potential overlap between the two measures. Further research could provide evidence of the conceptual overlaps between these items around the dimensions of relationships, social participation and loneliness (e.g. through in-depth qualitative interviews to explore these items).

In the analysis of the factor structure of the ASCOT-Carer, CES and EQ-5D-3 L items combined, it was found that the CES and ASCOT-Carer items partially measure the same construct, whereas the EQ-5D-3 L items load onto a separate factor. With regard to the ASCOT-Carer and CES overlap, the CES items of activities outside of caring and social support were found to load onto the same factor as the seven ASCOT-Carer items. These two attributes have some conceptual overlap with the ASCOT-Carer attributes of Occupation - doing things I value and enjoy and Social contact and participation, however, the CES items relate more specifically to caregiving experience. For example, ASCOT-Carer Social contact and participation captures the broad outcome state of an individual’s satisfaction with the quality and quantity of social contact and connectedness. By contrast, the CES social support item refers more narrowly to the level of social support from friends and family with respect to caregiving, yet still relates to social contact.

Three of the six CES items loaded onto a separate factor to the ASCOT-Carer. The attributes of fulfilment from caring, control over caring, and getting on with the care-recipient may be conceptualised as aspects of the caregiving experience [27], which are distinct from the construct of aspects of QoL that may be affected by social care interventions. For example, Getting on with the care-recipient may be indirectly and weakly influenced by social care services, however, the quality of the existing relationship and other factors are likely to be more important [26]. Likewise, the carer’s experience of fulfilment from caring is not a concept that carers are able to relate to the effect of social care services [26]. Unlike control over daily life, which may be affected by social care support [26], control over caring is the carers’ experience of their ability to influence the care provided by health, social care and other local services (see Table 1). The CES item that captures the amount of assistance from organisations and the government does not load onto either of the two factors identified in the analysis.

Four of the five EQ-5D-3 L items loaded onto a separate factor with no overlap with the ASCOT-Carer or CES. One of the five items (anxiety/depression) did not load onto any of the factors, with the minimum loading criterion of >.40. Of the three factors, this item had the highest loading for factor one, along with the seven ASCOT-Carer items and two CES items. Therefore, study provides evidence that the ASCOT-Carer, like the service user version of ASCOT [47], and also the CES, only captures physical health in a limited way; however, there may be more overlap with psychological health. This finding is consistent with the conceptual basis of the CES and ASCOT-Carer as measures of carers’ outcomes, beyond health.

The limitations of the study should be considered when interpreting the results. This paper presents analysis based on a sample of carers in England. Data was collected from a heterogeneous sample of carers identified through users of publicly-funded social care services. Due to the study methodology, the sample is not representative of carers in England. While some of the sample characteristics are consistent with population estimates [59], older adults are overrepresented. Even with this limitation, the study provides further evidence of the validity of the CES and ASCOT-Carer with a heterogeneous sample of carers that is consistent with other studies [25, 29] and how these measures compare to the EQ-5D-3 L. Further work is needed to establish the validity and comparability of the measures in other countries with different social policy, cultural and linguistic contexts.

In summary, this study provides evidence of construct and structural validity of the ASCOT-Carer, CES and EQ-5D-3 L as carer outcome measures. The items from these measures load onto three separate factors. These relate to (1) social care related quality of life (i.e. aspects of broader quality of life that may be affected by social care services); (2) carer experience related to the process of caregiving, which are outside the direct influence of social care services; and (3) physical health-related quality of life. The strong correlation between the ASCOT-Carer/CES suggests that (economic) evaluation studies could use one or other of these two measures, rather than both together. There is some evidence that the ASCOT-Carer may be better suited to the evaluation of social care interventions, however, further research would usefully inform an understanding of the conditions under which these two measures perform best. The EQ-5D-3 L captures the distinct construct of health-related quality of life, so may be used alongside the ASCOT-Carer or CES to consider both health-related and ‘beyond health’ outcomes. Further work is required to establish whether this approach may potentially double-count health effects, especially with regard to psychological health.

Conclusions

The results of this study indicate that the ASCOT-Carer and CES are promising measures for the evaluation of health and social care services for carers that capture aspects of quality of life ‘beyond health’. The choice of whether to use the ASCOT-Carer or CES depends on the study objectives. If the study seeks to evaluate social care services, defined as services ‘for’ the care-recipient and/or the carer, the ASCOT-Carer may be more suitable. If the study aims to measure carer experience more broadly, the CES may be more suitable. Further research is needed to establish the conditions under which each measure performs best, to inform the appropriate use of these measures in evaluation studies of health and social care interventions or policy.

Availability of data and materials

The raw data from the study reported in this article is not freely available because we do not have consent for publication of these data.

Abbreviations

ADLs:

Activities of daily living

ASCOT:

Adult Social care outcomes toolkit

ASCOT-Carer:

Adult social care outcomes toolkit for carers

CES:

Carer experience scale

EFA:

Exploratory factor analysis

EQ-5D:

EuroQol five-dimension questionnaire

EQ-5D-3 L:

EuroQoL five-dimension questionnaire with three levels of response

HRQoL :

Health-related quality of life

ICECAP-O:

ICEpop CAPability measure for Older people

INT4:

Interview with four levels of response

KMO:

Kaiser-Meyer-Olkin

QALY:

Quality-adjusted life years

QoL:

Quality of life

SCRQoL:

Social care-related quality of life

SDS:

Self-determination scale

References

  1. Malley J, Towers A, Netten A, Brazier J, Forder J, Flynn T. An assessment of the construct validity of the ASCOT measure of social care-related quality of life with older people. Health Qual Life Outcomes. 2012;10(21):1-14.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Netten A. Overview of outcome measurement for adults using social care services and support. London: NIHR School for Social Care Research; 2011.

    Google Scholar 

  3. Courtin E, Jemiai N, Mossialos E. Mapping support policies for informal carers across the European Union. Health Policy. 2014;118:84–94.

    Article  PubMed  Google Scholar 

  4. Pickard L. The supply of informal care in Europe. European Network of Economic Policy Research Institutes 2011. https://www.files.ethz.ch/isn/133676/ENEPRI%20RR94%20_ANCIEN%20WP%203_%20Supply%20of%20Informal%20Care%20in%20Europe.pdf.

  5. Pickard L, Wittenberg R, Comas-Herrera A, Davies B, Darton R. Relying on informal care in the new century? Informal care for elderly people in England to 2031. Ageing Soc. 2000;20:745–72.

    Article  Google Scholar 

  6. Department of Health. Caring about carers: a national strategy for carers. London: Department of Health; 1999.

    Google Scholar 

  7. Department of Health. Carers at the heart of 21st century families and communities: a caring system on your side, a life of your own. London: Department of Health; 2008.

    Google Scholar 

  8. Department of Health. Recognised, valued and supported: next steps for the carers strategy. London: Department of Health; 2010.

    Google Scholar 

  9. Department of Health. Carers Strategy: Second National Action Plan 2014–2016. London: Department of Health; 2014.

    Google Scholar 

  10. Bovaird T. Attributing outcomes to social policy interventions: ‘gold standard’ or ‘Fool's gold’ in public policy and management? Soc Policy Adm. 2012;48:1–23.

    Article  Google Scholar 

  11. Department of Health. Transparency in outcomes: a framework for quality in adult social care; a response to the consultation and next steps. London: Department of Health; 2010.

    Google Scholar 

  12. Department of Health. The adult social care outcomes framework 2018/19: a handbook of definitions. London: Department of Health; 2018.

    Google Scholar 

  13. Smith K, Wright K. Informal care and economic appraisal: a discussion of possible methodological approaches. Health Econ. 1994;3:137–48.

    Article  CAS  PubMed  Google Scholar 

  14. Brouwer W. Too important to ignore? Informal carers and significant others. Pharmacoeconomics. 2006;24:39–41.

    Article  PubMed  Google Scholar 

  15. Davidson T, Levin L-A. Is the societal approach wide enough to include relatives? Appl Health Econ Health Policy. 2010;8:25–35.

    Article  PubMed  Google Scholar 

  16. Al-Janabi H, Flynn T, Coast J. QALYs and Carers. Pharmacoeconomics. 2011;29:1015–23.

    Article  PubMed  Google Scholar 

  17. Hoefman RJ, van Exel J, Brouwer W. How to include informal Care in Economic Evaluations. Pharmacoeconomics. 2013;31:1105–19.

    Article  PubMed  Google Scholar 

  18. Devlin N, Brooks R. EQ-5D and the EuroQol group: past, present and future. Appl Health Econ Health Policy. 2017;15(2):127–37.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Brooks R. EuroQol: the current state of play. Health Policy. 1996;37:53–72.

    Article  CAS  PubMed  Google Scholar 

  20. The EuroQol Group. EuroQol - a new facility for the measurement of health-related quality of life. Health Policy. 1990;16:199–208.

    Article  Google Scholar 

  21. Dolan P. Modeling valuations for EuroQol health state. Med Care. 1997;35:1095–108.

    Article  CAS  PubMed  Google Scholar 

  22. Forder J, Caiels J. Measuring the outcomes of long-term care. Soc Sci Med. 2011;73:1766–74.

    Article  PubMed  Google Scholar 

  23. Forder J, Malley J, Rand S, Vadean F, Jones K, Netten A. Identifying the impact of adult social care: interpreting outcome data for use in the adult social care outcomes framework. Canterbury: PSSRU, University of Kent; 2016.

    Google Scholar 

  24. Rand S, Malley J. Carers’ quality of life and experiences of adult social care support in England. Health Soc Care Commun. 2014;22:375–85.

    Article  Google Scholar 

  25. Rand S, Malley J, Forder J, Netten A. Factor structure and construct validity of the adult social care outcomes toolkit for carers (ASCOT-carer). Qual Life Res. 2015;24:2601–14.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Rand S, Malley J, Netten A. Measuring the social care outcomes of carers: an interim technical report for the identifying the impact of adult social care (IIASC) study. Canterbury: PSSRU, University of Kent; 2012.

    Google Scholar 

  27. Al-Janabi H, Coast J, Flynn T. What do people value when they provide unpaid care for an older person? A meta-ethnography with interview follow-up. Soc Sci Med. 2008;67:111–21.

    Article  PubMed  Google Scholar 

  28. Al-Janabi H, Flynn T, Coast J. Estimation of a preference-based Carer experience scale. Med Decis Mak. 2011;31:458–68.

    Article  Google Scholar 

  29. Goranitis I, Al-Janabi H, Coast J. An investigation into the construct validity of the Carer experience scale (CES). Qual Life Res. 2014;23:1743–52.

    Article  PubMed  Google Scholar 

  30. Netten A, Burge P, Malley J, Potoglou D, Towers A, Brazier J, Flynn T, Forder J, Wall B. Outcomes of social care for adults: developing a preference weighted measure. Health Technol Assess. 2012;16:1–165.

    Article  CAS  PubMed  Google Scholar 

  31. Towers A, Smith N, Palmer S, Welch E, Netten A. The acceptability and feasibility of using the adult social care outcomes toolkit (ASCOT) to inform practice in care homes. BMC Health Serv Res. 2016;16(524):1-14.

  32. Turnpenny A, Caiels J, Whelton B, Richardson L, Beadle-Brown J, Crowther T, Forder J, Apps J, Rand S. Developing an easy read version of the adult social care outcomes toolkit (ASCOT). J Appl Res Intellect Disabil. 2018;31:e36–48.

    Article  PubMed  Google Scholar 

  33. Malley J, Rand S, Netten A, Towers A, Forder J. Exploring the feasibility and validity of a pragmatic approach to estimating the impact of long-term care: The 'expected' ASCOT method. Journal of Long-Term Care. 2019;1:67–83.

  34. Rand S, Malley J, Towers A, Netten A, Forder J. Validity and test-retest reliability of the self-completion adult social care outcomes toolkit (ASCOT-SCT4)with adults with long-term physical, sensory and mental health conditions in England. Qual Life Res. 2017;15(163):1-15.

  35. Rand S, Towers A, Razik K, Turnpenny A, Bradshaw J, Caiels J, Smith N. Factor structure and construct validity of the easy-read Adult Social Care Outcomes Toolkit (ASCOT-ER). J Intellect Dis Res. 2019. https://doi.org/10.3109/13668250.2019.1592126.

  36. Pickard L: The effectiveness and cost-effectiveness of support and services to informal carers of older people: a review of the literature prepared for the audit commission. In Audit Commission; 2004.

  37. Holder J, Smith N, Netten A. Outcomes and quality for social care services for carers: Kent County council carers survey development project 2007–2008. Canterbury: Personal Social Services Research Unit, University of Kent; 2009.

  38. Fox D, Holder J, Netten A. Personal social Services of Adult Carers in England 2009–10: survey development project – technical report. Canterbury: Personal Social Services Research Unit, University of Kent; 2010.

    Google Scholar 

  39. Malley J, Fox D, Netten A. Developing a carers’ experience performance indicator. Canterbury: Personal Social Services Research Unit, University of Kent; 2010.

    Google Scholar 

  40. Batchelder L, Malley J, Burge P, Lu H, Saloniki E, Linnosmaa I, Trukeschitz B, Forder J. Carer social care-related quality of life outcomes: estimating English preference weights for the adult social care outcomes toolkit for Carers (ASCOT-Carer). Value Health. 2019;22(12):1427–40.

    Article  PubMed  Google Scholar 

  41. Survey of carers in households 2009/10. http://digital.nhs.uk/catalogue/PUB02200. Accessed 12 Nov 2018.

  42. Sheldon KM, Ryan RM, Reis H. What makes for a good day? Competence and autonomy in the day and in the person. Personal Soc Psychol Bull. 1996;22:1270–9.

    Article  Google Scholar 

  43. Williamson GM, Shaffer DR. Relationship quality and potentially harmful behaviors by spousal caregivers: how we were then, how we are now. Psychol Aging. 2001;16:217–26.

    Article  CAS  PubMed  Google Scholar 

  44. Hughes M, Waite L, Hawkley L, Cacioppo J. A short scale for measuring loneliness in large surveys. Res Aging. 2004;26:655–72.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Statacorp: Stata data analysis statistical software: Release 13. 2013.

    Google Scholar 

  46. de Boer MR, Terwee CB, de Vet HCW, Moll AC, Voelker-Dieben HJM. Evaluation of cross-sectional and longitudinal construct validity of two vision-related quality of life questionnaires: the LVQOL and VCM1. Qual Life Res. 2006;15:233–48.

    Article  PubMed  Google Scholar 

  47. van Leeuwen K, Bosmans J, Jansen A, Hoogendijk E, van Tulder M, van der Horst H, Ostelo R. Comparing measurement properties of the EQ-5D-3L, ICECAP-O, and ASCOT in frail older adults. Value Health. 2015;18:35–43.

    Article  PubMed  Google Scholar 

  48. Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess. 1995;7:286–99.

    Article  Google Scholar 

  49. Coast J, Flynn T, Natarajan L, Sproston K, Lewis J, Louviere J, Peters TJ. Valuing the ICECAP capability index for older people. Soc Sci Med. 2008;67:874–82.

    Article  PubMed  Google Scholar 

  50. Grewal I, Lewis J, Flynn T, Brown J, Bond J, Coast J. Developing attributes for a generic quality of life measure for older people: preferences or capabilities? Soc Sci Med. 2006;62:1891–901.

    Article  PubMed  Google Scholar 

  51. Hackert MQN, Exel JV, Brouwer W. Valid outcome measures in Care for Older People: comparing the ASCOT and the ICECAP-O. Value Health. 2017;20:936–44.

    Article  PubMed  Google Scholar 

  52. Olsson U. Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika. 1979;44:443–60.

    Article  Google Scholar 

  53. Holgado-Tello F, Chacón-Moscoso S, Barbero-García IV, Ila-Abad E. Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Qual Quant. 2010;44:153–66.

    Article  Google Scholar 

  54. Bartlett MS. A note on the multiplying factors for various chi square approximations. J R Stat Soc Ser B. 1954;16:296–8.

    Google Scholar 

  55. Kaiser HF. An index of factor simplicity. Psychometrika. 1974;39:31–6.

    Article  Google Scholar 

  56. Mardia KV, Kent JT, Bibby JM. Multivariate analysis. London: Academic Press; 1979.

    Google Scholar 

  57. Fabrigar LR, Wegener DT, Maccallum RC, Trahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods. 1999;4:272–99.

    Article  Google Scholar 

  58. Hair JF, Anderson RE, Tatham RL, Black W. Multivariate data analysis. 5th ed. Prentice Hall: Upper Saddle River, NJ; 1998.

    Google Scholar 

  59. Department of Health. Survey of carers in households 2009/10. London: Department of Health; 2010.

    Google Scholar 

  60. Hutcheson G, Sofroniou N. The multivariate scientist. London: Sage; 1999.

    Book  Google Scholar 

  61. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrica. 1965;30:179–85.

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We are very grateful to all who participated in the research and to Accent, who undertook the fieldwork for this study.

Funding

This article is based on independent research commissioned and funded by the NIHR Policy Research Programme (Quality and Outcomes of person-centred care policy Research Unit). The views expressed in the publication are those of the authors, and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care, or its arm’s length bodies, or other government departments.

Author information

Authors and Affiliations

  1. Personal Social Services Research Unit (PSSRU), Cornwallis Building, University of Kent, Canterbury, CT2 7NF, UK

    Stacey Rand, Florin Vadean & Julien Forder

  2. The Care Policy and Evaluation Centre (CPEC), London School of Economics and Political Science, Houghton Street, London, WC2A 2AE, UK

    Juliette Malley

Authors
  1. Stacey Rand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juliette Malley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florin Vadean
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Julien Forder
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SR contributed to the design of the study, conducted the statistical analysis, and drafted the manuscript. JM and FV contributed to the drafting of the manuscript. JF and FV contributed to the design of the study. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Stacey Rand.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for the study was given by the national social care research ethics committee in England (12/IEC08/0049). Written or verbal informed consent was obtained from all participants prior to interview.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rand, S., Malley, J., Vadean, F. et al. Measuring the outcomes of long-term care for unpaid carers: comparing the ASCOT-Carer, Carer Experience Scale and EQ-5D-3 L. Health Qual Life Outcomes 17, 184 (2019). https://doi.org/10.1186/s12955-019-1254-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12955-019-1254-2

Keywords

Health and Quality of Life Outcomes

ISSN: 1477-7525

Contact us