Skip to main content
Download PDF
Download ePub

Assessing feasibility, construct validity, and reliability of a new aged care-specific preference-based quality of life instrument: evidence from older Australians in residential aged care

Health and Quality of Life Outcomes volume 20, Article number: 159 (2022) Cite this article

Abstract

Background

Quality of Life-Aged Care Consumers (QOL-ACC) is a new older-person-specific quality of life instrument designed for application in quality assessment and economic evaluation in aged care. The QOL-ACC was designed from its inception with older people receiving aged care services ensuring its strong content validity. Given that the QOL-ACC has already been validated in home care settings and a preference-weighted value set developed, we aimed to assess feasibility, construct validity and reliability of the QOL-ACC in residential aged care settings. 

Methods

Individuals living in residential aged care facilities participated in an interviewer-facilitated survey. The survey included the QOL-ACC, QCE-ACC (quality of aged care experience measure) and two other preference-based quality of life instruments (ASCOT and EQ-5D-5L). Feasibility was assessed using missing data and ceiling/floor effects. Construct validity was assessed by exploring the relationship between the QOL-ACC and other instruments (convergent validity) and the QOL-ACC’s ability to discriminate varying levels of self-rated health and quality of life. Internal consistency reliability was assessed using Cronbach’s alpha (α).

Results

Of the 200 residents (mean age, 85 ± 7.7 years) who completed the survey, 60% were female and 69% were born in Australia. One in three participating residents self-rated their health as fair/poor. The QOL-ACC had no missing data but had small floor effects (0.5%) and acceptable ceiling effects (7.5%). It demonstrated moderate correlation with ASCOT (r = 0.51, p < 0.001) and EQ-5D-5L (r = 0.52, p < 0.001) and a stronger correlation with the QCE-ACC (r = 0.57, p < 0.001). Residents with poor self-rated health and quality of life had significantly lower scores on the QOL-ACC. The internal consistency reliability of the QOL-ACC and its dimensions was good (α = 0.70–0.77).

Conclusions

The QOL-ACC demonstrated good feasibility, construct validity and internal consistency reliability to assess aged care-related quality of life. Moderate correlations of the QOL-ACC and other instruments provide evidence of its construct validity and signifies that the QOL-ACC adds non-redundant and non-interchangeable information beyond the existing instruments. A stronger correlation with the QCE-ACC than other instruments may indicate that quality of life is more intimately connected with the care experience than either health- or social-related quality of life in residential aged care settings.

Introduction

Similar to many other developed countries, older Australians with assessed care needs can access government-subsidised and funded aged care services either in their own home or in a residential care facility when they can no longer live independently. In 2020–21, Australian government expenditure on aged care was over AU$23.5 billion, with the largest proportion of funding ($14.3 billion or about 60%) allocated to residential aged care [1]. During this period, over one million older Australians received home-based care and support and just over one third of a million (371,000) accessed residential aged care services [2]. Due to a rapidly increasing ageing population, it is expected that about four million older Australians will need to access care and support from aged care providers either at home or residential aged care by 2050 [3].

A recent federal government commissioned independent investigation by the Royal Commission into Aged Care Quality and Safety in Australia found many systemic failures including substandard service quality, unsafe service delivery, abuse, neglect, and major quality and safety concerns leading to unnecessary distress, suffering, and poor well-being for older people using aged care services [4]. The COVID-19 pandemic has further exposed the cracks in the system with disproportionate levels of COVID-19-related morbidity and mortality in the sector; a situation that is being mirrored internationally, including in Europe, USA and Canada [5,6,7,8]. The Royal Commission has made a series of policy recommendations to overhaul the system to achieve transformational changes to ensure dignified, safe, transparent, and evidence-based person-centred care is provided to all older people in aged care. A key recommendation was to mandate routine collection and reporting of quality of life data to complement the existing mandatory clinical quality and safety indicators of care quality (e.g. pressure injury, medication management, unplanned weight loss) [4]. There is a growing consensus that whilst clinical quality indicators are useful, such indicators do not provide individuals’ lived experience of the quality of aged care and its influence on their quality of life [9, 10].

There can be no doubt that the quality of life of older people receiving aged care services is a concern for all Australians. However, to date, quality of life data has not been routinely collected nor reported nationally, and this has created a vacuum in the available evidence base outcomes across the board in terms of tangible improvements to people’s lives to inform policy and practice [11, 12]. In order to introduce person-centred and value-based care reforms across the aged care sector, there is a growing consensus regarding a need to collect data using valid, and reliable, measures of quality of life, where quality of life is encapsulated from the perspective of older people themselves. In order to address this need, our team has developed a new instrument (Quality of Life-Aged Care Consumers, QOL-ACC) for the measurement and valuation of quality of life in aged care [13]. From its inception and at every stage of development, [14,15,16] we have actively partnered with end users and industry partners. Consequently, the QOL-ACC uniquely incorporates salient quality of life dimensions that are important, meaningful, and preferred by older people receiving aged care services. Importantly, all the quality of life dimensions included in the QOL-ACC are also directly influenceable through the services and supports offered in home and residential care settings by aged care providers [15, 16].

The QOL-ACC is designed to serve as a standalone instrument to encapsulate aged-care specific quality life. It also has the potential to be applied as a preference-based utility measure applicable for assessing cost-effectiveness. In this way, the QOL-ACC can be used to generate evidence for much needed aged care policy reforms to drive efficiency improvements and ensure resources are allocated to maximise quality of life [13].

Although not developed specifically for application in aged care, there are several other existing preference-based instruments that have been widely applied in aged care settings including the EQ-5D-5L and the ASCOT [11, 12]. Our previous work has identified that these existing instruments have a more generic focus on health and social care, and were developed for application with adults of all ages rather than focusing on older people’s perceptions of important quality of life dimensions specific to aged care [11, 12, 17].

The development of a new quality of life instrument from its inception with older people requires a significant investment in time and resources to produce evidence to demonstrate that the instrument is valid, reliable, and produces meaningful scores. As the QOL-ACC is designed to be universally applicable for the aged care sector, assessing its performance in home and residential aged care settings is vital. The QOL-ACC has already been tested for its psychometric performance and construct validity among older people in home care setting [14]. Given its robust developmental origins and successful face validity testing across both home and residential aged care settings, coupled with its strong psychometric performance in home care settings, we hypothesised that the QOL-ACC is a feasible, valid, and reliable instrument to assess aged-care specific quality of life of older people in residential aged care facilities. Therefore, this study aimed to assess the feasibility, construct validity, and internal consistency reliability of the QOL-ACC in a residential aged care population in Australia.

Methods

Study sample and the survey

A multi-centre, cross-sectional study was designed and conducted in 22 residential aged care facilities across five Australian states and territories (Australian Capital Territory, Queensland, South Australia, Tasmania and Western Australia). Study participants were aged 65 years and older living in residential care facilities, able to speak and communicate in English, not currently diagnosed with severe cognitive impairment or dementia and determined by the facility manager to have the cognitive capacity to participate. The survey interviews were conducted by trained interviewers experienced in conducting interviews in aged care settings. All of the interviewers in the interview team were trained, skilled and experienced in quality of care experience/quality of life interviews with older aged care residents having recently previously undertaken a large data collection exercise in a similar population of aged care residents for the Royal Commission into Aged Care Quality and Safety investigation [18].

The project manager (CH) provided details of the residential aged care facilities who expressed an interest to participate in the study to the interviewing field teams. The field teams then made contacts with the facilities to arrange a suitable day and time to attend, and also inquired about any COVID-19 related visitor protocols that needed to be observed on each location. On the day of the visit, the facility manager provided the field team with a list of residents who had previously indicated an interest in participation and had given their verbal consent to be approached. Due to the ongoing COVID-19 restrictions in New South Wales and Victoria at the time of the study, interviews were unable to be conducted in these two states (Table 1). All participants provided written informed consent prior to participating in an interview-facilitated survey using a computer assisted personal interviewing (CAPI) device. Family members and next to kin of the residents who expressed their willingness to participate were informed about the research. The interviewers were police checked and trained on ethics and CAPI interview protocols. Each interview was conducted in the resident’s room or in another safe space one-to-one within the residential aged care facility. The study was approved by the Social and Behavioural Research Ethics Committee at Flinders University (Approval no: 5508). The participants completed the QOL-ACC, three other instruments (described below), a series of socio-demographic questions including age, gender, and global items asking participants to self-rate their overall general health (1 item, excellent to poor), and overall quality of life (1 item, excellent to poor) questions. Deidentified data from the CAPI was securely transferred to Flinders University for data storage and analysis.

Table 1 Socio-demographic characteristics of the respondents who were living in residential aged care facilities
Full size table

Instruments

QOL-ACC (Quality of Life-Aged Care Consumers)

The descriptive system for the QOL-ACC was developed using a multi-phase project including extensive work on content development and psychometric testing (including face, content, and construct validity testing) [14,15,16]. The final descriptive system for the QOL-ACC has 6 dimensions (independence, mobility, pain management, emotional wellbeing, social connection, and activities). Five response levels are attached to each dimension, ranging from the best level ‘all of the time’ to the worst level ‘none of the time’. Responses to the six domains can generate 63,500 possible combinations of response options representing QOL-ACC quality of life states. A discrete choice experiment comprising pairwise comparisons of QOL-ACC quality of life states was undertaken to develop a preference weighted scoring algorithm (value set) based upon the preferences of over 1000 older Australians accessing aged care services [19]. The utility weighted (index scores) for the QOL-ACC ranges from -0.56 (lowest possible score or pit state) to 1.00 (highest possible score). Negative QOL-ACC scores indicate that a small proportion of QOL-ACC states were considered by the majority of older Australians participating in the development of the value set as worse than being dead [19].

ASCOT (Adult Social Care Outcome Tool)

The ASCOT is a preference based social-care related quality of life instrument [20, 21]. The ASCOT descriptive system has 8 dimensions: personal cleanliness and comfort, food and drink, control over daily life, safety, accommodation cleanliness and comfort, social participation and involvement, occupation, and dignity. Each dimension is framed as “which of the following statements best describes…” and rated on a four response levels representing four different outcome status (‘ideal’, the preferred situation; ‘no needs’, where needs are met; ‘some needs’, where there are needs but no immediate/long-term health implications; and ‘high needs’, where needs have immediate and long-term health implications), [21] and the ASCOT preference weighted scores for English general population range from -0.17 to 1.0, with higher scores representing better social-care related quality of life [20].

QCE-ACC (Quality of Care-Aged Care Consumers)

The QCE-ACC is a preference-based measure of aged-care-specific quality of care validated in both home and residential aged care settings [18]. The QCE-ACC was developed from a study commissioned by the Royal Commission into Aged Care Quality and Safety in Australia [22]. The QCE-ACC has 6 dimensions (respect and dignity, services and supports, decision-making, staff skills and training, social relationships, and complaints). Each dimension is rated on a 5-response options (‘always ‘ to ‘never’). The QCE-ACC preference weighted scores for older people aged 65 years and above ranges from 0 to 1 with higher scores representing better aged-care specific quality of care experience [18].

EQ-5D-5L

The EQ-5D-5L is a generic health-related quality of life instrument with two sections: a descriptive system containing five dimensions of health status (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression) on a 5-level scale of severity (‘no problems’ to ‘extreme problems/unable’) and a visual analogue scale (EQ-VAS). It is one of the most widely used multi-attribute utility instruments and various sets of preference-based utility values have been developed across cross-national general population samples [23]. For this study, we used the Australian preference weights developed by Norman et al. [24]. The EQ-VAS is a vertical visual analogue scale of self-reported health which ranges from 0 (‘worst possible health one can imagine’) to 100 (‘best possible health one can imagine’).

Feasibility, validity, and reliability assessments

Feasibility, construct validity, and reliability assessments for the QOL-ACC were guided by the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines [25, 26]. Feasibility of the QOL-ACC was assessed in terms of missing data and floor/ceiling effects. The floor and ceiling effects occur when the QOL-ACC index score were clustered at the lowest (i.e., -0.57) and highest (i.e., 1.0) possible scores respectively. Low levels of missing data (≤ 5%) and floor/ceiling effect of ≤ 15% indicate good feasibility [27, 28].

Construct validity is the degree to which the scores from health/quality of life instruments represent the construct they purport to measure. This was assessed by testing whether the scores of QOL-ACC were consistent with a series of a priori hypotheses in relation to convergent and known-group validity. Convergent validity is the extent to which the QOL-ACC correlated with other instruments to the degree that was expected, and known group validity is the extent to which an instrument discriminates between groups known to be different. Assessing construct validity is an iterative process whereby a series of hypotheses are tested with an assumption that the instrument validly measures the construct it purports to measure. A total of seven a priori hypotheses were developed to appraise construct validity of the QOL-ACC (Table 2). A significant but medium (coefficient range of > 0.30 to 0.70) correlation between the QOL-ACC and other instruments is indicative of good convergent validity, with related constructs expected to have a stronger correlation than unrelated constructs [29]. For example, we hypothesised that the QOL-ACC would demonstrate a stronger relationship with the QCE-ACC (aged care quality experience) and ASCOT (social care related quality of life) than EQ-5D-5L (health related quality of life). Known group validity was assessed by testing the hypotheses that older people with poor self-reported health and quality of life would have lower scores on the QOL-ACC and vice versa. Construct validity was perceived as adequate if more than 75% of the hypothesised relationships, in terms of the directions and strengths of correlations, were supported by the analysis results [29, 30].

Table 2 A priori hypothesized association between the Quality of Life-Aged Care Consumer (QOL-ACC) instrument and other related constructs
Full size table

Reliability of the QOL-ACC was assessed by assessing internal consistency reliability using Cronbach’s alpha and item-scale correlations [31, 32]. Cronbach’s alpha demonstrates the degree to which all dimensions of the QOL-ACC tap into different aspects of the overall underlying construct (i.e., aged care-specific quality of life). Cronbach’s alpha between 0.70 and 0.90 indicate sufficient to good internal consistency reliability [31]. Further evidence of internal consistency was assessed by estimating how highly correlated each dimension was with the overall scale [32]. Ideally, the dimension-scale correlations should be similar, and good or higher item-scale correlations (i.e., ≥ 0.20 is good, and ≥ 0.40 is very good) support the notion that the dimensions were good contributors to the overall scale. Given that the QOL-ACC has six-dimensions but intends to produce a single index score, it was expected that all domains score similarly and correlate highly.

Statistical analysis

Data were analysed using STATA Version 15.1, Stata Corp LLC, Texas, USA. Descriptive statistics were used to summarise socio-demographics and presented as percentages for categorical variables, with mean (standard deviation) or median (interquartile range) for continuous variables. Socio-Economic Indexes for Areas (SEIFA) provides ranking of geographic areas in Australia by their relative socio-economic advantage and disadvantage based on information from the five-yearly census data collected by the Australian Bureau of Statistics. Two SEIFA indexes: Index of Relative Socio-economic Disadvantage (IRSD) and Socio-Economic Index of Education and Occupation (IEO) were estimated based on postcode of residency [33].

The Kruskal–Wallis Test was applied to assess the difference in the EQ-5D-5L, EQ-VAS, ASCOT, and QCE-ACC scores by QOL-ACC dimension levels (test of monotonicity: to indicate that the scores on other instruments increased by response levels across the QOL-ACC dimensions). For convergent validity (to assess the extent to which the QOL-ACC and other instruments measure related constructs), Spearman’s rank absolute correlation coefficients ( ρ and p values) were produced because the distribution of the index scores was not normal, as expected in this context (Additional file 1, Fig. 1). The size of correlation coefficients is interpreted as negligible (0.00 to 0.30), low (> 0.30 to 0.50), moderate (> 0.50 to 0.70) and high (> 0.70 to 0.90). These analyses were complemented by locally weighted scatterplot smoothing (LOWESS) techniques. The LOWESS is a form of non-parametric regression which plots a line of central tendency between two measures on a scatterplot (visually) to demonstrate relationship across all the possible score ranges without making assumption about the actual relationships. LOWESS plots were used to visually assess strength and direction of the relationship between the QOL-ACC and other instruments [34]. For known group validity, the Kruskal–Wallis test was used to test differences between the multiple groups. A post-hoc Dunn’s test was carried out following the Kruskal–Wallis test for multiple pairwise comparison between the groups [35]. A total of 7 hypotheses were tested to assess the construct validity of the QOL-ACC. To adjust for multiple testings, we used the Bonferroni technique to set the significance threshold at p ≤ 0.05/21 = 0.002.

Fig. 1
figure 1

The Quality of Life- Aged Care Consumers (QOL-ACC) LOWESS plots (clockwise) against the Adult Social Care Outcome Tool (ASCOT, A), Quality of Care-Aged Care Consumers (QCE-ACC, B), EuroQOL 5 dimensions and 5 levels (EQ-5D-5L, C) and EuroQOL Visual Analogue Scale (EQ-VAS, D)

Full size image

Results

Of the 279 residents approached, 55 (19.7%) declined to participate, and six (2.2%) were unable to participate due to sickness. Of the 218 who consented initially, 18 (8.3%) discontinued as they were struggling to comprehend and respond to the survey. Of the 200 respondents who fully completed the survey, 60% were female, mean age was 85 years (SD ± 7.7), 56.8% were 85 years and older, 68.5% were born in Australia and 95.5% had English as a first language. The demographic characteristics of the study population were comparable to the Australian national population in residential aged care as of 30th June 2021 in terms of gender, age, country of birth and language distributions (Table 1) [2]. Based on the one item general health and quality of life questions, a higher proportion of residents self-reported to have fair or poor health (33.5%) compared to fair or poor quality of life (25.5%). Based on the Australian Bureau of Statistics measures of socio-economic conditions, over one in five participating residents were originally from the least advantaged geographic areas (SEIFA IRSEAD quintile 1&2 = 21.5% and SEIFA-IEO quintiles 1 = 21.5%). Participating residents had a higher median index score on the QOL-ACC (median, IQR: 0.80, 0.65–0.91) and the QCE-ACC (median, IQR: 0.94, 0.86–0.99) than on the ASCOT (median, IQR: 0.73, 0.64–0.79) and the EQ-5D-5L (median, IQR: 0.64, 0.33–0.52) (Table 1).

All the respondents completed the QOL-ACC with no missing data, demonstrating its ease of use. The QOL-ACC demonstrated a small floor (0.5%) and acceptable ceiling effects (7.5%). The dimensions of the QOL-ACC mostly demonstrated a monotonic increment in scores on other instruments (ASCOT, QCE-ACC, EQ-5D-5L and EQ-VAS) by their response category levels, signifying that the QOL-ACC response categories were ordered and were able to distinguish between the respondents with low to high scores as measured by other instruments (Table 3).

Table 3 Mean and median scores for other instruments by QOL-ACC dimensions and their levels
Full size table

Construct validity

Convergent validity

The QOL-ACC and the scores from other instruments (QCE-ACC, ASCOT and EQ-5D-5L) were moderately correlated (Table 2 Hypothesis 1 & Table 4). Although it was moderate, the QOL-ACC demonstrated stronger correlation with the QCE-ACC (ρ = 0.57, p < 0.001) than with other instruments (Table 2 Hypothesis 2 & Table 4). The QOL-ACC demonstrated similar moderate correlations with the ASCOT (ρ = 0.51, p < 0.001; Table 2—Hypothesis 3) and the EQ-5D-5L (ρ = 0.52, p < 0.001; Table 2 Hypothesis 4). As expected, the QOL-ACC demonstrated a weaker correlation with the EQ-VAS (ρ = 0.36, p < 0.001; Table 2 Hypothesis 5).

Table 4 Relationship between Quality of Life-Aged Care Consumers (QOL-ACC) and other instruments
Full size table

LOWLESS graphs (Fig. 1: A, B, C, and D) added further evidence of convergent validity of the QOL-ACC by demonstrating upward trends in the relationship between the QOL-ACC and other instruments, indicating that higher quality of life measured by the QOL-ACC was associated with higher (better) scores on other instruments [34]. The trend was more pronounced between the QOL-ACC and QCE-ACC/ASCOT/EQ-5D-5L (Fig. 1: A, B, and C) than the EQ-VAS (Fig. 1D) across the full range of the QOL-ACC index scores, again mirroring the strength of correlations shown in Table 4.

Known group validity

Respondents with different ratings of health and quality of life had different scores on the QOL-ACC (Additional file 1, Table 1). More specifically, respondents with poor self-reported QOL had poor index scores on the QOL-ACC (Table 2 Hypothesis 6) with significant differences between all 5 categories of quality of life rating (Fig. 2; X2 = 64.4, p < 0.001). Similarly, respondents who reported poor health had poor index scores on the QOL-ACC (X2 = 69.1, df = 3, p < 0.001) with significant differences between all 5 categories of health ratings (Fig. 3, X2 = 69.1, df = 3, p < 0.001).

Fig. 2
figure 2

Quality of Life-Aged Care Consumers (QOL-ACC) index scores by self-rated quality of life

Full size image
Fig. 3
figure 3

Quality of Life-Aged Care Consumers (QOL-ACC) index scores by self-reported health

Full size image

All seven a priori hypotheses were confirmed (Table 1), signifying a strong evidence of construct validity of the QOL-ACC in residential aged care population.

Reliability

The internal consistency reliability of the overall QOL-ACC (alpha = 0.76) was good. All the QOL-ACC dimensions demonstrated adequate Cronbach’s alpha values with pain management (alpha = 0.77) and independence (alpha = 0.70) dimensions demonstrating the highest and lowest internal consistency respectively (Table 5). Dimension-scale correlations for all dimensions were excellent (≥ 0.40) adding further evidence of internal consistency reliability (Table 5).

Table 5 Overall and dimension level Cronbach’s alpha and Pearson’s correlations between individual items (dimensions) and the overall scale (Quality of Life-Aged Care Consumers, QOL-ACC)
Full size table

Discussion

This study provides comprehensive evidence of feasibility, construct validity and reliability of the QOL-ACC to assess quality of life among older people living in residential aged care facilities in Australia. These findings are not surprising given how meticulously the content of the QOL-ACC was developed, guided by direct feedback from older people receiving aged care services and industry partners to ensure adequate face and content validity of the QOL-ACC [11, 15, 16]. These findings add further evidence to our previous study in Australian home care settings, in which we demonstrated that the QOL-ACC was a valid instrument to measure quality of life in older people receiving aged care services at home [14]. Therefore, this study provides additional evidence to confirm the validity of QOL-ACC in measuring and valuing quality of life for older people receiving aged care services in Australian home and residential care settings.

Our analyses showed that the QOL-ACC has expected correlations with related constructs. As hypothesised (Table 2) the overall utility scores of the QOL-ACC positively and significantly correlated with the EQ-5D-5L and ASCOT, indicating that aged-care specific quality of life is a similar construct to health-related and social-care specific quality of life respectively. However, the strength of the correlations was moderate, indicating that the QOL-ACC adds new information and is measuring a construct sufficiently different from the constructs measured by existing instruments in aged care settings.

The relationship between the QOL-ACC and other instruments (ASCOT and EQ-5D-5L) in this study is consistent with findings in a parallel matched-design study in older Australian receiving aged care at home [14]. Albeit moderate, the strength of correlation of the QOL-ACC with both the ASCOT and the EQ-5D-5L was higher in the home care population than in this study [14]. These findings may indicate that quality of life among older people in long-term residential aged care is a more complex construct and is influenced by factors that are different than older people living in the community. Simply, when compared to people receiving aged care services at home, aged care residents are likely to be older, more impaired, frail, have multiple morbidities, be highly dependent and have less connection with their peers/loved ones than they would like to have, all of these factors may lead to differing perspectives on quality of life [2, 9, 36].

Older people living in residential aged care facilities are likely to have differing conceptions of quality of life based on their health status, care needs, experience of quality of care, and preferences. Supporting this argument, we found that the QOL-ACC demonstrated the strongest correlation with the QCE-ACC (a measure of quality-of-care experience in aged care) than with EQ-5D-5L and ASCOT (Table 4). These findings contrast with those from our prior work in older people receiving aged care services at home where we found that the QOL-ACC had a weaker correlation with the QCE-ACC than with ASCOT and EQ-5D-5L [14]. Older people in residential aged care facilities are more dependent on daily care supports provided by facilities than older people in the community and may not be able to maintain or improve their quality of life without those services and supports. Therefore, it is possible that quality of life perceptions for aged care residents may be more closely associated with care quality and the care experience than older people living in their own homes. However, the relationship between care experience and quality of life needs further investigation, particularly due to heterogeneity in care needs, as service provision becomes increasingly concentrated on individuals with the greatest care needs. A similar association between care experience and quality of life was reported in a study conducted in the UK [37]. However, it is also important to note that the level of association between the QOL-ACC and QCE-ACC was moderate. This finding supports the argument that these two constructs are distinctive and not directly interchangeable, reinforcing the value of measuring these two different concepts separately [37].

Known group analysis confirmed our prior hypotheses (Table 2) that the QOL-ACC scores were higher in people with higher on global self-reported quality of life (Fig. 2) and health (Fig. 3) items. These findings were mirrored in our previous study that assessed known group validity of the QOL-ACC in older Australians accessing aged care at home [18]. Strong evidence of convergent and known-groups validity demonstrates the construct validity of the QOL-ACC in older people accessing residential aged care services in Australia. Missing data, and ceiling and floor effects, for the QOL-ACC were within acceptable range, and are similar to other studies that have assessed feasibility of other preference-based instruments [28, 38, 39]. Internal consistency reliability for the QOL-ACC was good and the value was similar to literature reports for most of the other instruments, [40, 41] confirming that basic level of reliability of the instrument in residential aged care population. Further ongoing studies will establish other important psychometric properties of the QOL-ACC such as test–retest reliability and responsiveness.

Although this study was carried out in a relatively small sample of 200 older people living in residential aged care services, our study population was broadly representative in terms of age, gender, country of birth, and language distribution when compared with the national population using residential aged care services in Australia (Table 1) [2]. Despite ongoing COVID-19 restrictions at the time of the study, data was collected from five out of eight states and territories in Australia (data was not collected from New South Wales, Victoria and Northern Territory). As the QOL-ACC was primarily designed as a self-reported instrument, we were unable to include older people who had severe cognitive impairment. Although a proxy version of the QOL-ACC is available, due to COVID-19 restrictions operating at the time of data collection, we were unable to extend data collection activities to include proxies for those with severe cognitive impairment.

Our study has provided evidence to support the feasibility, reliability and construct validity of the QOL-ACC in residential aged care settings. Given that psychometric validation of any instrument is a journey than a destination, our future work will explore evidence of its content relevance, reliability (test–retest), validity, responsiveness (i.e., sensitive to detect change) and the utility of the QOL-ACC in a broader aged care population using a novel clinimetics approach. In essence, clinimetrics provides a more suitable conceptual and methodological framework by combining judgements from the stakeholders (e.g. aged care providers and end users) and modern psychometric methods (such as Rasch analysis and/or Item response theory) to assess important psychometric properties such as reliability, responsiveness, clinical validity and utility of the instruments [42, 43].

Conclusion

The QOL-ACC demonstrated good feasibility, construct validity, and internal consistency reliability to assess aged-care specific quality of life among aged care residents in Australia. With an increasing awareness of the central importance of person-centred outcomes for monitoring quality and safety in aged care, QOL-ACC has the potential to be applied as a key aged care quality indicator. The preference weighted scoring algorithm accompanying the QOL-ACC facilitates its application in economic evaluation for measuring and valuing quality of life to generate new evidence to guide value-based aged care policy and practice and facilitate sector-wide improvements.

Availability of data and materials

No (under embargo).

References

  1. Australian Institute of Health and Welfare. Canberra: Spending on aged care; 2022. (https://www.gen-agedcaredata.gov.au/Topics/Spending-on-aged-care). 

  2. Australian Institute of Health and Welfare. People using aged care: at 30 June 2021 (https://www.gen-agedcaredata.gov.au/Topics/People-using-aged-care). Canberra. 2022

  3. Parliament of Australia. Reforming Australia's aged care system: are we there yet? (https://www.aph.gov.au/About_Parliament/Parliamentary_Departments/Parliamentary_Library/pubs/BriefingBook44p/AgedCare). Canberra. 2013

  4. The Royal Commission into Aged Care Quality and Safety. Final Report: Care, Dignity and Respect- List of Recommendations (https://agedcare.royalcommission.gov.au/sites/default/files/2021-03/final-report-recommendations.pdf). Canberra Commonwealth of Australia. 2021

  5. Ibrahim JE, Ranson DL, Bugeja L. Premature deaths of nursing home residents: an epidemiological analysis. Med J Aust. 2018;208(3):143.

    Article  PubMed  Google Scholar 

  6. Lloyd L, Banerjee A, Harrington C, Jacobsen F, Szebehely M. It s a scandal! Comparing the causes and consequences of nursing home media scandals in five countries. Int J Social Soc Policy. 2014;34:2–18.

    Article  Google Scholar 

  7. Holroyd-Leduc JM, Laupacis A. Continuing care and COVID-19: a Canadian tragedy that must not be allowed to happen again. Can Med Assoc J. 2020;192(23):E632–3.

    Article  Google Scholar 

  8. McGilton KS, Escrig-Pinol A, Gordon A, Chu CH, Zuniga F, Sanchez MG, Boscart V, Meyer J, Corazzini KN, Jacinto AF, Spilsbury K, Backman A, Scales K, Fagertun A, Wu B, Edvardsson D, Lepore MJ, Leung AYM, Siegel EO, Noguchi-Watanabe M, Wang J, Bowers B. Uncovering the Devaluation of Nursing Home Staff During COVID-19: Are We Fuelling the Next Health Care Crisis? J Am Med Dir Assoc. 2020;21(7):962–5.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Armstrong P. Balancing the Tension in Long-Term Residential Care. Ageing Int. 2018;43(1):74–90.

    Article  Google Scholar 

  10. Gilbert AS, Garratt SM, Kosowicz L, Ostaszkiewicz J, Dow B. Aged Care Residents’ Perspectives on Quality of Care in Care Homes: A Systematic Review of Qualitative Evidence. Res Aging. 2021;43(7–8):294–310.

    Article  PubMed  Google Scholar 

  11. Cleland J, Hutchinson C, Khadka J, Milte R, Ratcliffe J. A Review of the Development and Application of Generic Preference-Based Instruments with the Older Population. Appl Health Econ Health Policy. 2019;17(6):781–801.

    Article  PubMed  Google Scholar 

  12. Bulamu NB, Kaambwa B, Ratcliffe J. A systematic review of instruments for measuring outcomes in economic evaluation within aged care. Health Qual Life Outcomes. 2015;13:179.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Ratcliffe J, Cameron I, Lancsar E, Walker R, Milte R, Hutchinson CL, Swaffer K, Parker S. Developing a new quality of life instrument with older people for economic evaluation in aged care: study protocol. BMJ Open. 2019;9(5):e028647.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Khadka J, Ratcliffe J, Hutchinson C, Cleland J, Mulhern B, Lancsar E, Milte R. Assessing the construct validity of the Quality-of-Life-Aged Care Consumers (QOL-ACC): an aged care-specific quality-of-life measure. Qual Life Res. 2022;31(9):2849-65.

  15. Hutchinson C, Ratcliffe J, Cleland J, Walker R, Corlis M, Cornell V, Khadka J. The integration of mixed methods data to develop the Quality of Life- Aged Care Consumers (QOL-ACC) measure. BMC Geriatr. 2021;21(1):702.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Cleland J, Hutchinson C, McBain C, Walker R, Milte R, Khadka J, Ratcliffe J. Developing dimensions for a new preference-based quality of life instrument for older people receiving aged care services in the community. Qual Life Res. 2021;30(2):555–65.

    Article  PubMed  Google Scholar 

  17. Easton T, Milte R, Crotty M, Ratcliffe J. Where’s the evidence? a systematic review of economic analyses of residential aged care infrastructure. BMC Health Serv Res. 2017;17(1):226.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Khadka J, Ratcliffe J, Chen G, Kumaran S, Milte R, Hutchinson C, Savvas S, Batchelor F. A new measure of quality of care experience in aged care: psychometric assessment and validation of the Quality of Care Experience (QCE) questionnaire. Caring Future Institue, South Australia: Flinders University; 2020.

    Google Scholar 

  19. Ratcliffe J, Bourke S, Li J, Mulhern B, Hutchinson C, Khadka J, Milte R, Lancsar E. Valuing the Quality of Life Aged Care Consumers (QOL-ACC) instrument for quality assessment and economic evaluation. Pharmacoeconomics. 2022:440(11):1069-79.

  20. Malley JN, Towers AM, Netten AP, Brazier JE, Forder JE, Flynn T. An assessment of the construct validity of the ASCOT measure of social care-related quality of life with older people. Health Qual Life Outcomes. 2012;10:21.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Netten A, Burge P, Malley J, Potoglou D, Towers AM, Brazier J, Flynn T, Forder J, Wall B. Outcomes of social care for adults: developing a preference-weighted measure. Health Technol Assess. 2012;16(16):1–166.

    Article  CAS  PubMed  Google Scholar 

  22. Cleland J, Hutchinson C, Khadka J, Milte R, Ratcliffe J. What defines quality of care for older people in aged care? A comprehensive literature review. Geriatr Gerontol Int. 2021;21(9):765–78.

    Article  PubMed  Google Scholar 

  23. Gerlinger C, Bamber L, Leverkus F, Schwenke C, Haberland C, Schmidt G, Endrikat J. Comparing the EQ-5D-5L utility index based on value sets of different countries: impact on the interpretation of clinical study results. BMC Res Notes. 2019;12(1):18.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Norman R, Cronin P, Viney R. A pilot discrete choice experiment to explore preferences for EQ-5D-5L health states. Appl Health Econ Health Policy. 2013;11(3):287–98.

    Article  PubMed  Google Scholar 

  25. Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, Terwee CB. COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures. Qual Life Res. 2018;27(5):1171–9.

    Article  CAS  PubMed  Google Scholar 

  26. Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, Bouter LM, de Vet HCW, Mokkink LB. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27(5):1159–70.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Lung T, Howard K, Etherton-Beer C, Sim M, Lewin G, Arendts G. Comparison of the HUI3 and the EQ-5D-3L in a nursing home setting. PLoS ONE. 2017;12(2):e0172796.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Toh HJ, Yap P, Wee SL, Koh G, Luo N. Feasibility and validity of EQ-5D-5L proxy by nurses in measuring health-related quality of life of nursing home residents. Qual Life Res. 2021;30(3):713–20.

    Article  PubMed  Google Scholar 

  29. Abma IL, Rovers M, van der Wees PJ. Appraising convergent validity of patient-reported outcome measures in systematic reviews: constructing hypotheses and interpreting outcomes. BMC Res Notes. 2016;9:226.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, Bouter LA, de Vet HCW. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42.

    Article  PubMed  Google Scholar 

  31. Gravesande J, Richardson J, Griffith L, Scott F. Test-retest reliability, internal consistency, construct validity and factor structure of a falls risk perception questionnaire in older adults with type 2 diabetes mellitus: a prospective cohort study. Arch Physiother. 2019;9(1):14.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Zijlmans EAO, Tijmstra J, van der Ark LA, Sijtsma K. Item-Score Reliability as a Selection Tool in Test Construction. Front Psychol. 2019;9:2298.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Australian Bureau of Statistics. Socio-Economic Indexes of Areas (SEFIA) 2016 (https://www.abs.gov.au/websitedbs/censushome.nsf/home/seifa). Canberra. 2018

  34. Cleveland WS. Robust locally weighted regression and smoothing scatterplots. J Am Stat Assco. 1979;74(368):829–36.

    Article  Google Scholar 

  35. Dinno A. Nonparametric Pairwise Multiple Comparisons in Independent Groups using Dunn’s Test. Stata J. 2015;15(1):292–300.

    Article  Google Scholar 

  36. Aspden T, Bradshaw SA, Playford ED, Riazi A. Quality-of-life measures for use within care homes: a systematic review of their measurement properties. Age Ageing. 2014;43(5):596–603.

    Article  PubMed  Google Scholar 

  37. Malley J, D’Amico F, Fernandez JL. What is the relationship between the quality of care experience and quality of life outcomes? Some evidence from long-term home care in England. Soc Sci Med. 2019;243:112635.

    Article  PubMed  Google Scholar 

  38. Marten O, Brand L, Greiner W. Feasibility of the EQ-5D in the elderly population: a systematic review of the literature. Qual Life Res. 2022;31(6):1621–37.

    Article  PubMed  Google Scholar 

  39. Rand S, Towers AM, Razik K, Turnpenny A, Bradshaw J, Caiels J, Smith N. Feasibility, factor structure and construct validity of the easy-read Adult Social Care Outcomes Toolkit (ASCOT-ER)*. J Intellect Dev Disabil. 2020;45(2):119–32.

    Article  Google Scholar 

  40. Perez-Ros P, Martinez-Arnau FM. EQ-5D-3L for Assessing Quality of Life in Older Nursing Home Residents with Cognitive Impairment. Life (Basel). 2020;10(7):100.

    PubMed  Google Scholar 

  41. Schwab CGG, Dichter MN, Berwig M. Item distribution, internal consistency, and structural validity of the German version of the DEMQOL and DEMQOL-proxy. BMC Geriatr. 2018;18(1):247.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Carrozzino D, Patierno C, Guidi J, Berrocal Montiel C, Cao J, Charlson ME, Christensen KS, Concato J, De Las Cuevas C, de Leon J, Eory A, Fleck MP, Furukawa TA, Horwitz RI, Nierenberg AA, Rafanelli C, Wang H, Wise TN, Sonino N, Fava GA. Clinimetric Criteria for Patient-Reported Outcome Measures. Psychother Psychosom. 2021;90(4):222–32.

    Article  PubMed  Google Scholar 

  43. Fleck MP, Carrozzino D, Fava GA. The challenge of measurement in psychiatry: the lifetime accomplishments of Per Bech (1942–2018). Braz J Psychiatry. 2019;41(5):369–72.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the field teams and trained interviewers who conducted interviews and collated data. We would also like to thank all the residential aged care facilities and residents who participated in the study. 

Funding

This work is supported by an Australian Research Council Linkage Project (Grant Number LP170100664). Additional funding support is from our partner organisations ECH, Helping Hand, Uniting Age Well, Uniting ACT NSW and Presbyterian Aged Care.

Author information

Authors and Affiliations

  1. Health and Social Care Economics Group, Caring Future Institute, College of Nursing and Health Sciences, Flinders University, Sturt North, GPO Box 2100, Adelaide, South Australia, 5001, Australia

    J Khadka, C Hutchinson, R Milte, J Cleland & J Ratcliffe

  2. Registry of Senior Australians, South Australian Health and Medical Research Institute, Adelaide, South Australia, Australia

    J Khadka

  3. College of Nursing and Health Sciences, Flinders University, Adelaide, South Australia, Australia

    A Muller

  4. Uniting AgeWell, Melbourne, VIC, Australia

    N Bowes

Authors
  1. J Khadka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C Hutchinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R Milte
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J Cleland
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A Muller
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. N Bowes
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. J Ratcliffe
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JK: data analysis and interpretation, drafted the paper, critical revision of the article and final approval of the version to be submitted; CH: project manager, data collection, data interpretation, critical revision of the article and final approval of the version to be submitted; RM: data analysis and interpretation critical revision of the article and final approval of the version to be submitted; JC: data collection, data interpretation, critical revision of the article and final approval of the version to be submitted; AM: data analysis and interpretation, critical revision of the article and final approval of the version to be submitted; NB: data interpretation, critical revision of the article and final approval of the version to be submitted; JR: project lead, concept or design of the work, data analysis and interpretation critical revision of the article and final approval of the version to be submitted. The author(s) read and approved the final manuscript. 

Corresponding author

Correspondence to J Khadka.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Social and Behavioural Research Ethics Committee at Flinders University (Approval no: 5508). The research to develop the final descriptive system for the QOL-ACC has sought to be as inclusive as possible, with project representation from older people with mild to moderate cognitive impairment or dementia (where informed consent could be provided) as well as those with good cognition. All the participants provided written informed consent.

Consent for publication

Yes.

Competing interests

None declared.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Table 1. Distribution of the Quality of Life- Aged Care Consumers (QOL-ACC) index scores by self-reported global health and quality of life items. Figure 1. Histograms showing distribution of scores of Quality of Life-Aged Care Consumers (QOL-ACC, 1A), Adult Social Care Outcome Tools (ASCOT, 1B), Quality of Care-Aged Care Consumers (QCE-ACC, 1C), EQ VAS (1D) and EQ-5D-5L (1E).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khadka, J., Hutchinson, C., Milte, R. et al. Assessing feasibility, construct validity, and reliability of a new aged care-specific preference-based quality of life instrument: evidence from older Australians in residential aged care. Health Qual Life Outcomes 20, 159 (2022). https://doi.org/10.1186/s12955-022-02065-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12955-022-02065-y

Keywords

Health and Quality of Life Outcomes

ISSN: 1477-7525

Contact us