Skip to main content

Responsiveness, construct and criterion validity of the Personal Care-Participation Assessment and Resource Tool (PC-PART)

Abstract

Background

The Personal Care-Participation Assessment and Resource Tool (PC-PART) was designed to measure participation restrictions in activities of daily living required for community life. Rasch analysis has confirmed that the PC-PART contains two unidimensional scales providing interval-level measurement: the Self Care and Domestic Life scales. This study investigated validity and responsiveness of these PC-PART scales using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) approach.

Methods

Thirteen hypotheses about Self Care and Domestic Life scale scores were established prior to conducting the analyses. Data from a prospective randomized controlled trial of additional (weekend) inpatient rehabilitation in Melbourne, Australia, were used. The 996 participants had a mean (SD) age of 74 (13) years and were admitted with orthopaedic (n = 581), neurological (n = 203) or other disabling impairments (n = 212). Self Care and Domestic Life scores were compared to functional independence (FIM), comorbidity (Charlson Comorbidity Index), whether activities of daily living goals were met, and discharge destination.

Results

Low to moderate correlations between FIM and PC-PART scales’ scores supported hypotheses that the PC-PART measures a different construct from functional independence: Self Care r s -0.52(95 % CI -.46 to -.57) and Domestic Life r s -0.32(95 % CI -.25 to -.38). The scales had low to moderate discriminative ability for discharge destination, with the area under the curve for Self Care, 0.70 (95 % CI 0.62-0.78), and Domestic Life, 0.72 (95 % CI 0.64-0.80). The discharge to community living cut-off scores for Self Care: 5.50 (sensitivity .83, specificity .53) and Domestic Life: 7.50 (sensitivity .75, specificity .60), represented patients having no participation restrictions. Change scores from admission to discharge demonstrated larger effect sizes for the Self Care (1.67) and Domestic Life (1.50) scales than for the FIM (1.10), supporting hypotheses about responsiveness. Ten of the 13 hypotheses were supported.

Conclusions

This study provided evidence supporting construct validity, criterion validity and responsiveness of the PC-PART Self Care and Domestic Life scales for inpatient rehabilitation. Clinicians, managers and researchers who wish to measure the patterns and extent of people’s participation restrictions in activities of daily living and the associated burden of care, before and/or after intervention, can be somewhat confident about the PC-PART’s validity and responsiveness for this purpose.

Trial registration

Data used in this research were gathered during a registered randomized controlled trial: Australian and New Zealand Clinical Trials Registry ACTRN12609000973213.

Background

Rehabilitation aims to improve activity performance and address barriers to patients’ participation in their life situations [1–3]. Rehabilitation services assist patients to adapt to challenges they face in their daily life as a result of their impairments. Participation is a key outcome of rehabilitation programmes [2, 4].

The International Classification of Functioning, Disability and Health (ICF) [5] is a commonly used framework in rehabilitation that informs assessment and measurement of patients’ functioning and health outcomes [6, 7]. The functioning and disability aspect of the ICF framework provides three separate constructs (impairments, activities, participation). However, only two components are described: one for impairments, and one for activities and participation, combined [5]. Researchers have commented on the lack of clarity in the interpretation of, and operationalization of the activity and participation concepts within the ICF framework [3, 7–11]. In particular, there is lack of consensus on interpretation of the definition for, and measurement of, participation-related constructs [7, 8, 10]. It seems accepted that measures eliciting information about an individual’s ability, level of difficulty or level of dependence in performing tasks, without inclusion of the modifying effects of the environment in the instrument’s metric, measure activity limitations [2, 3, 7]. With respect to measurement of participation restrictions, one view is that measures eliciting information about performance of tasks in natural environments and that include influences of the environment on performance in the instrument’s metric, measure participation restrictions [2–4, 12, 13].

The Personal Care-Participation Assessment and Resource Tool (PC-PART) [14–16] was designed to measure the presence or absence of participation restrictions experienced by individuals in self care and domestic activities of daily living (ADL) required for community life. It systematically identifies unmet ADL needs which persist in individuals’ living environments despite their own efforts, use of assistive devices, and supports or assistance from others [14, 16]. The PC-PART provides one conceptual perspective on the measurement of participation restriction in self care and domestic life domains.

The PC-PART differs from commonly used ADL instruments, such as the FIM [17, 18] and the Barthel Index (BI) [19], in a fundamentally important way. The FIM and BI measure patients’ level of dependence in self care and mobility, rating their abilities and their need for assistance or adaptive equipment or both. They can be considered to measure activity limitations [2]. Such instruments are not able to capture what ADL will actually be accomplished. The PC-PART differs in that it measures both the need for assistance or equipment and whether any required assistance is available and is provided in the patients’ living environment. Such information is critical, for example, for discharge planning from inpatient settings [20–23] and for admission decisions in emergency departments [24, 25].

The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) is an internationally recognised framework, developed through international consensus of experts in the measurement of health status outcomes [26–28]. The COSMIN checklist provides a framework of criteria for rating the methodological quality of research investigating the reliability, validity, responsiveness and interpretability of health measurement instruments. [27, 29]. It can also guide the development of rigorous methods to investigate measurement properties of health related outcome measures [29].

A systematic review of the measurement properties of the PC-PART using the COSMIN checklist showed that PC-PART items demonstrated good content validity [16, 30]. Other aspects of the instrument’s validity could not be confirmed from the systematic review. Subsequent research has demonstrated that the PC-PART has good inter-rater reliability for group applications but not for individual applications, such as in the clinical setting [31]. Using Rasch methods, a further study generated evidence supporting internal validity of 30 of the original 43 items, when grouped into separate Self Care (16 items) and Domestic Life (14 items) scales [32]. The objective of this present study was to evaluate the construct validity, criterion validity and responsiveness of the Rasch-derived Self Care and Domestic Life scales in an adult inpatient rehabilitation setting.

Methods

Design

This is an instrument validation study guided by the COSMIN framework, involving secondary analysis of existing data from a prospective randomized controlled trial (RCT). The RCT investigated what effect providing additional Saturday rehabilitation during inpatient rehabilitation had on functional independence, quality of life and length of stay, compared to 5 days per week of rehabilitation [33–35].

Participants

Participants were the 996 adults enrolled in the trial, conducted in two public hospital multidisciplinary inpatient rehabilitation units in Melbourne, Australia. Participants with orthopaedic (e.g. fractures, elective joint replacements), neurological (e.g. stroke, multiple sclerosis, Parkinson’s disease) or other disabling condition (e.g. cardiac, pulmonary, deconditioning) were included. Patients were not excluded if their primary language was different from English or if they had reduced cognition, but were excluded if they were admitted for ‘Geriatric Evaluation and Management’ (otherwise known as slow stream rehabilitation) or if they were enrolled in another trial. Patients are typically accepted for inpatient rehabilitation if assessed as being able to participate actively in rehabilitation with the expectation they will improve sufficiently to return to community living [33]. Ethics approval for this study, involving secondary analysis of the RCT data, was received from University and Health Service Human Research Ethics Committees. Participants gave written informed consent to take part in the RCT.

Measures

Data from all measures used in the RCT were available for this study and these are detailed elsewhere [33]. Hypotheses for this present study were generated prior to all analyses with knowledge of the available measures used for the RCT. Only data from measures relevant to the hypotheses for this present study were used.

The PC-PART was administered to gather data for the RCT at admission (baseline) and again at discharge from inpatient rehabilitation. It was administered by occupational therapists using a combination of patient interview, key informant interview and task observation (see Appendix A: Table 8 for item examples). Prior to commencement of data collection the occupational therapist assessors were provided with standardized education in the use of the PC-PART. This consisted of a one-hour training session. In addition, the occupational therapists were provided the PC-PART manual [14] and a recorded audiovidual presentation [15].

The PC-PART assessment was administered to identify participants’ existing participation restrictions in ADL in their discharge living environments. Items were scored as either OK by self (patients will manage the activity alone with or without aids in their living environments), OK with help (patients will manage the activity with help from others, and this help is available and provided in their living environments), or Not OK (patients will not manage the activity in their living environments despite their own efforts, use of aids and help from available support from others). Both OK by self and OK with help were scored 0 (no participation restriction present), and Not OK was scored 1 (participation restriction was present). Each Not OK represented an ADL participation restriction. These item response categories were shown to be valid using Rasch analysis [32]. When used clinically, the raw score for each scale is the total number of Not OK scores observed for an individual patient, with a range of possible scores of 0–16 (Self Care) and 0–14 (Domestic Life). Rasch-derived conversion scores for each scale use a 0–100 scale, where 0 reflects no ADL participation restriction and 100 represents complete ADL participation restriction. Self Care and Domestic Life scale total scores cannot be combined to form an overall PC-PART score [32].

Rasch-derived scores for the 16 Self Care scale items and 14 Domestic Life scale items were used for all analyses in this present study [32]. To aid clinical interpretation where relevant, Rasch-derived scores were related back to corresponding total raw Self Care and Domestic Life scores using a conversion table [32].

The FIM [36] consists of 18 items from motor (13 items) and cognitive (5 items) domains. Each item is rated on a 7-point scale, where 1 represents complete dependence and the need for total assistance and 7 represents complete independence. Scores range from 18 (complete dependence on all items) to 126 (complete independence on all items) [37]. Items cover activities such as eating, grooming, bathing, dressing, toileting, sphincter control, , transfers, locomotion, communication and social cognition. There is evidence from studies conducted in the past two decades across several countries and different patient populations, supporting reliability, validity and responsiveness of the FIM as a measure of disability for patients receiving rehabilitation [38]. Thus, the FIM was viewed as a suitable comparison instrument for the PC-PART. It is a measure of activity limitations according to ICF concepts and terminology [5, 39]. It has sufficient similarity in the content of its domains to the PC-PART, to generate hypotheses reflecting expected convergence and divergence between their scores at admission and discharge from inpatient rehabilitation. The FIM was administered as part of routine care by FIM trained assessors, including physiotherapists and occupational therapists. It was scored during multidisciplinary team meetings at admission (baseline) and at discharge from inpatient rehabilitation. At both points, the FIM was completed on a separate occasion to the PC-PART.

The Charlson Comorbidity Index [40] was selected as the best available measure used in the RCT to enable testing of hypotheses about the PC-PART’s scores related to the level of patients’ co-morbidity. The sum of the index score, adjusted for age, is an indicator of disease burden and an estimator of mortality [40]. It provided a mechanism to quantify the severity of a patient’s overall state of ill-health, given the number and seriousness of health conditions experienced. The index has been widely used and validated in population studies [41], but it is recognized that some conditions (e.g. rheumatological disease) are less accurately coded [42]. The score was calculated at admission.

ADL discharge goals were established by the patients and treating occupational therapists at admission. This information was gathered for the RCT but not with the structured approach of goal attainment scaling. Achievement of ADL goals was measured and recorded at discharge from inpatient rehabilitation by the treating occupational therapist as being either met/unmet. Partially met goals were categorized as unmet.

Patients’ discharge destinations, that is, patients’ living situations immediately following discharge from inpatient rehabilitation were categorised as home (usual place of residence), low-level residential care, high-level residential care, acute hospital transfer, or transitional care. The transitional care program involved continued inpatient care for either lower intensity rehabilitation activities or to await placement in residential care facilities.

Analysis

COSMIN checklist

The COSMIN checklist provided criteria for evaluating construct validity, criterion validity and responsiveness. In this study all design criteria were addressed [29, 43].

Construct validity

COSMIN stipulates that construct validity is the degree to which the scores of health related outcome instruments are consistent with hypotheses formulated prior to data analysis, based on the assumption that the instrument measures the construct of interest [27]. In accordance with COSMIN’s recommendations, construct validity was evaluated by five hypotheses of expected mean score differences between impairment groups, and expected correlations between PC-PART scores and FIM and Charlson Comorbidity Index scores [29]. The hypotheses and statistical test criteria used are presented in Table 1.

Table 1 Methods: Hypotheses, criteria and rationale used to test construct validity, criterion validity and responsiveness

Criterion Validity

COSMIN stipulates that criterion validity is the degree to which the scores of a health related patient reported outcomes instrument are an adequate reflection of a suitable gold standard [27]. In this case, the objectively observable dichotomous gold standard outcome was discharge destination (community living at home or in residential care versus inpatient acute or transitional care), reflecting an overall aim of rehabilitation to prepare patients for community living. Criterion validity was tested using three hypotheses, in accordance with COSMIN recommendedations. The hypotheses are presented in Table 1.

Receiver-Operator Characteristic curve data were used to estimate cut-off scores at discharge for the Self Care and Domestic Life scales that may discriminate patients discharged home or to residential care from those transferred to acute hospital or transitional care. Consideration was given to balancing sensitivity and specificity of the scales’ scores.

Responsiveness

COSMIN stipulates that responsiveness is the ability of an instrument to detect change over time when change has occurred [27]. In accordance with COSMIN’s recommendations, responsiveness was evaluated with five hypotheses about the relationship between change scores on the PC-PART and FIM and predicted magnitudes of effect sizes of each measure between admission and discharge (see Table 1).

Data analysis

Data were analyzed using IBM SPSS Statistics (Version 21.0.0) software. Missing study data were removed from analyses using pairwise methods in all analyses. According to the COSMIN rating scale [46], a sample size for testing measurement properties of n ≥ 100, is considered excellent; from n = 50-99 is good; from n = 30-49 is fair; and n < 30 is poor. It was expected that sample sizes, per analysis, using the RCT data (n = 996) would be excellent for evaluating construct validity, criterion validity and responsiveness of the Rasch-derived Self Care and Domestic Life scales.

Results

The 996 participants had a mean (SD) age of 74 (13) years, and 631 (63 %) were women (see Table 2). There were 581 (58 %) participants admitted with an orthopaedic diagnosis, 203 (20 %) with a neurological diagnosis and 212 (21 %) with other disabling impairments. Mean (SD) length of stay in the rehabilitation unit was 21 (16) nights. Most participants (93 %) were living at home prior to their acute hospital admission. Of the 7 % of participants not living at home prior to admission, 2 % (n = 27) lived in low-level residential care (LLC), 2 % (n = 23) lived in ‘other’ accommodation, and 2 % (n = 19) had missing data for this variable. Participants from LLC or ‘other’ accommodation (n = 50) showed average (median) improvement of 18 points on the FIM from admission to discharge.

Table 2 Participant characteristics and study data

Approximately 10 % of discharge PC-PART data for both Self Care and Domestic Life scales were missing (see Table 2). There were a number of these patients for whom most discharge PC-PART individual item data were available, but for whom Self Care scores (n = 64) and Domestic Life scores (n = 59) could not be calculated because there was between one and three missing values for individual items in the scale. To use a Rasch-derived scale and its associated conversion table, all items in the scale need to be completed to produce a valid score. There were 34 patients (3 % of the sample) with no discharge PC-PART data. Patients with no discharge PC-PART data (n = 34) had similar mean age (74 yrs, SD = 15, 95 % CI 67–81), length of stay (20 nights, SD = 21, 95 % CI 16–41), admission Self Care scale scores (Mean = 48.0, SD = 25.1, 95 % CI 40.7-63.4) and Domestic Life scale scores (Mean = 38.4, SD = 19.5, 95 % CI 29.3-50.2) compared to the rest of the sample. However, their admission FIM scores (median = 72) and discharge FIM scores (median = 71) were lower compared to the rest of the sample. A higher proportion of patients with no discharge PC-PART data were discharged to acute care (33 %), compared to 2 % for the whole sample.

Construct validity

Hypothesis 1

Table 3 shows that at admission, the large negative correlation between Self Care scores and FIM total scores, r s  = −.52 (95 % CI -.46 to-.57), and a moderate negative correlation between Domestic Life scores and FIM total scores, r s  = −.32 (95 % CI -.25 to-.38), supported the hypothesis about the magnitude and direction of these correlations. However, 95 % confidence intervals of the estimates included lower correlations than expected.

Table 3 Hypotheses 1 and 2 (construct validity): correlations between PC-PART scales and FIM at admission to inpatient rehabilitation

Hypothesis 2

Correlations by sex, age, and impairment between PC-PART scales and FIM generated 16 correlation values. The magnitude of 10 correlation values were as hypothesized, but six correlation values were lower than expected for both PC-PART scales (participants aged ≥80 years; those with orthopaedic or other disabling impairments) (see Table 3). Fifteen of the 16 lower bound 95 % confidence interval estimates were lower than predicted. Two upper bound 95 % confidence interval estimates for Domestic Life and FIM were higher than expected (participants aged ≤59 years; those with neurological impairment).

Hypothesis 3

There was a negligible (< .1) to small (.10 to.29) positive correlation between admission Self Care, r s  = .10 (95 % CI .04-.16), and Domestic Life, r s  = .04 (95 % CI .02-.10) scores and Charlson Comorbidity Index scores, suggesting a negligible relationship between the PC-PART scales and degree of comorbidity. This result did not support the hypothesis of a moderate positive correlation between the variables. Post hoc analysis showed that 75 % of participants’ comorbidity scores were ≤2 and 50 % of scores were ≤1, showing relatively low variation in scores across the sample.

Hypothesis 4

The hypothesis of no difference in Self Care and Domestic Life scale mean scores across impairment groups was not supported. The mean scores and 95 % confidence intervals from the group of patients with stroke [Self Care 56.5(95 % CI 52.5-60.5); Domestic Life 50.1 (95 % CI 46.4-53.8)] demonstrated higher admission scores on both PC-PART scales than patients in other impairment groups, with the closest group being patients with other neurological conditions [Self Care 43.0 (95 % CI 34.3-51.7); Domestic Life 39.3 (95 % CI 32.2-46.3)] (see Fig. 1).

Fig. 1
figure 1

Hypothesis 4 (construct validity): PC-PART Self Care and Domestic Life scores at admission for impairment groups, displaying mean and 95 % confidence interval for each group. (see separate file)

Hypothesis 5

The mean difference in PC-PART scores between patients who attained their ADL goals and patients who did not, was 9.3 (95 % CI 6.6-12.1) for Self Care and 12.2 (95 % CI 9.0-15.4) for Domestic Life ( see Table 4). As hypothesized, these values represented a clinically relevant difference in raw scores of at least one ADL participation restriction between groups on each scale.

Table 4 Hypothesis 5 (construct validity): PC-PART scores and 95 % CIs at discharge for variable ‘ADL goal met?’

Criterion validity

Hypothesis 6

Both Self Care and Domestic Life scale scores demonstrated low to moderate probability of correctly differentiating between patients discharged home or residential care (n = 815) versus patients discharged to acute hospital or transitional care (n = 86). The estimated area under the curve for the Domestic Life scale was .72 (95 % CI: .64-.80) and for the Self Care scale, was .70 (95 % CI: .62-.78). This result was modest, but supported the hypothesis of an area under the curve greater than .50, representing discriminative ability greater than chance (see Fig. 2).

Fig. 2
figure 2

Criterion validity: Area under the ROC curve (AUC) for the discharge PC-PART Self Care and Domestic Life scores, discriminating between patients discharged to home or residential care (community living) and patients discharged from inpatient rehabilitation to continued inpatient care. (see separate file)

Hypothesis 7

The hypothesis that those discharged home or to residential care would have Self Care and Domestic Life discharge scores representing less than three ADL participation restrictions, was supported (see Table 5). Those discharged to community living (home, low level-, high level residential care) had discharge mean Self Care scores of 2.7 (95 % CI 2.2-3.3), and Domestic Life scores of 6.2 (95 % CI 5.3-7.0), representing raw scores of no ADL participation restrictions on each scale.

Table 5 Hypotheses 7 and 8 (criterion validity): PC-PART Rasch-derived scores, raw scores and 95 % CIs at discharge, by discharge destination

Hypothesis 8

The hypothesis that those discharged to acute or transitional care would have Self Care and Domestic Life discharge mean scores representing three or more ADL participation restrictions, was partly supported. The 95 % confidence intervals included scores representing three ADL participation restrictions, but also included the possibility of two or one ADL participation restrictions. Those discharged to acute hospital or transitional care had discharge mean Self Care scores of 18.4 (95 % CI 11.5-25.3), and Domestic Life scores of 27.5 (95 % CI 20.1-34.8), representing raw scores of one to three ADL participation restrictions on each scale (see Table 5). Post-hoc analysis for this combined group showed that 13 of the 17 patients discharged to acute care had no discharge PC-PART data. The other four patients discharged to acute care for whom discharge PC-PART data were available, had at least 14 Self Care participation restrictions and 12 Domestic Life participation restrictions at discharge. Of the 69 patients discharged to transitional care, 30 (44 %) had no Self Care participation restrictions and 26 (38 %) had no Domestic Life participation restrictions.

Cut-off scores

Table 6 shows potential cut-off scores for each scale at several levels of sensitivity to correctly identify patients discharged to home or to residential care. Corresponding levels of specificity for scores to correctly identify patients discharged to acute hospital or transitional care are provided. Cut-off scores of zero on both PC-PART scales reflected optimal sensitivity but specificity values were relatively low : Self Care 5.50 (Sensitivity, .83, Specificity, .53), and Domestic Life 7.50 (Sensitivity, .75, Specificity, .60).

Table 6 Criterion validity: Discharge Self Care and Domestic Life scale ROC cut-off scores and their corresponding sensitivity/specificity in identifying discharge destination

Responsiveness

Hypothesis 9

As hypothesized, there was a low to moderate negative correlation (r s  ≤ −.49) across the sample, between FIM change scores and Self Care change scores, r s  = −.40 (95 % CI -.34 to -.45), and Domestic Life change scores, r s  = −.22 (95 % CI -.16 to-.30) (see Table 7).

Table 7 Hypotheses 9 and 10 (responsiveness): correlations between PC-PART scales’ change scores and FIM change scores, between admission and discharge

Hypothesis 10

Fifteen of 16 change score correlation values between the FIM and Self Care and Domestic Life scales by sex, age and major impairment groups were ≤ .49, as hypothesized (see Table 7). Participants aged ≤59 years had a value greater than .49 on the Self Care scale (r s  = .56), but the lower bound 95 % CI was lower than .49, (r s  = .41). Five upper bound 95 % confidence interval estimates were higher than .49: for Self Care (men, participants with neurological and other impairments); and for both Self Care and Domestic Life (participants aged ≤59 years).

Hypothesis 11

As hypothesized, there was a large effect size for both PC PART scales and FIM between admission and discharge with the FIM demonstrating the smallest effect size: Self Care scale (ES = 1.71; 95 % CI 1.60-1.82); Domestic Life scale (ES = 1.51; 95 % CI 1.40-1.61) and FIM (ES = 1.10; 95 % CI 1.01-1.20).

Hypothesis 12

Patients discharged home or to residential care had a large reduction (improvement) in mean PC-PART scores from admission to discharge on the Self Care scale (from 40.8 to 2.7; n = 810) and the Domestic Life scale (from 37.3 to 6.2; n = 814). These scores represented an average improvement from six Self Care participation restrictions at admission to none at discharge (ES = 1.73, 95 % CI 1.62-1.85), and from three Domestic Life participation restrictions at admission to none at discharge (ES = 1.56, 95 % CI 1.45-1.67). As hypothesized, both observed effect sizes were > 0.8.

Hypothesis 13

Patients discharged to acute hospital or transitional care had a large reduction in mean PC-PART scores from admission to discharge on the Self Care scale (from 52.4 to 18.4; n = 63) and the Domestic Life scale (from 53 to 27.5; n = 63). These scores represented an average reduction of nine Self Care participation restrictions at admission to two at discharge (ES = 1.54, 95 % CI 1.13-1.93), and from nine Domestic Life participation restrictions at admission to three at discharge (ES = 1.22, 95 % CI 0.83-1.59). Both effect sizes and their 95 % confidence intervalswere > 0.8. These results did not support the hypothesis of an effect size < .8 in this group.

COSMIN summary

Overall, for both Self Care and Domestic Life PC-PART scales, the number of hypotheses supported were: 3 of 5 for construct validity; 3 of 3 for criterion validity; and 4 of 5 for responsiveness. Overall 6 of 8 hypotheses about validity and 4 of 5 hypotheses about responsiveness were supported. Sample sizes for all analyses were good to excellent.

Discussion

This study evaluated construct validity, criterion validity and responsiveness of the PC-PART Self Care and Domestic Life scales for inpatient rehabilitation using the COSMIN framework. Overall, there was support for 10 of the 13 hypotheses.

Given that both the PC-PART and the FIM have provided evidence of reliability and validity, the lack of a strong negative correlation between the measures at admission could be interpreted as suggesting that the PC-PART measures a different construct to FIM. The FIM measures activity limitations [37]. The PC-PART scales performed in accordance with theoretical expectations, supporting construct validity of the PC-PART’s Self Care and Domestic Life scales as measures of ADL participation restriction.

To our knowledge, the PC-PART is the only instrument that specifically targets the transaction between people, their activity and the available supports in their living environments to record participation restrictions in activities of daily living required for community life. Other instruments seem similar, for example, the Assessment of Living Skills and Resources-Revised 2 (ALSAR-R2) [47]; Assessment of Life Habits (LIFE-H) [48]; Craig Handicap Assessment and Reporting Technique (CHART) [49]; and the Functional Autonomy Measurement System (SMAF) [50]. However, these assesments have applications in different areas of functioning than the PC-PART (e.g. performance in education, work, leisure tasks and body functions) and vary in the degree and manner in which they incorporate the need for, and availability of, supports, resources or assistance into their scoring [47–50]. The PC-PART therefore provides an important and unique contribution to health state measurement through its measurement of participation restrictions.

The relationship between comorbidity and PC-PART scores needs further investigation. Contrary to our expectations, the number and severity of comorbidities did not influence PC-PART scores (hypothesis 3). Overall, patients in this sample had relatively low comorbidity scores. Lack of variability in comorbidity scores may have affected the estimate of the correlation coefficient. It is possible that the Charlson Comorbidity Index was not sensitive to subclinical and chronic impairments that may impact people’s functioning (e.g. chronic pain or rheumatological conditions) [42]. Further evidence using prospective methods gathering more detailed information about comorbidity may add to our understanding about the measurement of participation restriction as related to the number and severity of co-existing impairments.

Admission PC-PART scores were shown to be higher for patients with stroke, compared to patients from other impairment groups, showing a lack of support for hypothesis 4, which postulated no difference between impairment groups. It may be that the sudden nature of stroke onset and combination of physical and cognitive impairments associated with stroke results in more participation restrictions in the accompishment of ADL than for people with other impairments. This result suggests that the PC-PART may be sensitive to impairment type, however this premise requires testing in a specifically designed study. If PC-PART scores are shown to differ between impairment groups, then it is possible the PC-PART may be useful for identifying groups of patients who are likely to require interventions focused on accomplishment of ADL required for community living as part of discharge planning.

The modest probability of both PC-PART scales’ scores ability to accurately reflect patients’ discharge destination shown in this study (hypothesis 6), seems likely to be an underestimation of their true discriminative ability. This result seems to have been influenced by the high proportion of missing PC-PART discharge data for patients discharged to acute inpatient care, as well as, a high proportion of patients with resolved participation restrictions in the transitional care group’s PC-PART discharge data [45]. The acute hospital and transitional care group discharge PC-PART scores were probably not representative of the group they were intended to represent, that is, patients transferred to acute care due to ongoing problems requiring medical management. On reflection, separation of patients discharged to acute hospital and transitional care into separate groups may have provided more robust validation data. Thus, these are preliminary findings. Prospective and specifically designed investigations of the PC-PART’s discriminative ability are required to produce more robust evidence about the ability of the Self Care and Domestic Life scale scores to accurately identify people who can return to community life from inpatient rehabilitation and those who continue to require inpatient services.

Both PC-PART scales appeared responsive. Their scores were shown to change in the direction expected when change had occurred, as indicated by other variables and instruments. Both scales demonstrated large effect sizes from rehabilitation admission to discharge. The correlation between change scores reflected a greater relative improvement in PC-PART scores than FIM scores between admission and discharge. This finding is consistent with theoretical expectations about PC-PART scores; that there should be a complete resolution of ADL participation restrictions prior to discharge home or to residential care (in residential care, the expectation is that all ADL needs are met). In contrast, it is possible for patients to be discharged to the community without complete independence scores on every FIM item, that is, without complete resolution of activity limitations, provided adequate supports are in place.

In this study, all patients’ Self Care and Domestic Life scores between admission and discharge showed large effect sizes, irrespective of discharge destination. For patients discharged to home or supported living environments, the large effect size of PC-PART scores between admission and discharge supported its responsiveness. It is possible that the patients discharged to transitional care (n = 69), who had no ADL participation restrictions at discharge because they were waiting for residential care placement, may have inflated the effect sizes for the acute and transitional inpatient care discharge group. Also, missing discharge PC-PART data for 13 of 17 patients transferred to acute care may have influenced the results by under-representing this group in the data. Thus, caution is advised when interpreting findings for hypothesis 13 due to limitations of the data as well as potential bias introduced during analyses. Responsiveness of the PC-PART scales should be further investigated in prospective, specifically designed studies.

One of the challenging decisions in validation research is whether to test hypotheses with the aid of structured guidelines, such as the COSMIN checklist, or whether to use more exploratory approaches. Formulating hypotheses prior to data analysis reduces the risk of bias when interpreting the results because criteria for validity are set before viewing the data. This avoids the temptation to think of alternative explanations for low correlations or no difference between groups, instead of concluding that the instrument may not be valid [27]. Limited existing validation research for the PC-PART influenced development of accurate hypotheses for this present study. The hypotheses used in this study were generated from some testing of the instrument [16, 32, 51–53], clinical knowledge and experience, combined with theoretical expectations of the instrument. The use of a more exploratory approach may have been useful for generating hypotheses for future testing, but would not have permitted the testing of evidence carried out in this study. Overall, the results of this study are positive, with the majority of hypotheses supported.

The COSMIN checklist provided a transparent, rigorous methodological structure for this research that assisted in minimizing methodological bias. It would be useful to use the COSMIN checklist to further evaluate the PC-PART scales in prospective, specifically designed research to build more evidence about the scales’ validity and responsiveness.

In clinical practice the PC-PART may aid discharge planning. The derived cut-off scores of zero Self Care and zero Domestic Life participation restrictions, desirable for discharge home or to residential care living situations, intuitively match clinical reasoning. The scales may be used to identify and prioritise areas for intervention and to ensure that patients who are discharged to community living environments do not have ADL participation restrictions at the time of discharge.

The PC-PART scales may be useful for clinical practice, clinical research and health care system management. In clinical practice, they may identify the presence of participation restrictions in ADL required for community life, enabling prioritisation of intervention and discharge planning. This may facilitate the reduction of barriers to discharge from inpatient care, which include issues of accommodation and supply of appropriate supports in community living environments [22]. In clinical research, changes that occur through interventions designed to reduce ADL participation restrictions, and their economic value, can be measured using the PC-PART scales. For the health care system, the PC-PART scales may be used to identify the nature and extent of participation restrictions experienced by populations in activities of daily living required for community life. This may aid understanding of the nature and extent of supports needed to enable people to live in the community and in turn, enable resources to be allocated where they are most needed.

Limitations

The retrospective use of existing data limited the scope of analysis to the type and nature of the existing variables, which were collected for a different purpose. Use of existing data also meant that specific methodological requirements for some analyses (criterion validity) were not favorable. The combined grouping of patients discharged to acute and transitional care may have resulted in an underestimation of the discriminative ability and responsiveness of the PC-PART scales. Therefore, the results of this study need to be interpreted in light of its limitations. Prospective studies could ensure more detailed, useful and specific data for comparison with PC-PART scores are gathered. Finally, testing of PC-PART scores in relation to assessments such as the ALSAR-R2, SMAF or LIFE-H, which all focus on accomplishment of some ADL as well as broader life activities, may provide opportunity for further validation research.

Conclusions

Overall, results of this rigorous validation study using the COSMIN checklist support the construct validity and criterion validity of the PC-PART’s Self Care and Domestic Life scales for inpatient rehabilitation and show they are responsive to clinical changes, as measures of ADL participation restrictions in activities of daily living required for community life. Evidence from this study adds to existing research establishing the PC-PART scales as unidimensional interval-level measures of participation restriction. Health service clinicians, managers and researchers may confidently use the PC-PART scales to measure the pattern and extent of people’s participation restrictions in activities of daily living required for community life, to gain an understanding of the burden of care associated with these needs and to aid resource allocation of services.

Abbreviations

ADL:

Activities of daily living

AUC:

Area under the curve

CCI:

Charlson comorbidity Index

COSMIN:

COnsensus-based Standards for the selection of health Measurement INstruments

ES:

Effect size

FIM:

Functional Independence Measure

HLC:

High level care

ICF:

International classification of functioning, disability and health

LLC:

Low level care

PC-PART:

Personal Care Participation Assessment and Resource Tool

ROC:

Receiver-operator characteristic curve

TCP:

Transitional care program

References

  1. Bilbao A, Kennedy C, Chatterji S, Ustün B, Barquero JLV, Barth JT. The ICF: applications of the WHO model of functioning, disability and health to brain injury rehabilitation. NeuroRehabilitation. 2003;18:239–50.

    PubMed  Google Scholar 

  2. Desrosiers J. Muriel Driver Memorial Lecture. Participation and occupation. Can J Occup Ther. 2005;72:195–204.

    Article  PubMed  Google Scholar 

  3. Dijkers MP. Issues in the conceptualization and measurement of participation: an overview. Arch Phys Med Rehabil. 2010;91:S5–S16.

    Article  PubMed  Google Scholar 

  4. Madden R, Fortune N, Cheeseman D, Mpofu E, Bundy A. Fundamental questions before recording or measuring functioning and disability. Disabil Rehabil. 2013;35:1092–6.

    Article  PubMed  Google Scholar 

  5. World Health Organisation. The International Classification of Functioning Disability and Health (ICF). Geneva: WHO; 2001.

    Google Scholar 

  6. Jelsma J. Use of the International Classification of Functioning, Disability and Health: a literature survey. J Rehabil Med. 2009;41:1–12.

    Article  PubMed  Google Scholar 

  7. Whiteneck G, Dijkers MP. Difficult to measure constructs: conceptual and methodological issues concerning participation and environmental factors. Arch Phys Med Rehabil. 2009;90:S22–35.

    Article  PubMed  Google Scholar 

  8. Heinemann AW, Tulsky D, Dijkers M, Brown M, Magasi S, Gordon W, et al. Issues in participation measurement in research and clinical applications. Arch Phys Med Rehabil. 2010;91:S72–6.

    Article  PubMed  Google Scholar 

  9. Badley EM. Enhancing the conceptual clarity of the activity and participation components of the International Classification of Functioning, Disability, and Health. Soc Sci Med. 2008;66:2335–45.

    Article  PubMed  Google Scholar 

  10. Coster W, Law M, Bedell GM, Khetani M, Cousins M, Teplicky R. Development of the participation and environment measure for children and youth: conceptual basis. Disability & Rehabilitation. 2012;34:238–46.

    Article  Google Scholar 

  11. Magasi S, Post MW. A comparative review of contemporary participation measures’ psychometric properties and content coverage. Arch Phys Med Rehabil. 2010;91:S17–28.

    Article  PubMed  Google Scholar 

  12. Mallinson T, Hammel J. Measurement of participation: intersecting person, task, and environment. Arch Phys Med Rehabil. 2010;91:S29–33.

    Article  PubMed  Google Scholar 

  13. Noreau L, Boschen K. Intersection of participation and environmental factors: a complex interactive process. Arch Phys Med Rehabil. 2010;91:S44–53.

    Article  PubMed  Google Scholar 

  14. Darzins P. PC-PART: Personal Care Participation Assessment and Resource Tool user’s manual. Melbourne: PART Group; 2004. Partgroup@bigpond.com.

    Google Scholar 

  15. Francis J. How to use the PC-PART to guide in-patient rehabilitation. Australia: The PART Group; 2004. p. 23. PARTgroup@bigpond.com.

    Google Scholar 

  16. Vertesi A, Darzins P, Lowe S, McEvoy E, Edwards M. Development of the Handicap Assessment and Resource Tool (HART). Can J Occup Ther. 2000;67:120–7.

    Article  CAS  PubMed  Google Scholar 

  17. Granger CV, Hamilton BB. UDS report. The Uniform Data System for Medical Rehabilitation Report of First Admissions for 1990. Am J Phys Med Rehabil. 1992;71:108–13.

    Article  CAS  PubMed  Google Scholar 

  18. Uniform Data Systems for Medical Rehabilitation. Adult FIM Workshop: Participant Manual Version 5.3 (Australia). Buffalo (NY): State University of New York at Buffalo; 2014.

    Google Scholar 

  19. Mahoney F, Barthel D. Functional evaluation: the Barthel Index. Md State Med J. 1965;14:62.

    Google Scholar 

  20. Moats G. Discharge decision-making, enabling occupations, and client-centred practice. Can J Occup Ther. 2007;74:91–101.

    Article  PubMed  Google Scholar 

  21. Moreland JD, Depaul VG, Dehueck AL, Pagliuso SA, Yip DWC, Pollock BJ, et al. Needs assessment of individuals with stroke after discharge from hospital stratified by acute Functional Independence Measure score. Disabil Rehabil. 2009;31:2185–95.

    Article  PubMed  Google Scholar 

  22. New P, Cameron PA, Olver JH, Stoelwinder JU. Defining barriers to discharge from inpatient rehabilitation, classifying their causes, and proposed performance indicators for rehabilitation patient flow. Archives of Physical Medicine & Rehabilitation. 2013;94:201–8.

    Article  Google Scholar 

  23. Shepperd S, McClaran J, Phillips C, Lannin N, Clemson L, McCluskey A, Cameron ID, Barras S: Discharge planning from hospital to home. Cochrane Database of Systematic Reviews 2010:Issue 1. doi:10.1002/14651858.CD14000313.pub14651853.

  24. Moss JE, Flower CL, Houghton LM, Moss DL, Nielsen DA, Taylor DM. A multidisciplinary Care Coordination Team improves emergency department discharge planning practice. Med J Aust. 2002;177:435–9.

    PubMed  Google Scholar 

  25. Veillette N, Demers L, Dutil E, McCusker J. Development of a functional status assessment of seniors visiting emergency department. Arch Gerontol Geriatr. 2009;48:205–12.

    Article  PubMed  Google Scholar 

  26. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments. BMC Med Res Methodol. 2006;6:2.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  27. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63:737–45.

    Article  PubMed  Google Scholar 

  28. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19:539–49.

    Article  PubMed Central  PubMed  Google Scholar 

  29. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010;10:22.

    Article  PubMed Central  PubMed  Google Scholar 

  30. Darzins S, Imms C, Di Stefano M. Measurement properties of the personal care participation assessment and resource tool: a systematic review. Disabil Rehabil. 2013;35:265–81.

    Article  PubMed  Google Scholar 

  31. Radia-George C, Imms C, Taylor N. The inter-rater reliability and clinical utility of the Personal Care Participation Assessment and Resource Tool (PC-PART) in an inpatient rehabilitation setting. Am J Occup Ther. 2014;68:334–43.

    Article  PubMed  Google Scholar 

  32. Darzins S, Imms C, Di Stefano M, Taylor N, Pallant J. Evaluation of the internal construct validity of the Personal Care Participation Assessment and Resource Tool (PC-PART) using Rasch analysis. BMC Health Serv Res. 2014;14:543.

    Article  PubMed Central  PubMed  Google Scholar 

  33. Taylor NF, Brusco NK, Watts JJ, Shields N, Peiris C, Sullivan N, et al. A study protocol of a randomised controlled trial incorporating a health economic analysis to investigate if additional allied health services for rehabilitation reduce length of stay without compromising patient outcomes. BMC Health Serv Res. 2010;10:308.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  34. Peiris C, Shields N, Brusco NK, Watts JJ, Taylor NF. Additional Saturday rehabilitation improves functional independence and quality of life and reduces length of stay: a randomized controlled trial. BMC Med. 2013;11:198.

    Article  PubMed Central  PubMed  Google Scholar 

  35. Peiris C, Shields N, Brusco NK, Watts JJ, Taylor NF. Correction: Additional Saturday rehabilitation improves functional independence and quality of life and reduces length of stay: a randomized controlled trial. BMC Med. 2013;11:262.

    Article  PubMed Central  Google Scholar 

  36. Hamilton BB, Granger CV. Disability outcomes following inpatient rehabilitation for stroke. Phys Ther. 1994;74:484–503.

    Google Scholar 

  37. Uniform Data Systems for Medical Rehabilitation. Adult FIM Workshop: Participant Manual Version 5.1 (Australia). Buffalo (NY): State University of New York at Buffalo; 2008.

    Google Scholar 

  38. Heinemann AW, Ehrlich-Jones L, Moore J: Rehabilitation Measures Database, Center for Rehabilitation Outcomes Research, Northwestern University Feinberg School of Medicine, Department of Medical Social Sciences Informatics group, Rehabilitation Institute of Chicago. vol. 2014; 2013.

  39. UDSMR Adult FIM Workshop - Participant Manual, Version 5.2 (Australia). Buffalo, NY14214: State University of New York at Buffalo; 2012.

  40. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–83.

    Article  CAS  PubMed  Google Scholar 

  41. Sharabiani M, Aylin P, Bottle A. Systematic review of comorbidity indices for administrative data. Med Care. 2012;50:1109–18.

    Article  PubMed  Google Scholar 

  42. Quan H, Li B, Couris C, Fushimi K, Graham P, Hider P, et al. Updating and Validating the Charlson Comorbidity Index and Score for Risk Adjustment in Hospital Discharge Abstracts Using Data From 6 Countries. Am J Epidemiol. 2011;173:676–82.

    Article  PubMed  Google Scholar 

  43. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. COSMIN checklist manual. Amsterdam: Executive Board of VU University Amsterdam; 2010.

    Google Scholar 

  44. Cohen JW. Statistical power analysisfor the behavioural Sciences, 2nd edn, Hillsdale, NJ: Lawrence Erlbum Associates;1988.

  45. Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. New York: Oxford University Press; 2015.

    Google Scholar 

  46. Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21:651–7.

    Article  PubMed Central  PubMed  Google Scholar 

  47. Clemson L, Bundy A, Unsworth C, Singh MF. Validation of the modified assessment of living skills and resources, an IADL measure for older people. Disability & Rehabilitation. 2009;31:359–69.

    Article  Google Scholar 

  48. Fougeyrollas P, Noreau L, Bergeron H, Cloutier R, Dion SA, St-Michel G. Social consequences of long term impairments and disabilities: conceptual approach and assessment of handicap. Int J Rehabil Res. 1998;21:127–41.

    Article  CAS  PubMed  Google Scholar 

  49. Whiteneck GG, Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil. 1992;73:519–26.

    CAS  PubMed  Google Scholar 

  50. Hébert R, Carrier R, Bilodeau A. The Functional Autonomy Measurement System (SMAF): description and validation of an instrument for the measurement of handicaps. Age Ageing. 1988;17:293–302.

    Article  PubMed  Google Scholar 

  51. Outcome Measures in Rehabilitation Project Report [http://www.health.vic.gov.au/subacute/outcome_phase1.pdf]

  52. Outcome measures in rehabilitation: Phase 2 Project [http://www.health.vic.gov.au/subacute/outcomefinal.pdf]

  53. Darzins P, McEvoy E, Lowe S, Vertesi A, Brock C, Edwards M, et al. Reliability and Validity of the Handicap Assessment and Resource Tool (HART). Quebec, Canada: The Canadian Association on Gerontology (CAG) Conference; 1996.

    Google Scholar 

Download references

Acknowledgements

SD received financial assistance to complete this research from an Australian Post Graduate Award Scholarship and from a La Trobe University Post Graduate Grant. NT and NS received funding to conduct the randomised controlled trial from the National Health and Medical Research Council of Australia (NH&MRC). Funding bodies had no role in the design, collection, analysis and interpretation of data; in the writing of the manuscript; nor in the decision to submit the manuscript for publication.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Susan W. Darzins.

Additional information

Competing interests

Susan Darzins is a Director of Darzins Consulting Pty. Ltd., which operates using the business name ‘The PART Group’. The PART Group distributes the PC-PART users’ manual, assessment worksheets and training DVDs. Darzins Consulting Pty Ltd is not financing this manuscript nor had any role in approving the final manuscript. During the last five years (January 2009–2014) Susan Darzins has not received reimbursements, fees, funding or salary associated with sales of PC-PART products, from Darzins Consulting Pty. Ltd. It is possible, but unknown to what extent, Susan Darzins could benefit financially in the future from the publication of this manuscript. Susan Darzins does not hold and is not applying for any patents relating to the contents of this manuscript. All other authors declare that they have no competing interests.

Authors’ contributions

SD conceived and designed the study and study methods, completed the data entry, performed all statistical analyses, drafted and revised the manuscript and formatted the manuscript for submission. CI assisted in design of the study and study methods, provided consultation and supervision during data analyses and participated in revision of the manuscript. NS assisted in design of the study and study methods, provided advice on data analyses and contributed to the revision of the manuscript. NT provided access to the study data, assisted in design of the study and study methods, provided advice on data analyses and contributed to revisions of the manuscript. All authors read and approved the final manuscript.

Authors information

SD is a PhD candidate at the School of Allied Health, Australian Catholic University. CI is Professor and Head of the School of Allied Health, Australian Catholic University. Both SD and CI are qualified occupational therapists, registered to practice in Australia. NT is Professor of Allied Health at Eastern Health/La Trobe University. NS is Professor of Clinical and Community Practice, Northern Health/ La Trobe University. Both NT and NS are registered physiotherapists.

Appendix A

Appendix A

Table 8 Example: two items from each of the PC-PART Self Care and Domestic Life scales

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Darzins, S.W., Imms, C., Shields, N. et al. Responsiveness, construct and criterion validity of the Personal Care-Participation Assessment and Resource Tool (PC-PART). Health Qual Life Outcomes 13, 125 (2015). https://doi.org/10.1186/s12955-015-0322-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12955-015-0322-5

Keywords