Psychometric performance of an assessment scale for strain in nursing care: The M-NCAS

Background Multiple instruments exist to measure dementia behaviors, but the nursing staff perspective on those behaviors and their level of burden has not been well measured. The goal of this study was to examine the psychometric performance of the Modified Nursing Care Assessment Scale (M-NCAS), a 28-item nurse rating of burden associated with care for institutionalized individuals with dementia. Nurses rate items in terms of extent to which the behavior or characteristic is present ("attitude" domain), and extent to which it is a burden ("strain" domain). Methods Data from 282 patients enrolled in a 12-week, double-blind, randomized clinical trial comparing risperidone treatment to placebo was used to evaluate M-NCAS item performance, internal consistency reliability, and construct validity. Empirical subscales were identified via exploratory factor analysis (EFA). Results Four poorly-performing items were deleted from further analyses. EFA identified 3 "attitude" subscales and 5 "strain" subscales. Cronbach's alphas were 0.65 and above. Correlation with the Cohen-Mansfield Agitation Inventory and the BEHAVE-AD, clinical ratings of dementia behaviors, were low to moderate. Conclusion The M-NCAS provides a valid and reliable means of obtaining care burden ratings from formal caregivers in long-term care, and provides a method for evaluating dementia interventions from the perspective of nursing staff.

with such care may contribute to staff turnover. Nursing staff turnover in nursing homes is high, ranging from 40%-96% annually in the United States [3]. The societal costs of nursing staff turnover (particularly in the current environment of nursing staff shortages) are great, and can be expected to increase as the number of institutionalized older adults increases. Studies of disturbing or distressing behaviors among institutionalized dementia patients demonstrate that these behaviors occur frequently, and are a source of great frustration to the nursing staff [4][5][6][7][8][9]. The prevalence of dementia in nursing homes, estimated to be close to 50% in the U.S. [10] and over 70% in Australia [11], coupled with the behaviors associated with dementia, combine to make caring for patients with dementia a particular challenge to nursing staff. Measuring this burden can provide another means of evaluating outcomes of dementia treatment in the nursing home setting.
There are measures relevant to the burden of paid/formal caregivers (i.e., nursing staff caring for dementia patients), but most focus on such overt behaviors as "forgetting what day it is" or "pull away" only (e.g., the Memory and Behaviors Problems Checklist [12] and the Resistiveness to Care Scale [2]). In their comprehensive examination of formal caregiver stress and cognitive impairment, Novak and Chappell [9] used an extensive individual face-to-face interview that lasted approximately one and a hours -a form of the Burden Interview [13] modified for institutional caregivers and Maslach's Burnout Inventory [14]. In their earlier work, they also asked 5 dementia-specific questions [15].
The Strain in Nursing Care Assessment Scale (NCAS) [9,10] was developed to address the need for a more comprehensive measure of nursing burden -specific to the long-term care setting -that includes sources of burden beyond overt behaviors. It is based on the conceptualization of burden as deriving from patient behaviors, as well as patient characteristics as perceived by the nurse carer. Unlike solely behavior-based measures completed by nurses, the NCAS addresses other aspects relevant to the provision of nursing care, such as nursing staff's perceptions regarding the meaningfulness of resident lives and the residents' level of gratefulness for care. The NCAS demonstrated the ability to capture changes in nurses' rating of difficulty regarding dementia patients' characteristics in a year-long study of a care intervention [16]. The Modified Strain in Nursing Care Assessment Scale (M-NCAS) was adapted from the NCAS developed and used in Swedish long-term care facilities [16,17]. The M-NCAS includes more items than the original instrument to capture the presence and impact of additional behaviors not represented in the original instrument, thus providing an even more comprehensive measure of nursing burden.
The M-NCAS was used in a clinical trial of an atypical antipsychotic, risperidone, for treatment of nursing home patients with behavioral and psychological signs and symptoms in dementia. This preliminary investigation describes the psychometric evaluation of the M-NCAS using data from the clinical trial; thus, it is a post-hoc assessment of reliability and validity. Assessment of psychometric properties included exploration of item performance, subscale assignment, and assessment of reliability and validity based on the available data from the clinical trial.

Design and Procedures
The psychometric evaluation used data from the baseline assessment of a multi-site, double-blind, placebo-controlled trial of risperidone in the treatment of behavioral and psychological symptoms among institutionalized patients with dementia. Fourteen sites throughout Australia and New Zealand recruited patients from the various nursing homes associated with each site. Patients were included in the clinical trial if they were 55 years of age or older, residing in a nursing home environment for one month, with a DSM-IV diagnosis of either dementia of the Alzheimer's type with behavioral disturbance, vascular dementia with behavioral disturbance, or mixed dementia, had scores less than or equal to 23 on the Mini-Mental Status Exam (MMSE) [18], and met aggressive item score criteria on the Cohen-Mansfield Agitation Inventory (CMAI) [19]. Aggressive item score criteria was defined as a score of at least 4 on one aggressive item, a frequency score of 3 on at least two aggressive items, or a frequency score of 2 on at least three aggressive items or two aggressive items occurring at a frequency of 2 and one at a frequency of 3 on the CMAI. These criteria were designed by the trial investigators to limit the sample to moderate to severe dementia patients having recognizable behavioral disturbances. Patients were required to have a carer who was able and committed to assisting the subject to comply with medication intake and the trial protocol, and who was willing to provide the information required at assessment interviews. Nursing home patients were excluded if they had medical or neurological conditions other than dementia which diminished cognitive function, dementia secondary to alcoholism, a diagnosis of major psychiatric comorbidity, substance abuse, tardive dyskinesia, clinicallyuncontrolled medical conditions or laboratory abnormalities, a history of adult seizures, an administration of a depot injection or a long-acting neuroleptic within two treatment cycles of selection, hypersensitivity to neuroleptic treatment, or a history of failure to respond to risperidone of four weeks duration or participation in clinical trials with investigational drugs within the past four weeks. All patients or their appropriate representatives provided written informed consent, and the research protocol was approved by the appropriate institutional review board and state guardianship boards where required. Nurse carers were identified for each patient participant.

Nursing Care Assessments
The Behavioral Pathology in Alzheimer's Disease Rating Scale (BEHAVE-AD), the CMAI, and the M-NCAS were administered to the nurse carers of the individual patients by the investigator or site coordinator at baseline, week 4, week 8, and week 12 (trial endpoints). Nurse carers could have more than one patient participating in the trial, and they completed questionnaires for each individual patient.

Modified Nursing Cares Assessment Scale (M-NCAS)
The M-NCAS is was adapted from the original NCAS instrument developed in Sweden, and contains 32 items based in part on the 21 original items of the NCAS. Additional items were selected based on comments made by long-term care nurses regarding their experience caring for dementia patients with dementia in long-term care facilities, and added to make the instrument more comprehensive.
The M-NCAS contains 32 items, with two domains per item: one addresses the occurrence and intensity of the behavior, in which the staff members indicate the extent to which they agree that the patient exhibits the behavior (the "attitude" domain); and one which addresses staff rating of the difficulty of coping with the behavior (the "strain" domain). Responses are measured on a fourpoint Likert-type scale (ranging from Agree to Don't Agree for the Attitude domain, and Very Easy to Very Difficult for the Strain domain). Lower scores are better for both domains. Domain total and subscale scores are calculated separately; total and subscale scores are calculated as an average of the individual item scores. The M-NCAS was translated from the original Swedish to English using standard forward and backward translation techniques.
For subscale analyses, if < 50% of the scale items were missing, the scale was scored with the mean score of the non-missing items for that individual used to impute a score for the missing items. If > 50% of the items were missing, no scale score was calculated; the subscale score was considered missing. Total scores were calculated only if < 50% of the items were missing. Missing item criteria was based on that recommended by Ware et al. [20].

Cohen-Mansfield Agitation Inventory (CMAI)
The CMAI is a 29-item scale specifically developed to assess agitated and disruptive behaviors, as well as carerelated problems, occurring in demented subjects residing in nursing homes. The scale measures the frequency of behaviors, and systematically assesses agitation. The scale's 29 activities are rated on a 7-point scale indicating the frequency of a particular activity (range; never to a few times per hour). The activities are organized into 4 subscales: physical/aggressive, physical/non-aggressive, verbal/aggressive, and verbal/non-aggressive. The total aggression score is the sum of the physical and verbal aggression subscales. The total non-aggression score is the sum of the verbal and physical non-aggression subscales. A recall period of one week was used. The CMAI was used to assess the construct validity of the M-NCAS.

Behavioral Pathology in Alzheimer's Disease Rating Scale (BEHAVE-AD)
The BEHAVE-AD is a 25-item instrument designed to assess the severity of behavioral disturbances in subjects with dementia, based upon the caregiver's observation and report of the subject's behavioral problems. It consists of 7 subscales: paranoid and delusion ideation, hallucinations, activity disturbances, aggressiveness, diurnal rhythm disturbances, affective disturbances, and anxieties and phobias. Symptoms are rated on a scale from 0 (absence of the symptom) to 3 (greatest severity). A global item is included, in which a single judgment is made as to how troubling or dangerous the subject's behavior has been to the caregiver. A recall period of one month was used. The BEHAVE-AD was used to assess the construct validity of the M-NCAS.

Data Analysis and Psychometric Evaluation
We evaluated the item performance, scaling characteristics, reliability, and construct validity of the M-NCAS using baseline assessment data.

Item Performance and Scaling Characteristics
Item performance was examined to identify the presence of items that may reduce the instrument's ability to detect changes over time, and to discriminate between groups. The characteristics of individual M-NCAS items examined were mean, minimum and maximum responses, percent missing, floor and ceiling effects, and item-total correlations. These data were used to determine if some items should be deleted for subsequent analyses (e.g., item-tototal correlations below 0.40 [21]). Exploratory factor analysis was performed to evaluate the underlying subscale structure of the M-NCAS.

Reliability
Internal consistency reliabilities of total and subscale scores were estimated using Cronbach's coefficient α. Data to examine test-retest reliability was not available.

Construct Validity
Correlations were determined for total and subscale scores for attitude and strain domains and the CMAI physical and verbal aggression and non-aggression subscales. Correlations were also calculated between M-NCAS scores and the total score, seven subscales, psychotic symptoms subtotal and global rating question of the BEHAVE-AD, and the CGI-S and CGI-C scores. Pearson correlations were used unless otherwise specified. Correlations were expected to be low to moderate (0.20 ≤ r < 0.40).
All statistical analyses were conducted using SAS statistical software version 8.0. A significance level of 0.05 and 2tailed tests were used unless otherwise noted. No adjustments were made for multiple comparisons.

Results
Two hundred and eighty-two patients had evaluable baseline data for the psychometric evaluation. The majority of patients were female (72.4%) and Caucasian (98.4%); 11.5% were between the ages of 65 and 74, while 87.1% were over age 74. Most subjects had a diagnosis of Alzheimer's Disease (59.1%), while 28.3% had vascular dementia and 12.5% had mixed dementia.
Nurse carers were primarily Caucasian. Further demographic data and information about the number and type of carers were not available.

Item Performance
Items and their distributional characteristics were reviewed. See Table 1 for a listing of items, mean scores, and floor (% of responses at 1) and ceiling effects (% of responses at 4). The attitude domain contained several items with high floor and ceiling effects. Item 12 had 79% of responses at floor -the highest among all items. Items 19 and 28 had the highest percentages of responses at ceiling (76% and 77%, respectively). Only one person had any missing data at the baseline visit. Examination of item correlation to total and subscale scores on the attitude domain demonstrated low or inverse correlations for four items (Item 5: tries to influence others in order to maintain control of his/her life; Item 12: is submissive; Item 27: patterns of behaviors you can foresee; and Item 19: has little control over his/her difficult behavior) (data not shown). patterns of behavior you can foresee, item 7 is apathetic/ seems to have limited emotions) did not perform well on the factor analysis -either overlapping substantially with another factor, or producing factor loadings less than 0.30.
Based on overall item analysis results, including floor and ceiling effects, item-to-item and item-to-total correlation results, and a review of factor analysis results, four poorlyperforming items were deleted from the attitude domain: item 5 tries to influence others in order to maintain control of his/her own life; item 12 submissive/gives in to everything done to him/her; item 19 has little control over his/her difficult behavior; and item 27 has patterns of behavior you can foresee. The removal of these items resulted in three distinct subscales for the attitude domain, demonstrating good approximation of simple structure. The three factors, all with variable loadings of 0.30 or above were "Attention-seeking" (e.g., "is manipulative", "needs someone close by all the time/is demanding"), "Autonomy" (e.g., "does things for a reason", "is apathetic/seems to have limited emotions"), and "Diffi-culty" (e.g. "is unpredictable" and "is friendly"). Items were reverse-coded as appropriate.
To ensure inter-domain consistency, we removed the same four items from the strain dimension, thus resulting in an acceptable five-factor solution for this domain. The five factors, all with variable loadings of 0.30 or above, are "Affect" (e.g., "is calm", 'is anxious"), "Job satisfaction" (e.g., "is grateful for what is done for him/her", "gives no job satisfaction"), "Neediness" (e.g., "is selfish", "is manipulative"), "Predictability" (e.g., "knows what he/ she wants and stands up for his/herself", "is aggressive/ resistive"), and "Self-direction" (e.g., "is frightened/vulnerable", "seems to experience the normal range of emotions"). See Tables 2 and 3 for a list of specific items in each domain.
Following the removal of these 4 items, item-to-subscale correlations were examined for all subscales. In all cases, items correlated moderately to highly with the subscales to which they were assigned via the factor analysis. See Tables 2 and 3.

Reliability
The internal consistency reliability (Cronbach's alpha) for the attitude and strain total scores was good -0.79 and 0.95, respectively. Internal consistency reliability for sub-scales in both domains was excellent in general. See Table  4. Table 5 summarizes the correlations between the M-NCAS attitude total and subscale scores and the CMAI total aggression, total non-aggression, and subscale scores. In general, correlations were low to moderate [22]. The highest correlation was between attention-seeking and verbal non-aggression (r = 0.68; p < 0.01). Table 5 also summarizes the correlations between the strain total and subscale scores and the CMAI total aggression, total non-aggression, and subscale scores. Correlations were generally low to moderate, with only one correlation exceeding 0.40 (verbal non-aggression and neediness; r = 0.46, p < 0.01). Table 6 presents the correlations between the M-NCAS attitude total and subscale scores and the BEHAVE-AD.

Construct Validity
With the exception of the activity disturbance subscale, most correlations between the BEHAVE-AD subscales and  the attitude total score were statistically significant. In general, the attention-seeking attitude subscale was related to BEHAVE-AD scores, but the autonomy subscale was not. Results were mixed for the difficulty subscale, with a correlation of 0.47 to the aggressiveness BEHAVE-AD subscale. Table 6 also presents results for the M-NCAS strain domain and BEHAVE-AD score correlations. The total and most of the strain subscale scores were significantly related to BEHAVE-AD total and subscale scores.

Discussion
The M-NCAS demonstrated good psychometric properties based on an analysis of baseline data collected during a clinical trial of risperidone versus placebo, as reported in this preliminary investigation. Item performance, particularly floor and ceiling effects (together with item-total correlation data), suggested that four items on the attitude domain were performing poorly from a psychometric standpoint. Two items performed poorly on item correlations as well as factor analysis, while item 12 had a high floor effect as well as poor correlations. The remaining 28 items still capture sufficiently comprehensive data on the domains of interest, based on content review and on the empirical performance of subscales derived from them. The collection of additional data would enhance the confidence of conclusions regarding M-NCAS performance. These results support the use of the M-NCAS for the collection of valid, reliable, and comprehensive information on the burden experienced by nurse carers when caring for dementia patients in long-term care settings. The instrument can assist in staff management by identifying nurse carers with the greatest levels of burden -ideally so that remedial action could be taken, either through extra support for staff members or through the shifting of caseloads from particularly difficult patients for nurses at risk for attrition.
There are several limitations to this study. Of major concern is the lack of information on the nurse carers themselves. No data was available as to the type of carer, the duration they had cared for the patient, and their sociodemographic characteristics. The inability to characterize the respondents to the M-NCAS is a limitation in interpreting generalizability of results. No control was attempted for nurse raters, so the possibility exists that measurement properties of the M-NCAS differed across nurse raterstherefore skewing the results. The extent to which rater effects limit the validity of results is likely to be minimal, however, given that multiple subjects were being rated by each nurse rater. Future examination of between-subject properties would be helpful. The sample size for the study is small, relative to the standard recommendations regarding the number of subjects per item for factor analysis.
However, we set criteria for relatively moderate to high factor loadings. Of note is that the final recommendations for the measure length are closer to that guideline. Because this was a post-hoc analysis of clinical trial data, the instruments selected for use in assessing construct validity may not have been optimal, in that they did not assess nursing strain per se. However, they did assess the propensity of exhibiting certain behaviors -as does the M-NCAS. Additional data on stability across time (test-retest reliability) is desirable for a full psychometric description of the M-NCAS, and was not available from this study. Finally, this sample of nurse carers was almost exclusively Caucasian. Therefore, the generalizability of these results to nurse carers of different ethnicities, and in different locales is limited.

Conclusions
The M-NCAS enables the detection of the presence or absence of specific behaviors similar to checklists (the attitude scale), but extends that information by providing a rating of the degree of burden of each aspect rated. It possesses good psychometric properties for use with nurse carers working with Alzheimer's patients in long-term care facilities. The M-NCAS provides a unique approach to identifying both positive and negative behaviors, and to quantifying the amount of stress felt by carers as a result of these behaviors. The nursing staff perspective on residents with dementia is unique [23], and the M-NCAS exploits this perspective by capturing the aspects of residents beyond overt behaviors.

List of Abbreviations
Behavioral Pathology in Alzheimer's Disease Rating Scale