Assessing the validity and intra-observer agreement of the MIDAM-LTC; an instrument measuring factors that influence personal dignity in long-term care facilities

Abstract

Background

Patients who are cared for in long-term care facilities are vulnerable to lose personal dignity. An instrument measuring factors that influence dignity can be used to better target dignity-conserving care to an individual patient, but no such instrument is yet available for the long-term care setting. The aim of this study was to create the Measurement Instrument for Dignity AMsterdam - for Long-Term Care facilities (MIDAM-LTC) and to assess its validity and intra-observer agreement.

Methods

Thirteen items specific for the LTC setting were added to the earlier developed, more general MIDAM. The MIDAM-LTC consisted of 39 symptoms or experiences for which presence as well as influence on dignity were asked, and a single item score for overall personal dignity. Questionnaires containing the MIDAM-LTC were administered face-to-face at two moments (with a 1-week interval) to 95 nursing home residents residing on general medical wards of six nursing homes in the Netherlands. Constructs related to dignity (WHO Well-Being Five Index, quality of life and physical health status) were also measured. Ten residents answered the questions while thinking aloud. Content validity, construct validity and intra-observer agreement were examined.

Results

Nine of the 39 items barely exerted influence on dignity. Eight of them could be omitted from the MIDAM-LTC, because the thinking aloud method revealed sensible explanations for their small influence on dignity. Residents reported that they missed no important items. Hypotheses to support construct validity, about the strength of correlations between on the one hand personal dignity and on the other hand well-being, quality of life or physical health status, were confirmed. On average, 83% of the scores given for each item’s influence on dignity were practically consistent over 1 week, and more than 80% of the residents gave consistent scores for the single item score for overall dignity.

Conclusion

The MIDAM-LTC has good content validity, construct validity and intra-observer agreement. By omitting 8 items from the instrument, a good balance between comprehensiveness and feasibility is realised. The MIDAM-LTC allows researchers to examine the concept of dignity more closely in the LTC setting, and can assist caregivers in providing dignity-conserving care.

Background

In light of the ageing population and the fact that people live a relatively longer period of time with chronic diseases and disabilities, concerns about losing personal dignity may increasingly arise [1, 2].

Personal dignity is a type of dignity which relates to a sense of worthiness, is individualistic, tied to personal goals and social circumstances, and can be taken away or enhanced by circumstances or acts from others [3–5]. It should be distinguished from basic dignity, which is the inherent dignity of each human being and can be regarded as a universal and inalienable moral quality [1, 6]. Earlier studies have shown that loss of personal dignity is associated with depression, hopelessness, a desire for death [7] and requests for euthanasia and physician-assisted suicide [8–11]. It is this type of dignity that is therefore important to understand, assess and preserve within the context of health care. The dignity concept can contribute to care in the last phase of life because it goes beyond assessment of physical and psychosocial health status and includes one’s perception of worthiness, both as an individual and in relation to close others and society [12–14]. By now, there is a substantial amount of knowledge on how patients nearing death [15–17] and older people in nursing homes [18–22] understand the concept of dignity. The majority of these studies had a qualitative design, and described the factors that can preserve or undermine personal dignity, and their interrelatedness.

Some of these empirical studies have served as a basis for the development of a measurement instrument for dignity. An example is the Patient Dignity Inventory, a 25-item list which was validated in patients in a palliative care program (predominantly cancer patients with a life expectancy of less than 6 months) [23, 24]. Another instrument targeting at dying patients is the dignity card-sort tool, which can be used to rank factors influential in the loss or preservation of dignity at life’s end [25, 26]. Recognizing the need for an instrument that is applicable to a more general patient population, our research group has developed the Measurement Instrument for Dignity AMsterdam (MIDAM) and examined its content validity in people with one or more advance directive(s) [27].

A setting for which no such measurement instrument is yet available is the long-term care setting. As compared to the general patient population, some aspects probably become more important for those who live permanently in an institution. Patients who are cared for in long-term care facilities not only face threats to dignity arising from functional and/or cognitive decline, they are also confronted with an unfamiliar living environment, little privacy, are often heavily reliant on staff and increasingly lack social networks, making them rather vulnerable to lose personal dignity [18, 28]. A measurement instrument can give insight regarding those who are most at risk of losing dignity, and can be used to better target more effective, dignity-conserving care to an individual patient. Therefore, the aim of this study is to provide the long-term care setting with a valid and reliable measurement instrument. In this article we describe how the already existing MIDAM was adapted to create the MIDAM-LTC, and how we tested its content validity, construct validity and intra-observer agreement in a sample of 95 nursing home residents. Furthermore, we examined possibilities to reduce the length of the instrument, in order to make it feasible for use in practice.

Methods

Design and study population

The starting point for this study was our earlier developed measurement instrument for self-perceived dignity, retrospectively named MIDAM [27]. This instrument consists of 26 items (symptoms or experiences) categorized in 4 domains: (I) evaluation of self in relation to others, (II) functional status, (III) mental state and (IV) care and situational aspects. On the basis of the results of an extensive qualitative interview study among 30 nursing home residents [22, 29], we added items specific for long-term care facilities to the MIDAM, i.e. items with regard to the way residents are treated by nursing home staff, living circumstances in the nursing home, living in a group, limited capacity of nursing home staff, sense of belonging and sense of meaning (see Table 1). These six themes were frequently mentioned by nursing home residents in the interview study, but not adequately reflected in the MIDAM. In a process of reflection and interaction, we formulated 13 items following the structure of the original MIDAM-items. We abundantly added items in order to be comprehensive. The extension “for Long-Term Care facilities” was added to the name of the instrument; MIDAM-LTC (see Additional file 1).

Table 1 Origin of items added to the MIDAM in order to create the MIDAM-LTC

To test the instrument’s psychometric properties, data were collected on the general medical wards (long-stay units for people with physical illnesses) of six nursing homes in the Netherlands. Nursing home residents were recruited with help from a unit manager, elderly care physician or the most important nurse on the ward. Eligible residents were all those on the wards who were cognitively able to understand the instructions and questions. Because many nursing home residents could not write anymore, an additional criterion was that a resident had to be able to communicate with an interviewer who would administer the questionnaire face-to-face and fill in the answers. All eligible residents received an information letter approximately one week before the interviewers came to the nursing home, and they could indicate whether or not they wanted to participate in the study at the time the interviewer visited them one week later. By consenting to participate in the study, the nursing home resident also gave us permission to ask his/her contact person in the care record (closest relative), elderly care physician and responsible nurse for information about the mental and physical health status of the resident.

The questionnaire

For each item (symptom or experience) in the MIDAM-LTC, respondents were first asked whether it applied to their life (by thinking about the past 2 days). Only if they answered affirmatively, they were asked to what extent this influenced their sense of dignity on a five-point scale (see Table 2). We deliberately distinguished between these two questions, in order to prevent respondents not suffering from a certain symptom or experience rating its putative association with dignity. In addition, we asked respondents to rate their sense of dignity on a 10-point scale (1 = sense of dignity completely lost, 10 = sense of dignity completely intact). Besides these questions about personal dignity, the questionnaire asked for respondent characteristics, and contained measurements of related constructs like the WHO-Five Well-being Index, the EQ-5D, and a question to rate quality of life (on a 10-point scale). In order to assess the resident’s mental and physical health status, the Cognitive Performance Scale [30], Barthel Index [31] and Karnofsky Performance Status Scale [32] were presented to the nurses. The elderly care physician and closest relative provided information about the resident’s diseases.

Table 2 Questions for each item in the MIDAM-LTC

Procedure

We first piloted the questionnaire with three nursing home residents, after which a few items of the MIDAM-LTC were reformulated. Subsequently, the questionnaire was administered face-to-face to all participants by four interviewers (among whom MOV and IG) and took approximately 30 minutes for all questions, of which the MIDAM-LTC lasted about 20 minutes. Residents were handed a card with the answering options to help them choosing their answer. Ten residents were asked to fill in the questionnaire by using the 'think aloud’ method [33]. This method was used to elicit data on nursing home residents’ thought processes as they responded to the items and to assess whether they understood the questions as we intended them. To be able to study intra-observer agreement, we visited the nursing homes one week later and asked 49 residents to fill in the questionnaire a second time with help from another interviewer than the week before. We intended to have two measurements of half of the participants, and asked the residents who were available at the time of our second visit. Only a few declined to participate twice. We chose for a time interval of one week, which we estimated to be long enough to prevent recall, though short enough to prevent that major changes in personal dignity occurred [34]. The study was approved by the Medical Ethics Committee of the VU University Medical Center.

Analysis of psychometric properties

Each item of the MIDAM-LTC can be considered a causal indicator, meaning that items are not expected to correlate with each other, and that even a single symptom or experience may suffice to undermine dignity, while a low sense of dignity need not necessarily imply that someone suffers from all the symptoms listed [35, 36]. Therefore, performing a factor analysis or calculating a total score of all items was not meaningful. Instead, each item had to be considered separately, while we kept in mind that the instrument needed to be comprehensive, though take as little effort and time as possible to be filled in, in order to be feasible for use in practice.

Content validity was determined through two approaches. To examine whether all items in the MIDAM-LTC were relevant for personal dignity, we first calculated the mean scores per item for influence on dignity and the percentage of respondents who indicated that the item (if present) influenced their dignity quite a lot or very much (score 4 or 5). Because a valid ground for omitting items in models with causal indicators is that they occur too infrequently to be worth reporting [36], we decided to omit those items which fulfilled two criteria: a mean score for influence on dignity lower than 2.50, and a percentage below 25% of respondents who indicated that the item influenced their dignity quite a lot or very much (score 4 or 5). Comprehensiveness was assessed by asking the residents who answered the questionnaire while thinking aloud whether they missed any items that influenced their dignity.

To assess construct validity, we tested several hypotheses about the relation between dignity and related constructs (Table 3), based on expectations arising from our qualitative interview study. Pearson’s correlation coefficients between the constructs were calculated. According to Cohen, we classified a correlation coefficient over 0.5 as a strong relation, 0.3 to 0.5 as moderate, 0.1 to 0.3 as small, and below 0.1 as no relation between constructs at all [37].

Table 3 Hypotheses to assess construct validity

In examining the intra-observer agreement after one week, we were especially interested in absolute measures of agreement. Since Intraclass Correlation Coefficients and Cohen’s kappa values are relative measures of agreement [38], we calculated agreement percentages between the two observations. First, we compared the single item scores for overall personal dignity on both measurements. Next, percentages of agreement were calculated for each item’s influence on dignity in two different ways; we distinguished between exact agreement and agreement if we allowed the scores to differ one point on the 5-point scale. We hereby reasoned that an item could not undermine dignity if not present in a resident’s life - a non-affirmative answer on the first question - and scored it accordingly on the second question (score 1 on the 5-point scale). In this way, we could also take the first question regarding presence into account in calculating the agreement percentages. In addition, we looked at each item’s mean score for influence on dignity on both measurements, and tested whether these differed from each other with paired sample t-tests. Finally, average scores and average agreement percentages across all items’ influence on dignity were calculated.

Results

Sample characteristics

In total, 131 residents were approached to participate in this study. Twenty-one residents declined to participate, two residents were absent on the days the interviewers were in the nursing home and two residents died after they had received the information letter. A further eleven residents were excluded by the time of the interview, as it turned out that they were cognitively unable to understand the questions. This resulted in 95 participating residents, whose characteristics are shown in Table 4. The majority of the respondents were female and they averagely resided 744 days (median 575 days) in the nursing home. The most frequently reported diseases were heart diseases, rheumatoid arthritis and stroke. Respondents rated their sense of dignity on average as 7.3 (SD 1.6) on the 10-point scale.

Table 4 Characteristics of the study population

Content validity

In Table 5, the items are ranked according to the mean scores given for influence on dignity, and ordered per domain. The percentage of nursing home residents who agreed that an item applied to their life ranged from 98.9% ('Using medical-technical aids’) to 5.3% ('Not looking well-groomed’ and 'Not made any meaning or lasting contribution’) and were highest in the domain 'Functional status’. Most of the 13 added items specific for long-term care facilities were frequently present in the study population. However, their mean influence on sense of dignity was generally rather small. The highest mean scores for influence on dignity could instead be found in the domain 'Evaluation of self in relation to others’, and ranged from 3.25 ('Feeling worthless for friends and family’ and 'Not treated with respect by caregivers’) to 1.87 ('A changed physical appearance’). Nine items barely exerted influence on dignity according to the formulated criteria (marked with an asterisk in Table 5). Five of them were items that were specifically added for the long-term care setting.

Table 5 Presence and influence on dignity per item in a group of nursing home residents (N = 95), ordered per domain

Before definitively removing these nine items from the instrument, we listened to the recordings of the 10 nursing home residents who filled in the MIDAM-LTC while thinking aloud, to find out the reason for the small extent to which an item averagely influenced dignity. This analysis revealed that some symptoms or experiences, although present in a resident’s life, did not undermine dignity because the resident was satisfied with the way it was. For example, the majority of the residents who agreed that they had a small room did not long for a bigger room (“Where do I need it for?”). More or less the same argumentation was given for the small influence on dignity for the items 'Little opportunity to shower’ (“I don’t want to shower more often, it makes me tired”) and 'Having little contact with other residents’ (“They are too demented or fighting with each other, so I’d rather be on my own”). The items 'Receiving little time from nurses’ and 'A lot of different nurses’ hardly undermined dignity, because residents ascribed the presence of both items to the circumstances rather than to nurses’ unwillingness (“They are terribly busy because of the lack of staff. If I really need them, they will make time for me”). 'Being forgetful’ and 'A changed appearance’ were regarded and accepted as belonging to the ageing process and therefore not undermining dignity (“Yes I have more wrinkles, grey hair and I forget things occasionally, but may I? I’m 82 years old!”). An extra reason why 'A changed appearance’ barely exerted influence on dignity was that some nursing home residents said they had lost weight, which they actually regarded as positive. There was one item for which no sensible explanation could be found for the low scores on a) presence and b) influence on dignity: 'Mentally unable to take decisions’. Possibly, these low scores were a consequence of our decision to include only respondents who were cognitively able to understand the questions. It might also be that the way the item was phrased could have discouraged respondents to indicate the item applied to them. We therefore chose to keep this item in the instrument and reformulate it more mildly into 'I feel unable to take major decisions’.

In addition, the thinking aloud method revealed that the item 'Feeling bored and experiencing every day as the same’ consisted of two different aspects; nursing home residents could experience every day as the same, while not feeling bored. Therefore, we decided to reformulate this item into 'All days seem colourless to me’. Furthermore, the items 'Not able to carry out usual activities’ and 'Lost fighting spirit’ were differently interpreted by the residents. For some, usual activities were activities they used to do in the past (e.g. cycling), whereas others thought of activities they did in the nursing home (e.g. reading and watching TV). To correct for these different interpretations, and to only include those activities participants have a current need for, this items needs to be reformulated into 'Not able to carry out activities I would like to do’. As for fighting spirit, some interpreted this as 'enjoying all organized activities in the nursing home’, whereas others considered this as 'standing up for themselves’. However, this latter item does not need to be reworded, as it concerns the same character trait that lies at the root of these two different manifestations. Since the purpose of the MIDAM-LTC is to give insight regarding those who are most at risk of losing dignity, high scores on items are merely a signal to start questioning the source of dignity-related distress. Finally, no items were missed by the nursing home residents.

Construct validity

We tested our first hypothesis (see Table 3) without the eight items of which we had decided to omit them, as described above. The number of items where residents indicated that it influenced their dignity (very) much (score 4 or 5) correlated moderately with the single item score for overall personal dignity (r = -0.49), just missing the threshold to be classified as a strong correlation. Our second hypothesis was supported by the data: Pearson’s r for the relation between the single item scores for quality of life and personal dignity was 0.50, and between the WHO Well-Being Scale and personal dignity 0.53. Unfortunately, we could test our third hypothesis only partially. According to our expectations, the correlation between the Barthel Index and personal dignity was low (r = 0.23). However, scores on the Karnofsky Performance Status Scale were sometimes unrealistically high (for some residents even '100’ which means perfectly healthy), indicating that some nurses did probably not understand the question or took the average nursing home residents as a reference in mind when answering this question. Calculating a correlation coefficient between this scale and the single item score for dignity was therefore not appropriate.

Intra-observer agreement

Table 6 shows the percentages of agreement for each item’s influence on dignity, as well as average agreement percentages across all items of the MIDAM-LTC (without the eight items of which we had decided to omit them). The average exact agreement percentage for all items combined was 70.6% and increased to 83.4% when we allowed one point difference on the five-point scale. In the latter condition, individual item’s agreement ranged from 59.6% to 95.8%. No significant differences between the mean scores for influence on dignity on both measurements existed; neither for all items combined, nor for any individual item (data not shown). Of all nursing home residents who rated their overall personal dignity on both measurements, 50.0% gave the exact same score on the single item 10-point scale. A further 30.4% of the residents differed only one point in their ratings.

Table 6 Intra-observer agreement after 1 week in a group of nursing home residents (N = 49)

Discussion

By implementing all changes and omitting some items, the revised MIDAM-LTC consists of 31 items (see Additional file 1), and a good balance between comprehensiveness and feasibility is realised. The MIDAM-LTC has good basic standard psychometric properties. Content validity refers to the extent to which the concept of interest is comprehensively represented by the items of an instrument. That no aspects were missed provides evidence for a good content validity of the MIDAM-LTC. In addition, the 13 added items specific for long-term care facilities were derived from an extensive qualitative interview study with 30 nursing home residents [22, 29], so these items were considered relevant by the target population. Given that we abundantly added items in our efforts to be comprehensive, it is therefore not surprising that five of these 13 added items were not found to have a large influence on personal dignity (although frequently present) and could be omitted from the instrument. In contrast, only three items from the general MIDAM [27] were found to barely influence dignity, demonstrating the validity of these already existing items across different settings.

Construct validity, applicable in situations in which there is no gold standard, refers to whether the instrument provides the expected scores, based on existing knowledge about the construct [39]. Our expectations regarding the extent to which personal dignity correlated with other constructs were virtually all supported by the data, and the one expectation that was not (formulated in the first hypothesis) came very close to confirmation. This shows that the MIDAM-LTC has good construct validity. Although dignity appears strongly related to quality of life, it is noteworthy that nursing home residents rated their dignity generally higher (with a mean of 7.3 out of 10) as compared to their quality of life (with a mean of 6.6 out of 10). Firstly, this finding suggests that personal dignity is a resilient construct. Most nursing home residents seem able to withstand the various physical and psychological challenges they face, making a great undermining of dignity rather the exception than the norm. Secondly, this higher score for dignity suggests that nursing home residents can distinguish between personal dignity and quality of life, despite the overlap in physical, socio-psychological and spiritual aspects reflected in both. Whereas overall quality of life exceeds health-related quality of life, which in turn is more than health status only [40], dignity may go beyond quality of life because it also brings one’s perception of being worth of respect from themselves and from others along. As quality of life is defined as a subjective integration of all aspects of one’s life deemed relevant [41], personal dignity may stress more importance on the evaluation of oneself in close relation to others. In our qualitative studies, we found that relational and societal aspects could undermine a resident’s dignity, but preserve or enhance it as well [22, 29]. For example, being socially involved with others, receiving good professional care and social support, and being amongst disabled others and therefore less prone to exposures of disrespect from the outer world could help residents to maintain or regain their dignity. Presence of these preservative factors may explain the relatively high score given for personal dignity in this study.

An adequate intra-observer agreement is attributable to more than 83% of the residents who gave a practically consistent score for the each item’s influence on dignity over a week, and to more than 80% of the residents who did this for the single item score for overall personal dignity. Some lower agreement scores were found for individual items that were more prone to fluctuate in time (e.g. 'feeling guilty about calling on the nurses a lot’). Similar good results were obtained in a study on the psychometric properties of the Patient Dignity Inventory [24]. Test-retest reliability was then measured over a 24-hour time frame, which might be too short to ensure no recall bias was introduced. That we found these high percentages even after one week implies that personal dignity and the items in the MIDAM-LTC are quite stable.

Strengths and limitations

The MIDAM-LTC is, to our knowledge, the first instrument measuring dignity specifically targeted at the population living in long-term care facilities. Although nursing homes are only one facility providing long-term care, we believe that the MIDAM-LTC is relevant for all people living in any kind of long-term care institution. Since the adjustment of the instrument is based upon the perspectives of nursing home residents - who are probably the most severely disabled patients within the long-term care setting - it is likely that the added items cover the whole range of relevant aspects influential to the dignity of persons living in institutions. By using the think-aloud method in 10 nursing home residents, we gained valuable insight in the thought processes they engaged in when responding to the questionnaire. However, in interpreting their answers, we must be aware that answers obtained by the think-aloud method may be more social-desirable than the answers received from the other nursing home residents. Our study was limited to the experiences of residents who were able to think and communicate about dignity. Although applicable to people who suffer from mild cognitive decline, the instrument cannot provide relevant information in people with more severe cognitive incapacities. Another limitation is that the MIDAM-LTC can only detect whether a certain symptom or experience undermines personal dignity, and not what preserves dignity. It might be that the absence of certain symptoms or experiences could actually have improved one’s sense of dignity, e.g. if a resident is one of the few still being able to go to the toilet independently. However, an instrument measuring both undermining and preservative factors would require a different structure, and would possibly become too complex to be understood by respondents. To provide the long-term care setting with a feasible instrument, we therefore only focused on the factors that undermine dignity, as they are more relevant for practice.

Conclusions

The MIDAM-LTC appears to be a reliable and valid instrument for the assessment of factors influencing personal dignity in the long-term care setting. By reducing the number of items listed, the feasibility of the instrument for use in practice has increased. It allows researchers to examine the concept of dignity more closely in long-term care, e.g. by investigating distributions of sources of dignity-related distress across various patients. Caregivers working in long-term care institutions could use the MIDAM-LTC to assist them in providing dignity-conserving care, by identifying the factors that undermine a patient’s personal dignity.