- Open Access
Validation of an individualised quality of life measure in older day hospital patients
Health and Quality of Life Outcomesvolume 6, Article number: 27 (2008)
To test the ease of use, reliability, responsiveness and construct validity of the Patient Generated Index, an individualised quality of life score, in older people attending a Medicine for Older People Day Hospital.
Prospective longitudinal study in patients attending a specialist Medicine for Older People Day Hospital in Scotland. The Patient Generated Index was administered at baseline, one week later, and at the end of Day Hospital attendance. Functional Limitations Profile, Hospital Anxiety and Depression Score, Barthel index and global subjective impressions of change were also collected and compared with baseline scores and change in Patient Generated Index scores. Reliability was assessed using intraclass correlation coefficients in subjects reporting no change in global quality of life; responsiveness was assessed using effect size and Guyatt coefficients in subjects reporting change in global quality of life. External validity was assessed via correlation with measures of physical function, comorbid disease and psychological state.
75 patients were enrolled, mean age 81 years. Mean completion time was 5.0 minutes at baseline. Reliability was moderate (intraclass correlation coefficient 0.72) but there were weak and inconsistent responses to change (effect sizes 0.02 to 0.15; Guyatt responsiveness coefficient 0.29). Patient Generated Index scores correlated with Functional Limitation Profile scores (r = 0.51, p < 0.001), baseline anxiety score (r = -0.25, P = 0.039) and baseline depression score (r = -0.37, P = 0.002) but displayed only weak, non-significant correlation with number of comorbid diseases (r = -0.22, P = 0.07), number of medications (r = -0.21, P = 0.08) and Barthel score (r = 0.09, p = 0.45).
The Patient Generated Index appears moderately reliable and easy to complete, but is poorly responsive to change, limiting its usefulness in clinical practice or research.
Quality of life is regarded as a key healthcare outcome by patients and by clinicians; both groups see improvement in quality of life as an important function of Medicine for Older People services such as Day Hospitals . Most Day Hospital services do not routinely measure quality of life however; the emphasis remains on the measurement of function [2, 3], and it is thus difficult to know whether such services are successful in improving quality of life.
Most 'quality of life' measures in current use focus either on health-related quality of life or health status – different concepts from overall quality of life . Whilst it is generally thought that health is an important determinant of overall quality of life, and that health services are best placed to address only this aspect of quality of life, such a compartmentalised approach is at odds with the holistic ethos of health and social care that is central to the activities of Day Hospitals for older people.
Most quality of life tools in current use are further limited by the use of a set series of questions. Quality of life is a highly individual, subjective concept , thus a single set of questions may lack face validity for many patients – i.e. the questions may not cover areas of quality of life important to the individual patients. In an attempt to overcome this issue, and to broaden the measurement of quality of life beyond health issues, individualised quality of life tools have been developed [6–8]. The Patient Generated Index (PGI) is such a tool, deriving in part from the hypothesis that quality of life can be depicted as the fit between expectation and reality at a given time .
If Day Hospitals are to be effective in improving and promoting quality of life, it is important to try and measure quality of life. Individualised quality of life tools have been used successfully in a variety of clinical settings, including in older people [7–9] and may overcome some of the limitations described above. Before such tools can be used in clinical practice, they require to be tested in the population of interest. Such tools must be easy to use, reliable (stable over time in stable patients), responsive to change, and have validity (correlate with either gold standard measures or with other measures of function, environment and psychosocial status and experience) . To be of use in clinical practice, tools should also collect data that are not collected by existing tools, and which have an impact on clinical activity.
Little previous work has been done using individualised quality of life tools in older people. We have previously shown that the PGI can be used in older people with heart failure, although the responsiveness of the tool was poor in this population. The SEIQoL has been studied in older hospitalised people , healthy older people  and in octogenarians in a longitudinal study ; the first group had some difficulties using the direct weighting device and took an impractically long time to use the tool. We were unable to find evidence that individualised quality of life tools have been assessed in a Medicine for the Elderly Day Hospital setting.
In this paper, we report the results of a study designed to test the acceptability, reliability, responsiveness and validity of an individualised quality of life tool – the Patient Generated Index – in a cohort of older patients attending a Medicine for the Elderly Day Hospital.
Patient selection and recruitment
Patients were recruited from the Medicine for Older People outpatient clinic at Royal Victoria Hospital, Dundee, UK, prior to their attendance at Royal Victoria Day Hospital. Patients are referred by primary care practitioners in the community or attend as follow-up after an in-patient stay in the hospital. Patients aged 65 years or over are referred with a wide range of presentations, including but not limited to falls, immobility, incontinence, breathlessness, weight loss and general debility. After assessment in the outpatient clinic, patients requiring multidisciplinary assessment and therapy attend the Day Hospital service once or twice a week. Attendance at Day Hospital usually lasts between four and six weeks.
Patients were eligible to be included in the study if they were due to attend Day Hospital. Patients with only a single planned Day Hospital attendance were excluded, as were those with a Folstein mini-mental state examination score of < 18/30, and those who were otherwise unable to give written informed consent. The study was approved by Tayside Local Research Ethics committee (application 05/S1401/107) and conformed to the principles of the Declaration of Helsinki.
Patients were given study information at their Medicine for the Elderly clinic appointment. Interviews were carried out during subsequent Day Hospital attendances to avoid the inconvenience of extra appointments or visits for the patient. Mini-mental state examination score  was recorded prior to obtaining written informed consent from all patients. The questionnaires administered were the Patient Generated Index  (higher score = better quality of life), the Functional Limitation Profile (FLP)  (a subjective measure of function; higher scores = worse function), and the Hospital Anxiety and Depression Score (HADS)  (higher scores = more anxious or depressed).
Questionnaires were administered at baseline (first visit to Day Hospital), one week later and at the final Day Hospital attendance. All questionnaires were administered by a trained nurse (RLF) with experience in caring for older people. Questionnaires were administered in the same order at each visit. At each time point, patients were asked to mark a visual analog scale to indicate their overall quality of life and at week one and discharge were also asked how their overall quality of life had changed since their last visit on a seven point Likert scale, ranging from 'much worse' through 'no change' to 'much better'. At the one week and final visits, patients answered the Patient Generated Index in two different ways. Firstly, they answered the questionnaire without reference to previous results. They were then presented with their selection of domains for quality of life from the baseline visit, which they rescored.
The Patient Generated Index has been described previously and has been tested in a variety of conditions including back pain, heart failure, colorectal cancer and postnatally [6, 12, 15–17]. Please see Additional File 1 for a description. Baseline data were obtained from the medical notes on age, sex, past medical history, patients' current medication as well as social circumstances. The Barthel Index, routinely collected by Day Hospital staff, was recorded on the first and last Day Hospital attendances.
Sample size calculation
Pilot work suggested that the correlations between the FLP and PGI, and between the PGI and the Barthel Index, were approximately r = 0.4. To detect this degree of correlation at the 0.05 significance level with 90% power requires 50 patients (one-sided test). We assumed a dropout rate of 30% which would therefore have required 72 patients to be recruited. To detect a 25% improvement or deterioration in quality of life score during attendance at Day Hospital, given a baseline PGI score of 65.8 (SD 36) as found in our pilot work, a sample size of 52 would be required (1 sample students t-test, p < 0.05, 90% power). Allowing for a 30% dropout rate required 75 patients to be recruited to the study.
Data were entered onto an Excel database and transferred to SPSS version 14 (SPSS, Chicago, USA) for statistical analysis. Reliability was assessed using those patients who reported 'no change' in quality of life on a Likert scale between baseline and one week. PGI scores at baseline and one week for this group were compared with intraclass correlation coefficients, using a one-way random effects model. Responsiveness was assessed using those patients who reported improvement or deterioration in overall quality of life on either the Likert scale or visual analog scale between week 1 and the final visit. Two methods of calculating responsiveness were used: a) effect size (test2-test1)/SD (test1) and b) Guyatt responsiveness coefficient, calculated as: minimally important difference/SD (change in scores of patients selecting 'no change' on Likert scale), where minimally important difference = mean of change scores in patients getting 'a little better' and 'a little worse' on the Likert score . Construct validity was tested by correlating questionnaire scores with functional outcome scores and with FLP scores within each time point.
127 patients were screened for suitability. Of these, two were admitted to hospital on their first visit to Day Hospital, three had significant cognitive impairment, 10 had only a single planned Day Hospital attendance, 9 declined to return to Day Hospital and 19 declined to enrol in the study without giving a reason. 75 patients were recruited to the study between July 2006 and March 2007 thus achieving the recruitment target. Baseline details are shown in Table 1. Of note, patients at baseline had anxiety scores just above the mean compared to normative population data, but significantly higher depression scores than seen in the general population (equivalent to the 80th/85th centile) .
72/75 (96%) of subjects attended the one-week follow up appointment, and 63/75 (74%) attended the final follow-up appointment. Reasons for non-attendance were ill-health or hospitalisation (5 patients), early discharge from Day Hospital (6 patients) and one patient who declined further participation as they could not complete the questionnaires. Subjects attended Day Hospital weekly a median of 5 (interquartile range 2) times.
Completion rates for the questionnaires
Completion rates were high for all questionnaires. 74/75 (99%) successfully completed the PGI at baseline, 72/72 (100%) completed successfully at week 1, and 63/63 (100%) completed the PGI successfully at the final visit.
Time to completion for the PGI
Mean (SD) time to completion for the PGI was 5.0 (1.4) minutes at the baseline visit, 4.6 (1.6) minutes at the one week visit, and 4.2 (1.3) minutes at the final visit.
Reliability was scored by comparing baseline and 1 week scores in patients who had noted 'no change' in their overall quality of life at 1 week; a total of 40 patients. The intraclass correlation coefficient (ICC) was 0.72 (95% CI 0.54 to 0.84) for the PGI when new domains could be selected, and was 0.61 (0.37 to 0.77) for the PGI when the baseline domains were presented a week later.
Change scores for each category of change contained on the Likert score were calculated and compared. The categories 'worse' and 'much worse' were aggregated due to small numbers; similarly, 'much better' contained only one respondent and was aggregated with 'better'. Change in overall quality of life as marked on a visual analog scale was correlated with change in questionnaire scores between week 1 and the final week of attendance. Results are given in Tables 2 and 3.
Effect sizes were calculated for patients who improved their overall quality of life between 1 week and the last visit at Day Hospital, and separately for those whose quality of life deteriorated. Results are shown in Table 4. Conventionally, an effect size of > 0.8 is regarded as large, 0.5 to 0.8 is moderate, and 0.2 to 0.5 is small. Responsiveness coefficients were also calculated for each tool; coefficients were 0.57 for the Barthel score, 0.26 for the FLP total, 0.29 for the PGI when new domains could be selected, and 0.004 for the PGI when baseline domains were presented at the final visit.
There is no gold standard with which to compare quality of life tools. However, common sense suggests that quality of life should worsen with worsening self-reported function and with worsening objectively assessed function. To test this, questionnaire scores were correlated with FLP and Barthel scores at each time point. The PGI showed moderate correlations with the FLP score (r = -0.44 to -0.51, P < 0.001) but weak correlations with the Barthel score (r = 0.09 at baseline, P = 0.45; r = 0.18 at final attendance, P = 0.2). The PGI was modestly correlated with the HADS anxiety score at baseline (r = -0.25, P = 0.039) and with the HADS depression score at baseline (r = -0.37, P = 0.002) but displayed only weak, non-significant correlation with number of comorbid diseases (r = -0.22, P = 0.07) or with the number of medications (r = -0.21, P = 0.08).
Choice of domains
There was considerable variability in the domains chosen at both follow-up times with few subjects choosing the exactly the same domains at follow-up. After one week, 22/72 (31%) chose completely different domains of quality of life; at final follow-up, 15/63 (24%) chose completely different domains to those chosen at the baseline assessment. Domains chosen by participants are summarised in Table 5.
This study has shown that the interviewer-administered Patient Generated Index (PGI) can be completed by the majority of older patients without significant cognitive impairment attending a Medicine for the Elderly Day Hospital. Completion rates were high, and the mean time to complete the tool was low. The reliability of the PGI was moderate, and interestingly the tool was less reliable when rescoring pre-chosen domains of quality of life. This suggests that re-presenting previously chosen domains of quality of life does not improve the psychometric properties of this tool, in contrast to findings with some other tools .
Responsiveness to change as measured by effect size was low for the PGI, although none of the tools used in this study displayed a large effect size. An effect size of > 0.8 is generally regarded as large, 0.5–0.8 is moderate, and 0.2 to 0.5 is small. Similarly, the responsiveness coefficients for the PGI suggested low responsiveness. Although correlation was evident between global change in quality of life as denoted by the Likert scale, correlations were weak and at times inconsistent – patients with worse overall quality of life scored similar PGI scores to those with improved global quality of life. There were no significant correlations between change in the PGI and change in the global visual analog score. This, in combination with the results from responsiveness to change analysis, suggests that the PGI cannot be relied upon as an index of change in quality of life in this patient population.
Correlation between the PGI and self-reported function (the FLP), anxiety and depression scores were moderate, suggesting that the PGI does have construct validity whilst collecting different information to the FLP. Correlation with the Barthel score was low and non-significant, suggesting that there was little overlap between the information collected by these tools, and that the PGI is collecting information that is not collected with standard Day Hospital assessment tools.
Strengths of our study include an adequately powered sample size and a representative group of attendees at Day Hospital including patients with mild to moderate cognitive impairment. We were able to test construct validity by comparing PGI scores with a variety of other measures of function and disease burden. Weaknesses of the study include the single centre nature of the study; results are not necessarily generalisable to other centres or other cultural settings.
Even though the PGI was originally designed to be a self-administered instrument, previous studies have found that interviewer administration of the PGI is necessary to ensure completion . The PGI has previously found to have moderate reliability and evidence of construct validity has also been found [21–23]. The evidence regarding responsiveness is conflicting however; whilst some studies report reasonable responsiveness , others have noted a lack of responsiveness when using the PGI with older people . The SEIQoL, a related individualised quality of life tool, has also accrued conflicting data regarding its responsiveness to change [24, 25]. It is still possible however that such individualised quality of life tools are able to detect change, but that comparator measures used to assess this do not correlate with changes in quality of life. It has been suggested that over time, patients may change the way that they appraise their quality of life, which could mask changes in real quality of life unless the underlying process that the individual uses to assess their quality of life is also measured .
This study has shown that the Patient Generated Index can be completed quickly and successfully when administered by interview to older people in Day Hospital. The tool showed moderate reliability, poor responsiveness to change, and variable external validity. Whilst the PGI may have some use as a baseline measure of quality of life, the lack of responsiveness makes it unsuitable for use as a measure of change in Day Hospital. The pursuit of individualised quality of life information remains a worthy goal however and future work should focus on ways of improving the responsiveness of the tool, whilst ensuring that it does not increase further in complexity.
Roberts H, Khee TS, Philp I: Setting priorities for measures of performance for geriatric medical services. Age Ageing 1994, 23(2):154–157. 10.1093/ageing/23.2.154
Black DA: The modern geriatric day hospital. Hosp Med 2000, 61(8):539–543.
Forster A, Young J, Langhorne P: Medical day hospital care for the elderly versus alternative forms of care. Cochrane Database Syst Rev 2000, (2):CD001730.
Bradley C: Importance of differentiating health status from quality of life. Lancet 2001, 357(9249):7–8. 10.1016/S0140-6736(00)03562-5
Calman KC: Quality of life in cancer patients – an hypothesis. J Med Ethics 1984, 10(3):124–127.
Ruta DA, Garratt AM, Leng M, Russell IT, MacDonald LM: A new approach to the measurement of quality of life. The Patient- Generated Index. Med Care 1994, 32(11):1109–1126. 10.1097/00005650-199411000-00004
Browne JP, O'Boyle CA, McGee HM, Joyce CR, McDonald NJ, O'Malley K, Hiltbrunner B: Individual quality of life in the healthy elderly. Qual Life Res 1994, 3(4):235–244. 10.1007/BF00434897
Seymour DG, Starr JM, Fox HC, Lemmon HA, Deary IJ, Prescott GJ, Whalley LJ: Quality of life and its correlates in octogenarians. Use of the SEIQoL-DW in Wave 5 of the Aberdeen Birth Cohort 1921 Study (ABC1921). Qual Life Res 2008, 17(1):11–20. 10.1007/s11136-007-9291-4
Mountain LA, Campbell SE, Seymour DG, Primrose WR, Whyte MI: Assessment of individual quality of life using the SEIQoL-DW in older medical patients. QJM 2004, 97(8):519–524. 10.1093/qjmed/hch081
Bowling A: Measuring Disease. 2nd edition. Buckingham: Open University Press; 2001:19–21.
Folstein MF, Folstein SE, McHugh PR: "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975, 12(3):189–198. 10.1016/0022-3956(75)90026-6
Ruta DA, Garratt AM, Russell IT: Patient centred assessment of quality of life for patients with four common conditions. Qual Health Care 1999, 8(1):22–29.
Pollard B, Johnston M: Problems with the Sickness Impact Profile: a theoretically based analysis and a proposal for a new method of implementation and scoring. Soc Sci Med 2001, 52: 921–934. 10.1016/S0277-9536(00)00194-5
Zigmond AS, Snaith RP: The hospital anxiety and depression scale. Acta Psychiatr Scand 1983, 67(6):361–370. 10.1111/j.1600-0447.1983.tb09716.x
Camilleri-Brennan J, Ruta DA, Steele RJ: Patient generated index: new instrument for measuring quality of life in patients with rectal cancer. World J Surg 2002, 26(11):1354–1359. 10.1007/s00268-002-6360-2
Symon A, McGreavey J, Picken C: Postnatal quality of life assessment: validation of the Mother-Generated Index. BJOG 2003, 110(9):865–868. 10.1111/j.1471-0528.2003.02030.x
Witham MD, Crighton LJ, McMurdo ME: Using an individualised quality of life measure in older heart failure patients. Int J Cardiol 2007, 116(1):40–45. 10.1016/j.ijcard.2006.03.026
O'Keeffe ST, Lye M, Donnellan C, Carmichael DN: Reproducibility and responsiveness of quality of life assessment and six minute walk test in elderly heart failure patients. Heart 1998, 80(4):377–382.
Crawford JR, Henry JD, Crombie C, Taylor EP: Normative data for the HADS from a large non-clinical sample. Br J Clin Psychol 2001, 40(4):429–434. 10.1348/014466501163904
Guyatt GH, Townsend M, Keller JL, Singer J: Should study subjects see their previous responses: data from a randomized control trial. J Clin Epidemiol 1989, 42(9):913–920. 10.1016/0895-4356(89)90105-4
Martin F, Camfield L, Rodham K, Kliempt P, Ruta D: Twelve years' experience with the Patient Generated Index (PGI) of quality of life: a graded structured review. Qual Life Res 2007, 16(4):705–715. 10.1007/s11136-006-9152-6
Tully M, Cantrill J: The test-retest reliability of the modified Patient Generated Index. J Health Serv Res Policy 2002, 7(2):81–89. 10.1258/1355819021927728
Tully MP, Cantrill JA: The validity of the modified patient generated index – a quantitative and qualitative approach. Qual Life Res 2000, 9(5):509–520. 10.1023/A:1008949809931
Smith HJ, Taylor R, Mitchell A: A comparison of four quality of life instruments in cardiac patients: SF-36, QLI, QLMI, and SEIQoL. Heart 2000, 84(4):390–394. 10.1136/heart.84.4.390
O'Boyle CA, McGee H, Hickey A, O'Malley K, Joyce CR: Individual quality of life in patients undergoing hip replacement. Lancet 1992, 339(8801):1088–1091. 10.1016/0140-6736(92)90673-Q
Schwartz CE, Rapkin BD: Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal. Health Qual Life Outcomes 2004, 2: 16. 10.1186/1477-7525-2-16
With thanks to the patients and staff who participated in this study
Funded by: Health Services Research Committee, Chief Scientist Office, Scottish Executive. Grant number CZG/2/233. The funder played no role in design, execution, analysis or writing of this manuscript.
The authors declare that they have no competing interests.
LW, CAL, METM and MDW produced pilot data and designed the study. RLF collected the data and helped analyse the data. MDW analysed the data and wrote the paper. All authors contributed to the writing and revising of the paper. All authors read and approved the final manuscript