- Short report
- Open Access
What is the relevance of quality of life assessment for patients with attention impairment?
Health and Quality of Life Outcomesvolume 11, Article number: 70 (2013)
Attention disturbances are frequently observed in multiple sclerosis (MS) patients. The aim of this study was to provide new evidence regarding the suitability of using self-reported QoL information in this impaired population by exploring the construct validity, reliability, and external validity of a MS-specific quality of life (QoL) instrument.
Design: cross-sectional study. Inclusion criteria: MS patients of any disease subtype. Data collection: sociodemographic (age, gender, marital status, education level, and occupational activity) and clinical data (Expanded Disability Status Scale, disease duration); QoL (MusiQoL and SF36); and attention performance (Wechsler Memory Scale and PASAT). According to the French norms, non-impaired and impaired populations were defined. For each population, suitability indices were provided to quantify how the structures matched with the initial structure of the reference population assessed in the validation study.
One hundred and twenty-four consecutive patients were enrolled. The factor analysis performed in the impaired populations showed that the questionnaire structure adequately matched the initial structure of the MusiQoL. The unidimensionality of the dimensions was preserved, and the internal/external validity indices were close to those of the reference population.
Our study suggests that attention impairment dysfunction did not compromise the reliability and validity of the self-reported QoL questionnaires.
The use of self-reported outcomes in subjects with cognitive dysfunction is of particular concern . The main argument against using self-reported quality of life (QoL) information from patients with cognitive dysfunction is that QoL instruments were not developed for use with cognitively impaired individuals. Some authors argue that cognitively impaired individuals suffering from multiple sclerosis (MS) are unable to produce valid QoL measures [2, 3], whereas other studies have suggested that cognitive decline in these patients does not compromise the reliability/validity of self-reported health measures [4–7]. In these previous studies, the assessment of cognitive function focused on aspects of memory [5, 7] and executive functions .
Attention plays a role in cognitive processing and learning. Attention as a basic cognitive function represents an essential part of conscious perception and higher order cognitive functions and plays an important role in normal functioning. Attention is a broad term that is used to designate the processes that mediate the appropriate allocation of cognitive resources to relevant stimuli, such as objects, locations or moments. The limitation of this capacity may result from deficits in the automatic or controlled processing of information, as well as the incapacity to store and manipulate temporary information, the function known as working memory. In MS patients, cognitive performance is globally affected and impacts divided attention, focused attention, sustained attention, attentional flexibility, and response speed [8, 9]. Despite the high occurrence of working memory and attention disturbances in MS patients [10, 11], even during the early stages of the disease , no study has specifically examined the extent to which MS patients with attention dysfunction can validly self-report their QoL. Herein, we provide new evidence regarding the suitability of using self-reported QoL information in this impaired population. The aim of this study was to explore the construct validity, reliability and external validity of a MS-specific QoL instrument in MS subjects with or without attention impairment.
This study relied on a cross-sectional design and was performed in the neurology department of a public French academic teaching hospital (Marseille, France). The inclusion criteria were as follows: a MS diagnosis according to the McDonald criteria , age ≥ 18 years, any subtype of MS, outpatient status, no neurological disease (other than MS), no history of severe mental illness (except depressive disorder), no dementia (Mini Mental State Examination score < 24), no history of alcohol/drug abuse, and native French-speaking. The French Ethics Committee approved the study (Comité Consultatif de Protection des Personnes dans la Recherche Biomédicale, Marseille 2, France) and patients gave their written consent to participate. Sociodemographic (age, gender, marital status, education level, and occupational activity) and clinical (MS subtype and disease duration) data were recorded for each patient. MS disability was assessed using the Expanded Disability Status Scale (EDSS).
QoL was assessed using the French version of the Multiple Sclerosis International Quality of Life questionnaire (MusiQoL) and the Short Form 36 (SF36). The MusiQoL is a well-validated MS-specific questionnaire that describes the nine dimensions and yields a global index score: activity of daily living (ADL), psychological well-being (PWB), symptoms (SPT), relationships with friends (RFr), relationships with family (RFa), relationships with health care system (RHCS), sentimental and sexual life (SSL), coping (COP), and rejection (REJ). The SF36 is a generic questionnaire  describing eight subscales: physical function (PF), social function (SF), role physical (RP), role emotional (RE), mental health (MH), vitality (V), bodily pain (BP), and general health (GH). Two composite scores (physical and mental, PCS-SF36 and MCS-SF36) were also calculated.
Attention performance was measured using two tests: the French version of the Paced Auditory Serial Addition Test (PASAT)  and the attention/concentration subscale of the French version of the Wechsler Memory Scale (attention_WMS) . The attention_WMS score was calculated using 3 subtests (mental control, digit span, and visual memory). By comparing with French normative values according to age group , a patient was considered cognitively impaired if their score was more than 1 standard deviation (SD) below the normative value. The PASAT score was generated using the PASAT-3 s version of the test (number of correct responses). By comparing with French normative values according to sex, age, and educational level , a patient was considered to be cognitively impaired if their PASAT score was more than 1 SD below the normative value.
Statistical analyses were performed on the populations defined above using the same procedure reported in the initial validation publication (reference population) . For each group, psychometric properties were compared to those reported from the reference population. The structures of the MusiQoL were explored in the non-impaired and impaired populations using principal component factor analyses with varimax rotation [19, 20] to determine how these structures matched with the initial structure of the MusiQoL. For each population, the proportion of factors identified from the initial nine-factor structure of MusiQoL and the proportion of items projected to their initial dimension were retrieved.
The multidimensional structure (construct validity) was verified using the multi-trait/multi-item analysis program . Internal structural validity was assessed by calculating item-dimension correlations. Item internal consistency (IIC) was calculated by correlating each item with its scale, and item discriminant validity (IDV) was assessed by determining the extent to which items correlated with the dimension they were hypothesized to represent compared to correlations with other dimensions. Floor and ceiling effects were determined to assess the distribution of the responses. For each dimension, internal consistency reliability was evaluated using Cronbach’s alpha coefficient .
The unidimensionality of each dimension was calculated by computing item goodness-of-fit statistics (INFIT) issued from Rasch analyses . INFIT values ranging from 0.7 to 1.2 ensure that all scale items tend to measure the same concept.
To assess external validity, Spearman’s correlation coefficients were used to determine relationships between the MusiQoL and SF36 dimensions in each group, and the associations between MusiQoL dimension scores and sociodemographic and clinical features were reported as in the validation study. For qualitative variables, the mean dimension scores of the MusiQoL were compared across patient groups (e.g., gender, educational level, marital status, and occupational status) using one-way analysis of variance. Quantitative variables (e.g., age, EDSS score, and MS duration) were analyzed using Spearman’s correlation coefficients. Acceptability was assessed by calculating the percentage of missing data per dimension. Data analyses were performed using SPSS 11.0, MAP-R, LISREL and WINSTEP software.
To quantify how each of the 4 structures matched with the initial structure (reference structure), suitability indices were calculated in accordance with a previous study . Decision rules were established by experts in QoL and used to define satisfactory properties according to appropriate standards [19, 20]. The means of different proportions were calculated to produce the suitability index of the ‘construct validity’ and the suitability index of the ‘external validity’.
A total of 124 patients participated in the study. The mean age was 45 years (SD 11), 71 (57,3%) of the patients were women, and 58 (47,2%) had more than 12 years of education. MS subtypes included 61 (49,2%) secondary progressive, 36 (29,0%) relapsing-remitting, 20 (16,1%) primary progressive, and 7 (5,6%) clinically isolated syndromes. The EDSS median was 4.75 (minimum 1.00, maximum 8.00), and the median disease duration was 9.86 years (minimum 0, maximum 31). After comparison to the French normative values, 75 non-impaired and 39 impaired individuals were identified based on the attention_WMS, and 32 non-impaired and 73 impaired individuals were identified based on the PASAT.
The 9-factor structure of the MusiQoL was clearly detected in the PASAT-impaired population, and 8 factors were detected in the attention_WMS-impaired population (the REJ dimension was not detected). The percentage of items well-projected to their initial dimension was higher in the impaired populations (range: 80,6 to 100%) compared to the non-impaired populations (Additional files 1 and 2). The proportion of dimensions with IIC and the proportion of dimensions with IDV that were greater than 0.2 from the reference dimension were moderately satisfactory. In contrast, the correlation of each item with its associated dimension was higher than with the others in 6 dimensions in the PASAT-impaired population and 7 dimensions in the attention_WMS-impaired populations. Cronbach’s alpha coefficients were satisfactory in at least 6 of 9 dimensions in the two impaired populations. The INFIT statistics were satisfactory in the attention_WMS-impaired population, but less satisfactory for the PASAT-impaired group. The proportions of dimensions with floor and ceiling effects less than 10% from the reference are described in Table 1. Additional details are provided in Table 2. The suitability indices of construct validity in the impaired populations were populations were detailed in Table 1.
Eight dimensions satisfied the 3 conditions defining the suitability of relationships between MusiQoL and SF36 scores, both for the PASAT- and attention_WMS-impaired populations (Tables 3 and 4). The suitability indices of age, disease duration, EDSS, gender, and marital status were satisfactory. Associations between the MusiQoL scores and educational level and occupational status were less satisfactory (Additional files 3 and 4). In summary, the suitability indices of external validity in the non-impaired populations were 83% (Table 1).
As reported in previous studies [4, 6, 7, 24], our results support the conclusion that cognitively impaired MS patients, as defined by attention dysfunction, reliably and consistently answer the MusiQoL questionnaire. The suitability indices of the impaired populations were close to those of the non-impaired populations, and they could be considered completely acceptable considering the small sample size of some of the defined populations. The assessment of QoL in MS patients could be more widely used without concern over the adequacy of this approach for cognitively impaired patients.
Some strengths and limitations of the current study should be considered:
i) Considering only one type of dysfunction may not have accurately reflected global cognitive functioning. It has been well documented that it is unusual to observe one deficit in isolation in clinical practice. However, there is little consensus on the definition of ‘global’ cognitive dysfunction . Despite this limitation, the findings of the present study support the relevance of self-reported quality of life assessments for patients with cognitive disorders. Future studies should provide similar explorations in MS populations with cognitive dysfunction defined with other definitions integrating combination of different composites.
ii) We did not consider factors previously associated to cognitive performance, such as depression , fatigue , and MS medications . However, the aim of this study was to provide evidence supporting the conclusion that cognitively impaired MS patients reliably answer a self-reported QoL questionnaire regardless of the presence or absence of other factors that could have influenced their performance.
iii) The suitability indices used to define satisfactory properties relied on debatable decision rules and non-standardized calculation methods. Analyses of sensitivity should be performed.
iv) Replication of these findings in larger groups of patients is required.
v) Similar studies should be performed in other populations suffering from cognitive impairment, such as severe mental illness and elderly individuals, to determine whether these findings are specific to patients with MS.
Riemsma RP, Forbes CA, Glanville JM, Eastwood AJ, Kleijnen J: General health status measures for people with cognitive impairment: learning disability and acquired brain injury. Health Technol Assess 2001,5(6):1–100.
Goverover Y, Chiaravalloti N, DeLuca J: The relationship between self-awareness of neurobehavioral symptoms, cognitive functioning, and emotional symptoms in multiple sclerosis. Mult Scler 2005,11(2):203–212. 10.1191/1352458505ms1153oa
Benedict RH, Cox D, Thompson LL, Foley F, Weinstock-Guttman B, Munschauer F: Reliable screening for neuropsychological impairment in multiple sclerosis. Mult Scler 2004,10(6):675–678. 10.1191/1352458504ms1098oa
Gold SM, Schulz H, Monch A, Schulz KH, Heesen C: Cognitive impairment in multiple sclerosis does not affect reliability and validity of self-report health measures. Mult Scler 2003,9(4):404–410. 10.1191/1352458503ms927oa
Marrie RA, Miller DM, Chelune GJ, Cohen JA: Validity and reliability of the MSQLI in cognitively impaired patients with multiple sclerosis. Mult Scler 2003,9(6):621–626. 10.1191/1352458503ms971oa
Baumstarck K, Pelletier J, Aghababian V, Reuter F, Klemina I, Berbis J, Loundou A, Auquier P: Is the concept of quality of life relevant for multiple sclerosis patients with cognitive impairment? Preliminary results of a cross-sectional study. PLoS One 2012,7(1):e30627. 10.1371/journal.pone.0030627
Baumstarck K, Reuter F, Boucekine M, Aghababian V, Klemina I, Loundou A, Pelletier J, Auquier P: Relevance of quality of life assessment for multiple sclerosis patients with memory impairment. PLoS One 2012,7(12):e50056. 10.1371/journal.pone.0050056
De Sonneville LM, Boringa JB, Reuling IE, Lazeron RH, Ader HJ, Polman CH: Information processing characteristics in subtypes of multiple sclerosis. Neuropsychologia 2002,40(11):1751–1765. 10.1016/S0028-3932(02)00041-6
McCarthy M, Beaumont JG, Thompson R, Peacock S: Modality-specific aspects of sustained and divided attentional performance in multiple sclerosis. Arch Clin Neuropsychol 2005,20(6):705–718. 10.1016/j.acn.2005.04.007
Krupp LB, Sliwinski M, Masur DM, Friedberg F, Coyle PK: Cognitive functioning and depression in patients with chronic fatigue syndrome and multiple sclerosis. Arch Neurol 1994,51(7):705–710. 10.1001/archneur.1994.00540190089021
Zakzanis KK: Distinct neurocognitive profiles in multiple sclerosis subtypes. Arch Clin Neuropsychol 2000,15(2):115–136.
Feuillet L, Reuter F, Audoin B, Malikova I, Barrau K, Cherif AA, Pelletier J: Early cognitive impairment in patients with clinically isolated syndrome suggestive of multiple sclerosis. Mult Scler 2007,13(1):124–127. 10.1177/1352458506071196
Polman CH, Wolinsky JS, Reingold SC: Multiple sclerosis diagnostic criteria: three years later. Mult Scler 2005,11(1):5–12. 10.1191/1352458505ms1135oa
Leplege A, Ecosse E, Verdier A, Perneger TV: The French SF-36 health survey: translation, cultural adaptation and preliminary psychometric evaluation. J Clin Epidemiol 1998,51(11):1013–1023. 10.1016/S0895-4356(98)00093-6
Gronwall DM: Paced auditory serial-addition task: a measure of recovery from concussion. Percept Mot Skills 1977,44(2):367–373. 10.2466/pms.19220.127.116.117
Wechsler D: WMS-R Echelle clinique de Mémoire de Wechsler-Révisée. Paris: Les Editions du Centre de Psychologie Appliquée; 1991.
Reuter F, Baumstarck-Barrau K, Loundou A, Pelletier J, Auquier P: Paced auditory serial addition test: normative data in a French population. Rev Neurol (Paris) 2010,166(11):944–947. 10.1016/j.neurol.2010.01.018
Simeoni MC, Auquier P, Fernandez O, Flachenecker P, Stecchi S, Constantinescu C, Idiman E, Boyko A, Beiske A, Vollmer T: Validation of the multiple sclerosis international quality of life questionnaire. Mult Scler 2008,14(2):219–230.
Juniper EF, Guyatt GH, Mesbah M, Ravaud P: Quality of life and pharmacoeconomics in clinical trials. Philadelphia: Lippincott-Raven; 1996.
Nunnaly JC, Bernstein IC: Psychometric theory. New York: Mc Graw-Hill; 1994.
Ware JE, Harris WJ, Gandek B, Rogers BW: MAP-R for windows: Multitrait-multi-item analysis program - revised User’s guide. Boston: Health Assessment Lab; 1997.
Cronbach LJ: Coefficient alpha and the internal structure of tests. Psychometrika 1951, 16: 297–334. 10.1007/BF02310555
Wright BD, Stone MH: Best test design: Rasch measurement. Chicago: Mesa press; 1979.
Marrie RA, Goldman M: Validity of performance scales for disability assessment in multiple sclerosis. Mult Scler 2007,13(9):1176–1182. 10.1177/1352458507078388
Achiron A, Barak Y: Cognitive changes in early MS: a call for a common framework. J Neurol Sci 2006,245(1–2):47–51.
Arnett PA, Randolph JJ: Longitudinal course of depression symptoms in multiple sclerosis. J Neurol Neurosurg Psychiatry 2006,77(5):606–610. 10.1136/jnnp.2004.047712
Debouverie M, Pittion-Vouyovitch S, Brissart H, Guillemin F: Physical dimension of fatigue correlated with disability change over time in patients with multiple sclerosis. J Neurol 2008,255(5):633–636. 10.1007/s00415-008-0761-6
Oken BS, Flegal K, Zajdel D, Kishiyama SS, Lovera J, Bagert B, Bourdette DN: Cognition and fatigue in multiple sclerosis: potential effects of medications with central nervous system activity. J Rehabil Res Dev 2006,43(1):83–90. 10.1682/JRRD.2004.11.0148
The authors are grateful to all the patients for their participation in the study. The authors thank Jessica Fernandez for her consequent contribution for the statistical analyses.
This work was supported by institutional grants from the French 2004 Programme Hospitalier Recherche Clinique and the French 2009 Institut de Recherche en Santé Publique (Concepts, Usages et Déterminants en Qualité de Vie, CUD-QV). The sponsor was represented by Assistance Publique, Hôpitaux de Marseille (Marseille, France). The funders had no role in study design, data collection and analysis, decision to publish, or manuscript preparation.
The authors declare that they have no competing interests.
Conception and design: PA, JP. Study coordination: PA, JP, KB. Inclusion and clinical data collection: IK, JP. Acquisition of cognitive data: FR, VA. Analysis of data: KB, MB, AL. Interpretation of data: KB, PA, JP, FR, VA. Drafting and writing of manuscript: KB, PA. Revision of manuscript: PA, JP, FR, VA, IK. All authors read and approved the final manuscript.