Association of psychological status and patient-reported physical outcome measures in joint arthroplasty: a lack of divergent validity
© Giesinger et al.; licensee BioMed Central Ltd. 2013
Received: 30 November 2012
Accepted: 16 April 2013
Published: 19 April 2013
Patient-reported outcome measures have become a well-recognised part of outcome assessment in orthopaedic surgery. These questionnaires claim to measure joint-specific dimensions like pain, function in activities of daily living, joint awareness or stiffness. Interference of the patient’s psychological status with these orthopaedic questionnaires however may make accurate interpretation difficult.
We recruited 356 patients after unilateral, primary THA or TKA and performed a postal survey including the Brief Symptom Inventory (psychological distress measure), the Catastrophising Scale (from the Coping Strategies Questionnaire), the WOMAC score (Western Ontario and McMaster Universities Osteoarthritis Index) and the Forgotten Joint Score – 12 (FJS-12). Associations between the different questionnaires were determined calculating Pearson correlation coefficients. Two multiple linear regression models were used to investigate the impact of socio-demographic variables, clinical variables and of the psychological scales (BSI and Catastrophising Scale) separately for the WOMAC score and the FJS-12.
WOMAC-Total score showed strong correlation to Catastrophising (r = 0.79), BSI-Somatisation (r = 0.63) and BSI-GSI (r = 0.54). The FJS-12 demonstrated modest to strong correlation with Catastrophising (r = −0.60), BSI-Somatisation (r = −0.49) and the BSI-GSI (Global Severity Index) (r = −0.44). BSI-GSI and Catastrophising explained 54.3% of variance in a multivariate regression model for the WOMAC score. The same two scales explained 30.0% of variance for the FJS-12.
There is a strong relationship between psychological status and orthopaedic outcome. The scale names of orthopaedic outcome measures suggest to measure specific dimensions like pain, stiffness, function or joint awareness. In fact they largely include patient’s psychological status indicating poor divergent validity.
There is widespread recognition that assessment of patient outcome following total hip and total knee arthroplasty (THA and TKA respectively) should employ patient-reported outcome (PRO) measures. These tools allow a more patient-centred view in treatment evaluation [1–3] and advocates suggest that they provide a remarkably sophisticated evaluation of whether a treatment has worked in the (important) sense of whether or not the patient feels better, and how much better . Consequently a number of disease and joint-specific PRO assessment instruments have been developed for use with orthopaedic conditions [5–8]. These outcome questionnaires focus mainly on the patients function in typical activities of daily living (ADLs), pain intensity or joint stiffness. They are often employed in tandem with more generic health outcome instruments such as the SF-36 which in addition to assessing physical health incorporates questions on psycho-social aspects of general health. Some generic tools such as the SF-12 have separate summary scores for physical and mental health. Tools such as this have been shown to display good divergent validity  in that there is very little interaction between physical and mental component questions and thus overall scores. Interestingly in disease-specific scores that do not have specific mental health components, significant correlation of psychological variables and disease specific variables has been demonstrated [10–12]. This interaction is somewhat expected as poor physical outcome and pain after THA/TKA can cause psychological distress and reduce quality of life, or alternatively, poor psychological status can result in worse physical outcome by interfering with the patients’ compliance to treatment  and affect pain coping strategies . Such causal dependency is probably bidirectional with the directions difficult to separate. An alternative explanation though to the overlap in mental and physical health parameters in these assessment tools is a failure of the patient-reported outcome measure to discriminate the overlapping constructs, and thus poor divergent validity [15, 16]. A lack of divergent validity means that interpretability of such scales is limited since the resulting scores blend different constructs. Poor outcome scores can then reflect poor physical outcome, poor psychological status, or both. It is clearly desirable to use a diagnostic tool that separates physical from psychological variables as well as possible if one wishes to assess physical function in isolation.
Thresholds for correlations as indicators of divergent validity are rarely explicitly stated in the literature. However, some studies suggest that correlations below 0.30 indicate divergent validity [17, 18], whereas correlations above 0.40 are considered as indicating convergent validity .
The aim of this study was to evaluate the divergent validity of the WOMAC score and the Forgotten Joint Score, and to investigate correlations with psychological variables after joint arthroplasty.
Patients and methods
All patients that underwent THA or TKA at our institution within the last five years were considered for enrolment in this study and approached for study participation at their follow-up visits in 2008.
Inclusion criteria were: unilateral THA (cemented Stuemer-Weber hip stem, uncemented Fitmore cup, Zimmer) or unilateral TKA (cemented LCS complete, DePuy), primary arthroplasty surgery, no previous THA or TKA surgery.
Sociodemographic and clinical data including sex, age, education, type and location of implant and time since surgery were collected. Patients were sent the questionnaires and an informed consent form via mail. A reminder call was made to those patients who did not send back the questionnaires within eight weeks. If there was no response for another four weeks they were excluded. Reasons for not participating in the study were recorded.
Ethical approval for this study was obtained from the ethics committee of the canton of St Gallen, Switzerland.
Forgotten joint score-12
The Forgotten Joint Score-12 (FJS-12) is a recently published PRO measure to assess joint awareness in hips and knees during various activities of daily living . It consists of 12 questions and is scored using a 5-point Likert response format with the raw scores transformed onto a 0–100 point scale. High scores indicate good outcome. The FJS has been shown to have a low ceiling effect and discriminates well between good, very good and excellent outcome after THA and TKA. It has shown high internal consistency (Cronbach’s Alpha 0.95) and discriminates well in known group comparisons .
Western Ontario and McMaster Universities Osteoarthritis Index
The Western Ontario and McMaster Universities (WOMAC) Osteoarthritis Index is a widely used outcome measure in patients with lower limb osteoarthritis (OA) . It consists of 24 questions covering three dimensions: pain (five questions), stiffness (two questions) and function (17 questions). Scale scores are derived from adding up the item scores. High scores indicate poor outcome. The WOMAC OA index has been extensively tested for validity, reliability, feasibility and responsiveness for measuring changes after different OA interventions [5, 20–22].
Brief symptom inventory
The Brief Symptom Inventory (BSI)  is a psychological self-report symptom scale developed as a short-form version of the SCL-90-R . It is widely used in various medical fields to assess current psychological status and distress. The 53 items are grouped in nine symptom scales (somatisation, obsessive-compulsive behaviour, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism) and three global indices, Global Severity Index (GSI) as a global distress measure, Positive Symptom Distress Index (PSDI), and Positive Symptom Total (PST). Scale scores are derived from mean item scores. High scores indicate high psychological symptom burden.
The catastrophising scale is part of the Coping Strategies Questionnaire developed by Rosenstiel and Keefe . It comprises six items assessing catastrophising as a pain-related coping strategy characterised by a feeling of being overstrained and a pessimistic future perspective. The scale scores are derived from adding up the items. A high score indicates poor coping.
Sample characteristics are presented as percentages or as means with standard deviations and ranges. For determining associations between the administered scales (WOMAC score, FJS-12, BSI, Catastrophising scale) Pearson-correlation coefficients were calculated. Two multiple linear regression models were used to investigate the impact of sociodemographic and clinical variables and of the psychological scales (BSI and Catastrophising scale) separately for the WOMAC and for the FJS-12 score. In these models adjusted R-Squared (R2) indicates the proportion of variance explained by the independent variables (predictors) in the model. Variables having a significant association with the WOMAC or the FJS-12 in univariate analyis were considered for inclusion into the multivariate regression model if p < 0.05. In a first block of predictors, the patient characteristics sex, education, and location were included. In a second block of predictors the psychological scales (BSI scales and the Catastrophising scale) were included using a forward selection procedure.
Descriptive statistics for clinical and socio-demographic variables (n = 243)
Time since surgery (months)
Correlations between FJS-12, WOMAC, BSI and the catastrophising-scale
Correlations between WOMAC, FJS-12, Catastrophising and BSI
BSI Interpersonal sensitivity
BSI Phobic anxiety
BSI Paranoid ideation
Multivariate analysis of the FJS-12 and the WOMAC score
Sex, education and location of implant (hip or knee) have previously been shown to impact on the FJS-12 and WOMAC-Total score . These variables were included as predictors in two separate linear regression models, with the WOMAC total score and FJS-12 as the dependant variables. The global distress scale of the BSI (BSI-GSI) as well as BSI-Somatisation and the Catastrophising scale were included as predictors in both models.
Multiple linear regression model for FJS-12 and WOMAC-Total
Adjusted R 2
Change adjusted R 2
Adjusted R 2
Change adjusted R 2
This study investigated the associations between psychological parameters and physical outcome assessed by two PRO instruments, the WOMAC score and the FJS-12. We found high correlations between disease-specific outcome measures and several of the assessed psychological domains. Multivariate regression showed that catastrophising, psychological distress and somatisation explained almost 60% variance of the WOMAC score beyond the known covariates of sex, implant location and education. We found the same predictor set for the FJS-12, however, psychological parameters accounted only for half the variance seen in the WOMAC score.
Our findings indicate a significant lack of divergent validity of the WOMAC score and, to a lesser extent, of the FJS-12. The variance proportions estimated with help of the regression model suggest a substantial overlap between the orthopaedic and psychological scales. The lack of divergent validity becomes even more evident when opposing the high correlations between the WOMAC subscales themselves (above 0.80) and the correlations of the WOMAC total score with the psychological scores (up to 0.79).
This significant overlap with psychological status is not reflected in the WOMAC scales’ names (pain, stiffness, function) which somewhat misleadingly suggest to just measure physical, joint-related characteristics. This is also true for the FJS-12 which refers to joint awareness. However, the term joint awareness seems more closely related to psychological aspects.
We also found that location of joint arthroplasty (hip or knee) explained less than 5% of variance of both FJS-12 and the WOMAC score. This is interesting as it is well accepted that outcome differs between total hip and total knee arthroplasty populations [26, 27]. In contrast, the psychological scales exceeded these proportions by a factor of 10 (for both FJS-12 and WOMAC). Thus, our data indicate a stronger association between psychological factors and joint-related outcomes than that between outcome and the type of joint replaced.
Our findings compare well to other results from literature. Escobar et al.  investigated the association between WOMAC scores and the different subscales of the SF-36. They showed that both psycho-social and physical SF-36 scales correlated to the WOMAC score in a similar way. The WOMAC function subscale demonstrated the same correlation with both SF-36 social and physical function scores. WOMAC stiffness was equally correlated with SF-36 role-physical function score and mental health score. Similarly Wolfe  highlighted that divergent validity of the WOMAC may be compromised by factors such as fatigue, symptom counts, depression, and low back pain.
The strong correlation between physical and psychological scales found here and in other studies [28–30] may partially be explained by causal interdepencies that have been suggested by several longitudinal studies.
Sharma et al.  demonstrated that mental health measured with the SF-36 predicted subsequent improvement in physical function in TKA, results in line with Brander et al. , who showed that preoperative depression substantially influences Knee Society Rating Scale function scores five years post-operatively. In contrast, Lingard et al.  found (in a large prospective observational study) that although psychological distress decreased post-operatively, pre-operative levels of distress were not related to post-operative improvement (change in pain and function).
Lopez-Olivo et al.  found a strong correlation between pre-operative psychological status and post-operative physical function at 6 months. Education, coping style and locus of control over health at baseline explained 22% of variance in WOMAC pain at follow-up. A similar predictor-set explained 19% of the WOMAC function scale and 36% of the total score of the Knee Society Rating Scale.
Our study was based on a cross-sectional design which is reasonable for the investigation of divergent validity. However, it does not allow for causal interpretation of the associations between orthopaedic outcomes and psychological variables. A limitation is the limited number of predictors in our model that left a large proportion of unexplained variance. Further interesting predictors that may be of future research interest include patient activity level, social support, cognitive function, range of motion and joint stability.
A particular strength of this study is the use of a comprehensive and detailed assessment of psychological status (BSI and the Catastrophising Scale from the Coping Strategies Questionnaire). These scales are more differentiated and comprehensive than other tools such as the SF-36 which has previously been employed to assess psychosocial characteristics of arthroplasty populations.
We found a substantial overlap between physical and psychological patient-reported symptoms in an arthroplasty population, i.e. orthopaedic PRO measures were strongly associated with psychological PRO measures indicating poor divergent validity.Whereas this may also reflect existing causal dependencies, it impairs valid measurement of orthopaedic outcome. Divergent validity is an important psychometric characteristic of PRO instruments that is required to guarantee accurate assessment of specific orthopaedic outcomes.
Problematically, the category names of the orthopaedic outcome scales suggest measurement of specific constructs such as pain, stiffness, function or joint awareness but they appear to be strongly associated with patients’ psychological status. Our findings suggest that the names of certain orthopaedic scales do not adequately reflect the constructs assessed with these scales.
The work of Johannes M. Giesinger was funded by a grant from the Austrian Science Fund (FWF L502).
- Wright RW: Knee injury outcomes measures. J Am Acad Orthop Surg 2009, 17:31–39.PubMed
- Suk M, Norvell DC, Hanson B, Dettori JR, Helfet D: Evidence-based orthopaedic surgery: what is evidence without the outcomes? J Am Acad Orthop Surg 2008, 16:123–129.PubMed
- Pollard B, Johnston M, Dixon D: Theoretical framework and methodological development of common subjective health outcome measures in osteoarthritis: a critical review. Health Qual Life Outcomes 2007, 5:14.PubMedView ArticlePubMed Central
- Timmins N: NHS goes to the PROMS. BMJ 2008, 336:1464–1465.PubMedView ArticlePubMed Central
- Bellamy N, Buchanan WW, Goldsmith CH, Campbell J, Stitt LW: Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee. J Rheumatol 1988, 15:1833–1840.PubMed
- Behrend H, Giesinger K, Giesinger JM, Kuster MS: The “Forgotten Joint” as the Ultimate Goal in Joint Arthroplasty Validation of a New Patient-Reported Outcome Measure. J Arthroplasty 2012, 27:430–436.PubMedView Article
- Dawson J, Fitzpatrick R, Carr A, Murray D: Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg Br 1996, 78:185–190.PubMed
- Dawson J, Fitzpatrick R, Murray D, Carr A: Questionnaire on the perceptions of patients about total knee replacement. J Bone Joint Surg Br 1998, 80:63–69.PubMedView Article
- Ware J, Kosinski M, Keller S: SF-12: How to Score the Sf-12 Physical and Mental Health Summary Scales. 3rd edition. Lincoln, RI: QualityMetric Incorporated; 1998.
- Axford J, Butt A, Heron C, Hammond J, Morgan J, Alavi A, Bolton J, Bland M: Prevalence of anxiety and depression in osteoarthritis: use of the Hospital Anxiety and Depression Scale as a screening tool. Clin Rheumatol 2010, 29:1277–1283.PubMedView Article
- Lavernia CJ, Alcerro JC, Brooks LG, Rossi MD: Mental health and outcomes in primary total joint arthroplasty. J Arthroplasty 2012,27(7):1276–1282.PubMedView Article
- Lopez-Olivo MA, Landon GC, Siff SJ, Edelstein D, Pak C, Kallen MA, Stanley M, Zhang H, Robinson KC, Suarez-Almazor ME: Psychosocial determinants of outcomes in knee replacement. Ann Rheum Dis 2011, 70:1775–1781.PubMedView Article
- Lenze EJ, Munin MC, Dew MA, Rogers JC, Seligman K, Mulsant BH, Reynolds CF 3rd: Adverse effects of depression and cognitive impairment on rehabilitation participation and recovery from hip fracture. Int J Geriatr Psychiatry 2004, 19:472–478.PubMedView Article
- Somers TJ, Keefe FJ, Pells JJ, Dixon KE, Waters SJ, Riordan PA, Blumenthal JA, McKee DC, LaCaille L, Tucker JM: Pain catastrophizing and pain-related fear in osteoarthritis patients: relationships to pain and disability. J Pain Symptom Manage 2009, 37:863–872.PubMedView ArticlePubMed Central
- Escobar A, Quintana JM, Bilbao A, Azkarate J, Guenaga JI: Validation of the Spanish version of the WOMAC questionnaire for patients with hip or knee osteoarthritis. Western Ontario and McMaster Universities Osteoarthritis Index. Clin Rheumatol 2002, 21:466–471.PubMedView Article
- Wolfe F: Determinants of WOMAC function, pain and stiffness scores: evidence for the role of low back pain, symptom counts, fatigue and depression in osteoarthritis, rheumatoid arthritis and fibromyalgia. Rheumatology (Oxford) 1999, 38:355–361.View Article
- de Groot IB, Favejee MM, Reijman M, Verhaar JA, Terwee CB: The Dutch version of the Knee Injury and Osteoarthritis Outcome Score: a validation study. Health Qual Life Outcomes 2008, 6:16.PubMedView ArticlePubMed Central
- Innes E, Straker L: Validity of work-related assessments. Work 1999, 13:125–152.PubMed
- Fayers P, Machin D: Quality of Life: The Assessment, Analysis and Interpretation of Patient-reported Outcomes. 2nd edition. Chichester, UK: John Wiley & Sons; 2007.View Article
- Impellizzeri FM, Mannion AF, Leunig M, Bizzini M, Naal FD: Comparison of the reliability, responsiveness, and construct validity of 4 different questionnaires for evaluating outcomes after total knee arthroplasty. J Arthroplasty 2011, 26:861–869.PubMedView Article
- Wolfe F, Kong SX: Rasch analysis of the Western Ontario MacMaster questionnaire (WOMAC) in 2205 patients with osteoarthritis, rheumatoid arthritis, and fibromyalgia. Ann Rheum Dis 1999, 58:563–568.PubMedView ArticlePubMed Central
- Terwee CB, Roorda LD, Knol DL, De Boer MR, De Vet HC: Linking measurement error to minimal important change of patient-reported outcomes. J Clin Epidemiol 2009, 62:1062–1067.PubMedView Article
- Derogatis LR: BSI Brief Symptom Inventory: Administration, Scoring and Procedures Manual (4th edition). National Computer Systems: Minneapolis, MN; 1993.
- Derogatis LR: SCL-90-R Revised Manual. Baltimore: John Hopkins School of Medicine; 1983.
- Rosenstiel AK, Keefe FJ: The use of coping strategies in chronic low back pain patients: relationship to patient characteristics and current adjustment. Pain 1983, 17:33–44.PubMedView Article
- Bourne RB, Chesworth B, Davis A, Mahomed N, Charron K: Comparing patient outcomes after THA and TKA: is there a difference? Clin Orthop Relat Res 2010, 468:542–546.PubMedView ArticlePubMed Central
- Hamilton D, Henderson GR, Gaston P, Macdonald D, Howie C, Simpson AH: Comparative outcomes of total hip and knee arthroplasty: a prospective cohort study. Postgrad Med J 2012, 88:627–631.PubMedView Article
- Pallant JF, Keenan AM, Misajon R, Conaghan PG, Tennant A: Measuring the impact and distress of osteoarthritis from the patients’ perspective. Health Qual Life Outcomes 2009, 7:37.PubMedView ArticlePubMed Central
- van Baar ME, Dekker J, Lemmens JA, Oostendorp RA, Bijlsma JW: Pain and disability in patients with osteoarthritis of hip or knee: the relationship with articular, kinesiological, and psychological characteristics. J Rheumatol 1998, 25:125–133.PubMed
- Salaffi F, Cavalieri F, Nolli M, Ferraccioli G: Analysis of disability in knee osteoarthritis. Relationship with age and psychological variables but not with radiographic score. J Rheumatol 1991, 18:1581–1586.PubMed
- Sharma L, Cahue S, Song J, Hayes K, Pai YC, Dunlop D: Physical functioning over three years in knee osteoarthritis: role of psychosocial, local mechanical, and neuromuscular factors. Arthritis Rheum 2003, 48:3359–3370.PubMedView Article
- Brander V, Gondek S, Martin E, Stulberg SD: Pain and depression influence outcome 5 years after knee replacement surgery. Clin Orthop Relat Res 2007, 464:21–26.PubMed
- Lingard EA, Riddle DL: Impact of psychological distress on pain and function following knee arthroplasty. J Bone Joint Surg Am 2007, 89:1161–1169.PubMedView Article
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.