Skip to main content

A Mokken analysis of the literacy in musculoskeletal problems questionnaire

Abstract

Background

Limited health literacy is known to impact on medication adherence, hospital readmission and potentially poorer health outcomes. The literature on the health literacy of those with musculoskeletal conditions suggests greater functional limitations and increased pain levels. There are a number of measures of health literacy. One that specifically relates to musculoskeletal complaints is the Literacy in Musculoskeletal Problems (LiMP) questionnaire. The LiMP contains 9 multiple choice items that cover anatomy, musculoskeletal conditions and the diagnosis of musculoskeletal complaints. The aim of the study was to evaluate the dimensionality and internal structure of the LiMP in patients attending for osteopathy care at a student-led clinic, as a potential measure of musculoskeletal health literacy.

Method

Three hundred and sixty-one (n = 361) new patients attending the Victoria University Osteopathy Clinic completed the LiMP and a demographic and health information questionnaire prior to their initial consultation. Mokken scale analysis, a nonparametric item response theory approach, was used to evaluate the dimensionality and structure of the LiMP in this population, to ascertain whether the questionnaire was measuring a single latent construct – musculoskeletal health literacy. McDonald’s omega and Cronbach’s alpha were calculated as the reliability estimations. The relationship between the LiMP and a single item screen of health literacy was also undertaken.

Results

The 9 items on the LiMP did not form a Mokken scale and the reliability estimations were below an acceptable level (alpha and omega <0.45). LiMP items 5 and 8 were more likely to be answered correctly by those with higher health literacy (p < 0.05), however the effect sizes were small (<0.20).

Conclusion

Calculation of a total score for the LiMP, as advocated by the original authors, is not supported based on data in the present study. Further research is required to explore the relationship of the LiMP items to demographic and clinical data, and to other broader measures of health literacy. Further research may also develop a health literacy measure that is specific to patients seeking manual therapy care for musculoskeletal complaints.

Introduction

The burden of musculoskeletal disorders in society is substantial [1,2,3] and one strategy to reduce this burden may be addressing health literacy (HL). Although HL has received considerable attention in the health professions literature [4, 5] there are few examples in the manual therapies. A systematic review by Loke et al. [6] suggests between 7 and 42% of those with chronic musculoskeletal complaints demonstrate low health literacy, with estimates varying depending upon the population, and type of measure utilised [7]. Vaughan et al. [8] evaluated the HL of patient’s attending an osteopathy, on-campus student-led clinic using a single item screening question, which identified 10% of patients attending this clinic had below adequate HL. Those patients who did not speak English at home, those with lower education levels, and those who were less satisfied with their life demonstrated lower levels of HL.

Although a range of measures exist to evaluate HL [9,10,11] they can be time-consuming to complete, or require administration by a research or practice assistant. Further, there are no specific questionnaires that exist in the literature to evaluate the HL of patient’s attending for manual therapy care. Rosenbaum et al. [12] state that “…musculoskeletal health literacy is thought to require a more sophisticated set of skills than those deemed crucial for general health literacy” (p. 608). This assertion is possibly supported by Briggs et al. [13] who identified that a cohort of chronic low back pain (LBP) patients had “...difficulties in seeking, understanding and utilising LBP information” (p. 275) despite having adequate HL scores using the Short Test of Functional Health Literacy in Adults.

The Literacy in Musculoskeletal Problems (LiMP) questionnaire was developed to evaluate the “...competencies that are integral to making informed decisions regarding musculoskeletal health” (p. 610) [12]. The LiMP consists of 9 items across three domains: anatomy and terminology, musculoskeletal conditions, and diagnosis and treatment. The questionnaire takes 5 min to complete. Patients are asked to select a response for each item (5 response options for items 1–8 and 4 response options for item 9) and only one response is permissible. The response to each item is scored as correct (1) or incorrect (0) and the score totalled for a possible score of 9. LiMP scores below 6 are reported to indicate limited musculoskeletal HL and “…scores of ≥6 can effectively rule out both limited musculoskeletal literacy and general health literacy” (p. 191) [14]. The LiMP has been used to evaluate the musculoskeletal HL of those with carpal tunnel syndrome [15], foot and ankle complaints [16], those presenting to an emergency department [14, 17] and an orthopaedic outpatients clinic [18]. It has a Flesch-Kincaid grade level of 4.2 (able to be read by those with a 4th grade education or above) [17].

There is limited information about the dimensionality and internal structure of the LiMP, both a part of the argument for the validity of the questionnaire. Initial analysis identified a moderate positive correlation (0.41) between Newest Vital Sign [19] scores and the LiMP, and a Cronbach’s α of 0.59 [17]. Beyond Cronbach’s alpha, there is limited information to provide support for the internal structure of the LiMP. The aim of the present study is to evaluate the dimensionality and internal structure of the LiMP to ascertain if it measures a single latent construct (musculoskeletal health literacy) in a patient population attending a student-led osteopathy clinic for the management of musculoskeletal complaints.

Methods

This study forms part of a larger investigation into the demographics and health screening of patient’s attending the VU Osteopathy Clinic, a student-led osteopathy teaching clinic at Victoria University (VU) in Melbourne, Australia. The VU Human Research Ethics Committee approved the study (HRE15–005) and the study was conducted between March 1 and June 30, 2016.

Participants

All new patients attending the VU Osteopathy Clinic were asked to complete a demographic and health screening questionnaire devised by the researchers, along with the LiMP. Patients completed the questionnaires in the reception area of the clinic prior to their initial osteopathy treatment, however completion was not required to receive care. The demographic data, health screening data and LiMP were de-identified by one of the authors (BV) prior to data entry.

Data analysis

Data were entered into SPSS (IBM Corp, USA) then exported to R [20] for analysis. Descriptive data analysis was undertaken in the psych package [21]. LiMP scores were entered as polytomous data and missing data for the LiMP were imputed using a two-way imputation [22] from the TestDataImputation package [23]. After imputation, scores were recoded to dichotomous to reflect a correct (1) or incorrect (0) response for the item. The focus of the analysis presented here is the psychometric properties of the LiMP using Mokken scale analysis (MSA). MSA is a non-parametric item response theory approach used to evaluate dimensionality and the internal structure of a scale or questionnaire [24, 25]. Readers are directed to other authors for a more in-depth review of MSA [25,26,27,28]. The mokken package [27] was used to perform a Mokken scale analysis (MSA) following the procedures and steps described by Stochl et al. [26] and Van der Ark [27], and outlined in Table 1. The mathematical basis for the scalability coefficients is found in Stochl et al. [26]

Table 1 Steps in the Mokken scale analysis for dichotomous items outlined by Stochl et al. [26]

McDonald’s omega [29] was the selected reliability estimation method to evaluate internal structure, and was calculated using the userfriendlyscience package [30]. This estimation was selected as it does not require tau-equivalence. Both McDonald’s omega total (ωt) and omega hierarchal (ωh) were calculated with values over 0.7 being acceptable [31]. Omega total (ωt) is the “…estimate of the proportion of total common variance in the test” (pp. 152) [32] whereas ωh is “…the extent to which a scale score estimates a latent variable common to all items” (pp. 5) [33]. Cronbach’s alpha was also calculated for comparison with the Rosenbaum et al. [17] study using the userfriendlyscience package [30]. The data likely violates the tau-equivalence assumption to calculate alpha, thereby limiting the ability to meaningfully interpret this value [33]. Further, alpha calculations often “…leads to overestimates of reliability and undercorrections for biases due to measurement error” (pp. 206) [34]. These issues with α support the use of ω as the reliability estimation method in the present study. The Mann-Whitney U test was used to evaluate whether the total LiMP score was significantly influenced by health literacy as measured by the single item “How confident are you completing medical forms?” [35]. Alpha was set at 0.05 and the effect size (r) was calculated and interpreted according to Yatani [36].

Results

Four hundred and forty (N = 440) new patients attended the student-led clinic between March and June 2016, with demographic and health information data available for 414 patients. The 26 patients whose data was not available for analysis included those under the age of 18 years (n = 2) and patients who indicated they did not consent for their data to be used in the study (n = 24). Fifty-four (n = 53, 13%) responses were removed as the patient did not complete the LiMP leaving 361 responses to analyse. Selected demographic data and the body region of the presenting complaint are presented in Table 2 and Fig. 1 respectively. Percentages for correct responses to each of the 9 LiMP items are presented in Table 3.

Table 2 Demographic data for patients participating in the study before and after removal of those patients who did not complete the Literacy in Musculoskeletal Problems (LiMP) questionnaire
Fig. 1
figure 1

Region of patient complaints presenting to the VU Osteopathy Clinic

Table 3 Literacy in Musculoskeletal Problems questions with percentage of correct responses

Initial scalability of the 9-item LiMP suggested that it was a ‘weak’ scale (0.14 ± 0.02) that is, the items together do not measure a single latent construct. Individual item scalability was also low (Hi < 0.24). A negative Hij coefficient was identified for item 9 with all other items. The aisp function was subsequently used to identify possible Mokken scales. With the lowerbound set at 0.3, the minimum for a Mokken scale, two potential scales were identified: one being items 3, 5, and 8; the second being items 6 and 7. A lowerbound value of 0.40 suggested items 3 and 8 only formed a Mokken scale. Setting the lowerbound value to 0.5 suggested that none of the 9 items would create a Mokken scale. McDonald’s ωt (0.52, 95%CI 0.45–0.59) and ωh (0.27), and Cronbach’s alpha (0.49, 95%CI 0.41–0.57) for the 9-item scale were all below acceptable values.

As a total score for the LiMP is not valid each individual item was evaluated to ascertain if the correct/incorrect answer for the individual LiMP item was significantly based on the single item health literacy screen “How confident are you completing medical forms?” [35]. Those patients with higher health literacy were more likely to answer item 5 What is sciatica? (p = 0.011, r = 0.13) and item 8 How does Rheumatoid Arthritis (RA) differ from Osteoarthritis (OA) (p < 0.01, r = 0.18) correctly by, although the effect sizes were small.

Discussion & conclusion

The present study sought to evaluate the dimensionality and internal structure of the LiMP in a patient population attending a student-led osteopathy clinic for the management of musculoskeletal complaints. Although Rosenbaum et al. [17] suggest “the LiMP is a valid tool for specifically assessing musculoskeletal health literacy…” (p. 405) the data presented here do not support this conclusion. Mokken scale analysis was used to evaluate the dimensionality of the LiMP when completed by patients presenting with a range of musculoskeletal complaints. This statistical approach provides evidence that the 9 LiMP items do not form a single scale. Rosenbaum et al. [17] report that whilst “…the LiMP does ultimately assess musculoskeletal literacy, it does so through an evaluation of several themes that cumulatively, not independently, determine one’s musculoskeletal literacy” (p. 404). This statement suggests that a total score for the LiMP is appropriate as a measure of a patients’ musculoskeletal health literacy. The calculation of a total score for the LiMP is not supported by the MSA or the McDonald’s ωh value. This raises questions about the use of the LiMP total score as a musculoskeletal HL screening tool.

Although one, or possibly two, Mokken scale(s) were identified, the fact these scales only contain two items severely limits their utility as a measurement tool. The value of the LiMP may be in the relationship of individual items to health screening data, rather than the creation of a total score for the questionnaire. Item 5 What is sciatica? and item 8 How does Rheumatoid Arthritis (RA) differ from Osteoarthritis (OA) were more likely to be answered correctly by those with higher self-reported health literacy. It is not clear why these two items only would be significantly different albeit with small effect sizes. Both items evaluate knowledge of musculoskeletal conditions as do item 1 A “fractured” bone is and item 7 Arthritis is therefore the difference may not be associated with knowledge level. Further research is required to ascertain if the significant difference is related to the population in the current study (possible given the small effect size), an issue with the structure of the LiMP items, or is a reflection of the patients’ knowledge of musculoskeletal complaints.

There are a number of limitations in the present study including convenience sample of patients attending the VU Osteopathy Clinic during the study period, and that patients were attending specifically for osteopathy care. Whether the results of the present study are generalizable to other professions managing musculoskeletal complaints, or attending a private practice setting, requires further investigation. Researchers are encouraged to utilise the LiMP and evaluate the dimensionality and internal structure in their clinical setting, to determine if these properties are consistent in other patient populations. Further research into the LiMP could also utilise parametric item response theory approaches such as Rasch analysis whereby stricter assumptions are placed on the data. The current research suggests that the LiMP in its current format is not appropriate for use in a population seeking osteopathy care. Use of a questionnaire that evaluates the health literacy of patients specifically seeking care for a musculoskeletal complaint, where the care is provided by a manual therapist may be of value. Continued efforts to evaluate the health literacy of those patients attending for the care of a musculoskeletal complaint is warranted given the substantial, and increasing, burden of musculoskeletal complaints worldwide.

References

  1. Mody GM, Brooks PM. Improving musculoskeletal health: global issues. Best Pract Res Clin Rheumatol. 2012;26:237–49.

    Article  PubMed  Google Scholar 

  2. March L, Smith EU, Hoy DG, Cross MJ, Sanchez-Riera L, Blyth F, Buchbinder R, Vos T, Woolf AD. Burden of disability due to musculoskeletal (MSK) disorders. Best Pract Res Clin Rheumatol. 2014;28:353–66.

    Article  PubMed  Google Scholar 

  3. Smith E, Hoy DG, Cross M, Vos T, Naghavi M, Buchbinder R, Woolf AD, March L. The global burden of other musculoskeletal disorders: estimates from the global burden of disease 2010 study. Ann Rheum Dis. 2014;73:1462–9.

    Article  PubMed  Google Scholar 

  4. Shum J, Poureslami I, Doyle-Waters MM, FitzGerald JM. The application of health literacy measurement tools (collective or individual domains) in assessing chronic disease management: a systematic review protocol. Systematic Reviews. 2016;5:1–8.

    Article  Google Scholar 

  5. Malloy-Weir JL, Charles C, Gafni A, Entwistle V. A review of health literacy: definitions, interpretations, and implications for policy initiatives. J Public Health Policy. 2016;37:334–52.

  6. Loke YK, Hinz I, Wang X, Rowlands G, Scott D, Salter C. Impact of health literacy in patients with chronic musculoskeletal disease–systematic review. PLoS One. 2012;7:e40210.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Saha S, Snyder A, Nugent S, Simon AB, Gralnek I, Provenzale D, et al. Variation in estimates of limited health literacy by assessment instruments and non-response bias. J Gen Intern Med. 2010;25:675–81.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Vaughan B, Mulcahy J, Fitzgerald K. Health literacy screening of patients attending a student-led osteopathy clinic: a pilot investigation. Complement Ther Clin Pract. 2016;24:41–4.

    Article  PubMed  Google Scholar 

  9. Altin SV, Finke I, Kautz-Freimuth S, Stock S. The evolution of health literacy assessment tools: a systematic review. BMC Public Health. 2014;14:1. doi:10.1186/1471-2458-14-1207.

  10. Jordan JE, Osborne RH, Buchbinder R. Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses. J Clin Epidemiol. 2011;64:366–79.

    Article  PubMed  Google Scholar 

  11. Collins SA, Currie LM, Bakken S, Vawdrey DK, Stone PW. Health literacy screening instruments for eHealth applications: a systematic review. J Biomed Inform. 2012;45:598–607.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Rosenbaum AJ, Uhl RL, Rankin EA, Mulligan MT. Social and cultural barriers: understanding musculoskeletal health literacy. J Bone Joint Surg Am. 2016;98:607–15.

    Article  PubMed  Google Scholar 

  13. Briggs AM, Jordan JE, Buchbinder R, Burnett AF, O’Sullivan PB, Chua JYY, Osborne RH, Straker LM. Health literacy and beliefs among a community cohort with and without chronic low back pain. Pain. 2010;150:275–83.

    Article  PubMed  Google Scholar 

  14. Rosenbaum AJ, Pauze D, Pauze D, Robak N, Zade R, Mulligan M, Uhl RL. Health literacy in patients seeking Orthopaedic care: results of the literacy in musculoskeletal problems (LIMP) project. Iowa Orthop J. 2015;35:187.

    PubMed  PubMed Central  Google Scholar 

  15. Rosenbaum AJ, Dunkman A, Goldberg D, Uhl RL, Mulligan M. A cross-sectional study of musculoskeletal health literacy in patients with carpal tunnel syndrome. Hand. 2016;11:330–35.

  16. Rosenbaum AJ, Tartaglione J, Abousayed M, Uhl RL, Mulligan MT, Alley M, DiPreta JA. Musculoskeletal health literacy in patients with foot and ankle injuries a cross-sectional survey of comprehension. Foot Ankle Specialist. 2016;9:31–6.

    Article  PubMed  Google Scholar 

  17. Rosenbaum AJ, Phillips N, Patel N, Uhl R, Mulligan M, Pauze D, Pauze D, Robak N. Assessment of orthopaedic literacy in an urban emergency department: pilot results of the literacy in musculoskeletal problems (LiMP) project. Curr Orthop Pract. 2013;24:400–5.

    Article  Google Scholar 

  18. Rosenbaum AJ, Uhl RL, Mulligan M, Pauze D, Pauze D, Robak N. Third place: do rates and risk factors for limited musculoskeletal health literacy differ between patients in the emergency department and patients in an outpatient orthopaedic surgery practice? Curr Orthop Pract. 2015;26:343–8.

    Article  Google Scholar 

  19. Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, Mockbee J, Hale FA. Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med. 2005;3:514–22.

    Article  PubMed  PubMed Central  Google Scholar 

  20. R Core Team. R: A language and environment for statistical computing. www.R-project.org/. Accessed 20 June 2016.

  21. Revelle W. psych; Procedures for Personality and Psychological Research. http://CRAN.R-project.org/package=psych. Accessed 20 June 2016.

  22. van Ginkel JR, Sijtsma K, van der Ark LA, Vermunt JK. Incidence of missing item scores in personality measurement, and simple item-score imputation. Methodology. 2010;6:17–30.

    Article  Google Scholar 

  23. Dai S, Wang X, Svetina D. TestDataImputation: Missing Item Responses Imputation for Test and Assessment Data. https://CRAN.R-project.org/package=TestDataImputation. Accessed 20 June 2016.

  24. Mokken RJ. A theory and procedure of scale analysis: With applications in political research, vol. 1: Walter de Gruyter; 1971.

  25. Sijtsma K, Meijer RR, van der Ark LA. Mokken scale analysis as time goes by: an update for scaling practitioners. Pers Individ Dif. 2011;50:31–7.

    Article  Google Scholar 

  26. Stochl J, Jones PB, Croudace TJ. Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers. BMC Med Res Methodol. 2012;12:1–16. doi:10.1186/1471-2288-12-74.

  27. Van der Ark LA. New developments in Mokken scale analysis in R. J Stat Softw. 2012;48:1–27.

    Google Scholar 

  28. Wind SA. An instructional module on Mokken scale analysis. Educ Meas Issues Pract. 2017;36:50–66.

  29. Zinbarg RE, Revelle W, Yovel I, Li W. Cronbach’s α, Revelle’s β, and McDonald’s ω H: their relations with each other and two alternative conceptualizations of reliability. Psychometrika. 2005;70:123–33.

    Article  Google Scholar 

  30. Peters G. _userfriendlyscience: Quantitative analysis made accessible_. https://CRAN.R-project.org/package=userfriendlyscience. Accessed 20 June 2016.

  31. Hermsen LA, Leone SS, Smalbrugge M, Knol DL, van der Horst HE, Dekker J. Exploring the aggregation of four functional measures in a population of older adults with joint pain and comorbidity. BMC Geriatr. 2013;13:119. doi:10.1186/1471-2318-13-119.

  32. Revelle W, Zinbarg RE. Coefficients alpha, beta, omega, and the glb: comments on Sijtsma. Psychometrika. 2009;74:145–54.

    Article  Google Scholar 

  33. Crutzen R, Peters G-JY. Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all. Health Psychol Rev. 2017;11:242–47.

  34. Schmidt FL, Le H, Ilies R. Beyond alpha: an empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs. Psychol Methods. 2003;8:206–24.

  35. Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, Bradley KA, Nugent SM, Baines AD, VanRyn M. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med. 2008;23:561–6.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Yatani K. Mann-Whitney’s U test. http://yatani.jp/teaching/doku.php?id=hcistats:mannwhitney#effect_size. Accessed 27 May 2014.

Download references

Acknowledgements

The authors would like to thank to Chris Fallu for his assistance with the data collection.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Availability of data and materials

Data pertaining to the study is available from the following DOI: 10.6084/m9.figshare.3494822

Author information

Authors and Affiliations

Authors

Contributions

BV and JM conceived the study. All authors undertook the literature review. BV undertook the data analysis. All authors contributed to the Discussion and approved the final manuscript.

Corresponding author

Correspondence to Brett Vaughan.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Victoria University Human Research Ethics Committee (HRE15–005).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vaughan, B., Mulcahy, J., Coffey, A. et al. A Mokken analysis of the literacy in musculoskeletal problems questionnaire. Health Qual Life Outcomes 15, 245 (2017). https://doi.org/10.1186/s12955-017-0826-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12955-017-0826-2

Keywords