A Mokken analysis of the literacy in musculoskeletal problems questionnaire
© The Author(s). 2017
Received: 22 July 2016
Accepted: 13 December 2017
Published: 21 December 2017
Limited health literacy is known to impact on medication adherence, hospital readmission and potentially poorer health outcomes. The literature on the health literacy of those with musculoskeletal conditions suggests greater functional limitations and increased pain levels. There are a number of measures of health literacy. One that specifically relates to musculoskeletal complaints is the Literacy in Musculoskeletal Problems (LiMP) questionnaire. The LiMP contains 9 multiple choice items that cover anatomy, musculoskeletal conditions and the diagnosis of musculoskeletal complaints. The aim of the study was to evaluate the dimensionality and internal structure of the LiMP in patients attending for osteopathy care at a student-led clinic, as a potential measure of musculoskeletal health literacy.
Three hundred and sixty-one (n = 361) new patients attending the Victoria University Osteopathy Clinic completed the LiMP and a demographic and health information questionnaire prior to their initial consultation. Mokken scale analysis, a nonparametric item response theory approach, was used to evaluate the dimensionality and structure of the LiMP in this population, to ascertain whether the questionnaire was measuring a single latent construct – musculoskeletal health literacy. McDonald’s omega and Cronbach’s alpha were calculated as the reliability estimations. The relationship between the LiMP and a single item screen of health literacy was also undertaken.
The 9 items on the LiMP did not form a Mokken scale and the reliability estimations were below an acceptable level (alpha and omega <0.45). LiMP items 5 and 8 were more likely to be answered correctly by those with higher health literacy (p < 0.05), however the effect sizes were small (<0.20).
Calculation of a total score for the LiMP, as advocated by the original authors, is not supported based on data in the present study. Further research is required to explore the relationship of the LiMP items to demographic and clinical data, and to other broader measures of health literacy. Further research may also develop a health literacy measure that is specific to patients seeking manual therapy care for musculoskeletal complaints.
The burden of musculoskeletal disorders in society is substantial [1–3] and one strategy to reduce this burden may be addressing health literacy (HL). Although HL has received considerable attention in the health professions literature [4, 5] there are few examples in the manual therapies. A systematic review by Loke et al.  suggests between 7 and 42% of those with chronic musculoskeletal complaints demonstrate low health literacy, with estimates varying depending upon the population, and type of measure utilised . Vaughan et al.  evaluated the HL of patient’s attending an osteopathy, on-campus student-led clinic using a single item screening question, which identified 10% of patients attending this clinic had below adequate HL. Those patients who did not speak English at home, those with lower education levels, and those who were less satisfied with their life demonstrated lower levels of HL.
Although a range of measures exist to evaluate HL [9–11] they can be time-consuming to complete, or require administration by a research or practice assistant. Further, there are no specific questionnaires that exist in the literature to evaluate the HL of patient’s attending for manual therapy care. Rosenbaum et al.  state that “…musculoskeletal health literacy is thought to require a more sophisticated set of skills than those deemed crucial for general health literacy” (p. 608). This assertion is possibly supported by Briggs et al.  who identified that a cohort of chronic low back pain (LBP) patients had “...difficulties in seeking, understanding and utilising LBP information” (p. 275) despite having adequate HL scores using the Short Test of Functional Health Literacy in Adults.
The Literacy in Musculoskeletal Problems (LiMP) questionnaire was developed to evaluate the “...competencies that are integral to making informed decisions regarding musculoskeletal health” (p. 610) . The LiMP consists of 9 items across three domains: anatomy and terminology, musculoskeletal conditions, and diagnosis and treatment. The questionnaire takes 5 min to complete. Patients are asked to select a response for each item (5 response options for items 1–8 and 4 response options for item 9) and only one response is permissible. The response to each item is scored as correct (1) or incorrect (0) and the score totalled for a possible score of 9. LiMP scores below 6 are reported to indicate limited musculoskeletal HL and “…scores of ≥6 can effectively rule out both limited musculoskeletal literacy and general health literacy” (p. 191) . The LiMP has been used to evaluate the musculoskeletal HL of those with carpal tunnel syndrome , foot and ankle complaints , those presenting to an emergency department [14, 17] and an orthopaedic outpatients clinic . It has a Flesch-Kincaid grade level of 4.2 (able to be read by those with a 4th grade education or above) .
There is limited information about the dimensionality and internal structure of the LiMP, both a part of the argument for the validity of the questionnaire. Initial analysis identified a moderate positive correlation (0.41) between Newest Vital Sign  scores and the LiMP, and a Cronbach’s α of 0.59 . Beyond Cronbach’s alpha, there is limited information to provide support for the internal structure of the LiMP. The aim of the present study is to evaluate the dimensionality and internal structure of the LiMP to ascertain if it measures a single latent construct (musculoskeletal health literacy) in a patient population attending a student-led osteopathy clinic for the management of musculoskeletal complaints.
This study forms part of a larger investigation into the demographics and health screening of patient’s attending the VU Osteopathy Clinic, a student-led osteopathy teaching clinic at Victoria University (VU) in Melbourne, Australia. The VU Human Research Ethics Committee approved the study (HRE15–005) and the study was conducted between March 1 and June 30, 2016.
All new patients attending the VU Osteopathy Clinic were asked to complete a demographic and health screening questionnaire devised by the researchers, along with the LiMP. Patients completed the questionnaires in the reception area of the clinic prior to their initial osteopathy treatment, however completion was not required to receive care. The demographic data, health screening data and LiMP were de-identified by one of the authors (BV) prior to data entry.
Steps in the Mokken scale analysis for dichotomous items outlined by Stochl et al. 
Dimensionality assessment (using the coefH function in the mokken package in R). Produces scalability coefficient for all items (H), item pairs (Hij), and individual items (Hi)). Values for each should be between 0 and 1.
Formation of Mokken scales (using the automated item search procedure or aisp function in the mokken package in R). Items form a coherent scale.
Assessment of monotonicity (using the check.monotonicity function in the mokken package in R). Score for an individual item should monotonically increase for an increase in the value associated with the latent trait).
Assessment of invariant item ordering (IIO) and non-intersection (using the check.iio function in the mokken package in R). The item characteristic curves for a score of 0 or should not cross as the latent trait changes. IIO is where the ‘difficulty’ of the item does not change as the value associated with the latent trait changes.
McDonald’s omega  was the selected reliability estimation method to evaluate internal structure, and was calculated using the userfriendlyscience package . This estimation was selected as it does not require tau-equivalence. Both McDonald’s omega total (ωt) and omega hierarchal (ωh) were calculated with values over 0.7 being acceptable . Omega total (ωt) is the “…estimate of the proportion of total common variance in the test” (pp. 152)  whereas ωh is “…the extent to which a scale score estimates a latent variable common to all items” (pp. 5) . Cronbach’s alpha was also calculated for comparison with the Rosenbaum et al.  study using the userfriendlyscience package . The data likely violates the tau-equivalence assumption to calculate alpha, thereby limiting the ability to meaningfully interpret this value . Further, alpha calculations often “…leads to overestimates of reliability and undercorrections for biases due to measurement error” (pp. 206) . These issues with α support the use of ω as the reliability estimation method in the present study. The Mann-Whitney U test was used to evaluate whether the total LiMP score was significantly influenced by health literacy as measured by the single item “How confident are you completing medical forms?” . Alpha was set at 0.05 and the effect size (r) was calculated and interpreted according to Yatani .
Demographic data for patients participating in the study before and after removal of those patients who did not complete the Literacy in Musculoskeletal Problems (LiMP) questionnaire
All 414 available data sets
After removal of those patients who did not complete the LiMP
Mean (±SD) years
Highest education level attended
High school (not completed)
High school (completed)
Trade or vocational education
Completing medical forms
Not at all confident
A little confident
General health (median, range)
Satisfaction with life (median, range)
Literacy in Musculoskeletal Problems questions with percentage of correct responses
1. A “fractured” bone is
2. All of the following facts about X-rays are true EXCEPT:
3. What is the name of the bone in your thigh?
4. An Orthopaedic Surgeon is
5. What is sciatica?
6. The knee is a
7. Arthritis is
8. How does Rheumatoid Arthritis (RA) differ from Osteoarthritis (OA)?
9. If you break your wrist, what might you doctor give you to help you heal?
Initial scalability of the 9-item LiMP suggested that it was a ‘weak’ scale (0.14 ± 0.02) that is, the items together do not measure a single latent construct. Individual item scalability was also low (Hi < 0.24). A negative Hij coefficient was identified for item 9 with all other items. The aisp function was subsequently used to identify possible Mokken scales. With the lowerbound set at 0.3, the minimum for a Mokken scale, two potential scales were identified: one being items 3, 5, and 8; the second being items 6 and 7. A lowerbound value of 0.40 suggested items 3 and 8 only formed a Mokken scale. Setting the lowerbound value to 0.5 suggested that none of the 9 items would create a Mokken scale. McDonald’s ωt (0.52, 95%CI 0.45–0.59) and ωh (0.27), and Cronbach’s alpha (0.49, 95%CI 0.41–0.57) for the 9-item scale were all below acceptable values.
As a total score for the LiMP is not valid each individual item was evaluated to ascertain if the correct/incorrect answer for the individual LiMP item was significantly based on the single item health literacy screen “How confident are you completing medical forms?” . Those patients with higher health literacy were more likely to answer item 5 What is sciatica? (p = 0.011, r = 0.13) and item 8 How does Rheumatoid Arthritis (RA) differ from Osteoarthritis (OA) (p < 0.01, r = 0.18) correctly by, although the effect sizes were small.
Discussion & conclusion
The present study sought to evaluate the dimensionality and internal structure of the LiMP in a patient population attending a student-led osteopathy clinic for the management of musculoskeletal complaints. Although Rosenbaum et al.  suggest “the LiMP is a valid tool for specifically assessing musculoskeletal health literacy…” (p. 405) the data presented here do not support this conclusion. Mokken scale analysis was used to evaluate the dimensionality of the LiMP when completed by patients presenting with a range of musculoskeletal complaints. This statistical approach provides evidence that the 9 LiMP items do not form a single scale. Rosenbaum et al.  report that whilst “…the LiMP does ultimately assess musculoskeletal literacy, it does so through an evaluation of several themes that cumulatively, not independently, determine one’s musculoskeletal literacy” (p. 404). This statement suggests that a total score for the LiMP is appropriate as a measure of a patients’ musculoskeletal health literacy. The calculation of a total score for the LiMP is not supported by the MSA or the McDonald’s ωh value. This raises questions about the use of the LiMP total score as a musculoskeletal HL screening tool.
Although one, or possibly two, Mokken scale(s) were identified, the fact these scales only contain two items severely limits their utility as a measurement tool. The value of the LiMP may be in the relationship of individual items to health screening data, rather than the creation of a total score for the questionnaire. Item 5 What is sciatica? and item 8 How does Rheumatoid Arthritis (RA) differ from Osteoarthritis (OA) were more likely to be answered correctly by those with higher self-reported health literacy. It is not clear why these two items only would be significantly different albeit with small effect sizes. Both items evaluate knowledge of musculoskeletal conditions as do item 1 A “fractured” bone is and item 7 Arthritis is therefore the difference may not be associated with knowledge level. Further research is required to ascertain if the significant difference is related to the population in the current study (possible given the small effect size), an issue with the structure of the LiMP items, or is a reflection of the patients’ knowledge of musculoskeletal complaints.
There are a number of limitations in the present study including convenience sample of patients attending the VU Osteopathy Clinic during the study period, and that patients were attending specifically for osteopathy care. Whether the results of the present study are generalizable to other professions managing musculoskeletal complaints, or attending a private practice setting, requires further investigation. Researchers are encouraged to utilise the LiMP and evaluate the dimensionality and internal structure in their clinical setting, to determine if these properties are consistent in other patient populations. Further research into the LiMP could also utilise parametric item response theory approaches such as Rasch analysis whereby stricter assumptions are placed on the data. The current research suggests that the LiMP in its current format is not appropriate for use in a population seeking osteopathy care. Use of a questionnaire that evaluates the health literacy of patients specifically seeking care for a musculoskeletal complaint, where the care is provided by a manual therapist may be of value. Continued efforts to evaluate the health literacy of those patients attending for the care of a musculoskeletal complaint is warranted given the substantial, and increasing, burden of musculoskeletal complaints worldwide.
The authors would like to thank to Chris Fallu for his assistance with the data collection.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Availability of data and materials
Data pertaining to the study is available from the following DOI: 10.6084/m9.figshare.3494822
BV and JM conceived the study. All authors undertook the literature review. BV undertook the data analysis. All authors contributed to the Discussion and approved the final manuscript.
Ethics approval and consent to participate
The study was approved by the Victoria University Human Research Ethics Committee (HRE15–005).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Mody GM, Brooks PM. Improving musculoskeletal health: global issues. Best Pract Res Clin Rheumatol. 2012;26:237–49.View ArticlePubMedGoogle Scholar
- March L, Smith EU, Hoy DG, Cross MJ, Sanchez-Riera L, Blyth F, Buchbinder R, Vos T, Woolf AD. Burden of disability due to musculoskeletal (MSK) disorders. Best Pract Res Clin Rheumatol. 2014;28:353–66.View ArticlePubMedGoogle Scholar
- Smith E, Hoy DG, Cross M, Vos T, Naghavi M, Buchbinder R, Woolf AD, March L. The global burden of other musculoskeletal disorders: estimates from the global burden of disease 2010 study. Ann Rheum Dis. 2014;73:1462–9.View ArticlePubMedGoogle Scholar
- Shum J, Poureslami I, Doyle-Waters MM, FitzGerald JM. The application of health literacy measurement tools (collective or individual domains) in assessing chronic disease management: a systematic review protocol. Systematic Reviews. 2016;5:1–8.View ArticleGoogle Scholar
- Malloy-Weir JL, Charles C, Gafni A, Entwistle V. A review of health literacy: definitions, interpretations, and implications for policy initiatives. J Public Health Policy. 2016;37:334–52.Google Scholar
- Loke YK, Hinz I, Wang X, Rowlands G, Scott D, Salter C. Impact of health literacy in patients with chronic musculoskeletal disease–systematic review. PLoS One. 2012;7:e40210.View ArticlePubMedPubMed CentralGoogle Scholar
- Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Saha S, Snyder A, Nugent S, Simon AB, Gralnek I, Provenzale D, et al. Variation in estimates of limited health literacy by assessment instruments and non-response bias. J Gen Intern Med. 2010;25:675–81.View ArticlePubMedPubMed CentralGoogle Scholar
- Vaughan B, Mulcahy J, Fitzgerald K. Health literacy screening of patients attending a student-led osteopathy clinic: a pilot investigation. Complement Ther Clin Pract. 2016;24:41–4.View ArticlePubMedGoogle Scholar
- Altin SV, Finke I, Kautz-Freimuth S, Stock S. The evolution of health literacy assessment tools: a systematic review. BMC Public Health. 2014;14:1. doi:10.1186/1471-2458-14-1207.
- Jordan JE, Osborne RH, Buchbinder R. Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses. J Clin Epidemiol. 2011;64:366–79.View ArticlePubMedGoogle Scholar
- Collins SA, Currie LM, Bakken S, Vawdrey DK, Stone PW. Health literacy screening instruments for eHealth applications: a systematic review. J Biomed Inform. 2012;45:598–607.View ArticlePubMedPubMed CentralGoogle Scholar
- Rosenbaum AJ, Uhl RL, Rankin EA, Mulligan MT. Social and cultural barriers: understanding musculoskeletal health literacy. J Bone Joint Surg Am. 2016;98:607–15.View ArticlePubMedGoogle Scholar
- Briggs AM, Jordan JE, Buchbinder R, Burnett AF, O’Sullivan PB, Chua JYY, Osborne RH, Straker LM. Health literacy and beliefs among a community cohort with and without chronic low back pain. Pain. 2010;150:275–83.View ArticlePubMedGoogle Scholar
- Rosenbaum AJ, Pauze D, Pauze D, Robak N, Zade R, Mulligan M, Uhl RL. Health literacy in patients seeking Orthopaedic care: results of the literacy in musculoskeletal problems (LIMP) project. Iowa Orthop J. 2015;35:187.PubMedPubMed CentralGoogle Scholar
- Rosenbaum AJ, Dunkman A, Goldberg D, Uhl RL, Mulligan M. A cross-sectional study of musculoskeletal health literacy in patients with carpal tunnel syndrome. Hand. 2016;11:330–35.Google Scholar
- Rosenbaum AJ, Tartaglione J, Abousayed M, Uhl RL, Mulligan MT, Alley M, DiPreta JA. Musculoskeletal health literacy in patients with foot and ankle injuries a cross-sectional survey of comprehension. Foot Ankle Specialist. 2016;9:31–6.View ArticlePubMedGoogle Scholar
- Rosenbaum AJ, Phillips N, Patel N, Uhl R, Mulligan M, Pauze D, Pauze D, Robak N. Assessment of orthopaedic literacy in an urban emergency department: pilot results of the literacy in musculoskeletal problems (LiMP) project. Curr Orthop Pract. 2013;24:400–5.View ArticleGoogle Scholar
- Rosenbaum AJ, Uhl RL, Mulligan M, Pauze D, Pauze D, Robak N. Third place: do rates and risk factors for limited musculoskeletal health literacy differ between patients in the emergency department and patients in an outpatient orthopaedic surgery practice? Curr Orthop Pract. 2015;26:343–8.View ArticleGoogle Scholar
- Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, Mockbee J, Hale FA. Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med. 2005;3:514–22.View ArticlePubMedPubMed CentralGoogle Scholar
- R Core Team. R: A language and environment for statistical computing. www.R-project.org/. Accessed 20 June 2016.
- Revelle W. psych; Procedures for Personality and Psychological Research. http://CRAN.R-project.org/package=psych. Accessed 20 June 2016.
- van Ginkel JR, Sijtsma K, van der Ark LA, Vermunt JK. Incidence of missing item scores in personality measurement, and simple item-score imputation. Methodology. 2010;6:17–30.View ArticleGoogle Scholar
- Dai S, Wang X, Svetina D. TestDataImputation: Missing Item Responses Imputation for Test and Assessment Data. https://CRAN.R-project.org/package=TestDataImputation. Accessed 20 June 2016.
- Mokken RJ. A theory and procedure of scale analysis: With applications in political research, vol. 1: Walter de Gruyter; 1971.Google Scholar
- Sijtsma K, Meijer RR, van der Ark LA. Mokken scale analysis as time goes by: an update for scaling practitioners. Pers Individ Dif. 2011;50:31–7.View ArticleGoogle Scholar
- Stochl J, Jones PB, Croudace TJ. Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers. BMC Med Res Methodol. 2012;12:1–16. doi:10.1186/1471-2288-12-74.
- Van der Ark LA. New developments in Mokken scale analysis in R. J Stat Softw. 2012;48:1–27.Google Scholar
- Wind SA. An instructional module on Mokken scale analysis. Educ Meas Issues Pract. 2017;36:50–66.Google Scholar
- Zinbarg RE, Revelle W, Yovel I, Li W. Cronbach’s α, Revelle’s β, and McDonald’s ω H: their relations with each other and two alternative conceptualizations of reliability. Psychometrika. 2005;70:123–33.View ArticleGoogle Scholar
- Peters G. _userfriendlyscience: Quantitative analysis made accessible_. https://CRAN.R-project.org/package=userfriendlyscience. Accessed 20 June 2016.
- Hermsen LA, Leone SS, Smalbrugge M, Knol DL, van der Horst HE, Dekker J. Exploring the aggregation of four functional measures in a population of older adults with joint pain and comorbidity. BMC Geriatr. 2013;13:119. doi:10.1186/1471-2318-13-119.
- Revelle W, Zinbarg RE. Coefficients alpha, beta, omega, and the glb: comments on Sijtsma. Psychometrika. 2009;74:145–54.View ArticleGoogle Scholar
- Crutzen R, Peters G-JY. Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all. Health Psychol Rev. 2017;11:242–47.Google Scholar
- Schmidt FL, Le H, Ilies R. Beyond alpha: an empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs. Psychol Methods. 2003;8:206–24.Google Scholar
- Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, Bradley KA, Nugent SM, Baines AD, VanRyn M. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med. 2008;23:561–6.View ArticlePubMedPubMed CentralGoogle Scholar
- Yatani K. Mann-Whitney’s U test. http://yatani.jp/teaching/doku.php?id=hcistats:mannwhitney#effect_size. Accessed 27 May 2014.