Effects of physical therapy on quality of life in osteoporosis patients - a randomized clinical trial
© Schröder et al.; licensee BioMed Central Ltd. 2012
Received: 2 August 2011
Accepted: 16 August 2012
Published: 24 August 2012
The aim of this prospective randomized single-center study was to investigate whether sling exercise therapy is superior to conventional exercises in osteoporosis patients.
Patients with osteoporosis frequently experience fractures of the vertebral body, which may cause chronic back pain and other symptoms. These, in turn, may lead to immobilization, muscular atrophy, and restrictions in activities of daily living. The situation can be improved with specific medication and physiotherapy. We explored the effects of a variety of physical treatments on activities of daily living in patients with osteoporosis.
Fifty patients were randomly allocated to two treatment groups. Group A received traditional physiotherapy (PT) while group B underwent sling exercise therapy (ST). Both treatments were given twice a week for three months. The results of the treatment were registered on the quality of life questionnaire (Qualeffo-41) devised by the International Osteoporosis Foundation. After a further three months with no specific exercise treatment, we again tested all patients in order to draw conclusions about the long-term effects of both types of exercise.
Forty-four patients (88%) completed the study. Patients were assigned to small groups (a maximum of 5 patients in each group) and thus received individual attention and motivation. Quality of life was improved in both groups; a significantly greater improvement was registered in patients who performed sling exercises (Global score Qualeffo: p = 0.002).
The test results confirm the known positive effects of physical therapy on the quality of life of osteoporosis patients, as well as the fact that sling exercises are a sound alternative treatment for this condition.
Osteoporosis is one of the ten most common illnesses . Approximately 30% of women and 10% of men older than 50 years of age suffer from the disease . Osteoporosis a systemic skeletal disease in which bone mass is reduced, the microstructure of bone destroyed, and the risk of fracture increased . As no early symptoms other than non-specific back pain are observed, osteoporosis is rarely diagnosed before the first fracture occurs . Vertebral fractures are followed by chronic malaise and are known to restrict the mobility of the trunk and lower extremities. This, in turn, increases the risk of falls [5, 6]. Fractures are frequently followed by isolation and restriction of independence, leading to loss of autonomy and diminished quality of life in advanced age .
A variety of interventions have been developed to address this condition. The benefits of regular physical exercise include reduction of pain, prevention of falls, and improvement of mobility and quality of life [8–13]. Fitness factors such as strength, stamina, and easy movement are encouraged. Owing to the geriatric aspect of osteoporosis, activation of the sensomotor system is important. Many elderly individuals suffer from comorbidities (such as deterioration of visual acuity and perception of contrast) which, however, culminate in physical imbalance only when combined with proprioception deficits.
As recent advancements in sports medicine have proven the beneficial effects of sensomotor exercise on the regulation of posture and/or the frequency of falls [14, 15], we decided to use sling exercise therapy. The procedure was developed by Kirkesola  to treat motor problems, and was established by Meier  for prevention as well as rehabilitation of professional German sportsmen.
To our knowledge, the question as to whether sling exercise improves quality of life in osteoporosis patients has not been previously investigated.
Design and randomization
Patients were recruited from the osteoporosis outpatient clinics of Klinikum Südstadt hospital and the University hospital of Rostock. All study participants had proven osteoporosis (T-score ≤ −2.5). Patients who were unable to complete the entire training program during the period of the investigation were excluded from analysis.
Patients were randomized on a 1:1 basis. The permuted block design was used for randomization . The block size was randomly selected. The randomization envelopes were numbered in ascending order. A proband to be randomized opened the envelope with the lowest number among all sealed envelopes.
Over a period of 3 months, 44 patients with osteoporosis completed a twice-weekly 30-minute intensive exercise program designed to stabilize the trunk. Both groups completed a training program that consisted of 5 phases. Exemplified in Figure 1 are the 4 main phases of the Sling training represented. Phase 2 had in the course of the intervention no longer be practiced explicitly. The PT group completed similar exercises without slings (eg chair-rising exercises pelvic lift, step-ups).
Phase 1: Systematic cardiovascular and neuromuscular warm-up (PT group: general keep-fit exercises; ST group: step aerobics)
Phase 2: PT and ST groups: functional strength exercises focusing on correct posture
Phase 3: PT and ST groups: functional strength exercises for global surface muscles of the torso; ST group: dynamic sling exercise
Phase 4: Segmental stabilization (SST), both static and dynamic (PT group: exercise/medical ball; ST group: sling)
Phase 5: Stretching and relaxation
Each exercise session included all of the five phases.
Measurement of quality of life using the Qualeffo-41 questionnaire
Domain scores are calculated by averaging the answers of one domain and transforming the scores to a score from 0 to 100. The total score is calculated by summing all answers of questions 1–41. The raw total score ranges from 41 to 205 and this is transformed to scores from 0 to 100. All answers are standardized so that 1 represents the best and 5 (or 3, or 4) represents the worst quality of life.
All patients completed the questionnaire before the start of intervention, after 3 months of exercise, and at the 3-month follow-up investigation after cessation of the exercise program.
Ethics committee approval
Informed written consent was obtained from all patients. The study was approved by the regional ethics committee for medical research.
All data were stored and analyzed using the SPSS statistical package 19.0 (SPSS Inc. Chicago, Illinois). Descriptive statistics were computed for variables of interest. The computed statistics included means and standard deviations of continuous variables, and are presented as mean ± SD, frequencies, and relative frequencies of categorical factors.
Testing for differences in continuous variables between the two study groups was achieved by the two-sample t-test for independent samples or the Mann–Whitney U-test by ranks as appropriate. Selection of the test was based on evaluation of variables for normal distribution, employing the Kolmogorov-Smirnov test. Fisher's exact test was used for between-group comparisons of categorical variables.
Comparisons within groups between the time points of evaluation were performed with regard to percentage changes versus baseline by one-sample t-tests against 0, and for percentage change between the time points named “follow-up” and “after training” by a paired t-test. Adjustments of alpha levels were carried out using the Bonferroni correction, i.e. the level of significance was lowered to 0.05/3 = 0.017.
All p-values resulted from two-sided statistical tests; the level of significance was set to p≤0.05 when no Bonferroni correction was required.
Patient characteristics: gender: male (m)/female (f), age in years, body mass index in kg/m 2 ; bone mineral density in T-score; vertebral body fractures in numbers (n); physiotherapy (PT); sling exercise therapy (ST)
PT (n = 21)
ST (n = 23)
69.7 ± 3.7
71.0 ± 6.1
Body mass index
23.9 ± 2.9
25.6 ± 3.5
Bone mineral density
−2.8 ± 0.83
−2.8 ± 0.77
Vertebral body fractures
After three months of exercise, significant differences were registered between the PT and ST groups. These were particularly evident on pre-post comparison in the sling training group, and are interpreted as individual responses to the intervention.
Global scores as well as scores for the domains were standardized to a 100-point scale: a score of 0 indicates no difficulties due to osteoporosis while a score of 100 indicates maximum difficulties.
The difficulties experienced by the patients were generally low (baseline: 26.0 ± 11.2 in the PT group and 29.7 ± 9.8 in the ST group; p = 0.256). The total score was significantly improved after the intervention in the ST group (29.7 ± 9.8 baseline and 24.5 ± 7.7 after the program; p = 0.002). In contrast, the overall figure for the PT group changed just slightly (26.0 ± 11.2 baseline to 25.7 ± 10.6 after training; p = 0.477). Comparison of baseline and follow-up data showed improvement in both exercise groups (26.0 ± 11.2 and 23.9 ± 10.0 in the PT group, p = 0.766; 29.7 ± 9.8 and 21.8 ± 8.1 in the ST group, p < 0.001). The change in the ST group was highly significant.
Test results: global score, pain, physical function, social function, health perception, mental function; significance between groups; physiotherapy (PT); sling exercise therapy (ST)
PT (n = 21)
ST (n = 23)
26.0 ± 11.2
29.7 ± 9.8
25.7 ± 10.6
24.5 ± 7.7
23.9 ± 10,0
21.8 ± 8.1
46.2 ± 19.3
50.2 ± 23.3
41.2 ± 23.6
35.7 ± 22.0
38.8 ± 23.3
29.6 ± 24.0
16.2 ± 10.2
20.1 ± 10.4
14.0 ± 9.7
13.8 ± 8.1
12.8 ± 8.4
10.9 ± 6.2
18.4 ± 15.2
20.3 ± 11.3
16.3 ± 12.8
16.0 ± 10.1
17.7 ± 12.9
17.1 ± 10.4
50.0 ± 19.0
57.6 ± 15.5
49.2 ± 21.1
47.5 ± 11.6
44.4 ± 15.2
49.3 ± 17.4
31.5 ± 11.6
34,3 ± 12.0
38.8 ± 9.3
37.7 ± 7.7
34.4 ± 10.3
32.9 ± 11.6
Test results of percentage changes: global score, pain, physical function, social function, health perception, mental function; significance within each group; physiotherapy (PT); sling exercise therapy (ST)
After training vs. Baseline
Follow-up vs. Baseline
PT (n = 21)
4.7 ± 29.9
−2.1 ± 32.3
ST (n = 23)
−14.9 ± 19.9
−25.4 ± 18.4
PT (n = 21)
−8.0 ± 45,6
−15.8 ± 46.6
ST (n = 23)
−28.8 ± 29.7
−47.1 ± 36.7
PT (n = 21)
−9.8 ± 42.2
−14.1 ± 41.3
ST (n = 23)
−27.2 ± 34.9
−44.0 ± 31.5
PT (n = 21)
−7.0 ± 44.9
18.8 ± 109.7
ST (n = 23)
8.0 ± 110.1
32.0 ± 182.3
PT (n = 21)
−1.6 ± 25.3
1.3 ± 55.6
ST (n = 23)
−16.1 ± 19.7
−12.5 ± 26.7
PT (n = 21)
34.9 ± 41.4
15.4 ± 35.5
ST (n = 23)
19.8 ± 36.5
3.4 ± 43.7
The items of the social function domain included hobbies and activities that involved interaction with others. The data reveal little difficulties in both groups before the program (18.4 ± 15.2 in the PT group, 20.3 ± 11.3 in the ST group; p = 0.625). A marginal change was observed after the intervention (16.3 ± 12.8 in the PT group and 16.0 ± 10.1 in the ST group; p = 0.527). Similar data were recorded at the follow-up investigation (17.7 ± 12.9 in the PT group and 17.1 ± 10.4 in the ST group; p = 0.864).
Health perception is based on the patients' comprehensive personal assessment of their own quality of life. Patients relate their current state of health to their age, or look back on their quality of life and evaluate it. The groups were quite similar as regards their perception of health before the start of the program (50.0 ± 19.0 in the PT group, 57.6 ± 15.5 in the ST group; p = 0.151). The groups reported moderate difficulties. An improvement was observed in both groups after the intervention, but was only significant in the ST group (57.6 ± 15.5, 47.5 ± 11.6; p = 0.001). The PT group perceived their health as being improved at the follow-up investigation compared to immediately after the intervention (see Table 2). Comparing baseline and follow-up data in the PT group revealed no significant differences (50.0 ± 19.0, 44.4 ± 15.2; p = 0.916). Notable but non-significant (after Bonferroni correction) effects were registered in the sling exercise group immediately after the intervention, but were diminished thereafter (57.6 ± 15.5 baseline, 49.3 ± 17.4 on follow-up; p = 0.034).
This domain of the questionnaire is focused on moods and emotional states of osteoporosis patients. The difficulty experienced by patients of both groups was similar before the start of the intervention (31.5 ± 11.6 in the PT group, 34.3 ± 12.0 in the ST group; p = 0.434). Moods were deteriorated by several points in both groups (38.8 ± 9.3 in the PT group, 37.7 ± 7.7 in the ST group; p = 0.678), but initial values were nearly restored at follow-up (34.4 ± 10.3 in the PT group, 32.9 ± 11.6 in the ST group; p = 0.645). Comparisons within groups revealed a very significant difference in the PT group at the end of the intervention (31.5 ± 11.6 before the program to 38.8 ± 9.3 at the end of the intervention; p = 0.001).
Given a society in which the proportion of the elderly is on the increase, physicians and physiotherapists are focussing an increasing share of their attention on back pain and osteoporosis. Various exercises programs are available for patients with osteoporosis. Their efficacy as regards improving of bone density and reducing the risk of falls has been proven [8, 13]. Conclusions drawn by the Cochrane Collaboration refer - among other aspects - to the improvement of vertebral bone density in postmenopausal women by means of aerobic exercises, weight-bearing exercises, and resistance exercises . Our data concerning sling exercises for osteoporosis patients revealed a significant reduction in the perception of pain, a significant increase in physical performance, and a significant sensation of "feeling more healthy" among our patients. Especially worthy of mention is the enduring effect of this type of exercise on "feeling healthy" and pain relief. The effect persisted during the 3-month control period after completion of the training program. We have found no studies in the published literature reporting the results of sling exercises in patients with osteoporosis. The so-called conventional physiotherapy regimen used in our second group failed to achieve similar effects.
The relatively mild symptoms (50) experienced by our patients despite having undergone fractures is not easily explained (Table 3). Bianchi et al.  investigated 100 postmenopausal patients with osteoporosis, aged 50 to 85 years, in respect of quality of life and depression. The results of the study showed that osteoporosis exerted a negative impact on the psyche and quality of life of the patients. The group that experienced fractures fared the worst. One explanation for the results achieved in our study might have been the fact that 88% of our patients performed regular sports twice a week. Furthermore, our patients went for a walk at least once daily. Accordingly, the initial score for the domain physical function was rather low - which is indicative of a high degree of mobility in activities of daily living and household chores. This is contrasted by the relatively high level of pain before the start of the intervention. This was not significantly different in the sling exercises group (Table 3). Thus, patients benefited most in the domains of pain und physical function. The effect of training in this domain was perceptible even three months after completion of the program. One explanation could be the plasticity of the human neuromuscular system, which is forced to adapt consistently to new environmental conditions. In our case the unstable slings fulfill this "new" requirement. The degree of elasticity is such that the person doing the exercises is exposed to a number of stimuli. In other words, he/she has a very small supportive surface and has to regulate and control motions very rapidly and with full concentration. The deep muscles of the spine that lie close to the joints (musculus multifidus, musculus transversus abdominis, pelvic floor muscles and the diaphragm) need to be activated. This is very significant for stabilization of the lumbar spine and pain relief in this region [17, 23–25]. Local stabilizers appear to be atrophied in the presence of pain, and cannot be regulated or controlled properly [25–29]. This causes instability in the respective segments of the spine. In the absence of exercise stimuli, the deep muscles are only regulated in a phasic manner. As a result an individual may lose his/her sensitivity to correct posture [26, 27]. This explanatory approach has been established in the treatment of patients with back pain and the rehabilitation of sportsmen. Obviously, this fact cannot be immediately transferred to the treatment of patients with osteoporosis. However, the results of a study performed by Hongo et al.  show that even exercises of mild intensity may improve the strength of spinal muscles and thus exert a positive effect on quality of life in patients with osteoporosis. Circuit training as performed in Bergland et al.'s  study proved to be much more intensive. The authors had women with osteoporosis aged 60 to 84 years perform the training program for three months, twice a week, one hour per session, under the supervision of a physiotherapist. Based on their data Bergland et al. conclude that circuit training may exert a positive impact on mobility and quality of life. In other words, the regularity of exercise is a major factor. In order to design intensive exercises and provide individual care for patients with osteoporosis, it appears meaningful to limit the size of the group to a maximum of five persons. Immediate feedback from physiotherapists ensured that the patients were motivated, that they performed the exercises correctly (i.e. with no error), and this also increased the intensity of training. In sling exercises, the diversity of experience obtained in performing motions with the sling appears to exert a sustained effect on the sensomotor system of osteoporosis patients. Activities of daily living become easier when patients experience less pain. Improvement of sensomotor abilities appears to exert a positive impact on quality of life. Schwesig et al.  performed sensomotor exercises with the Aero-Step in 20 healthy elderly persons and 27 patients with osteoporosis aged on average 66.6 years. After conclusion of the training program the authors observed improvements in sensomotor performance and general wellbeing. How the authors registered the gain in quality of life is not clearly discernible from this report.
It is interesting that the domain of Mental function deteriorated in both groups after the intervention. We suspect that the loss of ‘new’ social contact plays a significant role.
Our study demonstrated a positive effect of sling exercises on pain, physical fitness and general body perception. Thus, sling exercises appear to be a favorable alternative to conventional physiotherapeutic procedures.
It was possible with the physical therapy to improve the pain status and the functional capability of the osteoporosis patients, with the sling exercise patients benefiting most.
The combination of pain mitigation and functional improvement resulted in improved perception of health and this in turn improved the sling exercise group’s quality of life.
- Bartl R, Gradinger R: Aktuelle Diagnostik und Therapie der Osteoporose auf der Basis der "European Guidance 2008". Der Orthopäde 2009, 4: 365–378.View ArticleGoogle Scholar
- Pollähne W, Minne HW: Epidemiologie, Diagnostik und klinisches Bild der Osteoporose. Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz 2001, 44(1):32–36. 10.1007/s001030050407View ArticleGoogle Scholar
- WHO Scientific Group on the Burden of Musculoskeletal Conditions at the Start of the New Millennium: The burden of musculoskeletal conditions at the start of the new millennium. World Health Organ Tech Rep Ser 2003, 919: 1–218.Google Scholar
- Faßbender WJ, Pfeilschifter J: Osteoporose kompakt. Leitlinienbasierte Prävention, Diagnostik und Therapie; mit 26 Fallbeispielen; mit 18 Tabellen. Stuttgart: Schattauer; 2008.Google Scholar
- KazKaz H, Johnson D, Kerry S, Chinapplee U, Tweed K, Patel S: Fall related risk factors and osteoporosis in women with rheumatoid arthritis. Rheumatology 2004, 43(10):1267–1271. 10.1093/rheumatology/keh304View ArticleGoogle Scholar
- Armstrong C, Swarbrick CM, Pye SR, O'Neill TW: Occurrence and risk factors for falls in rheumatoid arthritis. Ann Rheum Dis 2005, 64: 1602–1604. 10.1136/ard.2004.031195PubMed CentralView ArticlePubMedGoogle Scholar
- Salkeld G: Quality of life related to fear of falling and hip fracture in older women. a time trade off study. BMJ 2000, 320(7231):341–346. 10.1136/bmj.320.7231.341PubMed CentralView ArticlePubMedGoogle Scholar
- Preisinger E: Physiotherapie und Bewegung bei Osteoporose und Folgeerkrankungen. Z Rheumatol 2009, 68(7):534–538. 10.1007/s00393-009-0445-3View ArticlePubMedGoogle Scholar
- Bergland A, Thorsen H, Kåresen R: Effect of exercise on mobility, balance, and health-related quality of life in osteoporotic women with a history of vertebral fracture: a randomized, controlled trial. Osteoporos Int 2011, 22(6):1863–1871. 10.1007/s00198-010-1435-7View ArticlePubMedGoogle Scholar
- Teixeira LEPP, Silva KNG, Imoto AM, et al.: Progressive load training for the quadriceps muscle associated with proprioception exercises for the prevention of falls in postmenopausal women with osteoporosis: a randomized controlled trial. Osteoporos Int 2010, 21(4):589–596. 10.1007/s00198-009-1002-2View ArticlePubMedGoogle Scholar
- de Kam D, Smulders E, Weerdesteyn V, Smits-Engelsman BC: Exercise interventions to reduce fall-related fractures and their risk factors in individuals with low bone density: a systematic review of randomized controlled trials. Osteoporos Int 2009, 20(12):2111–2125. 10.1007/s00198-009-0938-6View ArticlePubMedGoogle Scholar
- Iwamoto J, Suzuki H, Tanaka K, et al.: Preventative effect of exercise against falls in the eldery: a randomized controlled trial. Osteoporos Int 2009, 20(7):1233–1240. 10.1007/s00198-008-0794-9View ArticlePubMedGoogle Scholar
- Uhlemann C, Lange U: Differenzialindikative physikalische Therapiestrategien der Osteoporose. Empfehlungen für den Praxisalltag. Z Rheumatol 2006, 65(5):407–416. 10.1007/s00393-006-0084-xView ArticlePubMedGoogle Scholar
- Schwesig R, Müller K, Hottenrott K, et al.: Sensomotorisches Training im Alter und bei Osteoporose. Bewegungstherapie und Gesundheitssport 2006, 22(2):62–68. 10.1055/s-2006-921564View ArticleGoogle Scholar
- von Stengel S, Kemmler W, Engelke K, et al.: Effects of whole body vibration on bone mineral density and falls: results of the randomized controlled ELVIS study with postmenopausal women. Osteoporos Int 2011, 22(1):317–325. 10.1007/s00198-010-1215-4View ArticlePubMedGoogle Scholar
- Kirkesola G: Sling Exercise Therapy – ein Konzept zur Behandlung von Beschwerden am Bewegungssystem. Zeitschrift für Physiotherapeuten 2001, 53(3):428–438.Google Scholar
- Meier HP: Neue Aspekte der Gelenkstabilisation – das Sling-Training. Leistungssport 2006, 2: 19–23.Google Scholar
- Rosenberger WF, Lachin JM: Randomization in Clinical trials. Theory and practice. New York: Wiley; 2002.View ArticleGoogle Scholar
- Lips P, Cooper C, Agnusdei D, et al.: Quality of life as outcome in the treatment of osteoporosis. The development of a questionnaire for quality of life by the European Foundation for Osteoporosis. Osteopor Int 1997, 7: 36–38.View ArticleGoogle Scholar
- Lips P, Cooper C, Agnusdei D, et al.: Quality of life in patients with vertebral fractures: validation of the quality of life questionnaire of the European Foundation for Osteoporosis (QUALEFFO). Osteopor Int 1999, 10: 150–160. 10.1007/s001980050210View ArticleGoogle Scholar
- Moher D, Schulz KF, Altman DG, Das CONSORT Statement: Überarbeitete Empfehlungen zur Qualitätsverbesserung von Reports randomisierter Studien im Parallel-Design. Der Schmerz 2005, 19(2):156–62. 10.1007/s00482-004-0380-9View ArticlePubMedGoogle Scholar
- Bianchi ML, Orsini MR, Saraifoger S, Ortolani S, Radaelli G, Betti S: Quality of life in post-menopausal osteoporosis. Health Qual Life Outcomes 2005, 3: 78. 10.1186/1477-7525-3-78PubMed CentralView ArticlePubMedGoogle Scholar
- O'Sullivan PB, Twomey LT, Allison GT: Evaluation of specific stabilizing exercise in the treatment of chronic low back pain with radiologic diagnosis of spondylolysis or spondylolisthesis. Spine 1997, 22(24):2959–2967. 10.1097/00007632-199712150-00020View ArticlePubMedGoogle Scholar
- O'Sullivan PB, Twomey LT, Allison GT, Sinclair J, Miller K, Knox J: Altered patterns of abdominal muscle activation in patients with chronic low back pain. Aust J Physiother 1997, 43(2):91–98.View ArticlePubMedGoogle Scholar
- O'Sullivan PB, Burnett A, Floyd AN, Gadsdon K, Logiudice J, Miller-Quirk H: Lumbar repositioning in a specific low back pain population. Spine 2003, 28(10):1074–1079.View ArticlePubMedGoogle Scholar
- Silfies SP, Squillante D, Maurer P, Westscott S, Karduna AR: Trunk muscle recruitment in spezific chronic low back pain populations. Clin Biomech 2005, 20: 465–473. 10.1016/j.clinbiomech.2005.01.007View ArticleGoogle Scholar
- Hodges PW, Richardson CA: Inefficient muscular stabilization of the lumbar spine associated with low back pain. Spine 1996, 21(22):2640–2650. 10.1097/00007632-199611150-00014View ArticlePubMedGoogle Scholar
- Hodges PW, Richardson CA: Feedforward contraction of transversus abdominis is not influenced by the direction of arm movement. Exp Brain Res 1997, 114(2):362–370. 10.1007/PL00005644View ArticlePubMedGoogle Scholar
- Zhao WP, et al.: Histochemistry and morphology of the multifidus muscle in lumbar disc herniation. comparative study between diseased and normal sides. Spine 2000, 25(17):2191–2199. 10.1097/00007632-200009010-00009View ArticlePubMedGoogle Scholar
- Hongo M, Itoi E, Sinaki M, et al.: Effect of low-intensity back exercise on quality of life and back extensor strength in patients with osteoporosis: a randomized controlled trial. Osteopor Int 2007, 18: 1389–1395. 10.1007/s00198-007-0398-9View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.