Development of a new occupational balance-questionnaire: incorporating the perspectives of patients and healthy people in the design of a self-reported occupational balance outcome instrument

Background Self-reported outcome instruments in health research have become increasingly important over the last decades. Occupational therapy interventions often focus on occupational balance. However, instruments to measure occupational balance are scarce. The aim of the study was therefore to develop a generic self-reported outcome instrument to assess occupational balance based on the experiences of patients and healthy people including an examination of its psychometric properties. Methods We conducted a qualitative analysis of the life stories of 90 people with and without chronic autoimmune diseases to identify components of occupational balance. Based on these components, the Occupational Balance-Questionnaire (OB-Quest) was developed. Construct validity and internal consistency of the OB-Quest were examined in quantitative data. We used Rasch analyses to determine overall fit of the items to the Rasch model, person separation index and potential differential item functioning. Dimensionality testing was conducted by the use of t-tests and Cronbach’s alpha. Results The following components emerged from the qualitative analyses: challenging and relaxing activities, activities with acknowledgement by the individual and by the sociocultural context, impact of health condition on activities, involvement in stressful activities and fewer stressing activities, rest and sleep, variety of activities, adaptation of activities according to changed living conditions and activities intended to care for oneself and for others. Based on these, the seven items of the questionnaire (OB-Quest) were developed. 251 people (132 with rheumatoid arthritis, 43 with systematic lupus erythematous and 76 healthy) filled in the OB-Quest. Dimensionality testing indicated multidimensionality of the questionnaire (t = 0.58, and 1.66 after item reduction, non-significant). The item on the component rest and sleep showed differential item functioning (health condition and age). Person separation index was 0.51. Cronbach’s alpha changed from 0.38 to 0.57 after deleting two items. Conclusions This questionnaire includes new items addressing components of occupational balance meaningful to patients and healthy people which have not been measured so far. The reduction of two items of the OB-Quest showed improved internal consistency. The multidimensionality of the questionnaire indicates the need for a summary of several components into subscales.


(Continued from previous page)
Conclusions: This questionnaire includes new items addressing components of occupational balance meaningful to patients and healthy people which have not been measured so far. The reduction of two items of the OB-Quest showed improved internal consistency. The multidimensionality of the questionnaire indicates the need for a summary of several components into subscales.
Keywords: Qualitative research, Rasch analysis, Occupational science, Patient perspective

Background
The use of self-reported outcome instruments in health care research has become increasingly important over the last decades, because the perspective of patients is an essential part regarding the effectiveness or non-effectiveness of a treatment [1][2][3]. Currently, it is recommended not only to measure outcomes from the perspectives of the patients, but also to involve them into the development of instruments. Patients should be asked for feedback on the wording to detect problems of understanding. Additionally, for already existing instruments, qualitative research is suitable in facilitating that the items are relevant to the target population [4][5][6].
Occupational balance largely guides the clinical practice of occupational therapists [7,8]. "Occupations" refer to goal-directed, meaning-and purposeful everyday activities that people do as individuals and in their social contexts [7]. Occupational therapists focus on occupations as a means, but also as an outcome of therapy. Occupational balance is one important construct that linksin the view of occupational therapists -"occupation" and health [9,10].
Occupational balance is defined diversely. One definition which is grounded in the beginning of occupational therapy refers to a balance between different occupational areas, such as work, play, rest and sleep [7]. However, the definitions are mainly derived from the perspectives of occupational therapists rather than of the perspectives of patients and healthy people (without a diagnosed health condition) [11]. Occupational balance may thus be an "academically defined concept" which lacks a link to the experiences of "real" people.
Up to now, only one questionnaire exists (Wilcock's twopage "questionnaire on involvement in physical, mental, social and rest occupations") which was developed and used to assess occupational balance [12]. However, this questionnaire has not been developed based on qualitative data, and was not used in further research. One essential aspect of the validity of an instrument is the content validity, referring to its ability to measure those underlying components of the construct which it intends to measure. This requires a conceptual definition of the construct to be measured and a specification of its components [13]. To assess occupational balance in patients of different health conditions, a generic self-reported outcome instrument, based on qualitative data on the perspectives and experiences of patients and healthy people, is required. Additionally, reliable and valid (occupational balance) instruments are prerequisites for the evaluation of outcomes in occupational therapy practice.
Therefore, the aim of the study was to develop a generic self-reported outcome instrument to assess occupational balance based on the experiences of patients and healthy people including an examination of its psychometric properties.

Design
We conducted a mixed-methods study that started off with qualitative analyses of the life stories of people with and without chronic autoimmune diseases to identify components of occupational balance. Based on these components, we developed the Occupational Balance-Questionnaire (OB-Quest). A German version was designed first and then forward and back translated into English according to standard translation procedures [14]. Construct validity and internal consistency of the OB-Quest were examined in quantitative data using Rasch analyses and Cronbach's alpha. This study was part of a larger study, the Gender, Occupational Balance and Immunology (GOBI) study [15]. A flow chart is depicted in Figure 1.

Participants
Patients of two outpatient clinics of the Medical University of Vienna, Austria, diagnosed with Crohn's disease (CD) [16], rheumatoid arthritis (RA) [17], systemic lupus erythematosus (SLE) [18], systemic sclerosis (SSc) [19] or diabetes mellitus type one (T1D) [20] were asked to participate. Only the qualitative data from patients with CD and RA were used from previous studies which employed the same methodological approach [11,15,21]. Additionally, healthy people were asked to participate via personal invitation by patients, such as "friends" of a similar age, and announcements in public places such as supermarkets, universities and the general hospital. Sex, age, employment status and health condition, if applicable were recorded. Furthermore, information about disease duration was obtained from patient files where appropriate.
A qualitative analysis of life stories with relation to occupational balance A biographical narrative approach was used to explore the experience of occupational balance. The participants' own perspectives on their lives, expressed in their life stories, were investigated in relation to current and biographic experiences [22]. The life stories, collected in two interview sessions, were transcribed verbatim and analysed with the biographic narrative interpretative method [22]. With this method, we compered the "lived life" (the biographical data) to the "told story" (the present perspective of the interviewee on her or his life). Hypotheses were developed by a research panel which were then verified or falsified depending on the course of each life story. Through this process, so-called typologies, which were common themes found in more than one life story, were identified. Typologies related to occupational balance were then extracted, defined as components of occupational balance and used as basis for the development of the questionnaire items.

Item development
Each component resulted in one item. Items were generated by the first author in collaboration with patients and healthy people [4]. After this initial process a final version of the items was formulated based on the feedback of additional patients and healthy people and on the discussion of the research panel (TAS, BP, AB and MD). Patients from the rheumatology outpatient clinic and internal medicine ward, as well as healthy people ("friends" of a similar age and visitors) were invited to give feedback. Each item included a numerical rating scale consisting of three response categories. "1" indicated a positive score, such as "having a good variety of activities", and "3" indicated a negative score such as "having little or no variety of activities". In addition, a German version was designed first and then the English version was developed by the use of a standard methodology with forward and back translation and proof reading by a total of four English or German native speakers (PG, AJ, LL and VN-D) [14].

Examination of construct validity and internal consistency
Construct validity and internal consistency of the OB-Quest were explored. Therefore, patients with RA or SLE and healthy people completed the OB-Quest, as well as a questionnaire on demographic data. The Statistical Package for the Social Sciences (SPSS) [23] was used for descriptive and RUMM 2030 [24] for Rasch analyses. Concerning construct validity we examined overall fit of the items to the Rasch model as suggested by Tennant et al. [25]. Therefore, the mean item log residual test of fit as well as the item-trait interaction chi-square statistics were assessed [25]. Non-significant residuals between -2.5 and +2.5 were interpreted as item fit, non-significant item-trait interaction chi-square values as overall fit [25][26][27]. Additionally, we calculated potential differential item functioning (DIF) of sex, age (above and below the median) and health condition (RA, SLE and healthy) concerning construct validity. Furthermore, we used an approach for unidimensionality testing proposed by Smith [28], namely the combination of principal component analysis (PCA) followed by a series of t-tests to assess if subsets of items result in different estimates of person parameters. As prerequisites for the t-tests, "easy" and "hard" subsets of items were selected based on PCA. The sets of items whose residual factors loaded most strongly (positively or negatively) on the first principal component factor were used because these are most likely to violate the assumption of unidimensionality. Easy items were defined as having fit residuals that loaded negatively on the first component [29]. We used Bonferroni adjustment for multiple testing regarding the level of significance of the results of the Rasch analyses. Internal consistency is an estimate of an instruments' reliability and was assessed with Cronbach's alpha (α) and Rasch reliability statistics (person separation index, PSI). PSI refers to the reproducibility of relative measure location. A high PSI value (≥ 0.7) is preferred, since this indicates a high probability that people with enhanced performance will achieve high measures (sensitivity) and vice versa [30,31]. We interpreted Cronbach's α of ≥ 0.9 as excellent, ≤ 0.89 and ≥ 0.8 as good, ≤ 0.79 and ≥ 0.7 as acceptable and ≤ 0.69 and ≥ 0.6 as questionable [23].

Ethical considerations
Participants were informed about study procedures, and confirmed their voluntary participation with written and oral informed consents. Furthermore, we guaranteed confidentiality and changed the names in the given quotes. Approval of the Ethics Committee of the Medical University of Vienna, Austria, was obtained.

Participants
Ninety people participated in the qualitative part of this study (15 people each with CD, SLE, SSc, RA or T1D and 15 healthy people). The data of 251 people (132 people with RA, 43 with SLE and 76 healthy people) were collected in the quantitative part of this study. Demographic characteristics of the participants are shown in Table 1.

Components of occupational balance
Eight components of occupational balance were identified in the analysis of the qualitative data, as described in Table 2 in the left column. In the following we give one example of how the components relate to the transcribed data. Sabine was diagnosed with systemic sclerosis at the age of 59 years. Sabine maintained occupational balance through a balance of challenging and relaxing activities: This year I was tandem parachuting four times, out of an airplane of 4000 meters height (first interview, lines 170-171). I love such challenges. I need something like this [parachuting, long distance motor cycling] in addition to my daily routine (first interview, lines 223-224).
Yesterday, we have played tennis. I must say that afterwards I need some relaxation (second interview, lines 330-331).

Occupational Balance-Questionnaire (OB-Quest)
The eight components of occupational balance identified in the qualitative analysis were used for the development of the questionnaire items, as shown in Table 2.
The first draft of the OB-Quest was piloted in 20 additional patients and healthy people. Based on their feedback some questions were reworded. The item activities intended to care for oneself and for others was deleted because patients complained that this question would not be relevant for those who did not care for others. The same applied to the example changed family circle in the item on the component adaptation of activities according to changed living conditions. Finally, seven items were formulated as presented in Table 2, second column.

Construct validity and internal consistency
The results of the overall fit, item and fit statistics referring to construct validity are shown in Table 3.
The following sets of items whose residual factors loaded most strongly were used for t-tests: variety of activities, adaptation of activities according to changed living conditions, impact of own health condition on activities, involvement in stressful and fewer stressful activities (negative loading), and challenging and relaxing activities, activities with acknowledgement by the individual and by the sociocultural context and rest and sleep (positive loading). The result of the t-test disproved the equivalence of test scores within the OB-Quest (t = 0.58, non-significant) and thus confounded our hypothesis on unidimensionality of the questionnaire. The item on the component rest and sleep showed DIF regarding the categories health condition and age. Furthermore, PSI was 0.51 and Cronbach's α 0.38, revealing low internal consistency. Therefore, we decided to remove the items on the components challenging and relaxing activities and adaptation of activities according to changed living conditions.
After deleting these items (components challenging and relaxing activities and adaption of activities to changed living conditions) the t-test and Cronbach's α changed. The new results of the t-test (t = 1.66, nonsignificant) were based on the use of the sets of items on the components involvement in stressful and fewer stressful activities and impact of health condition on activities (positive loading), activities with social acknowledgement, rest and sleep and variety of activities (negative loading). Furthermore, Cronbach's α changed to 0.57 indicating an improved internal consistency.
Because the item on the component challenging and relaxing activities confounded the unidimensionality of the OB-Quest, we decided to split this item into two concepts: too much and too little demand. Consequently, the two new items read now as follows: "1. Do you generally find your activities in your everyday life under-demanding? 2. Do you generally find your activities in your everyday life over-demanding?" Additionally, the item on the component adaptation of activities to changed living conditions misfitted the model. Accordingly this item was split into two concepts also, as shown in Table 2 right column. Furthermore due to DIF (as described above), the item on the component rest and sleep was divided into two items: one on rest and the other on sleep. Additionally, several changes were made based on the feedback of patients, e.g. particular words were reformulated (difficult to understand and/or score; Table 2). The final OB-Quest is proposed as ten items addressing seven components. Subsequently, the revised German version was forward and back translated;  Rodnan skin score = total remission < 10 (RODNAN) [37], IQR interquartile range,not applicable.
the final English and German versions of the OB-Quest are presented in Tables 4 and 5, respectively.

Discussion
This article describes the development of a new selfreported outcome instrument to assess occupational balance. Item development was based on an exploration of occupational balance in qualitative data and the involvement of patients with chronic autoimmune diseases and healthy people. Furthermore we validated and revised the OB-Quest based on quantitative data of patients with RA or SLE and healthy people. The OB-Quest includes items which have not been covered so far by instruments used to assess occupational balance. Challenging and relaxing activities and adaptation of activities have already been identified as components of occupational balance [21,38,39]. Additionally, stress has also been related to occupational balance earlier [40]. However, the four components challenging and relaxing activities, involvement in stressful activities and fewer stressing activities, impact of health condition on activities, and adaptation of activities to changed living conditions have not been included in occupational balance instruments. Different instruments have been used to assess single components of occupational balance, as identified in the current study. For example, the Short-Form 36-items Health Survey (SF-36) [41] assesses, besides other aspects, the impact of a health condition on activities of daily living (questions on health limiting daily activities). Another example is the Adolescent Stress Questionnaire which includes items on having enough time for activities others than school tasks [42]. Frequently used instruments, such as the Perceived Stress Questionnaire [43] or the Perceived Stress Scale [44] do not capture potential relations between stress and activities. However, a "composition" of the seven components, identified in our study within one instrument, has not been developed.
The results of this study indicate that occupational balance might be a multidimensional construct. Additionally, the results of the t-tests (0.58 and 1.66, respectively), indicated that the test scores vary within the OB-Quest items. Furthermore, the authors of previous studies came to the conclusion that occupational balance might be a multidimensional construct [45,46]. Moreover, the two models of occupational balance which are based on empirical quantitative and qualitative data [10,47] also challenge a unidimensional conceptualisation of occupational balance. The 8. Activities intended to care for oneself and for others multidimensionality indicates that a total score of the instrument is not justified and thus requests the summary of several of the components into subscales, as this is the case in the SF-36 [41]. Further research including components analysis is suggested to verify the various components of the OB-Quest. The concept of "occupational imbalance" should be addressed in further research. Studies of occupational imbalance are scarce and findings are diverse [48]. For example, occupational imbalance, defined as interference among occupations, was negatively associated with subjective wellbeing [49] and life satisfaction [45]. However, this was not the case in another study on occupational imbalance [48]. Currently, it is unclear whether occupational balance and occupational imbalance represent opposite poles of one single dimension or two distinct concepts [45,48].
The component activities intended to care for oneself and for others is important for the maintenance of occupational balance in people who have someone they care about. When assessing occupational balance in people who do care for others, the following question could be used: "Could you take sufficient care of yourself while caring for another (such as a family member, loved one, etc.)?" The inclusion of optional items was found to be justified in specific settings or circumstances [6]. However, due to the feedback of patients and healthy people, we decided to exclude this item of the OB-Quest.

Strengths and limitations
The development together with patients and healthy people, as well as the base on experiences of people with and without chronic autoimmune diseases strengthens the questionnaire's construct validity [5,13]. Additionally, this process followed common recommendations for the development of self-reported outcome instruments [5,50] and resulted in the identification of new components of occupational balance. Cronbach's α or the internal consistency of the questionnaire turned out to improve by the reduction of the items on the components challenging and relaxing activities and adaptation of activities according to changed living conditions. In the current study three of the four native speakers were without a health professional or patient perspective and conducted the forward and back translations of the different questionnaire versions. Another study could include  patients of various diagnoses according to the International Classification of Diseases [19]. Moreover, a translation into different languages and an examination of the instrument's psychometric properties in different countries all over the world, could allow an evaluation of its culture fairness. Additionally, a conceptual definition of occupational balance would be of great value and importance. We extracted components of occupational balance from qualitative data. However, we did not aim to provide a conceptual definition of occupational balance. The conceptualization of a concept requires a complex process of synthesizing the findings of several studies [51]. We suggest further studies on the conceptualization of occupational balance. "Occupations" or "activities" refer to all the things that you do, including very simple things, such as bathing or getting dressed. The definition of "activities" includes professional actions, free-time and relaxation activities (such as reading or sleeping), as well as childcare and the support of dependents. Please put an 'x' next to the most applicable answer to each question.

Conclusions
This questionnaire includes new items addressing components of occupational balance meaningful to patients and healthy people which have not been measured so far. The OB-Quest showed improved internal consistency after the reduction of two items, based on data obtained from patients with chronic autoimmune diseases and healthy people. The multidimensionality of the questionnaire indicates the need for a summary of several components into subscales.

Competing interests
There are no declared competing interest of the authors.
Authors' contributions MD and TS were involved into conception and design, the acquisition of data, the analysis and interpretation of data, wrote the draft manuscript, and gave final approval of the manuscript. GS contributed to the conception and design of the study, assisted the data acquisition, analysis and interpretation, and the draft version, and finally gave advice on editing of manuscript. AK-W, VF-M, CD, and JS, gave substantial contributions to conception and design, supported the acquisition of the data, have been involved in revising the draft manuscript critically, and finally approved the manuscript considered for publication. BP was involved into conception, design of the study, and item development, supported the analysis and interpretation of the data, contributed substantially to the draft manuscript and approved the final version. AB was involved into the conception and design, item development, the acquisition and the interpretation of the data, the writing of the draft manuscript and gave final approval of the manuscript. MAS was involved into the acquisition and the interpretation of the data, the writing of the draft manuscript and gave final approval of the manuscript. The authors have taken an active part in the study and take responsibility for its contents. The FWF had no influence on the manuscript. All authors read and approved the final manuscript.