An instrument to assess quality of life in relation to nutrition: item generation, item reduction and initial validation

Background It is arguable that modification of diet, given its potential for positive health outcomes, should be widely advocated and adopted. However, food intake, as a basic human need, and its modification may be accompanied by sensations of both pleasure and despondency and may consequently affect to quality of life (QoL). Thus, the feasibility and success of dietary changes will depend, at least partly, on whether potential negative influences on QoL can be avoided. This is of particular importance in the context of dietary intervention studies and in the development of new food products to improve health and well being. Instruments to measure the impact of nutrition on quality of life in the general population, however, are few and far between. Therefore, the aim of this project was to develop an instrument for measuring QoL related to nutrition in the general population. Methods and results We recruited participants from the general population and followed standard methodology for quality of life instrument development (identification of population, item selection, n = 24; item reduction, n = 81; item presentation, n = 12; pretesting of questionnaire and initial validation, n = 2576; construct validation n = 128; and test-retest reliability n = 20). Of 187 initial items, 29 were selected for final presentation. Factor analysis revealed an instrument with 5 domains. The instrument demonstrated good cross-sectional divergent and convergent construct validity when correlated with scores of the 8 domains of the SF-36 (ranging from -0.078 to 0.562, 19 out of 40 tested correlations were statistically significant and 24 correlations were predicted correctly) and good test-retest reliability (intra-class correlation coefficients from 0.71 for symptoms to 0.90). Conclusions We developed and validated an instrument with 29 items across 5 domains to assess quality of life related to nutrition and other aspects of food intake. The instrument demonstrated good face and construct validity as well as good reliability. Future work will focus on the evaluation of longitudinal construct validity and responsiveness.


Background
The intake of food is a basic human need. This basic need is accompanied by sensations of both pleasure (e.g. related to taste, social interaction) and despondency (e.g. related to indigestion, gastrointestinal disturbances, weight gain). These sensations may affect quality of life (QoL) and may be influenced by different composition and nutrient content of food stuff.
Furthermore, certain nutrients and types of diets may be associated with other patient important outcomes such as longevity, mortality and morbidity. For instance, the Mediterranean diet and high fruit and vegetable intake may lead to a range of positive health outcomes (e.g. reduction in myocardial infarction, stroke and pulmonary disease) [1]. One could therefore argue that modification of diet, given its potential for positive health outcomes, should be widely advocated and adopted. However, the feasibility and success of dietary changes will depend, at least partly, on whether potential negative influences on QoL can be avoided. It is therefore important to assess how food intake and dietary changes relate to QoL. This is of particular importance in the context of dietary intervention studies and for the development of new food products to improve health and well being. In addition, QoL related to nutrition may potentially serve as a predictor of compliance with specific dietary interventions. Social context of nutrition, such as eating together, may impact on domains such as satisfaction and happiness [2].
Instruments to measure the impact of nutrition on quality of life in the general population, however, are few and far between [3]. For example, Hatton et al. found that a prepared diet improved nutritional health perceptions and affect and reduced hassles related to nutrition in patients with cardiovascular disease [4]. The authors used four tools that were modeled on disease specific quality of life and well-being instruments. While the instruments showed face validity and indicated that the measured outcomes improved, details of item generation and item reduction for these instruments were not described. Furthermore Hatton did not address quality of life related to social aspects of life, such as interaction with others during meals, in particular in societies that place high emphasis on diet and food intake.
Therefore, the aim of this project was to develop an instrument for measuring QoL related to nutrition and food intake in the general population.

Methods
We followed standard methodology for quality of life instrument development based on the following six phases described in the framework by Guyatt and Kirshner [5]: a. Identification of Population b. Item Selection c. Item Reduction d. Item Presentation e. Pretesting of Questionnaire f. Validation of Questionnaire We focused on discriminative properties in this study, but aimed to develop an instrument that eventually will be also useable for evaluative purposes. However, we did not investigate evaluative properties in this study. The flow of the study is described in figure 1.

Search for existing instruments and identification of the population
We began our work by searching the literature for instruments measuring QoL related to food and food instruments related to QoL that could inform our work. We searched three databases (Medline, Health and Psychosocial Instruments (HAPI), CAB Abstracts) up to November 2007 and reviewed references from relevant articles (see search strategies in appendix 1). Of the 2083 citations resulting from the electronic search, there were two food related quality of life scales of particular interest that we reviewed prior to beginning the work on item generation [6,7]. Another citation described the use of questionnaires related to the impact of prepared diets on quality of life [4]. We also identified an abstract describing a nutrition QOL survey [8]. While providing potentially relevant items, these scales primarily focus on quality of life related to dietary therapy. Other instruments related to food provided potentially relevant themes such as food involvement, preparation, purchase, food diversity and social constraints related to food, but they focused on special populations. However, our focus was on the general population that might be exposed to general health messages regarding food intake and diet changes. Thus, we aimed to recruit a representative sample of the general population applying the following inclusion criteria: age greater than 18; no dietary restrictions (with the exception of vegetarian diet); able to read and speak Italian. We excluded participants with psychiatric, emotional or cognitive problems that could prevent reliable completion of the questionnaire; a diagnosis of a disease that is likely to influence completion of the questionnaire or selection of items; a major illness that substantially influences the patient's quality of life; distance of residence from recruitment centre of greater than 1 hour drive.

Item generation and selection
This phase consisted of extraction of items from the reviewed literature, feedback from nutrition experts and semi-structured personal interviews (figure 1). Three investigators interviewed participants using a semistructured questionnaire allocating up to 90 minutes to each interview. We recruited 24 participants in November and December 2007 through a consumer research agency in Rome, Italy. This consumer research agency holds a database of approximately 7000 individuals recruited since 1995 through public relation and publically available databases. Potential candidates were selected at random and invited by telephone or email to participate. Exclusion criteria for this phase of the study were work activity in the following professions: public relation, journalism, market research, marketing, food production or sales, psychology or sociology. Participants should not have participated in other interviews for at least 3 months prior to recruitment in this study. Participants received reimbursement for their travel expenses and provided informed consent. Table 1 shows the demographic characteristics of these participants.
The item generation phase generated 187 items that were partially overlapping.

Item reduction and pretesting
We grouped the collected items into similar themes before recruiting 81 participants in four Italian cities (Rome, Naples, Parma and Milan) for item reduction and initial item presentation using the same agency as for the item generation phase in addition to participants who worked in food production and sales (figure 1). Table 2 shows the demographic characteristics of these participants. Participants used 7 point Likert-type scales to rate the relative importance (not important (1) to extremely important (7)), agreement with (complete disagreement (1) to complete agreement (7)), or frequency of occurrence (never (1) to always (7)) for each of the selected items. We also began testing comprehension of draft items in the final third of the 81 participants by obtaining feedback about readability and clarity of question.
We then conducted descriptive analysis, factor analysis, bivariate correlations and item-item correlations. After eliminating items of low importance, low agreement between participants about the importance or infrequent occurrence, we grouped the remaining 105   An additional 12 items related to taste did not load on any of the domains. We then reduced the number of items in each domain by removing items with high inter-item correlations (r > 0.5) or items that covered similar aspects (e.g. separate items indicating that healthy food should prevent cancer, cardiovascular disease, hypertension and diabetes were grouped into preventing disease). We made the latter judgments through discussion and consensus of three investigators. The final list of items prior to item presentation included 31 items, two of which we considered possibly difficult to understand.

Item presentation
We presented the 31 items resulting from item reduction to an additional 12 volunteers. All items required answers on a 7 point Likert-type scale and inquired about the past two weeks. As anticipated, participants did not easily understand two items and we dropped those items before pretesting and validation of the questionnaire. Participants showed ease of understanding and there were no obvious ceiling or floor effects for the remaining 29 items (the questionnaire, named Qualcibo, is shown in appendix 2). The instrument required less than approximately 12 minutes for completion.

Validation of the questionnaire
We performed a number of validation exercises (figure 1). First, we recruited a sample of 2576 participants from the general population during introduction of a new food product in temporary shops in the cities of Milan and Rome. Potential participants entering the shops were approached by employees of the temporary shop to participate in the study. They were asked to sit down and complete a questionnaire, but we did not select information on participants who declined to participate. Participants were informed of the purpose of the study and completed the questionnaire on a computer touch screen.
Second, we recruited a sample of 128 participants on the internet through advertisement who completed the questionnaire online together with the Short Form 36 (SF-36) [9]. This recruitment was done in the context of an advertising campaign for the same new food product that was offered in the temporary shops. Advertisement was sent by email through a marketing agency that was responsible for marketing the new food product.
Third, 20 volunteers participated in a reliability study (figure 1). These participants were recruited as part of a study investigating biochemical markers of nutrient intake. They completed the questionnaire twice, approximately four weeks apart in the context of a clinic visit. No lifestyle changes were suggested to these individuals.

Statistical analysis for validation and reliability study
We calculated Pearson correlation coefficients between the items selected for presentation of the initial validation set of 2576 participants. We based the allocation of the items of Qualcibo into domains on factor analysis (principal component analysis with varimax rotation) and face validity as judged by the investigators [10]. To investigate the internal consistency of Qualcibo domains, we calculated Cronbach alpha values [10]. We evaluated convergent and discriminant cross sectional construct validity by calculating Pearson's correlation coefficients between Qualcibo domains and the SF-36 domains in 128 participants. We considered correlations of less than 0.2 as very weak, from 0.2 to 0.35 as weak, from greater than 0.35 to 0.5 as moderate, and of more than 0.5 as strong. For interpretation of the data, we used blinded a priori ratings about the strength of the correlation between the 8 domains on the SF36 and the Qualcibo from four of the authors with significant experience in quality of life research. Finally, we evaluated the test-retest reliability of Qualcibo using repeated measurements in the sample of 20 individuals who completed the instrument twice. We calculated mean scores at the two administrations and compared these scores for all 29 questions using a paired t-test and calculated Pearson's correlation coefficient. We then calculated intra-class correlation coefficients by domain where the between-rater variance estimates at the two times of administration were in the numerator and the betweenrater variance in addition to the within-rater variance of the two ratings in the denominator [11,12]. We used SPSS for Windows 14.0 and 17.0 for the statistical analyses (SPSS, Inc, Chicago, Ill).

Ethics
The item generation and item reduction phase was approved by the ethics board of the Italian National Cancer Institute "Regina Elena" in Rome, Italy. Recruited participants signed an informed consent. For the latter part of the study, the requirement for informed consents was waived by the institutional review board and the reliability data were provided by one of the investigators (CM) as part of an ongoing study that had received ethics approval by the University of Parma.

Development
Descriptive characteristics of the participants enrolled for the item generation and selection phase (n = 24) are shown in Table 1. 81 participants completed the extended item questionnaire of 187 items and their descriptive characteristics are shown in Table 2. We reduced this set of items to 29 through statistical analyses, discussion and item presentation in the item reduction phase.

Pretesting and validation
Thirty-nine percent of the 2576 participants were men. The mean age of the recruited participants was 42.2 years, with a mean weight of 66.2 kg and height of 169.7 cm. Table 3 shows the results of the factor analysis for the 29 items. The items loaded on five factors that were related to the initial clusters we identified: healthy lifestyle (n = 10 items), symptoms (n = 6), sensations (n = 6), social and role function (n = 4), taste (n = 3). Table  4 shows the internal consistency reliability for the entire set of questions and the single domains. Table 5 shows the mean scores for the five domains and table 6 the correlations among domains. The mean scores were above 4 (the mean of the score range) but the standard deviation was approximately 1.0 for all domains. We found that two items (becoming upset in relation to food intake and a feeling of happiness after a rich breakfast) were not loading uniquely on only one factor. Furthermore, one of these items (becoming upset in relation to food intake) showed a flat distribution indicating that this item may have been misunderstood by participants.

Cross-sectional construct validity
The 128 participants (35.9% male) who participated in the internet survey and completed both the Qualcibo and the SF-36 had a mean age of 35.3 (SD 10.2) years, mean weight of 63.9 (SD 13.2) kg, and were 169.7 (SD 8.9, data missing on 14 individuals) cm tall. The correlations between the domains on the Qualcibo and the SF-36 ranged from -0.078 to 0.562 (table 7). Of the 40 tested correlations, 19 were statistically significant (p < 0.01 for 12 correlations and p < 0.05 for 7 correlations). The correlations in the Qualcibo symptoms domain with all of the SF36 domains were higher (primarily in the moderate to strong category) than the other Qualcibo domains. Most of the correlations were weak to very weak. However, except for the correlations with the symptoms domain for which we expected slightly lower correlations, the direction and magnitude of the associations were generally in line with the predictions by the authors with expertise in quality of life research. In fact, 24 correlations were predicted correctly, 15 were either higher or lower by one category and one lower by two categories (predicted as moderate correlation but resulted as very weak).

Reliability
The 20 participants who completed the Qualcibo twice had a mean age of 65.7 (SD 4.4) years, weighted 76.6 (SD 8.0) kg and were 166 (SD 7.8) cm tall. Despite performing 29 tests, there were no significant differences in the mean scores for any of the 29 questions between the two administrations. The correlation coefficients between the two administrations ranged from 0.03 to 0.82 and 16 correlations were above 0.5. The lowest correlation coefficient was largely driven by one respondent who reported a 7 on the first administration and a 1 on the second administration (item 15). The correlation coefficients by domain ranged from 0.55 to 0.84 (p < 0.05 for all domains). The intra-class correlation coefficients by domain were 0.84 for healthy lifestyle, 0.71 for symptoms, 0.90 sensations, 0.77 for social and role function, 0.73 for taste.

Discussion
Applying standard methodology following an established framework, we created an instrument that evaluates quality of life related to nutrition [5]. The 29 items of the Qualcibo are simple to complete, show good face   validity, and internal consistency reliability. Evaluation of construct validity generally indicated correlations with the SF-36 of expected magnitude and direction. Reliability of the instrument is also adequate. Our literature search indicated that validated instruments for the general population in this area are absent. We identified one abstract that described the development of a nutrition quality of life screening tool [8]. We therefore believe that this instrument may find application in nutrition surveys and clinical studies.

Strength
We believe our study has several strengths. We started our work with a thorough review of the literature on existing items and generated a large list of candidate items. The extensive subsequent phases following standard methodology and using large sample sizes are another strength of this study.

Limitations
This study has some limitations. First, the generalizability of the results need to be evaluated in an international context because this study was performed in only one Mediterranean country in only one language. Second, two items showed loading on more than one factor. For example, the item dealing with satisfaction and agreement on food taste (item 21) loaded on both the sensations and the social and role function domain. One possibility for this and similar instances is that the item actually does relate to more than one domain. Alternatively, despite the detailed efforts to ensure optimal phrasing of the item the intended question may not have been specific enough. This could result in differing understanding of the item across respondents. Third, the recruitment strategy might have favoured participants with an interest in nutrition. We believe that this could have led to higher than average scores on some of the domains. Finally, we only performed cross-sectional validation, but did not address longitudinal construct validity and responsiveness.

Instrument properties
The instrument has 29 items with 5 domains: healthy lifestyle (n = 10 items), symptoms (n = 6 items), sensations (n = 6 items), social and role function (n = 4 items), and taste (n = 3 items). Mean scores were   above 4 in the large validation set of 2576 participants who likely possessed above average interest in nutrition. Further work in other large representative populations is required to establish the mean score in the general population. However, we believe that the score distribution indicates that both deterioration and improvement will be detectable in most populations.
Correlations with the SF-36 domains were very weak to strong. Although most of the correlations were weak, we expected these low correlations because our instrument focuses on domains that are only partially related to those of the SF-36 and more specific for food intake. Given that we made a priori predictions about the strength of the associations, the observed correlations indicated good construct validity. Finally, both internal consistency reliability and test-retest reliability indicate that this instrument has good psychometric properties.

Context
In the context of recommendations about diet and clinical interventions to alter risk factors, the need for instruments to assess the impact of nutrition related lifestyle changes exists [3]. This instrument is one of the first to tackle the gap of validated tools to assess the relation between nutrition and quality of life. We found that sensations, symptoms healthy lifestyle, family function, social and role function are important in the context of nutrition. The impact on those domains should be considered in the prescription of dietary interventions to patients in both the clinical and the research settings. It will be important to explore whether potential small benefits in morbidity outcomes as a result of dietary interventions studies outweigh potential negative outcomes on quality of life and vice versa. Our instrument should allow this assessment. The instrument might also be able to predict the compliance of subjects with specific dietary interventions based on reported change in QoL with the introduction of those diets. The instrument will require additional work to ensure proper translation and cultural adaptation.

Conclusions
We developed and validated an instrument to assess quality of life related to nutrition and other aspects of food intake. The instrument demonstrates promising validity and will be a suitable questionnaire for population based research on diet changes and the impact of nutrition on Qol. It can be used to determine whether dietary interventions negatively or positively influence individuals' perception of QoL related to nutrition. Further work will focus on the instruments longitudinal construct validity and responsiveness.