Skip to main content
Download PDF
Download ePub

Content validity of the National Comprehensive Cancer Network – Functional Assessment of Cancer Therapy – Breast Cancer Symptom Index (NFBSI-16) and Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function Short Form with advanced breast cancer patients

Health and Quality of Life Outcomes volume 17, Article number: 92 (2019) Cite this article

Abstract

Background

The purpose of this study is to evaluate the content validity of the National Comprehensive Cancer Network – Functional Assessment of Cancer Therapy – Breast Cancer Symptom Index (NFBSI-16) and the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function Short Form 10b among patients with hormone receptor positive (HR+)/human epidermal growth factor receptor 2 negative (HER2-) advanced breast cancer.

Methods

Cognitive debriefing interviews sought to evaluate patients’ ability to read, understand, and meaningfully respond to the questionnaires, as well as to evaluate the questionnaires’ relevance in the target patient population. Interviews were conducted by telephone and lasted approximately 90 min. Audio recordings were transcribed, anonymized, and analyzed using qualitative data analysis software.

Results

Fifteen cognitive debriefing interviews were conducted with women (mean age 66.0 years [standard deviation = 12.4]). Patients reported metastases in the bone (86.7%), liver (20.0%), lung (13.3%), skin (6.7%), and lymph node (6.7%) (not mutually exclusive). All patients for whom data were available demonstrated understanding of the instructions and the recall period of the NFBSI-16 (n = 14/14, 100.0%) and the PROMIS (n = 14/14, 100.0%). Greater than 90% of patients demonstrated understanding of each of the items in the NFBSI-16 and the PROMIS. Greater than 70% of patients demonstrated understanding of the response options of the NFBSI-16, > 90% understood response options of PROMIS Items 1–6, and ≥ 50% understood response options of PROMIS Items 7–10. Conceptual relevance was supported for most items in both questionnaires based on patients’ reports of experiencing the concepts as part of their breast cancer experience.

Conclusions

The results of the cognitive debriefing interviews provide evidence that the NFBSI-16 and PROMIS Physical Function Short Form 10b have content validity in the HR+/HER2- advanced breast cancer patient population. Patients may benefit from additional instructions at the point the response options reverse direction in the PROMIS.

Background

For patients with advanced breast cancer, both the disease process and its treatment give rise to numerous signs and symptoms that have negative impacts on their day-to-day lives (e.g., pain, skin irritation, etc.) [1]. Endocrine therapies are most commonly used in the treatment of hormone receptive-positive (HR+) advanced breast cancers, although recent studies suggest that quality of life is preserved longer in patients receiving combination therapies [2,3,4,5]. Common disease- and treatment-related symptoms experienced by patients with advanced breast cancer include pain and fatigue [6,7,8,9,10], which may be associated with more distal impacts on physical function such as decreased mobility and ability to carry out daily activities [6, 9, 11]. Few patient-reported outcome (PRO) questionnaires have been used to measure patient experience in advanced breast cancer subtypes.

In order to evaluate treatment benefits in advanced breast cancer, it is necessary to first understand the disease and treatment experience from the perspective of the patient. PRO measures can provide valuable insight into the patient experience, and their use in evaluating treatments in oncology has increased rapidly over the last several years. In considering the selection of a suitable PRO instrument to measure study endpoints, attention should be given to whether the instrument is content valid in the target patient population. Content validity refers to evidence that the PRO instrument measures concepts that are relevant to a disease and important to patients with that disease, and that the items are constructed in such a way that respondents can easily read and comprehend and to which they can provide meaningful responses [12, 13]. As stated in the United States Food and Drug Administration (FDA) Guidance for Industry – Patient-Reported Outcomes Measures: Use in Medical Product Development to Support Labeling Claims, such evidence can be established by conducting concept elicitation interviews with the target patient population to identify and describe the relevant and important concepts of a disease, and by conducting cognitive debriefing interviews with the target patient population to evaluate the comprehensibility, readability, and relevance of a PRO instrument [14].

In the current study, systematic reviews of the literature in advanced breast cancer and of available PRO instruments were conducted to identify potentially suitable PRO measures for use in an HR+/human epidermal growth factor receptor 2 negative (HER2-) advanced breast cancer population. Characteristics of the instruments of interest (including an evaluation of development histories and psychometric properties) and concepts that are considered directly related to disease status were assessed during the review [14]. As a result of these research activities, the National Comprehensive Cancer Network – Functional Assessment of Cancer Therapy – Breast Cancer Symptom Index (NFBSI-16) and the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function Short Form 10b were selected as being most suitable to measure the important and relevant concepts of interest related to disease symptoms, treatment side effects, and physical functioning impacts in this patient population. The content validity of both the NFBSI-16 and PROMIS Physical Function Short Form 10b has been evaluated previously in breast cancer and cancer populations more generally [15,16,17,18,19], but not in an HR+/HER2- advanced breast cancer population specifically. Due to differences in disease trajectory and treatments among HR and HER2 subgroups, it is important to examine content validity in this specific subtype.

The purpose of this article is to describe the content evaluation of the PRO questionnaires (NFBSI-16 and PROMIS Physical Function Short Form 10b) through cognitive debriefing interviews with patients with HR+/HER2- advanced breast cancer.

Methods

Development history of measures

Cognitive debriefing interviews sought to evaluate patients’ ability to read, understand, and meaningfully respond to the questionnaires, as well as to evaluate the questionnaires’ overall relevance and ease of completion in the HR+/HER2- advanced breast cancer patient population. Prior to describing the cognitive debriefing methods for this study, we describe the development history (including any prior evaluation of the content validity or psychometric properties) of each instrument.

The National Comprehensive Cancer Network – Functional Assessment of Cancer Therapy – Breast Cancer Symptom Index (NFBSI-16)

The NFBSI-16 is a 16-item assessment of disease-related symptoms, treatment side effects, and general function and well-being. The instrument has three subscales: Disease-Related Symptom (DRS) – nine items; Treatment Side-Effect (TSE) – four items; and General Function and Well-Being (F/WB) – three items. All items have a seven-day recall period and a five-point verbal descriptive response scale [15].

The NFBSI-16 was developed as part of a larger project to create patient-reported symptom indexes for 11 different cancer types and builds upon the original Functional Assessment of Cancer Therapy (FACT) Breast Cancer Symptom Index (FBSI), and other components of the Functional Assessment of Chronic Illness Therapy (FACIT) measurement system [16]. Open-ended concept elicitation interviews were conducted with patients diagnosed with stage III or stage IV breast cancer (N = 52) to identify symptoms or concerns most important and relevant to them in their assessment of treatment value for their breast cancer [16]. Specifically, patients were first asked to generate a list of up to 10 symptoms and rank the importance of each symptom from 0 (“not important”) to 10 (“extremely important”) [16]. Additional concept elicitation interviews were conducted with patients diagnosed with stage III or stage IV breast cancer who had experience with chemotherapy (N = 52) to identify important and relevant concepts of their advanced breast cancer experience [16]. Patients also completed the FACT-Breast questionnaire [16]. Final item selection for the NFBSI-16 was driven by quantitative (i.e., frequency counts and measure of chance endorsement) and qualitative (i.e., review of the open-ended patient interviews) evaluation of the data [15]. Psychometric evaluations of the NFBSI-16 in advanced breast cancer populations, including convergent validity analyses and known-groups methods analyses, demonstrated acceptable measurement properties [15].

The Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function Short Form 10b

The PROMIS Physical Function Short Form 10b is a 10-item measure developed from the larger PROMIS physical function item bank of 124 items [20]. PROMIS defines the physical function latent trait as the ability to perform activities of daily living (ADLs; both general and instrumental) [21]. Items align with four subcategories: the functioning of one’s upper extremities (dexterity), lower extremities (walking or mobility), and central regions (neck, back), as well as instrumental ADLs (such as running errands) [22]. Items can assess more than one of these subcategories, but generally can be assigned to one predominant category [23]. Items are meant to be answered using the present tense and have a five-point verbal rating scale.

The PROMIS Physical Function Short Form 10b was developed consistent with all PROMIS item banks and for use across disease areas [24, 25]. Further research activities were conducted to examine applicability to cancer populations. Content analysis of data from a diverse sample of patients with cancer in focus groups (N = 21) and cognitive interviews (N = 40) was used to inform domain experts’ qualitative item review. Item modifications were made to reflect cancer-specific concerns (for example, adding items regarding neuropathic pain, which was supported by data analysis and expert consensus) [17]. Similar versions of the PROMIS Physical Functioning Short Form were shown to have acceptable measurement properties (including internal consistency reliability, test-retest reliability, and construct-related validity) among advanced solid tumor populations (including breast cancer) [18, 19].

Patient recruitment

The patients who were recruited for this study, following independent review board approval, were identified at four clinical sites located in New Orleans, Louisiana (n = 10); St. Louis, Missouri (n = 3); Chicago, Illinois (n = 1); and Detroit, Michigan (n = 1). Clinicians confirmed patients’ diagnosis of HR+/HER2- advanced breast cancer, among other inclusion and exclusion criteria (Table 1), prior to patients being included in the study.

Table 1 Study inclusion/exclusion criteria
Full size table

Conduct of interviews

The cognitive debriefing interviews were conducted with patients one-on-one, over the telephone, by trained researchers. Each interview lasted approximately 90 min and was audio recorded with the subject’s prior written and verbal consent. The questionnaires were cognitively debriefed with patients following a concept elicitation exercise; the concept elicitation portion of the interviews has been reported elsewhere. The cognitive debriefing portion of the interview lasted approximately one hour.

Patients were provided electronic copies of both the NFBSI-16 and PROMIS just prior to the interview. While patients were cognitively debriefed on both questionnaires during the same interview, the order in which they were debriefed was rotated so that the same questionnaire was not debriefed first in all interviews. During the interview, patients were first asked to complete the questionnaires using a “think aloud” method. This method allowed patients the opportunity to complete each questionnaire without any interruption from the interviewer, and to describe aloud how they arrived at each answer, which helps to identify words, terms, or concepts that they may not understand or might interpret differently than intended [26]. After the patient completed the first questionnaire using the “think aloud” method, the interviewer then followed a semi-structured cognitive debriefing interview guide to elicit specific feedback on that questionnaire that might not have been covered during the think-aloud process. For example, the following types of questions were asked by the interviewer to assess patient understanding after the think-aloud exercise:

  • What does this [instruction/question/response option] mean to you?

  • Can you put this [instruction/question/response option] into your own words?

  • What made you choose [response selected by subject]?

In addition, the interviewer asked about the questionnaires’ relevance and overall ease or difficulty of completion. After cognitive debriefing was completed on the first questionnaire, the same process was followed for the second questionnaire.

Coding and analysis

Interviews were audio-recorded, transcribed, and anonymized (i.e., identifying information was removed). Transcripts were imported into a computerized qualitative data analysis package to facilitate the storing, coding and retrieval of qualitative data using Boolean operators [27]. A codebook was developed based on the components of the questionnaires and interview questions. For example, codes were created to tag patient quotes representing their interpretation of Item 1 of the NFBSI-16 and to distinguish those patients who interpreted the item as intended from those who did not: “NFBSI::Item 1::Interpreted as intended::Yes” or “NFBSI::Item 1:: Interpreted as intended::No.” A similar coding structure was followed for each component of the questionnaires and for each type of interview question.

The research team reviewed patient quotes as they related to the study objectives. Patients’ interpretations of each instruction, item, and response option in the questionnaires were evaluated, and a determination was made by the research team as to whether each patient demonstrated understanding of each component of the questionnaire per the criteria described in the study protocol and consistent with recommended practices [13].

Results were tabulated, summarized, and presented to support the content validity of the questionnaires and to identify potential areas for improvement in measurement consistent with recommended practices [13].

All frequencies and percentages reported were based on the number of patients who provided sufficient data that could be used in analysis; some data were not collected from every patient, or responses may have been insufficient or uninterpretable as determined by the research team. Therefore, total frequency and percent calculations are reported based on the number of patients for whom data were available, and not necessarily the total sample of 15.

Results

Study sample

All 15 patients were women, with an age range from 45.3 to 87.6 years (mean = 66.0, standard deviation = 12.4). The majority were white (n = 8/15, 53.3%), and the greatest number had attended some college or certificate program (n = 7/15, 46.7%) and were retired (n = 7/15, 46.7%) at the time of the interview. Clinically, the majority of patients were post-menopausal (n = 12/15, 80%), had breast cancer metastasized to the bone (n = 13, 80.0%), and had an Eastern Cooperative Oncology Group (ECOG) score of 1 (n = 11, 73.3%). Full patient demographic and health information can be found in Tables 2 and 3.

Table 2 Patient-reported demographic and health information
Full size table
Table 3 Clinician-reported health information
Full size table

Cognitive debriefing interview results

National Comprehensive Cancer Network – Functional Assessment of Cancer Therapy – Breast Cancer Symptom Index (NFBSI-16)

All patients for whom data were available (n = 14/14, 100.0%) demonstrated understanding of the instructions and the recall period of the NFBSI-16. Overall, greater than 90% of patients demonstrated understanding of each of the item questions. Specifically, all patients who provided sufficient data (n ≥ 12) demonstrated complete (100%) understanding of 14 of the 16 items of the NFBSI-16. For the remaining two items, 14 out of 15 patients (93.3%) demonstrated understanding of Item 3 (feeling ill), and 11 out of 12 patients (91.7%) demonstrated understanding of Item 10 (nausea). Additionally, greater than 70% of patients for whom data were available demonstrated understanding of all of the response options: “Not at all” (n = 12/13, 92.3%), “A little bit” (n = 10/14, 71.4%), “Somewhat” (n = 11/11, 100.0%), “Quite a bit” (n = 11/13, 84.6%), and “Very much” (n = 9/11, 81.8%). In terms of item relevance and comprehensiveness, ≥80% of patients reported experiencing at least 11 out of the 16 concepts as a part of the HR+/HER2- advanced breast cancer experience (lack of energy, pain, feeling ill, shortness of breath, family role, fatigue, pain, worry, ability to work, ability to enjoy life, and quality of life), while between 42 and 67% of patients endorsed the remaining five concepts (bone pain, nausea, side effects, hair loss, mouth sores). Overall, the majority of patients reported that they did not believe additional concepts should be added to the questionnaire and that the questionnaire was easy to complete (n = 11/14, 78.5% and n = 12/13, 92.3%, respectively) (Table 4).

Table 4 NFBSI-16 cognitive debriefing summary Table (N = 15)
Full size table

Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function Short Form 10b

All patients for whom data were available (n = 14/14, 100.0%) interpreted the instructions of the PROMIS as intended. Overall, greater than 70.0% of patients demonstrated understanding of each item as intended. Specifically, all patients who provided sufficient data (n ≥ 11) demonstrated complete understanding (100%) of eight of the 10 PROMIS items. For the remaining two items, 13 out of 14 patients (92.9%) demonstrated understanding of Item 7 (vigorous physical activities), and 11 out of 12 patients (91.7%) demonstrated understanding of Item 9 (putting trash outside).

Additionally, two sets of response options were debriefed. The first set of response options, “Without any difficulty, With a little difficulty, With some difficulty, With much difficulty, Unable to do,” used for Items 1–6, were understood by > 90% of patients for whom data were available (n ≥13). The second set of response options was less well understood by patients; four of the five response options were interpreted as intended by ≥50% of patients for whom data were available (n ≥ 10) (“Not at all, Somewhat, Quite a lot, and Cannot do”), and only 25.0% for whom data were available (n = 12) demonstrated understanding of the response option “Very little.” In terms of item relevance, ≥75.0% of patients reported at least nine out of the 10 concepts as part of their HR+/HER2- advanced breast cancer experience (ability to do chores, ability to go up and down stairs, ability to run errands or shop, ability to bend down, ability to lift weight above shoulders, vigorous physical activities, bathing or dressing, moderate physical activities, and ability to get in and out of car), while 57.1% of patients endorsed the remaining concept (putting trash outside). Additionally, while the PROMIS has no stated recall period, almost all patients (n = 13, 86.7%) answered the items based on their current status. Overall, the majority of patients (n = 11/13, 84.6%) did not believe additional concepts should be added to the questionnaire. All of the patients (n = 15/15, 100.0%) indicated that the questionnaire was easy to complete (Table 5).

Table 5 PROMIS Physical Function Short Form 10b cognitive debriefing summary Table (N = 15)
Full size table

Discussion

For the NFBSI-16, cognitive debriefing results showed strong support for the content validity of the measure in a HR+/HER2- advanced breast cancer population; all of the patients in the sample for whom data were available understood the instructions and the majority of the items (14 out of 16), while the remaining two items were understood by at least 90% of the patient sample. Further, the majority of the concepts measured by the NFBSI-16 were relevant to at least 80% of the sample, and all of the concepts were relevant to at least 40%. Similarly, at least 90% of the patient sample demonstrated understanding of the instructions and items of the PROMIS Physical Function Short Form 10b. Almost all of the concepts were relevant to at least 75% of the sample. The cognitive debriefing results for the NFBSI-16 and PROMIS Physical Function Short Form 10b demonstrated that HR+/HER2- advanced breast cancer patients were able to complete and comprehend the questionnaires in ways consistent with developer and researcher expectations, and that overall, the concepts covered by the questionnaires are relevant to the patient experience of HR+/ HER2- advanced breast cancer.

While some patients did not utilize the response options as expected for some items in the PROMIS, they demonstrated understanding of the item-concepts and the response scale overall. For example, Item 7 asked “Does your health now limit you in doing vigorous activities…?” and a subject selected “Not at all” with the rationale that she could not do vigorous activities at all, though the correct response option for her ability would have been “Cannot do.” The misunderstanding of the response scale may potentially have been a result of the negatively worded items, which switched direction from the previous items in the questionnaire and may be more prone to response error. This may also explain why the response options “Very little” and “Quite a lot” were less well understood by patients (n = 3/12, 25.0% and n = 6/10, 60.0%, respectively); however, the response option “Somewhat” was well understood (n = 8/10, 80.0%), as middle response options may be understood regardless of response scale direction. Another possible explanation could be that both ends of the response scale have negatively worded anchors (i.e., “Not at all” and “Cannot do”), which may have contributed to some of the misinterpretation.

In general, if a majority of patients understand the content as it is intended, there is justification to leave the questionnaire as is. Despite cognitive interviews being a widely used method, there are various approaches to interpreting data. For instance, there are no established and agreed upon thresholds for determining whether modifications are needed to a PRO instrument based on cognitive interview results. Some studies, such as the current one, do not employ any a priori threshold, and modifications might be deemed beneficial and necessary even though the majority demonstrated understanding. In its paper on establishing content validity, the International Society For Pharmacoeconomics and Outcomes Research (ISPOR) Task Force acknowledges the lack of clarity around whether modifications to an instrument are warranted if only a minority of patients misinterpret the content [13]. Ultimately, the decision to make any modifications should be made in consideration of whether the proposed change would increase the overall content validity of the instrument. In the current study, additional instructions before the second set of response options in the PROMIS Physical Function Short Form 10b could be one suggestion to help orient the respondent to the change in the directional nature of the items and response scale, and potentially help alleviate response error.

The limitations of the study center on the clinical and demographic characteristics of the patient sample, which may impact generalizability of results. Findings on the content validity of the questionnaires may not be generalizable to other subtypes of breast cancer or be comprehensive across all treatment experiences or tumor types, alongside the fact that the more severely ill may be less likely to participate in the study. In addition, all patients in this sample had completed some college, and most were treated at a single center in Louisiana.

Conclusions

The results of the cognitive debriefing interviews provide evidence that the NFBSI-16 and PROMIS Physical Function Short Form 10b assess the disease-related symptoms, treatment-related side effects, and physical functioning impact concepts that are important and relevant to HR+/HER2- advanced breast cancer patients, and do so in ways that patients can understand and to which they can meaningfully respond. Additional instructions on the PROMIS Physical Function Short Form 10b to orient the respondent to the response scale direction may increase respondent understanding. These findings add to the evidence of content validity of the NFBSI-16 and PROMIS Physical Function Short Form 10b in an advanced breast cancer patient population. Future research should further confirm the questionnaires as being “fit for purpose” in the target patient population via psychometric evaluation and score interpretation in regulated clinical trials.

Availability of data and materials

The datasets generated and/or analyzed during the current study are generally available from the corresponding author upon reasonable request. However, portions of the content may not be available due to confidentiality reasons.

Abbreviations

ADLs:

activities of daily living

DRS:

Disease-Related Symptom

ECOG:

Eastern Cooperative Oncology Group

F/WB:

General Function and Well-Being

FACIT:

Functional Assessment of Chronic Illness Therapy

FACT:

Functional Assessment of Cancer Therapy

FBSI:

Functional Assessment of Cancer Therapy Breast Cancer Symptom Index

HER2-:

human epidermal growth factor receptor 2

HR + :

hormone receptive-positive

ISPOR:

International Society for Pharmacoeconomics and Outcomes Research

NFBSI-16:

National Comprehensive Cancer Network – Functional Assessment of Cancer Therapy – Breast Cancer Symptom Index

PRO:

patient-reported outcome

PROMIS:

Patient-Reported Outcomes Measurement Information System

TSE:

Treatment Side Effect

References

  1. National Cancer Institute. Stages of breast Cancer. 2014. http://www.cancer.gov/cancertopics/pdq/treatment/breast/Patient/page2. Accessed 2/11/2016.

  2. Baselga J, Campone M, Piccart M, Burris HA 3rd, Rugo HS, Sahmoud T, et al. Everolimus in postmenopausal hormone-receptor-positive advanced breast cancer. N Engl J Med. 2012;366(6):520–9. https://doi.org/10.1056/NEJMoa1109653.

    Article  CAS  PubMed  Google Scholar 

  3. Cardoso F, Costa A, Norton L, Cameron D, Cufer T, Fallowfield L, et al. 1st international consensus guidelines for advanced breast cancer (ABC 1). Breast. 2012;21(3):242–52. https://doi.org/10.1016/j.breast.2012.03.003.

    Article  CAS  PubMed  Google Scholar 

  4. Burris HA 3rd, Lebrun F, Rugo HS, Beck JT, Piccart M, Neven P, et al. Health-related quality of life of patients with advanced breast cancer treated with everolimus plus exemestane versus placebo plus exemestane in the phase 3, randomized, controlled, BOLERO-2 trial. Cancer. 2013;119(10):1908–15. https://doi.org/10.1002/cncr.28010.

    Article  CAS  PubMed  Google Scholar 

  5. Harbeck N, Iyer S, Turner N, Cristofanilli M, Ro J, Andre F, et al. Quality of life with palbociclib plus fulvestrant in previously treated hormone receptor-positive, HER2-negative metastatic breast cancer: patient-reported outcomes from the PALOMA-3 trial. Ann Oncol. 2016;27(6):1047–54. https://doi.org/10.1093/annonc/mdw139.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Beaumont T, Leadbeater M. Treatment and care of patients with metastatic breast cancer. Nurs Stand. 2011;25(40):49–56. https://doi.org/10.7748/ns2011.06.25.40.49.c8566.

    Article  CAS  PubMed  Google Scholar 

  7. Badr H, Laurenceau JP, Schart L, Basen-Engquist K, Turk D. The daily impact of pain from metastatic breast cancer on spousal relationships: a dyadic electronic diary study. Pain. 2010;151(3):644–54. https://doi.org/10.1016/j.pain.2010.08.022.

    Article  PubMed  Google Scholar 

  8. Bender CM, Ergyn FS, Rosenzweig MQ, Cohen SM, Sereika SM. Symptom clusters in breast cancer across 3 phases of the disease. Cancer Nurs. 2005;28(3):219–25.

    Article  PubMed  Google Scholar 

  9. Irvin W Jr, Muss HB, Mayer DK. Symptom management in metastatic breast cancer. Oncologist. 2011;16(9):1203–14. https://doi.org/10.1634/theoncologist.2011-0159.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Walberg MP, Ravnan SL. Metastatic breast cancer: a review of current and novel pharmacotherapy. Formulary. 2011;46:130–46.

    Google Scholar 

  11. Mosher CE, Johnson C, Dickler M, Norton L, Massie MJ, DuHamel K. Living with metastatic breast cancer: a qualitative analysis of physical, psychological, and social sequelae. Breast J. 2013;19(3):285–92. https://doi.org/10.1111/tbj.12107.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity-establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1-eliciting concepts for a new PRO instrument. Value Health. 2011;14(8):967–77.

    Article  PubMed  Google Scholar 

  13. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity-establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 2-assessing respondent understanding. Value Health. 2011;14(8):978–88.

    Article  PubMed  Google Scholar 

  14. Center for Drug Evaluation and Research, Center for Devices and Radiological Health, Center for Biologics Evaluation and Research. 12/2009. https://www.fda.gov/media/77832/download. Accessed 05/15/2019.

  15. Garcia SF, Rosenbloom SK, Beaumont JL, Merkel D, Von Roenn JH, Rao D, et al. Priority symptoms in advanced breast cancer: development and initial validation of the National Comprehensive Cancer Network-Functional Assessment of Cancer therapy-breast Cancer symptom index (NFBSI-16). Value Health. 2012;15(1):183–90. https://doi.org/10.1016/j.jval.2011.08.1739.

    Article  PubMed  Google Scholar 

  16. Cella D, Rosenbloom SK, Beaumont JL, Yount SE, Paul D, Hampton D, et al. Development and validation of 11 symptom indexes to evaluate response to chemotherapy for advanced cancer. J Natl ComprCancNetw. 2011;9(3):268–78.

    Google Scholar 

  17. Garcia SF, Cella D, Clauser SB, Flynn KE, Lad T, Lai JS, et al. Standardizing patient-reported outcomes assessment in cancer clinical trials: a patient-reported outcomes measurement information system initiative. J Clin Oncol. 2007;25(32):5106–12 doi:25/32/5106 [pii];10.1200/JCO.2007.12.2341 [doi].

    Article  PubMed  Google Scholar 

  18. Jensen RE, Potosky AL, Reeve BB, Hahn E, Cella D, Fries J, et al. Validation of the PROMIS physical function measures in a diverse US population-based cohort of cancer patients. Qual Life Res. 2015;24(10):2333–44. https://doi.org/10.1007/s11136-015-0992-9.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Yost KJ, Eton DT, Garcia SF, Cella D. Minimally important differences were estimated for six patient-reported outcomes measurement information system-Cancer scales in advanced-stage cancer patients. J Clin Epidemiol. 2011;64(5):507–16. https://doi.org/10.1016/j.jclinepi.2010.11.018.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, et al. The patient-reported outcomes measurement information system (PROMIS): progress of an NIH roadmap cooperative group during its first two years. Med Care. 2007;45(5 Suppl 1):S3–S11. https://doi.org/10.1097/01.mlr.0000258615.42478 55 [doi];00005650-200705001-00002 [pii].

    Article  PubMed  PubMed Central  Google Scholar 

  21. Fries JF, Cella D, Rose M, Krishnan E, Bruce B. Progress in assessing physical function in arthritis: PROMIS short forms and computerized adaptive testing. J Rheumatol. 2009;36(9):2061–6. https://doi.org/10.3899/jrheum.090358.

    Article  PubMed  Google Scholar 

  22. HealthMeasures. List of adult measures. 2016. http://www.healthmeasures.net/explore-measurement-systems/neuro-qol/intro-to-neuro-qol/list-of-adult-measures. Accessed 10/27/2016.

  23. Rose M, Bjorner JB, Becker J, Fries JF, Ware JE. Evaluation of a preliminary physical function item bank supported the expected advantages of the patient-reported outcomes measurement information system (PROMIS). J Clin Epidemiol. 2008;61(1):17–33. https://doi.org/10.1016/j.jclinepi.2006.06.025.

    Article  CAS  PubMed  Google Scholar 

  24. DeWalt DA, Rothrock N, Yount S, Stone AA. Evaluation of item candidates: the PROMIS qualitative item review. Med Care. 2007;45(5 Suppl 1):S12–21. https://doi.org/10.1097/01.mlr.0000254567.79743 e2 [doi];00005650-200705001-00003 [pii].

    Article  PubMed  PubMed Central  Google Scholar 

  25. Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010;63(11):1179–94. https://doi.org/10.1016/j.jclinepi.2010.04.011.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Ericsson KA, Simon HA. Protocol analysis: verbal reports as data. Cambridge, MA: MIT Press; 1993.

    Google Scholar 

  27. Lewins A, Silver C. Using software in qualitative research: a step-by-step guide. London: Sage Publications; 2007.

    Book  Google Scholar 

Download references

Acknowledgments

N/A

Funding

Funding for this manuscript was provided by Novartis Pharmaceuticals.

Author information

Authors and Affiliations

  1. Adelphi Values, 290 Congress St, 6th Floor, Boston, MA, 02210, USA

    Meaghan Krohe, Brittany Klooster & Nina Galipeau

  2. Novartis Oncology, One Health Plaza, 345/5130E, East Hanover, NJ, 07936, USA

    Derek H. Tang

  3. Evidera, 7101 Wisconsin Avenue Suite 1400, Bethesda, MD, 20814, USA

    Dennis Revicki

  4. Northwestern University, 633 N. St Clair, 19th Floor, Chicago, IL, 60611, USA

    David Cella

Authors
  1. Meaghan Krohe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Derek H. Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brittany Klooster
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dennis Revicki
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nina Galipeau
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David Cella
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MK, NG, and BK contributed to study design, management, data collection, and data analysis. DT, DR, and DC all contributed to study design and helped to draft the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Meaghan Krohe.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from Quorum Review IRB under QR# 31726/1.

Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent: Informed consent was obtained from all individual participants included in the study.

Consent for publication

Not applicable.

Competing interests

Nina Galipeau, Brittany Klooster, and Meaghan Krohe were employees of Adelphi Values at the time this manuscript was written, which received payment from Novartis Pharmaceuticals to assist with manuscript preparation. Derek Tang was an employee of Novartis Pharmaceuticals. Dennis Revicki and David Cella were paid consultants to Novartis related to this project.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krohe, M., Tang, D., Klooster, B. et al. Content validity of the National Comprehensive Cancer Network – Functional Assessment of Cancer Therapy – Breast Cancer Symptom Index (NFBSI-16) and Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function Short Form with advanced breast cancer patients. Health Qual Life Outcomes 17, 92 (2019). https://doi.org/10.1186/s12955-019-1162-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12955-019-1162-5

Keywords

Health and Quality of Life Outcomes

ISSN: 1477-7525

Contact us