This qualitative study was performed through interviews with 25 adult participants with lupus from the Washington D.C. area. The interviews were conducted by Evidera, a healthcare research firm with expertise in outcomes research, instrument development, and validation. The first phase included 10 patients in 45–60 min concept elicitation interviews conducted by telephone. The second phase included 15 patients with in-person or over the phone cognitive debriefing interviews.
Prior to the interviews, IRB approval was obtained. Written informed consent and a medical release of information form were obtained from the participants. Interviews were performed by experienced qualitative interviewers. A semi-structured interview guide for each phase was used to ensure consistency across interviews. In addition to the questions asked during the interviews, participants were invited to complete the LFA-REAL PRO questionnaire, a health-related quality of life questionnaire, and a socio-demographic form. All interviews were audio-recorded. Participants’ treating physicians were contacted to complete a brief clinical form to confirm lupus diagnosis.
Recruitment of participants
The patient registries and networks of the Lupus Foundation of America (LFA) were used to identify potential participants. Efforts were made to recruit a diverse sample representative of the lupus population, including men, women, and people of African American and Hispanic origin. Interested respondents were directed to the LFA website for more information and to complete an online interest form containing pre-screening questions. If the person seemed likely to meet the eligibility requirements (see below) based on the online interest form, a member of the LFA staff conducted a full screening over the phone using a standardized recruitment script to verify the information provided in the survey. LFA staff tracked recruitment efforts using a secure, password-protected participant recruitment tracking log. Enrolled participants were scheduled for interviews either by telephone, on-site at Evidera, or at the LFA office.
Participant flow chart is included in the Additional file 1 section. A total of 25 participants were recruited to participate in the study (Phase I and Phase II). Out of the 665 that responded to the online screening questions, 435 completed the online screening form successfully and of those, 219 were found to be ineligible. The most common reason for ineligibility was the participant’s willingness to participate in a telephone or in-person interview. Among the 216 who were found to be eligible based on the online pre-screening questions, we were unable to reach 33 and did not need to contact 153, as additional participants were not needed once content saturation was achieved. The remaining 30 were eligible and immediately enrolled in the study. Only 26 of the 30 were successfully scheduled, with 25 fully completing their interview. The remaining participant did not show up for the interview. Of the 25 people interviewed in the study, ten participated in phase I concept elicitation telephone interviews and 15 participated in phase II cognitive interviews (n = 11 by telephone, n = 2 in-person at the LFA office in Washington D.C. and n = 2 in-person at the Evidera office in Bethesda, MD).
Participants who met the following inclusion/exclusion criteria were invited to participate in the study.
Inclusion Criteria: ≥18 years of age at the time of consent; medical diagnosis of SLE; able to read, speak, and understand English sufficiently to complete all assessments; willing and able to provide written informed consent to participate in research; and willing and able to provide a signed medical release form to verify SLE diagnosis with the treating physician.
Exclusion Criteria: the presence of a cognitive impairment, hearing difficulty, visual impairment, acute psychopathology, or insufficient knowledge of the interview language that, in the opinion of the investigator/interviewer, could interfere with his or her ability to provide written consent and complete an interview; previous participation in any study related to the development of the LFA-REAL PRO; currently participating in an interventional research study; unable to complete the study procedures; or unwilling to have the interview audio-recorded.
Phase I (concept elicitation)
Participants were asked questions about their experience with lupus, including symptoms, treatment, and how lupus impacts their daily life. Participants were mailed the forms which they completed and returned the forms to Evidera using a pre-paid mailing envelope.
The primary objective of the concept elicitation interviews was to elicit emergent descriptions of the participants’ experience of lupus. These included concepts related to kinds of lupus symptoms, information regarding disease activity and worsening of symptoms, the impact of lupus features on daily function and quality of life, and other aspects of the lupus condition that patients consider important. Participants were also asked to provide feedback on the draft LFA-REAL PRO questionnaire that they completed during the interview.
Phase II (cognitive debriefing interviews)
Cognitive debriefing interviews focused on eliciting feedback regarding the following items: 1) clarity of each item; 2) interpretability of each items 3) ease of completion of the instrument; 4) comprehensiveness of the instrument; and 5) appropriateness of the format, instructions, response scales, and recall period used.
A semi-structured interview guide was used to guide the interview. Participants were asked to complete the LFA-REAL PRO and to provide feedback on its content, structure, response options, and understandability. At the completion of the interview, participants completed the SF-36 questionnaire and the socio-demographic and clinical questionnaire. All interviews were audio-recorded.
Semi-structured interview guide
Standardized semi-structured interview guides were used in the concept elicitation and cognitive debriefing interview phases. The semi-structured interview guide for Phase I was developed based on the results of preliminary concept identification work conducted by the LFA. The semi-structured interview guide for Phase II was developed based on the results from Phase I and the content of the LFA-REAL PRO.
Measures
LFA-rapid evaluation of activity in lupus PRO (LFA-REAL PRO)
With a recall period of four weeks, the LFA-REAL PRO asks patients to evaluate the severity and progress of symptoms associated with their current lupus activity along a series of anchored visual analog scales (VAS). Items evaluated separately include rash, symptoms of arthritis (i.e., joint pain, joint swelling, and joint stiffness), a summary arthritis score that integrates those three features, muscle pains or aches, body symptoms (e.g., shortness of breath), fatigue, fever, and hair loss. These items evolved and were modified in Phase II based on patient concept elicitation results, and the final instrument incorporated feedback from Phase II. A pre-defined component of the instrument was to break arthritis and mucocutaneous symptoms down into discrete components that might allow patients to describe similar features to those that physicians consider in the LFA REAL™ CLINRO. For example, a score for overall arthritis is not elicited until after the patient scores each component of joint pain, joint stiffness, and joint swelling, in effect, teaching patients through this process how physicians evaluate arthritis. The LFA-REAL was not shown to the patients until after the concept elicitation and cognitive debriefing interviews were completed, both of which confirmed the validity of the descriptors and provided edits to the wording used for the instructions and labels. The current version of the PRO is presented in Fig. 1.
SF-36 (version 1.0 standard; Rand scoring) questionnaire
This 36-item questionnaire reports current experiences or experiences over the past four weeks using Likert-type response scales to assess patient health status and quality of life. Scores range from 0 to 100, with higher scores indicating higher levels of functioning. Scores were generated for each of the eight domains (i.e., physical functioning, role limitations due to physical problems, bodily pain, general health perceptions, vitality, social functioning, and role limitations due to emotional problems and mental health) and two summary domains (i.e., physical component summary [PCS] and mental component summary [MCS]).
The SF-36 was chosen to describe the population based on its extensive use in SLE studies [25]. We acknowledge the shortcomings of the SF-36 in SLE and its potential for floor effects and in the validation study we plan to include several other PROs [26]. SF-36 scoring was based on the original orthogonal rotation [27, 28].
“Questions about You” form (a socio-demographic and clinical questionnaire)
Participants completed the brief “Questions About You” form, a socio-demographic and clinical questionnaire, after completing the semi-structured interviews. The “Questions About You” form asked participants to provide socio-demographic information including, but not limited to, age, race, employment, education, and basic clinical information. This information was used for descriptive purposes to characterize the population sample.
Clinical form
For eligible participants, treating physicians were asked to complete a clinical form to confirm and date the diagnosis of lupus.
Data analysis
Quantitative data
A database for all quantitative data collected during the study was developed, tested, and validated using DataFax system, a direct fax-to-computer software using optical character recognition (OCR) to collect study case report forms (CRFs). DataFax is an FDA Title 21 Code of Federal Regulations Part 11 Compliant system that provides a time-stamped electronic audit trail for the creation, modification, or deletion of electronic data.
Electronic images of the faxed CRFs (sociodemographic, clinical data and questionnaire item scores) were entered into the database and reviewed by project scientific staff. Data discrepancies were identified and resolved.
Qualitative data
All interviews were digitally recorded, and audio recordings were transcribed. An Evidera project team member reviewed the interview transcripts for content and removed any participant-identifying information. The de-identified transcript was used for analysis. Qualitative data was analyzed using ATLAS.ti qualitative data analysis software version 7.5, which is designed for the qualitative analysis of textual, graphical, audio, and video data. It is a concept database that allows the researcher to create and enter names of concepts, or “codes,” to be used for conceptualizing large amounts of qualitative data. The program allows the analyst to organize and relate these concepts to each other in order to evaluate the underlying structure of the qualitative data.
A coding dictionary was developed by Evidera based on the structure of the interview guide and other concepts of interest, and imported to ATLAS.ti. Two different staff members independently reviewed and coded the first interview transcript, and the results were compared and reconciled. Once there was sufficient agreement between the initial coders, the rest of the transcripts were processed. Concepts mentioned within each transcript were assigned codes from the coding dictionary. If new codes were identified, they were added to the coding dictionary. Outputs were generated based on these codes, which were entered into a saturation grid in order to compare and tally the amount of novel information observed in each subsequent interview. Saturation was achieved when no novel information was gathered. Concepts or concerns that emerged were used to determine revisions made to the LFA-REAL PRO as described above. All revisions—and the rationale behind the revisions—were tracked using an item-tracking matrix.