Skip to content

Advertisement

  • Research
  • Open Access

Instrument completion and validation of the patient-reported apnea questionnaire (PRAQ)

Health and Quality of Life Outcomes201816:158

https://doi.org/10.1186/s12955-018-0988-6

  • Received: 5 March 2018
  • Accepted: 26 July 2018
  • Published:

Abstract

Background

We previously developed the preliminary version of the Patient-Reported Apnea Questionnaire (PRAQ), a questionnaire measuring health-related quality of life in patients with (suspected) obstructive sleep apnea (OSA). This questionnaire was developed for clinical practice, where it can potentially serve two goals: use on an individual patient level to improve patient care, and use on an aggregate level to measure outcomes for quality improvement at a sleep center. In this study we aim to finalize the PRAQ, make a subselection of items and domains specifically for outcome measurement, and assess the validity, reliability and responsiveness of the PRAQ.

Methods

Patients with suspected OSA were included and asked to complete the PRAQ and additional questionnaires one or more times. The collected data was used to perform the final item selection for clinical practice and for outcome measurement, create the domains for outcome measurement, and assess the measurement properties internal consistency, test-retest reliability, convergent validity and responsiveness.

Results

180 patients were included in the study. The final version of the PRAQ for use in clinical practice contains 40 items and 10 domains. A subselection of 33 items in 5 domains was selected for optimal outcome measurement with the PRAQ. The results for the outcome measurement domains were: Cronbach’s alpha 0.88–0.95, ICC 0.81–0.88, and > 75% of hypotheses correct for convergent validity and responsiveness.

Conclusions

The PRAQ shows good measurement properties in patients with (suspected) OSA.

Keywords

  • Obstructive sleep apnea
  • Instrument development
  • Patient-reported outcome measure
  • Psychometrics

Background

Patients with obstructive sleep apnea (OSA) experience breathing stops while asleep, causing symptoms during the day such as excessive sleepiness, tiredness, and irritability. This can have a large impact on daily functioning of patients, and often affects a patient’s relationships and psychological wellbeing [13]. Furthermore, OSA is a known risk factor for comorbidities such as diabetes and heart failure [46], and is also associated with depression and anxiety [79]. Gaining an overview of the problems that OSA patients may experience, before, during, and in evaluating treatment, may be a challenge.

Patient-reported outcome measures (PROMs) are questionnaires for patients about symptoms or daily functioning. Most PROMs have been developed for use in clinical trials, but interest in their use in daily practice is growing [10, 11]. There, PROM scores can be used on an individual patient level to help bring patients’ problems to the forefront during consultations and to monitor treatment response, or on an aggregate level across groups of patients for quality improvement purposes [12]. Use of a PROM on an individual patient level may be especially relevant when patient symptoms are multiple and complex. We therefore believe that it would be beneficial to employ a PROM for patients with OSA in daily clinical practice.

In a recently published article, we described the item generation and preliminary item selection of a PROM for patients with (suspected) OSA: the Patient-Reported Apnea Questionnaire (PRAQ) [13]. We used the input of patients with OSA and healthcare professionals to select topics and items important for measuring quality of life for this patient group, which are also useful to discuss during an intake or follow-up consultation.

There are two ways in which the preliminary version of the PRAQ requires further development. First, the item reduction for the topic “sleepiness” has not yet taken place. During the item selection process of the PRAQ, patients indicated that the number of items on the topic of sleepiness could be reduced. Since the patients had no preference for which items to exclude, we decided to perform the final item reduction after studying the psychometric properties of the items. Second, the factor structure of the PRAQ has not yet been studied, and we wanted to find the optimal way to group (a subset of) the items of the PRAQ into domains for the purpose of outcome measurement.

Our aim is for the PRAQ to be employed in the following way: patients complete all items of the PRAQ before their consultation, the results of which can be discussed with a healthcare professional; and the aggregate outcomes of groups of patients can then be studied by making use of a subset of the completed items. This is beneficial for patients, who get feedback from clinicians on their results; for physicians, who get a quick insight into their patients’ main problems; and for sleep centers that wish to collect outcome data for quality improvement, because it ensures a steady stream of data due to integration in clinical practice.

In this article we describe the further development of the preliminary PRAQ. In addition, we aim to determine the reliability, validity and responsiveness of the PRAQ, with a focus on the domains that will be used for outcome measurement.

Methods

Population & method of completion of the PROMs

Baseline measurement

Patients referred to the sleep center of the Albert Scheitzer Hospital in Dordrecht, The Netherlands for suspected OSA received an invitation by email to complete the PRAQ and additional PROMs, 2–3 weeks before their intake consultation. They were informed that the results of the PRAQ would be discussed during their intake consultation. A reminder was sent one week later if the PROMs were not yet completed at that time. Patients who had not completed the PRAQ at home were offered the option of completing the PRAQ at the sleep center before their consultation.

Retest measurement

In order to assess test-retest reliability, patients who had completed the baseline measurement at home were asked to complete it again immediately before their intake consultation, on a computer in a private area of the sleep center. Only patients who had completed the retest no less than 7, and no more than 21 days after the baseline measurement were included for assessment of test-retest reliability.

Follow-up measurement

A common measure to express the number of (partial) breathing stops experienced while asleep is the Apnea-Hypopnea Index (AHI). We measured the responsiveness of the PRAQ in patients with an AHI ≥ 15, which indicates moderate to severe sleep apnea [14], and who were prescribed continuous positive airway pressure (CPAP) after their intake consultation. CPAP is the preferred treatment for OSA [15]. If the patients were still using CPAP at the time of the first follow-up consultation (6–8 weeks after start of CPAP), they were included for responsiveness. They were asked to complete the PRAQ and the additional PROMs immediately before their follow-up consultation at the sleep center. Ideally, responsiveness should be determined in a patient group in which CPAP therapy is successful and therefore a substantial change is expected with regard to the patient’s symptoms. CPAP therapy is generally considered successful when compliance is ≥4 h nightly [16].

A secure website was used for the completion of the PROMs. For any of the measurements, patients who were unable or unwilling to use a computer were offered the option of completing a paper copy of the PROMs.

Final stage of PRAQ development

The development article of the preliminary PRAQ [13] shows how the initial 43 items were selected based on their relevance for clinical practice and were sorted into preliminary domains: symptoms at night (6 items), sleepiness (8 items), tiredness (3 items), daily activities (5 items), unsafe situations (2 items), memory and concentration (2 items), quality of sleep (2 items), emotions (6 items), social interactions (8 items), and health concerns (1 item) (Appendix 1). All items are scored on a 7-point Likert scale (higher scores indicate worse problems), and the average item scores in a domain form its domain score.

First, we performed item reduction on the sleepiness domain, as 8 items was deemed too much by patients. Then, we looked at how the PRAQ could be best used for outcome measurement. It is important that all items fit into a domain that is either “coherent” in terms of clinical relevance, or (preferably) in terms of covariance matrix as determined by principal component analysis (PCA). Therefore, our aim was to identify which items of the PRAQ can be grouped into domains for outcome measurement after use of the results of the PRAQ for an individual patient. We describe below how we first reduced the number of items for the domain ‘sleepiness’, and then how from the remaining items a subset of items was selected for outcome measurement.

Item reduction of the sleepiness domain

During the development of the PRAQ, patients indicated that they felt that the number of items on the topic of sleepiness could be reduced. Because they had no preference for which items should be excluded, we took a statistical approach. We first looked for items with a high inter-item correlation (> 0.9), indicating that one of these items can be removed without a substantial loss of information [17]. As a second step, we used exploratory factor analysis to identify potential items with lower factor loadings (< 0.5), indicating that they do not cover the construct as well as the other items and are therefore more suitable for removal [17, 18].

Creating domains for the PRAQ-outcome

Two of our preliminary domains, ‘symptoms at night’ and ‘social interactions’, we considered formative rather than reflective domains: they do not aim to measure aspects of the same latent construct, but the items are grouped together based on clinical relevance. Grouping items in this way can be considered a “clinimetric” approach, as opposed to a “psychometric” approach which uses statistical methods to determine the dimensionality of a PROM [19]. We wanted to group these items together irrespective of their covariance matrix, because for content reasons we did not consider it desirable to combine these items with any of the other (potential) domains. Therefore, we excluded them from the PCA and kept these domains as they were.

We performed a PCA with oblique rotation (because correlations between the different patient complaints were expected) on the 26 items of the other preliminary domains. Items that did not load on any domain with a factor loading of at least 0.5 or that had a factor loading of > 0.3 on more than one factor [17], were then one by one removed from the analysis, starting with those items that for content reasons did not seem to fit well with the items they were grouped with in the PCA. Additionally, since domains should ideally consist of at least three items, we used this as a requirement for the PRAQ-outcome domains [17]. The one-dimensional domains that were identified by the analysis were added to the two clinimetric domains. Together, these domains form the subset of the PRAQ that can be used for outcome measurement.

Assessment of measurement properties

We studied the distribution of the individual items and the PRAQ domain scores at baseline to check for floor and ceiling effects (i.e. whether < 15% of the respondents achieved the highest or lowest possible scores [20]). We assessed the reliability, validity and responsiveness of the PRAQ following the taxonomy of measurement properties as constructed by the COSMIN panel [21].

We calculated the internal consistency parameter Cronbach’s alpha which should have a value between 0.70 and 0.95 [20]. We assessed test-retest reliability by calculatingthe intraclass correlation coefficient (ICCconsistency) for each PRAQ domain. ICC values of 0.7 are considered acceptable, but values of ≥0.8 are preferred [17]. Additionally, we calculated the standard error of measurement (SEM).

We used hypothesis testing to assess convergent validity, which involves studying the correlations of the scores of the PROM under study with the scores of other PROMs. We hypothesized on the size and direction of the (Spearman’s) correlations of the PRAQ domains with the (subscales of) PROMs with similar constructs (Appendix 2). We also hypothesized which PROMs should have a lower correlation with the PRAQ domain. Good convergent validity means that 75% of hypotheses are correct [20]. We used the following (subscales of) PROMs for convergent validity in their official Dutch translations:
  • The Epworth Sleepiness Scale (ESS) [22], measuring daytime sleep propensity. For eight situations, a patient indicates the likelihood that they would fall asleep while in that situation. The measurement properties of the ESS have been studied in a sleep apnea population [23].

  • The “vitality” domain of the RAND-36 [24]. The (freely available) RAND-36, which is the predecessor of the well-known SF-36, measures general quality of life in several domains. The vitality domain of the RAND-36 contains 4 items about a patient’s perceived energy level. The items are identical to the items of the vitality domain of the SF-36, and the domain’s measurement properties have been studied in a sleep apnea population in that context [23].

  • The following short-forms of the Patient-Reported Outcomes Measurement Information System (PROMIS) databank [2527]: sleep disturbance (5 items), sleep-related impairment (6 items), fatigue, satisfaction with participation in social roles, ability to participate in social roles, anger, anxiety and depression (the latter 6 all contained 4 items per short-form) [2831]. For “sleep disturbance” and “anger” these were custom short-forms with fewer items than the standard short forms, in order to reduce the number of items that patients had to complete for this study.

To assess responsiveness, we constructed hypotheses about the change scores of PRAQ in correlation to the change scores of the same instruments that were employed for hypothesis testing in construct validity (Appendix 2).

Results

Population characteristics

The baseline population consisted of 180 patients with suspected OSA who completed the baseline measurement. Of these patients, 105 completed the retest between 7 and 21 days (average 14 days), and 53 patients completed the follow-up measurement after 6–8 weeks of treatment with CPAP. Characteristics of these respective (sub)populations can be found in Table 1.
Table 1

Baseline characteristics of the study populations

 

Baseline population (n = 180)

Test-retest population (n = 105)

Population with follow-up after CPAP (n = 53)

Gender

31.7% female

38.1% female

25.0% female

Age (mean (SD))

50.1 (12.6)

50.4 (13.0)

55.8 (10.9)

Baseline AHI (mean (SD))

25 (23) (n = 160a)

27 (25) (n = 96a)

41 (22)

BMI (mean (SD))

28.9 (4.7)

28.3 (4.6)

30.4 (4.2)

ESS score (mean (SD))

9.9 (4.7)

9.6 (4.4)

9.8 (4.7)

ESS score ≥ 11

43%

42%

40%

Sleep study (type)

43% PG /57% PSG (n = 160*)

39%PG /61% PSG (n = 96*)

43%PG/ 57%PSG

CPAP compliance (mean (SD))

N/A

N/A

6:46 h (1:40 h)

CPAP compliance ≥4 h/night

N/A

N/A

96%

AHI with CPAP (mean(SD))

N/A

N/A

2.6 (3.4)

AHI Apnea-Hyponea Index, BMI Body Mass Index, ESS Epworth Sleepiness Scale, PG = polygraphy, PSG polysomnography

a. 20 patients with suspected OSA of the total study population did for various reasons (choose to) not undergo a sleep study to determine their AHI

Missing data

Patients completing the online PRAQ were not allowed to leave any items open (no missings allowed). Eleven patients completed the PRAQ on paper one or more times, and in one of these completed PRAQs (for follow-up after CPAP), item 33 (Appendix 1) was missing from the domain “social interactions”. We computed the domain score for this patient as the average of the remaining items.

Seven items allowed the response item “not applicable” (see Appendix 1). Between 19 and 46% of respondents selected this response category for the respective items.

Final stage of PRAQ development

Finishing the item selection of the sleepiness domain

None of the inter-item correlations in the preliminary ‘sleepiness’ domain was higher than 0.9. Principal component analysis showed that the lowest factor loading was 0.65, well above 0.5. Therefore, we took practical elimination decisions: the two items with a “not applicable” option were removed (about sleepiness while reading, and while driving a car) as well as an item about napping in the afternoon that had a different answering scale than the other items. This improves the homogeneity of the domain for patients. The final version of the PRAQ consists of 10 domains and 40 items (Appendix 1).

Identification and grouping of items for outcome measurement

The results of the final PCA can be found in Table 2. The items of the PRAQ domains “memory & concentration”, “sleep quality”, and “concerns about health” were removed because they did not have sufficient loading on any of the factors found in the PCA, or because the items loaded on more than one factor. The items of the PRAQ domains “tiredness” and “daily activities” loaded on a single factor rather than on two separate factors: these items were therefore combined in one domain called “energy & daily activities” for the goal of outcome measurement. The items of the PRAQ domain “unsafe situations” both loaded on one separate domain. However, since this domain contained only two items it was not added to the PRAQ-outcome.
Table 2

Results of the principal component analysisab

Items

Factor 1

Factor 2

Factor 3

During the past 4 weeks, did you have a problem with:

Fighting to stay awake during the day?

.290

.727

 

Suddenly falling asleep?

 

.836

 

Difficulty staying awake during a conversation?

 

.636

.222

Difficulty staying awake while watching something? (concert, movie, television)

 

.858

 

Falling asleep at inappropriate times or places?

 

.802

 

Feeling very tired?

.785

  

Lacking energy?

.856

  

Still feeling tired when you wake up in the morning?

.790

  

In the past 4 weeks:

How difficult was it for you to do your most important daily activity? (such as your job, studying, caring for the children, housework)

.841

  

How often did you use all your energy on only your most important, daily activity? (such as your job, studying, caring for the children, housework)

.940

  

How often did you use all your energy to accomplish only your most important daily activity? (such as your job, studying, caring for the children, housework)

.825

  

How much difficulty did you have finding energy for your hobbies?

.770

  

How difficult was it for you to get your chores done?

.849

  

How often did you feel depressed or hopeless?

.266

 

.677

How often did you feel anxious?

  

.793

How often did you lose your temper?

  

.803

How often did you feel that you could not cope with everyday life?

  

.746

How often did you feel irritated?

  

.889

How often did you have a strong emotional reaction to everyday events?

  

.875

a. The bold font numbers indicate the highest factor loading for that item

b. Absolute factor loadings < 0.2 are not shown in the table

The 19 remaining items in the PCA form three one-dimensional domains: sleepiness, energy & daily activities, and emotions, which together explain 73% of the variance. The PCA showed intercorrelations of these domains of 0.36–0.57. The domains are added to the two formative domains “symptoms at night” and “social interactions”, resulting in subset of 33 items in five domains. Figure 1 illustrates how the items and domains of the PRAQ result in the subselection of PRAQ items for outcome measurement. The domains that are present in both the full 40-item PRAQ and in the 33-item outcome subset overlap to a great extent.
Fig. 1
Fig. 1

The subselection of items and domains of the PRAQ for outcome measurement

Measurement properties

In this section we describe the measurement properties of the domains that are used for outcome measurement; the results for the domains of the 40-item version can be found in Additional file 1. The average baseline scores, standard deviations, and percentages of lowest and highest scores of the five outcome domains can be found in Table 3. No floor- or ceiling effects were found, except for a floor effect in the ‘sleepiness’ domain (20% of subjects scored 1–1.5). The results of the different aspects of reliability (internal consistency with Cronbach’s alpha, test-retest reliability with ICC, SEM) are also shown in Table 3. The values of Cronbach’s alpha and the ICC values are all above 0.8, indicating that these measurement properties are of good quality.
Table 3

PRAQ outcome domains: scores and reliability parameters (n = 180)

Domain name

Average (range 1–7)

Standard deviation

Lowest score (1–1.5)

Highest score (6.5–7)

Cronbach’s α

ICCa

SEMa

Sleepiness

3.13

1.57

20%

2.2%

0.88

0.81

0.69

Energy&daily activities

4.52

1.59

4.4%

7.8%

0.95

0.86

0.60

Emotions

2.89

1.28

13.3%

0.0%

0.92

0.85

0.50

Symptoms at night

3.48

1.27

3.9%

1.1%

- b

0.88

0.44

Social Interactions

3.11

1.42

13.9%

0.6%

- b

0.86

0.53

ICC = intraclass correlation efficient, SEM = standard error of measurement

a. n = 105

b. These domains are formative, and Cronbach’s α is only relevant when a domain is one-dimensional (36)

The correlations of the outcome domains with comparator instruments, which were used to determine convergent validity, are presented in Table 4. The correlations with the (somewhat) similar constructs were all within the ranges that we hypothesized (n = 14 hypotheses), and the correlations of selected PRAQ-domains with the dissimilar constructs were all lower than those with the similar constructs (n = 3 hypotheses), as expected.
Table 4

Correlations PRAQ outcome domains and comparator instruments a (n = 180)

 

ESS

PROMIS Sleep-related impairment

RAND vitality

PROMIS fatigue

PROMIS Ability to participatie in Social Roles and Activities

PROMIS Satisfaction Social Roles

PROMIS anger/ anxiety/ depression

PROMIS sleep disturbance

Sleepiness

0.67

0.60

−.40

.52

    

Energy & daily activities

0.45

0.83

−.77

0.86

−0.78

−0.60

  

Emotions

0.28

 

−0.59

0.56

−0.60

−0.42

0.69–0.76

 

Symptoms at night

       

0.47

Social interactions

 

0.56

  

−0.52

−0.39

  

ESS = Epworth Sleepiness scale, PROMIS = patient-reported outcomes measurement information system

a. Correlations in bold are considered similar constructs, for which detailed hypotheses were created. The other correlations are of (somewhat) different constructs and are expected to be weaker than the bold font correlations for that PRAQ domain (for details, see Appendix 2).. Correlations for which we had no specific expectations are not shown

The absolute change scores of the PRAQ outcome domains after patients were treated with CPAP ranged from 0.76 (domain “emotions”) to 1.96 (domain “energy & daily activities”) (Appendix 3). The correlations of the change scores of the PRAQ and the change scores of the comparator instruments (Table 5) were generally in agreement with our hypotheses (n = 17 hypotheses). The exception was the “emotions” domain of the PRAQ, which did not correlate as strongly with the change scores of the PROMIS domains about emotions (anger, anxiety, depression; r = 0.26–0.43) as we had expected. When a hypothesis is not met, it is important to identify why the results are different than expected [32]. To gain more insight into these unexpected scores, we therefore ran an additional analysis on the correlation of the PRAQ scores and the PROMIS scores at the follow-up measurement, showing results of r = 0.62–0.71. This shows that the discrepancy lies with the change score itself and not the absolute score of the follow-up measurement.
Table 5

Correlations between change scores PRAQ outcome domains and comparator instruments a (n = 53)

 

ESS

PROMIS Sleep-related impairment

RAND vitality

PROMIS fatigue

PROMIS Ability to participatie in Social Roles and Activities

PROMIS Satisfaction Social Roles

PROMIS anger/ anxiety/ depression

PROMIS sleep disturban-ce

Sleepiness

.62

.55

−.35

.35

    

Energy&daily activities

.52

.62

−.69

.70

−.74

−.61

  

Emotions

0.06

.23

−.32

.14

−.29

 

.26–.43

 

Symptoms at night

       

.58

Social interactions

 

.45

  

−.36

−.26

  

ESS = Epworth Sleepiness scale, PROMIS = Patient-Reported Outcomes Measurement Information System

a. Correlations in bold are considered similar constructs, for which detailed hypotheses were created. The other correlations are of (somewhat) different constructs and are expected to be weaker than the bold font correlations for that PRAQ-outcome domain (for details, see Appendix 2). Correlations for which we had no specific expectations are not shown

Discussion

In this article we present the finalized Patient-Reported Apnea Questionnaire (PRAQ). The PRAQ has a unique approach with regard to the integration of use on an individual patient level and for aggregate outcome measurement: patients complete all items of the PRAQ before their consultation, the results of which can be discussed with a healthcare professional; and the aggregate outcomes of groups of patients can then be studied by making use of a subset of the completed items. The PRAQ contains all topics and items that patients and healthcare providers consider important to discuss in practice, and for this purpose includes 40 items in 10 domains. For outcome measurement, a subset of 33 items of the PRAQ was selected, divided into two formative domains (items grouped together based on what makes sense clinically) and three one-dimensional subscales. These five outcome domains generally have good measurement properties in terms of internal consistency, test-retest reliability, convergent validity and responsiveness.

PCA showed that items of the PRAQ domains “tiredness” and “daily activities” load on the same factor, which is why the items of these preliminary domains are combined into one domain for the purpose of outcome measurement. For use an individual patient level, however, we decided to keep the two domains separate. Even though we acknowledge that feeling tired (a symptom), and the extent to which daily activities can be performed normally (a consequence of that symptom), are closely related concepts, they may be relevant to discuss separately for an individual patient in clinical practice. We will test this assumption in future research, in which the PRAQ will be employed and studied empirically.

The domains that are used for outcome measurement show good responsiveness. The one exception is the domain “emotions”, the change score of which showed a much weaker correlation than expected with the change scores of PROMs with similar constructs. We hypothesize that the discrepancy between expectation and results caused by the low scores of this domain at baseline (average 2.89) and the subsequent relatively small improvement that is achieved after treatment with CPAP (average 0.76). We do not doubt the construct validity of the domain, because the comparator PROMs show the same pattern in terms of low scores and small change scores, and because the correlation of the absolute scores after treatment with CPAP shows good convergent validity. However, because the change scores are small, it is likely that measurement error plays a relatively large role in the change scores of both the PRAQ domain and the comparator instruments, reducing the accuracy of the change scores and therefore also diffusing the correlation size. This means that that in terms of outcome/quality measurement, emotional problems appear to be of less importance than the topics of the other domains and more difficult to accurately measure, because relatively few people with (suspected) OSA experience severe problems.

Surprisingly, 20% of the study population had low scores (1–1.5) on the domain ‘sleepiness’, while sleepiness is one of the main complaints of OSA. We think that this is due to a relatively high difficulty of the sleepiness items of the PRAQ (such as falling asleep during a conversation) in combination with a generally low sleepiness in this population (average ESS < 10). This reason for the low sleepiness in the population is probably twofold. First, the main complaint of some patients who were referred for suspected OSA in this study is probably (socially problematic) snoring rather than sleepiness or tiredness during the day. OSA treatment will reduce snoring and is reimbursed by healthcare insurers, making it beneficial for these patients to visit the sleep center and get an OSA diagnosis. Second, for logistical reasons some patients with suspected severe OSA were not included in the study. These patients a fast- track procedure to bypass the sleep center’s the waiting list, which meant they were in practice not always asked to join the study. This is a limitation of the study. What we can derive from the current results is that the sleepiness domain of the PRAQ seems more useful to detect cases of severe sleepiness, which definitely requires treatment, than to distinguish mild and moderate sleepiness. However, future research should take place in a more representative patient group to study how the sleepiness domain performs in this population.

The PRAQ is designed for use in clinical practice, to help focus consultations on the problems that individual patients encounter. When using a PROM for this purpose, the ICC should preferably be very high (0.9–0.95 at individual level vs. 0.8 or higher at group level for aggregate outcome measurement [17]). The ICC values of the PRAQ are lower (0.81–0.88). However, the PRAQ is meant to open the conversation about a patient’s symptoms and functioning, not to serve as a “cut-off” score. Any elevated score could therefore result in conversation about this topic, and we believe that the PRAQ can serve its purpose despite the slightly lower ICCs.

Methodological considerations

The domains for outcome measurement were created with a combination of the “clinimetric” approach, in which items are grouped together based on clinical relevance; and the “psychometric” approach, which groups items together based on PCA [3335]. The combination of these two approaches is uncommon. We believe that scores of psychometric domains, with a clear one-dimensional construct, are more meaningful than formative domains because they have a clear interpretation. However, this approach is not always feasible when items have been selected to be part of a quality-of-life or symptoms questionnaire based on their deemed importance by the target population [36]. Items which cover symptoms of the same disease or treatment will often share covariance and thus appear to be covering the same latent construct, even when looking at the content of the items this makes no apparent sense (e.g. lack of appetite and decreased sexual interest in patients undergoing cancer treatment [36]). Therefore, we considered the best approach grouping together the different symptoms patients experience at night, as well as the variety of different ways in which sleepiness, tiredness and emotions might influence a patients’ social life, without subjecting them to PCA.

To aid the use of the PRAQ in clinical practice, we developed a patient-friendly digital report (the PRAQ-report) together with patients and healthcare professionals (Fig. 2). When using the PRAQ in clinical practice, it can be useful to look at individual item scores as well as the domain scores, especially in the formative domains in which item scores will generally differ more from each other. Therefore, both domain and individual item scores are shown in the report.
Fig. 2
Fig. 2

The first page of the PRAQ-report

Conclusions

In conclusion, we have shown that the PRAQ-practice and PRAQ-outcome generally have acceptable measurement properties and appear to be suitable PROMs for their respective purposes. However, further validation research is needed in patients who suffer from higher levels of sleepiness, to study the validity of the sleepiness domain. The applicability of a PROM for use in clinical practice and for measuring outcomes on aggregate level, may be of great importance for the further implementation of PROMs in healthcare.

Abbreviations

AHI: 

apnea-hypopnea index

CPAP: 

continuous positive airway pressure

ESS: 

Epworth sleepiness scale

ICC: 

intraclass correlation coefficient

OSA: 

obstructive sleep apnea

PCA: 

principal component analysis

PRAQ: 

patient-reported apnea questionnaire

PROM: 

patient-reported outcome measure

PROMIS: 

patient-reported outcomes measurement information system

SEM: 

standard error of measurement

Declarations

Funding

The first author is funded by a Radboudumc grant (project number R0002257).

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Author’s contributions

All authors were involved in the conception of the research and research design. IA and BH collected the data. IA and PW analyzed and interpreted the data. IA drafted the manuscript, PW and MR critically revised it. All authors read and approved the final manuscript.

Ethics approval and consent to participate

For this type of study ethical approval is not required under Dutch law. We obtained an official exemption from the the local Medical Ethics Committee CMO Arnhem-Nijmegen (file number 2015–2219). Informed consent was obtained from all participants included in the study. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Radboud Institute of Health Sciences, IQ healthcare, Radboud University Medical Center, PO box 9101, huispost 114, 6500, HB, Nijmegen, the Netherlands
(2)
Radboud Institute of Health Sciences, Departments for Health Evidence and Operating Rooms, Radboud University Medical Center, Nijmegen, The Netherlands
(3)
ApneuVereniging, Doorn, The Netherlands
(4)
Albert Schweitzer Ziekenhuis, Sleep Center, Dordrecht, The Netherlands

References

  1. O'Donoghue N, McKay E. Exploring the impact of sleep apnoea on daily life and occupational engagement. Br J Occup Ther. 2012;75:609–516.View ArticleGoogle Scholar
  2. Reishtein JL, Pack AI, Maislin G, Dinges DF, Bloxham TJ, George CF, et al. Sleepiness and relationships in obstructive sleep apnea. Issues Ment Health Nurs. 2006;27:319–30.View ArticlePubMedGoogle Scholar
  3. Rodgers B. Breaking through limbo: experiences of adults living with obstructive sleep apnea. Behav Sleep Med. 2014;12:183–97.View ArticlePubMedGoogle Scholar
  4. Bradley TD, Floras JS. Obstructive sleep apnoea and its cardiovascular consequences. Lancet. 2009;373:82–93.View ArticlePubMedGoogle Scholar
  5. Chan AS, Phillips CL, Cistulli PA. Obstructive sleep apnoea--an update. Intern Med J. 2010;40:102–6.View ArticlePubMedGoogle Scholar
  6. Young T, Peppard PE, Gottlieb DJ. Epidemiology of obstructive sleep apnea: a population health perspective. Am J Respir Crit Care Med. 2002;165:1217–39.View ArticlePubMedGoogle Scholar
  7. Bjornsdottir E, Benediktsdottir B, Pack AI, Arnardottir ES, Kuna ST, Gislason T, et al. The prevalence of depression among untreated obstructive sleep apnea patients using a standardized psychiatric interview. J Clin Sleep Med. 2016;12:105–12.View ArticlePubMedPubMed CentralGoogle Scholar
  8. Gupta MA, Simpson FC, Lyons DC. The effect of treating obstructive sleep apnea with positive airway pressure on depression and other subjective symptoms: a systematic review and meta-analysis. Sleep Med Rev. 2016;28:55–68.View ArticlePubMedGoogle Scholar
  9. Mokhlesi B, Ham SA, Gozal D. The effect of sex and age on the comorbidity burden of OSA: an observational analysis from a large nationwide US health claims database. Eur Respir J. 2016;47:1162–9.View ArticlePubMedGoogle Scholar
  10. Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.View ArticlePubMedGoogle Scholar
  11. Van Der Wees PJ, Nijhuis-Van Der Sanden MW, Ayanian JZ, Black N, Westert GP, Schneider EC. Integrating the use of patient-reported outcomes for both clinical practice and performance measurement: views of experts from 3 countries. Milbank Q. 2014;92:754–75.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Greenhalgh J, Dalkin S, Gooding K, Gibbons E, Wright J, Meads D, et al. Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care. Health Serv Deliv Res. 2017;5:1–163.View ArticleGoogle Scholar
  13. Abma IL, Rovers M, IJff M, Hol B, Westert GP, van der Wees PJ. The development of a patient-reported outcome measure for patients with obstructive sleep apnea: the Patient-Reported Apnea Questionnaire (PRAQ). J Patient Rep Outcomes. 2017;1:14.View ArticlePubMedPubMed CentralGoogle Scholar
  14. Mannarino MR, Di Filippo F, Pirro M. Obstructive sleep apnea syndrome. Eur J Intern Med. 2012;23:586–93.View ArticlePubMedGoogle Scholar
  15. Epstein LJ, Kristo D, Strollo PJ Jr, Friedman N, Malhotra A, Patil SP, et al. Clinical guideline for the evaluation, management and long-term care of obstructive sleep apnea in adults. J Clin Sleep Med. 2009;5:263–76.PubMedGoogle Scholar
  16. Grunstein RR. Sleep-related breathing disorders. 5. Nasal continuous positive airway pressure treatment for obstructive sleep apnoea. Thorax. 1995;50:1106–13.View ArticlePubMedPubMed CentralGoogle Scholar
  17. De Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine. Cambridge: Cambridge University Press; 2011.View ArticleGoogle Scholar
  18. Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess. 1995;7:286–99.View ArticleGoogle Scholar
  19. De Vet HCW, Terwee CB, Bouter LM. Clinimetrics and psychometrics: two sides of the same coin. J Clin Epidemiol. 2003;56:1146–7.View ArticleGoogle Scholar
  20. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42.View ArticlePubMedGoogle Scholar
  21. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63:737–45.View ArticlePubMedGoogle Scholar
  22. Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14:540–5.View ArticlePubMedGoogle Scholar
  23. Abma IL, van der Wees PJ, Veer V, Westert GP, Rovers M. Measurement properties of patient-reported outcome measures (PROMs) in adults with obstructive sleep apnea (OSA): a systematic review. Sleep Med Rev. 2016;28:18–31.View ArticlePubMedGoogle Scholar
  24. Hays RD, Sherbourne CD, Mazel RM. The RAND 36-item health survey 1.0. Health Econ. 1993;2:217–27.View ArticlePubMedGoogle Scholar
  25. DeWalt DA, Rothrock N, Yount S, Stone AA, Group PC. Evaluation of item candidates: the PROMIS qualitative item review. Med Care. 2007;45:S12–21.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Riley WT, Rothrock N, Bruce B, Christodolou C, Cook K, Hahn EA, Cella D. Patient-reported outcomes measurement information system (PROMIS) domain names and definitions revisions: further evaluation of content validity in IRT-derived item banks. Qual Life Res. 2010;19:1311–21.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Terwee CB, Roorda LD, de Vet HC, Dekker J, Westhovens R, van Leeuwen J, et al. Dutch-Flemish translation of 17 item banks from the patient-reported outcomes measurement information system (PROMIS). Qual Life Res. 2014;23:1733–41.PubMedGoogle Scholar
  28. Buysse DJ, Yu L, Moul DE, Germain A, Stover A, Dodds NE, et al. Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep. 2010;33:781–92.View ArticlePubMedPubMed CentralGoogle Scholar
  29. Yu L, Buysse DJ, Germain A, Moul DE, Stover A, Dodds NE, et al. Development of short forms from the PROMIS sleep disturbance and sleep-related impairment item banks. Behav Sleep Med. 2011;10:6–24.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Pilkonis PA, Choi SW, Reise SP, Stover AM, Riley WT, Cella D, et al. Item banks for measuring emotional distress from the patient-reported outcomes measurement information system (PROMIS(R)): depression, anxiety, and anger. Assessment. 2011;18:263–83.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Pilkonis PA, Yu L, Dodds NE, Johnston KL, Maihoefer CC, Lawrence SM. Validation of the depression item bank from the patient-reported outcomes measurement information system (PROMIS) in a three-month observational study. J Psychiatr Res. 2014;56:112–9.View ArticlePubMedPubMed CentralGoogle Scholar
  32. Abma IL, Rovers M, van der Wees PJ. Appraising convergent validity of patient-reported outcome measures in systematic reviews: constructing hypotheses and interpreting outcomes. BMC Res Notes. 2016;9:226.View ArticlePubMedPubMed CentralGoogle Scholar
  33. Marx RG, Bombardier C, Hogg-Johnson S, Wright JG. Clinimetric and psychometric strategies for development of a health measurement scale. J Clin Epidemiol. 1999;52:105–11.View ArticlePubMedGoogle Scholar
  34. Newcombe PA, Sheffield JK, Juniper EF, Marchant JM, Halsted RA, Masters IB, et al. Development of a parent-proxy quality-of-life chronic cough-specific questionnaire: clinical impact vs psychometric evaluations. Chest. 2008;133:386–95.View ArticlePubMedGoogle Scholar
  35. Ribera A, Permanyer-Miralda G, Alonso J, Cascant P, Soriano N, Brotons C. Is psychometric scoring of the McNew quality of life after myocardial infarction questionnaire superior to the clinimetric scoring? A comparison of the two approaches. Qual Life Res. 2006;15:357–65.View ArticlePubMedGoogle Scholar
  36. Fayers PM, Hand DJ. Factor analysis, causal indicators and quality of life. Qual Life Res. 1997;6:139–50.PubMedGoogle Scholar

Copyright

© The Author(s). 2018

Advertisement