- Open Access
The CONSORT Patient-Reported Outcome (PRO) extension: implications for clinical trials and practice
Health and Quality of Life Outcomesvolume 11, Article number: 184 (2013)
To inform clinical guidelines and patient care we need high quality evidence on the relative benefits and harms of intervention. Patient reported outcome (PRO) data from clinical trials can “empower patients to make decisions based on their values” and “level the playing field between physician and patient”. While clinicians have a good understanding of the concept of health-related quality of life and other PROs, evidence suggests that many do not feel comfortable in using the data from trials to inform discussions with patients and clinical practice. This may in part reflect concerns over the integrity of the data and difficulties in interpreting the results arising from poor reporting.
The new CONSORT PRO extension aims to improve the reporting of PROs in trials to facilitate the use of results to inform clinical practice and health policy. While the CONSORT PRO extension is an important first step in the process, we need broader engagement with the guidance to facilitate optimal reporting and maximize use of PRO data in a clinical setting. Endorsement by journal editors, authors and peer reviewers are crucial steps. Improved design, implementation and transparent reporting of PROs in clinical trials are necessary to provide high quality evidence to inform evidence synthesis and clinical practice guidelines.
Randomized controlled clinical trials (RCTs) can provide high-quality data regarding the impact of study interventions on patient outcomes, and remain the ‘gold standard’ of evidence regarding both the benefits and harms associated with an intervention. Over the last twenty years, the number of clinical trials that assess patient reported outcomes (PROs) has substantially increased . PROs can be defined as a “measurement of any aspect of a patient’s health status that comes directly from the patient (i.e. without the interpretation of the patient’s responses by a physician or anyone else”  and include health-related quality of life (HRQL), symptoms, satisfaction and adherence to medication. These subjective measures of outcome help evaluate the burden of disease and treatment from the patients’ perspective. In the conceptual framework developed by Till and colleagues adapted in (Figure 1) , PRO data from clinical trials may directly inform patients and practitioners, or may indirectly inform clinical practice through evidence synthesis into clinical practice guidelines.
The quality of data, including PROs, from trials may be threatened by the trial design, execution, and reporting. Like any other outcome being considered, PROs should be pre-specified, relevant and appropriate measure(s) . Failure to address a number of specific issues at the time of protocol writing or in the final trial report, such as the PRO hypothesis, timing and mode of assessment and strategies to manage missing data, can undermine the clinical validity of findings and limit translation of these into clinical practice .
All RCT outcomes, including PROs, should be completely and transparently reported. Anything less may diminish the usability of the trial results by clinicians and is an inefficient use of resources. High-quality reporting of PROs in clinical trials is required to allow readers to assess the validity of the PRO findings and assess any potential sources of bias. Evidence suggests, however, that reporting of PROs remains sub-optimal across RCTs [6, 7]. Previous studies estimating the proportion of PRO-RCTs that were likely to robustly inform clinical decision-makings in prostate cancer, for example, have found that only one third of PRO-RCTs published between 1980 and 2001 did so .
In a review of 794 RCTs published up to 2008 across a range of clinical settings, only 56% of articles provided a rationale for the selected PRO, 50% provided a PRO hypothesis, 28% provided information about PRO missing data and 64% discussed the PRO findings in the context of the other trial outcomes . Although the PROs of some RCTs were reported in an expanded supplementary publication, this only occurred for 15% of trials at the time of review, and the detailed PRO publications often appeared in methodologically focused journals. In cancer RCTs methodological drawbacks have often hampered a critical appraisal of results [9, 10].
Additional qualitative evidence suggests that these patterns of PRO reporting negatively impact on the effective use and dissemination of RCT PRO findings to clinical practice. A study of 30 academic oncologists examining their attitudes and behaviors regarding HRQL data from clinical trials indicated that, while these data were typically valued by clinicians and felt to benefit their patients, problems with the manner in which these data were reported in the literature limited their use in clinical practice . The data were considered to be less accessible than outcomes such as survival or the adverse effects of treatment. Many clinicians felt that HRQL/PRO data were frequently reported in a manner that is difficult to interpret and integrate with clinical outcomes, and were often reported months or years after the primary trial publication. Clinicians also wanted to be assured that the appropriate measurement instrument was used, that the results were not biased by missing data, that results were not “cherry picked” (i.e., chance findings arising from multiple testing), and that reported changes in PROs were clinically meaningful.
In this paper, we focus on how the recently developed CONSORT (Consolidated Standards of Reporting Trials) PRO extension  may be implemented to inform individual patient care and clinical guidelines as illustrated in (Figure 1).
The CONSORT PRO extension as a vehicle for change and challenges to its implementation
The CONSORT statement aims to encourage transparent and complete reporting of clinical trials and is associated with improved reporting of trials . Importantly, the CONSORT group has a proven ability to reach consensus on reporting guidelines and to promote their successful implementation. Extensions to the original CONSORT statement have provided guidance for specific trial designs, interventions, and specific outcomes but until recently did not provide specific guidance for PROs . Such guidance was clearly required since multiple factors threaten the quality of PRO reporting: researchers must design their RCTs with appropriate PRO implementation and execution, and authors must find a balance between providing sufficient detail to adequately report PRO results within the context of other study outcomes while still adhering to journal requirements for manuscript length and structure. Reviewers and editors must find the optimal balance between demanding that certain reporting standards be included in the context of the same journal constraints.
The development of a CONSORT PRO extension was thus undertaken, led by a Task Force of the International Society for Quality of Life Research (ISOQOL) and guided by the methodological framework proposed by the EQUATOR Network . Briefly, a systematic search was used to identify candidate reporting standards for PROs . An electronic survey of stakeholder groups was used to obtain experts’ judgments of the importance of each candidate standard for PRO reporting and the survey results were then debated by a face-to-face meeting of stakeholders to achieve consensus on items that should comprise a CONSORT extension. The meeting agenda included: a review of relevant evidence, discussion of the rationale for including or excluding items in a PRO reporting standards checklist from the point of view of the multiple stakeholders at the table, anonymous voting on the proposed standards, the development of a draft CONSORT extension, and the development of a knowledge translation and dissemination strategy [13, 14].
The group was ultimately able to reach consensus on five “extension” statements to the 2010 CONSORT guidance that each address a key reporting item considered critical for quality reporting from all types of RCTs where PROs are a primary or “important” secondary outcome as shown in Table 1. Important secondary outcomes refer to pre-specified domains that are focus of the principal PRO analyses . In addition, critical components of the existing 2010 CONSORT guidance that were critically relevant to PRO reporting were expanded by “elaboration” statements. The rationale for each of the 5 items comprising the CONSORT PRO extension are briefly summarized in Table 1. Readers are referred to the CONSORT PRO Extension for further information on the rationale for each item, examples of reporting and elaboration of the existing CONSORT 2010 statement in relation to PROs . These guidance statements, if implemented by authors and journal reviewers/editors, could potentially improve reporting and facilitate interpretation of PRO results for use in clinical practice.
Experience with the CONSORT Statement suggests that efforts to implement the CONSORT PRO Extension will face a number of challenges. First and foremost is obtaining journal endorsement for the CONSORT PRO Extension. A 2008 analysis of “Instructions to Authors” for 165 high impact factor medical journals found that only 38% mentioned the main CONSORT statement in their online instructions . Moreover, of the four CONSORT extensions that were examined, no single extension was mentioned on more than 3% of journal websites. Evidence suggests that reporting is improved in those journals with a policy of active implementation of CONSORT .
Theory and research on dissemination and implementation in health care settings suggests ways to promote greater adoption of the CONSORT statement and its extensions . Success can be enhanced when efforts extend beyond typical “push” approaches (i.e., publishing and publicizing new information), to include “pull” approaches (i.e., creating demand for change among the target audience). One possible way to create demand for endorsement is to have regular public reporting audit of those journals that do and do not mention CONSORT or its extensions in their online Instructions to Authors.
Even when CONSORT is endorsed there is considerable variation in how it is implemented at journals. Without strong direct language, for example, in a journal’s Instructions to Authors section, prospective authors will not have clear guidance about how they need to comply with journal endorsement. Plans to develop online resources which provide specific examples taken from publications should facilitate adherence to the new CONSORT-PRO checklist. Problems may be encountered; however, with consistency in the operationalization of “important secondary outcomes” since this term is not currently part of the standard clinical trial lexicon. In addition, problems may be encountered with how authors interpret the requirement that evidence of PRO instrument reliability or validity be provided or cited if available. Several questions arise in this regard. Is it sufficient to cite the original article that describes an instrument’s psychometric properties? Ideally further information on reliability should be reported in relation to the population of study in the clinical trial. Should supporting evidence also be provided for an instrument’s responsiveness to change? Should additional psychometric information be provided for translations of instruments? In addressing these issues, authors will need to stay abreast of evolving standards for PROs and their use in clinical research [2, 19].
Potential impact of the CONSORT PRO extension on the quality of clinical trial reporting
The need for improvement in the reporting of PROs is one of the principal reasons for developing the CONSORT PRO Extension. How likely is it, however, that publication and implementation of the CONSORT PRO Extension will have the desired positive impact? A recent systematic review and meta-analysis provides some insight. The review encompassed published studies evaluating the completeness of reporting of randomized controlled trials in medical journals based on CONSORT Statement criteria . Data for the review were drawn from 53 publications encompassing over 16,000 randomized clinical trials. Findings showed that 25 of 27 comparisons of the completeness of reporting favored CONSORT-endorsing journals over non-endorsers, with five of these comparisons yielding statistically significant differences. These findings strongly suggest that endorsement of the CONSORT PRO Extension will over time result in improvements in the reporting of PROs in clinical trials.
A Real-World Example from Oncology: evidence from prostate cancer RCTs evaluated against the CONSORT PRO extension items
In an effort to provide current evidence on the level of reporting according to the new CONSORT PRO extension items, in Table 2 we provide the proportion of recently conducted PRO-RCTs in prostate cancers that have addressed such issues. A systematic literature search, identifying all RCTs published with a PRO component (published between 2004 and 2012) in prostate cancers found 65 studies . All these RCTs were scrutinized based on a number of methodological criteria, including the CONSORT PRO extension items and data are summarized in Table 2. A remarkable finding is that only 18% of studies documented statistical methods for addressing PRO missing data. Also, only 23% reported methods for PRO administration. These very low percentages, underscore that some specific items reported in the new CONSORT PRO extension, are in urgent need of improvement to raise confidence in the solidity of PRO data stemming from these RCTs.
Implications of the CONSORT PRO extension for RCT design
The CONSORT PRO extension has important implications for trial design, although design decisions may go beyond simply what should be reported. The problem of PRO missing data illustrates this distinction. Difficulties with PRO data collection and compliance have historically been considered the major barriers to the successful implementation of PRO in clinical trials . The CONSORT PRO extension states that the amount of missing data and way that this has been addressed in the analysis should be reported. The analysis plan should clearly be pre-specified at the design stage, but in addition methods to prevent or minimize missing data in the trial should be stated. In addition the CONSORT PRO Extension stipulates that evidence of PRO instrument validity and reliability be included as part of the manuscript submission. This criterion has the potential to foster greater consideration of a PRO measure’s psychometric properties at the time of protocol design, thereby promoting greater use of measures with demonstrated reliability and validity. Likewise, consider the CONSORT PRO Extension item stipulating that the PRO hypothesis be stated in the manuscript, with relevant domains identified if applicable. This criterion has the potential to foster greater pre-specification of PRO hypotheses at the time of protocol development, thereby reducing the likelihood of selective reporting of PRO results in manuscript. It should be noted, however, that specification of outcomes in a protocol may not be sufficient to eliminate the problem of selective result reporting. Indeed, an audit comparing protocols submitted to a leading medical journal and the published reports based on those protocols found numerous discrepancies between the pre-specified outcomes and the reported outcomes . Additional efforts, such as providing manuscript reviewers with access to the original protocols, would appear to be necessary to meaningfully address the issue of selective reporting.
Within this framework, it is possible to envisage a positive impact of endorsement of the CONSORT PRO Extension not only in improving to the quality of clinical trial reporting but also on PRO trial design. The quality of certain aspects of clinical trials incorporating PROs are likely to improve as a direct result of widespread adoption of the CONSORT PRO Extension, while others will require more multi-faceted efforts on the part of the research community.
Improved PRO trial design and reporting: implications for guideline development
Individual randomized trials may either by chance or as a result of bias not represent the best possible estimates of an effect of an intervention on PROs. Thus, single small RCTs are rarely informative for evidence based health care decisions. Evidence-based health care decisions require systematic reviews to more completely consider the possible influence of bias and chance on health care decisions.
Systematic reviews of intervention effects alone, however, are also insufficient for evidence-based health care decision-making. In addition to information about effects of interventions, health care decisions require consideration of issues around the clinical state and circumstances including baseline risks for a certain outcome, the populations and societal values and preferences, estimates of effect and expertise to bring all of that together. Practice guidelines are recommendations that are intended to assist providers, recipients of health care and other stakeholders to make informed decisions and can take on the role of adequate support tools .
PROs can inform evidence-based recommendations in a number of ways. PROs address what matters to patients and their use in clinical trials can provide the information that is critical for decision making in practice guidelines. PROs also can provide information about the importance of other outcomes, non PROs, by conducting explicit value and preference elicitation exercises. Transparent communication of both the confidence in an estimate of effect on a PRO, which is also called the quality of the evidence [24, 25], and the estimate itself remains challenging. The GRADE Working Group has developed a system to assess the confidence and the likelihood of an estimate of effect on PRO that facilitates communication, which may be incorporated formally in the review process. The group is now focusing on a structured approach to assessing the confidence in estimates of values and preferences that influence the development of health care recommendations. The system is used by over 70 organizations worldwide. Using such systems has become a requirement in most standards on clinical practice guideline development [26, 27].
In conclusion, robust methodology and accurate reporting of data are crucial when evaluating PROs in clinical trials in order to provide health care providers and policy-makers with a transparent message about the impact of a given drug or a novel therapeutic approach on the patient’s health status. Scientific publications stemming from a poor study design or simply reporting insufficient information can mislead readers when interpreting study outcomes. Such publications are wasteful, both in terms of opportunities for providing better evidence, research funding, the effort of everyone involved from concept to paper – including but not confined to investigators, ethics committees, patients, trial co-ordinators, statisticians, journal staff, reviewers and finally readers. The new CONSORT PRO extension aims to raise quality standards and help bridge the gap between PRO-trial based outcomes and clinical practice by increasing clinician confidence in published PRO evidence.
Sanders C, Egger M, Donovan J, Tallon D, Frankel S: Reporting on quality of life in randomised controlled trials: bibliographic study. BMJ 1998, 317: 1191–1194. 10.1136/bmj.317.7167.1191
US Food and Drug Administration: Guidance for Industry: Patient-reported outcome measures: use in medical product development to support labeling claims. U.S. Department of health and human services food and drug administration. 2009.
Till JE, Osoba D, Pater JL, Young JR: Research on health-related quality of life: dissemination into practical applications. Qual Life Res 1994, 3: 279–283. 10.1007/BF00434902
Chassany O, Sagnier P, Marquis P, Fullerton S, Aaronson N: Patient-reported outcomes: the example of health related quality of life-a European guidance document for the improved integration of health related quality of life assessment in the drug regulatory process. Drug Inf J 2002, 36: 209–238. 10.1177/009286150203600127
Brundage M, Bass B, Jolie R, Foley K: A knowledge translation challenge: clinical use of quality of life data from cancer clinical trials. Qual Life Res 2011, 20: 979–985. 10.1007/s11136-011-9848-0
Efficace F, Osoba D, Gotay C, Sprangers M, Coens C, Bottomley A: Has the quality of health-related quality of life reporting in cancer clinical trials improved over time? Towards bridging the gap with clinical decision making. Ann Oncol 2007, 18: 775–781.
Brundage M, Bass B, Davidson J, Queenan J, Bezjak A, Ringash J, Wilkinson A, Feldman-Stewart D: Patterns of reporting health-related quality of life outcomes in randomized clinical trials: implications for clinicians and quality of life researchers. Qual Life Res 2011, 20: 653–664. 10.1007/s11136-010-9793-3
Efficace F, Bottomley A, Osoba D, Gotay C, Flechtner H, D’Haese S, Zurlo A: Beyond the development of health-related quality-of-life (HRQOL) measures: a checklist for evaluating HRQOL outcomes in cancer clinical trials–does HRQOL evaluation in prostate cancer research inform clinical decision making? J Clin Oncol 2003, 21: 3502–3511. 10.1200/JCO.2003.12.121
Bottomley A, Therasse P: Quality of life in patients undergoing systemic therapy for advanced breast cancer. Lancet Oncol 2002, 3: 620–628. 10.1016/S1470-2045(02)00876-8
Efficace F, Bottomley A, van Andel G: Health related quality of life in prostate carcinoma patients: a systematic review of randomized controlled trials. Cancer 2003, 97: 377–388. 10.1002/cncr.11065
Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD: Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA 2013, 309: 814–822. 10.1001/jama.2013.879
Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, Dias S, Schulz KF, Plint AC, Moher D: Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev 2012., 11: MR000030
The CONSORT statement 2010 http://www.consort-statement.org/consort-statement/overview0/
EQUATOR network http://www.equator-network.org/
Brundage M, Blazeby J, Revicki D, Bass B, de Vet H, Duffy H, Efficace F, King M, Lam CL, Moher D, et al.: Patient-reported outcomes in randomized clinical trials: development of ISOQOL reporting standards. Qual Life Res 2013, 22: 1161–1175. 10.1007/s11136-012-0252-1
Hopewell S, Altman DG, Moher D, Schulz KF: Endorsement of the CONSORT statement by high impact factor medical journals: a survey of journal editors and journal ‘Instructions to Authors’. Trials 2008, 9: 20. 10.1186/1745-6215-9-20
Hopewell S, Ravaud P, Baron G, Boutron I: Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ 2012, 344: e4178. 10.1136/bmj.e4178
Kerner JF, Guirguis-Blake J, Hennessy KD, Brounstein PJ, Vinson C, Schwartz RH, Myers BA, Briss P: Translating research into improved outcomes in comprehensive cancer control. Cancer Causes Control 2005,16(Suppl 1):27–40.
Rothman M, Burke L, Erickson P, Leidy NK, Patrick DL, Petrie CD: Use of existing patient-reported outcome (PRO) instruments and their modification: the ISPOR good research practices for evaluating and documenting content validity for the Use of existing instruments and their modification PRO task force report. Value Health 2009, 12: 1075–1083. 10.1111/j.1524-4733.2009.00603.x
Efficace F, Feuerstein M, Fayers P, Cafaro V, Eastham J, Pusic A, Blazeby J, EORTC Quality of Life Group (Patient Reported Outcome Measurements Over Time In ONcology-PROMOTION Project): Patient-reported outcomes in randomized controlled trials of prostate cancer: methodological quality and impact on clinical-decision-making. Eur Urol 2013. in press
Fayers P, Hays R: Assessing quality of life in clinical trials. New York: Oxford University Press; 2005.
Al-Marzouki S, Roberts I, Evans S, Marshall T: Selective reporting in clinical trials: analysis of trial protocols accepted by the lancet. Lancet 2008, 372: 201.
World health organization. Guidelines for development of WHO guidelines 2003.
Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, Liberati A, O’Connell D, Oxman AD, Phillips B, et al.: Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches the GRADE working group. BMC Health Serv Res 2004, 4: 38. 10.1186/1472-6963-4-38
Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, Schünemann HJ, GRADE Working Group: Rating quality of evidence and strength of recommendations: going from evidence to recommendations. BMJ 2008, 336: 1049–1051. 10.1136/bmj.39493.646875.AE
Qaseem A, Forland F, Macbeth F, Ollenschlager G, Phillips S, van der Wees P: Guidelines international network: toward international standards for clinical practice guidelines. Ann Intern Med 2012, 156: 525–531. 10.7326/0003-4819-156-7-201204030-00009
IOM (Institute-of-Medicine): Clinical Practice Guidelines We Can Trust. Washington, DC: The National Academies Press; 2011.
The authors gratefully acknowledge the CONSORT PRO Executive members (in addition to MB and MC): Profs Jane Blazeby, Dennis Revicki, David Moher and Doug Altman and the CONSORT PRO group as detailed in  including the ISOQOL Reporting Guidelines Task Force for their role in developing the CONSORT PRO Extension. We also thank Alessandro Perreca (from the GIMEMA Health Outcomes Research Unit) for editorial assistance.
The CONSORT PRO Extension was supported by funding from the UK Medical Research Council Hubs for Trials Methodology Research and the Canadian Institutes of Health Research.
MC, MB, PJ and FE all contributed to the development of the CONSORT PRO Extension and publication.
All authors contributed to drafts of the manuscript, read and approved the final version for publication.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.