Open Access

Meaningful health outcomes for paediatric neurodisability: Stakeholder prioritisation and appropriateness of patient reported outcome measures

  • Christopher Morris1, 2Email author,
  • Astrid Janssens1, 2,
  • Valerie Shilling1, 2,
  • Amanda Allard3,
  • Andrew Fellowes3,
  • Richard Tomlinson4,
  • Jane Williams5,
  • Jo Thompson Coon2,
  • Morwenna Rogers2,
  • Bryony Beresford6,
  • Colin Green2,
  • Crispin Jenkinson7,
  • Alan Tennant8 and
  • Stuart Logan1, 2
Health and Quality of Life Outcomes201513:87

DOI: 10.1186/s12955-015-0284-7

Received: 16 April 2015

Accepted: 11 June 2015

Published: 25 June 2015

Abstract

Background

Health services are increasingly focused on measuring and monitoring outcomes, particularly those that reflect patients’ priorities. To be meaningful, outcomes measured should be valued by patients and carers, be consistent with what health professionals seek to achieve, and be robust in terms of measurement properties.

The aim of this study was (i) to seek a shared vision between families and clinicians regarding key aspects of health as outcomes, beyond mortality and morbidity, for children with neurodisability, and (ii) to appraise which multidimensional patient reported outcome measures (PROMs) could be used to assess salient health domains.

Methods

Relevant outcomes were identified from (i) qualitative research with children and young people with neurodisability and parent carers, (ii) Delphi survey with health professionals, and (iii) systematic review of PROMs. The International Classification of Functioning Disability and Health provided a common language to code aspects of health. A subset of stakeholders participated in a prioritisation meeting incorporating a Q-sorting task to discuss and rank aspects of health.

Results

A total of 33 pertinent aspects of health were identified. Fifteen stakeholders from the qualitative and Delphi studies participated in the prioritisation meeting: 3 young people, 5 parent carers, and 7 health professionals. Aspects of health that emerged as more important for families and targets for health professionals were: communication, emotional wellbeing, pain, sleep, mobility, self-care, independence, mental health, community and social life, behaviour, toileting and safety. Whilst available PROMs measure many aspects of health in the ICF, no single PROM captures all the key domains prioritised as for children and young people with neurodisability. The paucity of scales for assessing communication was notable.

Conclusions

We propose a core suite of key outcome domains for children with neurodisability that could be used in evaluative research, audit and as health service performance indicators. Future work could appraise domain-specific PROMs for these aspects of health; a single measure assessing the key aspects of health that could be applied across paediatric neurodisability remains to be developed.

Keywords

Children Neurodisability Patient reported outcome Prioritisation Core outcome set

Health services are increasingly focused on measuring and monitoring outcomes, particularly those that reflect patients’ priorities [1, 2]. Such outcomes include routine data indicators, clinical assessments, and patient reported outcome measures (PROMs in UK; PROs in USA). PROMs are also advocated for use in clinical trials [3, 4], and to incorporate individualised feedback from PROMs into routine clinical consultations [5]. To be meaningful, outcomes measured should be valued by patients and carers, be consistent with what health professionals seek to achieve, and be robust in terms of measurement properties. In the UK, but probably salient elsewhere, there is recognition that families and health professionals do not always share a vision for what services are seeking to achieve, and efforts should establish a common focus [6]. Recent policies underline the growing prominence of outcomes as a mechanism for improving services for children and young people [7, 8]. There appears a strong case for agreeing key health outcomes for pragmatic groupings of children and young people, and identifying appropriate and robust PROMs to measure these areas.

Addressing these objectives is the focus of the Core Outcome Measures in Effectiveness Trials (COMET) Initiative, seeking to overcome the problems of synthesising research when different outcomes and measures have been used [9]. The COMET process engages key stakeholders to seek consensus on ‘what’ domains to measure and ‘how’ to measure them. A World Health Organization initiative to establish International Classification of Functioning, Disability and Health (ICF) ‘core sets’ for specific conditions pursues much the same goals [10].

Neurodisability is an umbrella term for conditions associated with impairment of the nervous system and includes cerebral palsy, autism, epilepsy [11]. Individually, many conditions that result in neurodisability are rare, whereas when grouped together they are common; many of the conditions give rise to similar health issues. Although it is reasonable to develop core outcome sets for more common neurodisability conditions, such as have been proposed recently for cerebral palsy [12], it would also be efficient to devise core outcome sets that could be used across paediatric neurodisability. This would avoid expending limited resources to develop multiple core outcome sets for the vast number of different neurodisability conditions and syndromes.

This paper describes the culmination of research that sought a shared vision regarding a core suite of health outcomes for children and young people with neurodisability, beyond mortality and morbidity. Preliminary work comprised three work streams: (i) qualitative research with service users to identify valued aspects of health from the perspectives of young people and parents [13]; (ii) a Delphi survey with health professionals to identify the aspects of health they target commonly [14]; (iii) a systematic review of existing multidimensional PROMs that could be used across paediatric neurodisability conditions [15]. The ICF Version for Children and Youth (ICF-CY) [16] provided a common language to analyse and compare aspects of health across the different work streams (Fig. 1). The study was conceived to inform health services through informing the development of the NHS Outcomes Framework [1], and was subsequently catalogued by the COMET database [17].
Fig. 1

Illustration of study design

In the qualitative study 54 children with neurodisability, and 53 parents participated in either focus groups or interviews [13]. The outcomes discussed by families were: communication, mobility, pain, self-care, temperament, interpersonal relationships and interactions, community and social life, emotional wellbeing, and gaining independence/future aspirations. Parents also discussed sleep, behaviour and safety.

In the Delphi survey the key domains agreed as relevant by over 200 health professionals were: mental health, confidence/emotional stability, anxiety/attention, sleep, pain, toileting, movement and manual ability, acquiring skills, communication, mobility, self-care, recreation and leisure [14]. Although professionals highlighted participation outcomes, aspects of social functioning were perceived generally as less of a responsibility for the NHS.

The systematic review identified 41 multidimensional PROMs that assess various aspects of health in the ICF-CY [15]. Domains covered more commonly are: mental functions, interpersonal interactions and relationships, major life areas, and community, social and civic life [15].

The final stages involved a meeting with a diverse group of stakeholders to seek agreement on core set of ‘shared priorities’ regarding the aspects of health identified. Then to appraise whether any existing PROM could measure the key outcomes prioritised.

Methods

Public involvement

The research had a strong ethos of public engagement throughout [18]. Five parents volunteered to be involved; all were parent carers of children with neurodisability (including cerebral palsy, acquired brain injury and epilepsy). Parents worked alongside members of the research team at various stages of the research in co-investigator meetings, parent advisory meetings and in the dissemination of findings. The group did not receive formal training for their involvement but were supported by members of the team; their contribution and traveling expenses were reimbursed.

Stakeholder prioritisation

We convened a representative group of stakeholders who had taken part in earlier stages of the research. Participants included a range of health professions, young people and parent carers. Health professionals were selected from those who had participated in all four rounds of the online Delphi survey, and a further selection was made to include a range of the different professions by drawing names out of a hat.

The stakeholders were in essence a ‘nominal group’ [19], and the main activity was a Q-sorting prioritisation task [20, 21]. We did not set out to apply the statistical methodology of Q-sorting. Rather, we used the Q-sorting task to observe the processes and discussions that the participants went through when prioritising health outcomes in order to gain insight in to their various attitudes and beliefs.

In advance of the meeting, a list of the health outcomes was compiled from those issues identified (i) from the qualitative research with children and young people and parents; (ii) through the online survey with health professionals; and (iii) from the content of eligible PROM questionnaires identified in the systematic review (Table 1). Duplicate aspects of health were removed. The final list of 33 ‘aspects of health’ or ‘health outcomes’ were represented on laminated cards, with an illustration inspired by the signs from the Talking Mat system used with young people with communication difficulties in the qualitative research. There were some aspects of health that overlapped conceptually, but a decision was made to present these items individually and allow the participants to decide if they should be collapsed as a grouped concept. Nominal group methods involve structured discussion in which participants articulate and clarify their personal views to other members of the group [19]; in our meeting each aspect of health was presented consecutively by a participant followed by facilitated discussion and negotiation about its importance relative to other aspects of health.
Table 1

Aspects of health used in the prioritising exercise and sources

Cards depicting:

Qualitative work with children and parents

PROMs

Delphi survey with health care professionals

Play

x

x

x

Sport

x

 

x

Independent

x

x

x

Communication

x

x

x

Memory

x

 

x

Concentration

 

x

 

Emotional wellbeing

x

x

x

Fitness and stamina

 

x

x

Breathing

  

x

Learning

x

x

x

Let me decide

x

x

x

Muscle strength

  

x

Moving my body

x

x

x

Manipulating objects

 

x

x

Moving about

x

x

x

Pain

x

x

x

Worried

x

x

 

Personality/confidence and self esteem

x

x

x

Family

x

x

x

Friends

x

x

x

Hearing and seeing

x

x

x

Self care

x

x

x

Eating nutrition

x

x

x

Self care Hygiene

x

x

x

Safety

x

x

x

Sexual health

 

x

x

Sleep

x

x

x

Social life/go out

x

x

x

Continence

x

x

x

Education

x

x

 

Drool, swallowing, constipation

x

 

x

Body structures

x

 

x

Control behaviour

x

 

x

Change body position

  

x

The card depicting “safety” was not printed and therefore not used in the exercise

To prioritise the aspects of health we split the participants into two groups, each with a mix of clinician and family representatives. They were directed to work collectively to order the 33 cards on a ‘forced choice frequency distribution’ grid (Fig. 2). The column boxes (left to right, x-axis) indicated priority ranking from ‘less important’ to ‘more important’ for the NHS; boxes within columns (y-axis) were assumed to be of equivalent importance, and no item or box indicated unimportance. The final positions of the cards on the grids were used to indicate shared higher (right side columns) and lesser priorities (left side columns). The groups worked through the exercise for 90 min at separate ends of the same large room, and then broke for lunch. The groups reconvened after lunch for 15 min to review their decisions about relative placements of cards on the grids. Finally, the groups came together to compare which aspects of health had been prioritised, and make any final comments.
Fig. 2

Q-sorting grid used to prioritise aspects of health

Synthesis of findings

We compared the aspects of health identified through the qualitative research and Delphi survey, and those prioritised in the stakeholder meeting. To bring together a core suite of health outcomes relevant to the NHS we identified (i) higher priority aspects of health from the stakeholder meeting, (ii) aspects of health more valued by participants in the qualitative study, (iii) aspects of health targeted by professionals in the Delphi study. Finally, we examined whether these aspects of health can be assessed for children with neurodisability using current multidimensional PROMs.

The NRES Committee North East - County Durham & Tees Valley approved the procedures for this study (reference 11/NE/0364).

Results

Stakeholder group

Participants in the stakeholder prioritisation meeting were three young people with neurodisability (one with autism, two with neuromuscular conditions who were wheelchair users), five parent carers of children with various complex conditions including one or more of cerebral palsy, autism, epilepsy, learning difficulties, and seven health professionals: two paediatricians, physiotherapist, occupational therapist, nurse, paediatric surgeon, and a child and adolescent psychiatrist.

Group dynamics and discussion

The groups engaged well and completed the task (Fig. 3), but found it challenging. There was debate about several aspects of health, for example whether or not they were eligible as being ‘morbidities’ (e.g. seizures) or perceived as self-reportable.
Fig. 3

Completed prioritisation grid from one group

Various issues arose in the process of prioritising concepts as an important focus for the NHS. Participants recognised some concepts as being more readily influenced by health services. They distinguished other concepts as broader ‘life outcomes’ that would be influenced by many factors. Physical outcomes were perceived as more amenable for health services to influence, including ‘pain’, ‘communication’, ‘continence’ and ‘moving my body’. Broader life outcomes that might be harder for health services to influence included ‘emotional well-being’, ‘making decisions’, ‘friends’, ‘family’, ‘social life’ and ‘play’.

Prioritisation task

There was variation in the ordering of the aspects of health between groups, but communication and emotional wellbeing were both ranked highest. Also ranked highly in both groups were pain, sleep, moving about, independence and worry.

Synthesis of findings

Aspects of health selected by stakeholders as shared priorities were communication, emotional wellbeing, pain, sleep, mobility, self-care, independence, mental health, and social activities. In addition, the findings of the qualitative and Delphi studies both support behaviour, toileting, and safety as outcomes that are important for parent carers and also targeted by health professionals. Thus we propose these as a suite of important health domains that should be assessed for children with neurodisability, defined with illustrative examples to aid interpretation (Table 2).
Table 2

Proposed core suite of aspects of health to be assessed using PROMs for children and young people with neurodisability

ICF domain

Example areas

Communication

Receiving and producing messages, including conversations and using technology

Emotional Wellbeing

Feelings of contentment, enjoyment, happiness

Pain

General or localised discomfort

Mobility

Moving around and changing locations

Self-care

Diet, exercise, washing, dressing

Independence

Expressing preferences, making choices

Community and social life

Recreation, sports and leisure

Mental Health

Anxiety, mood

Sleep

Onset, duration and quality of sleep

Behaviour

Managing expression of emotions, maintaining social interactions

Safety

Managing risks that lead to injury or harm

Toileting

Managing continence, constipation

We examined whether existing multidimensional generic PROMs identified in our systematic review [15] measure these health outcomes. Whilst available instruments measure many aspects of health across the ICF [15], no single PROM captures all the domains prioritised as key health outcomes for children and young people with neurodisability. The paucity of scales for assessing communication was notable. In addition, measures of mobility tend to focus on walking and running rather than the ability to move around independently, which might include using assistive technology to achieve participation in this domain.

Discussion

Agreement emerged regarding a core set of health outcomes appropriate to assess for children and young people with neurodisability: communication, emotional wellbeing, pain, sleep, mobility, self-care, independence, social activities, mental health, behaviour, toileting, and safety. In defining these concepts we drew on the WHO terminology; each aspect of health is defined in the ICF except emotional wellbeing and independence. No existing multidimensional PROM that can be used across neurodisability assesses all the key constructs. The paucity of scales to assess communication in multidimensional PROMs is a notable omission for paediatric neurodisability. This was an aspect of health that all stakeholders appeared to value highly, as it enables people to express preferences and make choices. The domain-specific ‘Focus on the Outcomes of Communication Under Six’ is a recently developed parent-reported measure of communication participation that informs how this domain can be measured [22]. It is interesting to note that the ICF core set proposed for cerebral palsy did not include communication [12], which was commented on at the time [23]. Given the high priority given to communication in our study this may merit reconsideration.

Our findings provide an incremental step towards a shared vision between families and clinicians of health outcomes for children with neurodisability. Our stakeholder prioritisation meeting showed that disabled young people, parents and professionals can work collaboratively, given appropriate motivation, environment and support. This approach would seem a valuable and important exercise that could be replicated.

Children and young people with neurodisability depend on services across health care, from primary and community care to specialist centres. The findings are important for a wide audience of clinicians, managers and commissioners, and especially those responsible for ensuring health services meet their goals efficiently.

Increasingly, integrated education, health and social care services are advocated as likely to be more efficient and family-centred [24], which creates complexity. Many health professionals in our research felt that there were limitations to the extent to which health services could be responsible for aspects of social participation. This view is consistent with Wilson and Cleary’s model linking clinical variables to measures of health related quality of life [25]. This also confirms our hypothesis based on results from the Delphi survey, where we stated that professionals might perceive themselves ‘accountable’ more for medical issues rather than social participation and well-being [14]. From the perspectives of families who receive support from a range of services, partitioning NHS outcomes may lack credibility; especially as they articulated how aspects of health were perceived as inter-related [13].

Our methods could be modified when seeking agreement on core outcome sets for other conditions, as advocated by the COMET Initiative [17], and when seeking PROMs to measure those outcome domains. The cognitive task for our prioritisation was challenging due to the large number of concepts that had to be understood, recalled and ranked. It has long been recognised that people find it difficult to discern more than a handful of categories [26]. Therefore it would be preferable to conduct a preliminary activity to reduce the number of issues for the final sorting task. Our methodology can be contrasted with procedures recommended to devise ICF core sets for particular conditions that involve qualitative research, Delphi survey and a consensus meeting [10]. The consensus meeting in the ICF process is a residential event with time allocated to training, and iterative voting to rank aspects of health. However, while the experts invited to these meetings come from diverse professions, and often parts of the world, the voices of patients and carers appear less represented; although we note one parent participated in the meeting focusing on cerebral palsy. Sticking strictly to ICF health domains also runs the risk of omitting outcomes such as emotional wellbeing that are conceptually not classified in the ICF.

Our methodology was successful and we engaged a diverse group of stakeholders in the qualitative study, Delphi survey and prioritisation meeting. However, a criticism that can be levelled at our study is that those who took part in the earlier stages were self-selecting, and the prioritisation meeting was a small sample of stakeholders at a single event. One could also debate the right proportions of each stakeholder group. Hence the extent to which our findings can be generalised is debateable, and we welcome further discussion with the broader stakeholder community. The COMET group have identified a number of ‘issues to consider’ when seeking consensus in this context, but do not provide specific guidance [27]. Hence it is difficult to know the ‘correct proportion’ of stakeholders to consult, or how many meetings are required. Therefore testing and reporting transparently our methods and experiences in this study adds to the methodological research in this area.

Another potential limitation is that our approach was non-categorical across neurodisability; therefore our proposed list of key outcome domains may omit aspects of health specifically important for particular conditions. Nevertheless, our findings accord with other research examining key outcomes for children and young people with complex healthcare needs, autism, cerebral palsy, and those who do not use speech for communication [2831]. Further research could explore whether our proposed core outcomes are replicated in other studies. Such work might explore whether priorities vary for subgroups, for example acquired versus developmental neurodisability and in different age groups.

In our appraisal of generic PROMs, like others, we found poor evidence for reliability of proxy-reports compared to children in many aspects of health [32]. There will always be children and young people who do not have the developmental cognitive capacity to self-report, and it is usually parents and carers who seek health care for their children. Therefore, parent-proxy report is appropriate and may provide important insights. For each domain we need to consider to what extent a proxy (primary carer) is able to report on that construct; in our earlier study parents suggested they had little idea about ‘how their child feels’ when looking at example items in existing PROMs. [13] Parent carers also raised different issues to children/young people that they think are important to measure, such as toileting, behaviour and safety. The potential for a primary carer measure based on the domains of more importance to parents in ways that they feel they can respond accurately merits consideration.

One way forward could be to review existing ‘domain-specific’ PROMs, i.e. questionnaires that seek to specifically measure the core outcomes identified in our study. It is evident that potential respondents should have a greater role in the development process to enhance the validity and acceptability of any PROM. Rasch analysis can also be used to test for invariance of how items perform across age groups, sex and between different diagnoses; the latter being pertinent with the range of neurodisability conditions. One application for PROMs is to measure outcomes over time. Ensuring that scales are robust to measure across different age-groups is important to enable outcomes to be monitored longitudinally. One area requiring further thought is how use PROMs to measure and monitor outcomes for young people as they transition into adulthood, and whether existing adult PROMs assess appropriate aspects of health. A fuller discussion of some of the issues can be found in the full report for this project [33].

Conclusions

There is much work to be done to realise the potential benefits of using PROMs for children generally [34], and especially with children with neurodisability conditions.

There appears a strong case for further research to develop PROMs for children and young people with neurodisability, especially if their application as robust routine indicators and outcome measures for evaluative research is to be realised. As a next step, it would seem appropriate to appraise existing ‘domain-specific’ PROMs for use with children and young people with neurodisability; that is, those questionnaires that specifically measure the key outcome domains. It is evident from our previous work, and other associated research, that service users should have a greater role in the development of any PROM. The International Society for Pharmacoeconomics and Outcomes Research has recommended good practices for developing PROMs for children and young people which are a valuable resource to guide research [35]; most important, perhaps, is that stakeholders be included as partners from the outset.

Declarations

Acknowledgements

We are grateful to the parents who were involved with us as partners in the research, and to the stakeholders who participated in the prioritisation meeting. This paper is part of a full report published by the National Institute for Health Research Library (www.journalslibrary.nihr.ac.uk/hsdr/volume-2/issue-15). Where possible, we have made changes to avoid direct duplication; however some content remains the same.

Funding

This study was part of research funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme (Project 10/2002/16 http://www.nets.nihr.ac.uk/projects/hsdr/10200216). The work also benefited from support from NIHR Collaboration for Leadership in Applied Health Research and Care of the South West Peninsula (PenCLAHRC), and the charity Cerebra. The views and opinions expressed in this paper are those of the authors and not necessarily those of the NHS, the NIHR, the Department of Health, or Cerebra.

Authors’ Affiliations

(1)
PenCRU, Child Health Group, University of Exeter Medical School, University of Exeter
(2)
PenCLAHRC, University of Exeter Medical School, University of Exeter
(3)
Council for Disabled Children, National Children’s Bureau
(4)
Department of Child Health, Royal Devon and Exeter NHS Foundation Trust
(5)
Department of Child Health and Paediatrics, Nottingham University Hospitals NHS Trust
(6)
Social Policy Research Unit, University of York
(7)
Nuffield Department of Population Health, University of
(8)
Department of Rehabilitation Medicine, University of Leeds

References

  1. Department of Health. The NHS Outcomes Framework 2013/14. 2012.Google Scholar
  2. Fitzpatrick R: Patient-reported outcome measures and performance measurement. In Performance measurement for health system improvement: experiences, challenges and prospects. Edited by Smith PC, Mossialos E, Papanicolas I, Leatherman S: Cambridge University Press; 2009: Health Economics, Policy and Management].Google Scholar
  3. US Food and Drug Administration. Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Rockville, MD: Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research; 2009.Google Scholar
  4. National Institute for Health and Care Excellence: Guide to the methods of technology appraisal 2013. Available at http://www.nice.org.uk/article/pmg9/resources/non-guidance-guide-to-the-methods-of-technology-appraisal-2013-pdf [Last accessed: 23/09/2014]; 2013.
  5. Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ. 2015;350:g7818.Google Scholar
  6. Kennedy I, Sir: Getting it right for children and young people: overcoming cultural barriers in the NHS so as to meet their needs. Commissioned by the Department of Health. London: The Stationery Office. Available at http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/@ps/documents/digitalasset/dh_119446.pdf [Last accessed 03/04/2014]; 2010.
  7. Department of Health. Improving Children and Young People’s Health Outcomes: a system wide response. 2013.Google Scholar
  8. Department of Health. Children and Young People's Health Outcomes Forum: Recommendations to improve children and young people’s health results. 2013.Google Scholar
  9. Williamson P, Altman D, Blazeby J, Clarke M, Gargon E. Driving up the quality and relevance of research through the use of agreed core outcomes. J Health Serv Res Policy. 2012;17:1–2.PubMedView ArticleGoogle Scholar
  10. Selb M, Escorpizo R, Kostanjsek N, Stucki G, Ustun B, Cieza A. A guide on how to develop an international classification of functioning, disability and health core set. Eur J Phys Rehabil Med 2015;51(1):105-17Google Scholar
  11. Morris C, Janssens A, Tomlinson R, Williams J, Logan S. Towards a definition of neurodisability: a Delphi survey. Dev Med Child Neurol. 2013;55(12):1103–8.PubMedView ArticleGoogle Scholar
  12. Schiariti V, Selb M, Cieza A, O’Donnell M. International Classification of Functioning, Disability and Health Core Sets for children and youth with cerebral palsy: a consensus meeting. Dev Med Child Neurol. 2015;57:149–58.PubMedView ArticleGoogle Scholar
  13. Allard A, Fellowes A, Shilling V, Janssens A, Beresford B, Morris C. Key health outcomes for children and young people with neurodisability: qualitative research with young people and parents. BMJ Open. 2014;4:e004611.PubMed CentralPubMedView ArticleGoogle Scholar
  14. Janssens A, Williams J, Tomlinson R, Logan S, Morris C. Health outcomes for children with neurodisability: what do professionals regard as primary targets? Arch Dis Child. 2014;99(10):927–32.PubMedView ArticleGoogle Scholar
  15. Janssens A, Thompson Coon J, Rogers M, Allen K, Green C, Jenkinson C, et al. A Systematic Review of Generic Multidimensional Patient-Reported Outcome Measures for Children, Part I: Descriptive Characteristics. Value Health. 2015;18:315–33.PubMedView ArticleGoogle Scholar
  16. World Health Organisation. International Classification of Functioning, Disability and Health - Children and Youth Version. ICF-CY. vol. resolution WHA 54.21. Geneva, Switzerland: WHO; 2007.Google Scholar
  17. COMET Initiative: Core Outcome Measures in Effectiveness Trials. wwwcomet-initiativeorg [accessed 16 June 2014].Google Scholar
  18. Morris C, Shilling V, McHugh C, Wyatt K. Why it is crucial to involve families in all stages of childhood disability research. Dev Med Child Neurol. 2011;53:769–71.PubMedView ArticleGoogle Scholar
  19. Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74:979–83.PubMed CentralPubMedView ArticleGoogle Scholar
  20. Watts S, Stenner P. Doing Q Methodological Research: Theory, Method & Interpretation. London: SAGE Publications Limited; 2012.Google Scholar
  21. Stephenson W. The study of behavior: Q-Technique and its methodology. Chicago: Univiversity of Chicago Press; 1953.Google Scholar
  22. Thomas-Stonell N, Oddson B, Robertson B, Rosenbaum P. Validation of the Focus on the Outcomes of Communication under Six outcome measure. Dev Med Child Neurol. 2013;55:546–52.PubMed CentralPubMedView ArticleGoogle Scholar
  23. Wright FV. What needs to happen before an International Classification of Function, Disability and Health Core Set is ready for clinical use? Dev Med Child Neurol. 2015;57:112–3.PubMedView ArticleGoogle Scholar
  24. Law M, Hanna S, King G, Hurley P, King S, Kertoy M, et al. Factors affecting family-centred service delivery for children with disabilities. Child Care Health Dev. 2003;29:357–66.PubMedView ArticleGoogle Scholar
  25. Wilson IB, Cleary PD. Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA. 1995;273:59–65.PubMedView ArticleGoogle Scholar
  26. Miller GA. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev. 1956;63:81–97.PubMedView ArticleGoogle Scholar
  27. Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.PubMed CentralPubMedView ArticleGoogle Scholar
  28. Young B, Rice H, Dixon-Woods M, Colver AF, Parkinson KN. A qualitative study of the health-related quality of life of disabled children. Dev Med Child Neurol. 2007;49:660–5.PubMedView ArticleGoogle Scholar
  29. Davis E, Shelly A, Waters E, Mackinnon A, Reddihough D, Boyd R, et al. Quality of life of adolescents with cerebral palsy: Perspectives of adolescents and parents. Dev Med Child Neurol. 2009;51:193–9.PubMedView ArticleGoogle Scholar
  30. Beresford B, Tozer R, Rabiee P, Sloper P. Desired Outcomes for Children and Adolescents with Autistic Spectrum Disorders. Child Soc. 2007;21:4–16.Google Scholar
  31. Rabiee P, Sloper P, Beresford B. Desired outcomes for children and young people with complex health care needs, and children who do not use speech for communication. Health Soc Care Community. 2005;13:478–87.PubMedView ArticleGoogle Scholar
  32. Janssens A, Rogers M, Thompson Coon J, Allen K, Green C, Jenkinson C, et al. A Systematic Review of Generic Multidimensional Patient-Reported Outcome Measures for Children, Part II: Evaluation of Psychometric Performance of English-Language Versions in a General Population. Value Health. 2015;18:334–45.PubMedView ArticleGoogle Scholar
  33. Morris C, Janssens A, Allard A, Thompson-Coon J, Shillling V, Tomlinson R, Williams J, Fellowes A, Rogers M, Allen K, et al. Informing the NHS Outcomes Framework: evaluating meaningful health outcomes for children with neurodisability using multiple methods including systematic review, qualitative research, Delphi survey and consensus meeting. Health Serv Deliv Res 2014, 2(15).Google Scholar
  34. Huang IC, Revicki DA, Schwartz CE. Measuring pediatric patient-reported outcomes: good progress but a long way to go. Qual Life Res. 2014;23:747–50.PubMed CentralPubMedView ArticleGoogle Scholar
  35. Matza LS, Patrick DL, Riley AW, Alexander JJ, Rajmil L, Pleil AM, et al. Pediatric patient-reported outcome instruments for research to support medical product labeling: report of the ISPOR PRO good research practices for the assessment of children and adolescents task force. Value Health. 2013;16:461–79.PubMedView ArticleGoogle Scholar

Copyright

© Morris et al. 2015

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement