Health-related quality of life by allergy symptoms in elementary school students
Health and Quality of Life Outcomes volume 16, Article number: 93 (2018)
Globally, allergic diseases are very common in childhood and may affect children’s quality of life. This study aimed to explore health-related quality of life of elementary school students with allergy symptoms using the EQ-5D-Y and to examine the validity and feasibility of the EQ-5D-Y.
The study subjects were the students within 50 elementary schools which voluntarily participated in research project. In this sample population, the EQ-5D-Y questionnaire was self-administered by the students, and demographic and health information were collected from the student’s parents. The parents’ information was used to investigate the proportion of students with allergic symptoms (wheezing, runny or blocked nose, and itchy rash) in the past 12 months. In addition, we analyzed the correlation of symptom reporting and EQ-5D-Y including EQ-VAS.
The overall return was 9117 responses, of which 198 (2.2%) lacked responses on the EQ-5D-Y dimension and 1258 (13.8%) on the VAS score. There were significant differences in symptom reporting in all EQ-5D-Y dimensions between groups with or without allergic symptoms. Particularly, there was a large difference in reporting rates in ‘having pain or discomfort’ and ‘feeling worried, sad or unhappy’ dimensions. As the number of allergic symptoms increased, in all dimensions also the problem reporting rate tended to increase.
As expected, the presence of allergic symptoms is inversely correlated with the quality of life of children. The EQ-5D-Y instrument proved to be useful in terms of feasibility and construct validity in assessing the quality of life of Korean elementary school students.
Asthma, rhinitis, and atopic dermatitis in childhood are common chronic diseases worldwide including Korea [1, 2]. Globally, in 2013, in the 6-7 year old group, the prevalence of asthma was 11.7%, of rhinoconjunctivitis 8.5% and of atopic dermatitis 7.9%, although there were wide variations in the prevalence and severity according to the world region . These allergic diseases not only increase the economic burden but also affect the quality of life of children, and their parents, respectively [2,3,4,5,6].
Interest in self-reported health-related quality of life [HRQOL] as an outcome measure in children and adolescents has grown in recent decades in both hospital and outpatient settings [7,8,9]. Consequently, a number of instruments have been developed to assess HRQOL in children and adolescents. Generic instruments can be applied to various population groups, allowing comparison of the relative effects of healthcare interventions and the quality of life between healthy and sick children . Assessment tools for HRQOL for adults may not be appropriate for children, especially if the children’s language and cognitive abilities are not yet well enough developed. In addition, there may be some discrepancies between children’s self-report and proxy reports e.g. by parents [11,12,13,14].
The EuroQoL group developed a child-friendly version of the already established EQ-5D 3 level version [EQ-5D-3 L] , the EQ-5D for youth [EQ-5D-Y] , to assess HRQOL in children and adolescents . The new instrument is simple and easy to self-administer. Previous studies supported the feasibility of EQ-5D-Y as an instrument with low missing or ambiguous answers [6, 18,19,20,21,22,23,24].
In Jelsma’s study comparing EQ-5D-Y and EQ-5D-3 L in children, the EQ-5D-Y led to fewer missing responses than the EQ-5D-3 L and response ranges in the EQ-5D-Y were broader than that in the EQ-5D-3 L, suggesting that the responsiveness in EQ-5D-Y may be better . The validity [6, 18, 20,21,22,23, 25, 26], reliability [20,21,22, 26] and responsiveness  of the EQ-5D-Y were demonstrated in populations including children and adolescents with chronic diseases including asthma. The Korean version of the EQ-5D-Y was launched in 2015 by the EuroQol group, and HRQOL data for elementary school students obtained using the EQ-5D-Y instrument in Korea was published firstly in 2017 .
The purpose of this study was to explore health-related quality of life of elementary school students with allergy symptoms using the EQ-5D-Y and to examine the validity and feasibility of the EQ-5D-Y.
Study design and procedure
This study is a cross-sectional survey. The subjects of the study were students in 50 primary schools which voluntarily participated in research project in Gyeongsangbuk Province in eastern South Korea. All schools were general schools, that is, no special schools were included. Of the participating schools, 39 were located in the city and 11 were located in the countryside. The study started with 24 schools in 2015, and another 26 were added in 2016. Whereas in the former 24 schools only the first grade students participated in the survey, in the latter 26 schools the survey was conducted in all grades.
Written questionnaires for both children and parents, together with an informative letter and a consent form for the children as well as for their parents or guardians, were provided in sealed envelopes. These envelopes then were distributed in the classes by teachers, to be taken home by the children and to be filled at home. One copy of the written consent form was signed by both the parents/guardians, and the children, respectively.
The questionnaires consisted of three parts: 1) questions on socio-demographic data (i.e., gender, age, and grade), 2) the Korean-translated modified version of the International Study of Asthma and Allergies in Childhood (ISAAC) questionnaire [i.e., physician diagnosis of asthma, allergic rhinitis and atopic dermatitis since birth, treatment and symptoms of the asthma, allergic rhinitis and atopic dermatitis in the past 1 year, school absence due to asthma, allergic rhinitis and atopic dermatitis past 12 months, and emergency room (ER) visits due to asthma] , and 3) the EQ-5D-Y. Socio-demographic and health information was requested to be filled in by the parents/guardians, and the EQ-5D-Y was requested to be self-completed by the students. After the EQ-5D-Y questionnaire, information was requested regarding who actually completed the EQ-5D-Y, whether the parents, the students with the help of their parents, or the students themselves. The completed questionnaires were submitted to the school by students. It took from 2 to 7 days. No further reminders to reply were sent. The survey was conducted between February and September, 2016.
The study was approved by the Institutional Review Board of Dongguk University (110757-201,701-HR-02-03).
The EQ-5D-Y consists of the EQ-5D-Y descriptive system and the EQ visual analogue scale (EQ-VAS). The EQ-5D-Y descriptive system asks a child to check its best description in each dimension about his or her health on the present day. The descriptive system comprises the same five dimensions as does the EQ-5D-3 L, except that it uses child-friendly wording (mobility, looking after myself, doing usual activities, having pain or discomfort, and feeling worried, sad or unhappy) to the target population 8 to 15 years old . Each dimension has three response levels: ‘no,’ ‘some,’ and ‘a lot of’ problems in four of the dimensions, and ‘not,’ ‘a bit,’ and ‘very’ in the fifth dimension . The additional EQ-VAS records the respondent’s self-rated health (“How good is your health TODAY?”) on a vertically numbered, visual analogue scale from 0 to 100. Its endpoints are labelled ‘The worst health you can imagine’ at 0 and ‘The best health you can imagine’ at 100 . A health state summary descriptor is defined by recording the problem level from each of the five dimensions using the EQ-5D-Y. Each state is referred to by a 5-digit code. For example, the 11111 state represents a full health state with no problems in any dimension, whereas 11312 represents no problems with mobility or looking after oneself, a lot of problems with usual activities, no pain or discomfort, and being a bit worried, sad or unhappy . Thus, with this questionnaire, 243 (35) possible health states can be defined. The use of the EQ-5D-Y questionnaire in this study was approved by the EuroQoL group.
To evaluate the feasibility of the EQ-5D-Y, all returned questionnaires (n = 9117) were analyzed. However, 198 missed responses, which is not completing at least one dimension of the EQ-5D-Y, of the total replies were excluded to the primary analysis, including evaluation of the validity of the EQ-5D-Y. The feasibility of using the EQ-5D-Y to measure the HRQOL of Korean elementary school students was analyzed using the completion rate of the EQ-5D-Y questionnaires and the actual responders. We explored demographic characteristics and missing responses categorized by the person filling out the questionnaire (i.e., parents, the students with the help of their parents, and the students themselves).
To explore the construct validity of the EQ-5D-Y, the differences in reported problem rate in EQ-5D-Y dimensions and EQ-VAS score according to allergic symptoms (i.e., wheezing or whistling in the chest, runny or blocked nose, and itchy rash) were analyzed. Differences in problem reporting rates were tested with chi-square test and EQ-VAS score differences with independent t-test. In addition, the difference in the proportion of respondents reporting problem in EQ-5D-Y dimensions and their EQ-VAS score depending on the number of allergic symptoms reported were determined using chi-square test or ANOVA with post hoc Tukey test as appropriate. According to previous studies, the reporting rate in the group with allergic symptoms was expected to be higher than that of the group with no allergic symptoms, and the higher, the higher the problem reporting rate in EQ-5D-Y [5, 18, 20, 28, 29]. P-value of less than 0.05 was considered statistically significant. All statistical analyses were performed using SAS software version 9.1 (SAS Institute, Cary, NC, USA).
Of 9949 handed out questionnaires, 9117 (91.6%) came back to the researchers. The median school response rate was 95%, with a minimum of 68.8% and a maximum of 100%.
Demographic characteristics and health information of the students are in Table 1. The age of participants ranged from 7 to 13 years, with an average age of 10.2 years (SD, 1.8). Age differed significantly (P < 0.001) between valid responders (mean age = 10.2) and responders who did not complete EQ-5D-Y descriptive system (mean age = 9.0) (data not shown). The biggest proportion of respondents – some 25% – was in 1st grade, whereas each some 15% were in grades 2 through 6. Of the missing respondents, 125 (63.1%) were in the 1st grade. Over the past 12 months, wheezing or whistling in the chest (asthma) were reported by 259 (2.9%), runny or blocked nose (allergic rhinitis) by 2477 (27.8%), and itchy rash symptoms (atopic dermatitis) by 1801 (20.2%).
The missing proportion of the EQ-5D-Y for all respondents and by actual responders of the EQ-5D-Y are shown in the Table 2. Of the 9117 responses, 198 (2.2%) had missing data for at least one dimension of the EQ-5D-Y, and the missing proportion for each dimension ranged from 155(1.7%) in ‘looking after myself’ dimension to 162 (1.8%) in the ‘feeling worried, sad, or unhappy’ dimension. There were 1258 (13.8%) missing responses on the EQ-VAS. There was no invalid answer on the EQ-5D-Y descriptive system or EQ-VAS. Of the respondents of EQ-5D-Y, 3627 (39.8%) were the students with the help of their parents, followed by 3371 (37.0%) parents and 1443 (15.8%), the students themselves. When students themselves responded, the missing proportion was somewhat lower, but not significantly.
Responses with missing values for the EQ-5D-Y descriptive system (198 cases, 2.2%) were excluded from the analysis. The remaining 8919 cases (97.8%) were used for the final analysis.
The proportions of reported problems by dimension and EQ-VAS score by grade (low vs. high grade) are summarized in Table 3. The proportion of reported problems in the 4th dimension ‘having pain or discomfort’ and in the 5th dimension ‘feeling worried, sad, or unhappy’ was higher than in the remaining first three dimensions. The mean VAS score was 91.2 (SD, 11.1) in grade 1st through 3rd, and 88.2 (SD, 13.0) in grade 4th through 6th. A total of 71 distinct EQ-5D-Y health states were reported. Among EQ-5D-Y health states, ‘11111’ (i.e., full health) states accounted for 82.0%, followed by the ‘11121’ and ‘11112’ health states at 5.0 and 4.8%, respectively (data not shown).
Comparisons of reported problems and VAS scores on the EQ-5D-Y by allergic symptoms are shown in Table 4. There were significant differences in the problem reporting proportion in all EQ-5D-Y dimensions between groups with and without allergic symptoms. There was a particularly large difference in problem reporting rates in ‘having pain or discomfort’ and ‘feeling worried, sad or unhappy’ dimensions compared with the other dimensions of the EQ-5D-Y. Among the allergic symptoms, the group experiencing symptoms of wheezing or whistling in the chest reported significantly more problems than the group without it. The problem reporting rate in the group experiencing wheezing or whistling in the chest in the respective dimensions was 21.1% in ‘having pain or discomfort’ and 18.8% in ‘feeling worried, sad or unhappy’. The average of EQ-VAS score for children experiencing wheezing was 84.5, for runny or blocked nose 88.8, and for itchy rash 88.6, respectively. Students with allergic symptoms had a lower EQ-VAS scores compared to those without. There were similar patterns of problem reporting of the EQ-5D-Y by the diagnosis and treatment of allergic diseases (data not shown).
Finally, the proportions of reported problems and VAS scores on the EQ-5D-Y by the number of allergic symptoms are shown in Table 5. As the number of allergic symptoms increased, the problem reporting rate tended to increase in all dimensions. In particular, the problem reporting rates in having pain or discomfort’ and ‘feeling worried, sad or unhappy’ dimensions increased significantly. As the number of allergic symptoms increased, the EQ-VAS score decreased significantly.
This study compared HRQOL according to allergy symptoms in Korean elementary school students and focused on the importance of school-based allergic disease management. Our study is a large population-based sample study. This study showed that the use of EQ-5D-Y to measure the quality of life of allergic children in elementary school students was feasible and showed construct validity with allergic symptoms.
We compared the quality of life measured with EQ-5D-Y in students with allergic symptoms versus those without. As expected, the quality of life measured with EQ-5D-Y was significantly lower in students with allergic symptoms than in those without, and the quality of life tended to decrease as the number of allergic symptoms increased. Raven-Sieberer et al.  indicated that the EQ-5D-Y might not be very useful for discriminating between respondents in the general population. However, our results indicate that EQ-5D-Y can be useful in distinguishing the quality of life of respondents in the general population.
In the EQ-5D-Y descriptive system, we found a high prevalence of problems reported of ‘having pain or discomfort’ and ‘feeling worried, sad or unhappy’ dimensions in all allergic symptoms. Students with wheezing symptoms reported the most problems in all dimensions of EQ-5D-Y, and also the lowest VAS scores. Thus, among the investigated allergic diseases, asthma appears to have a major effect on the HRQOL of children. Trends in the EQ-5D-Y problem reporting rate and EQ-VAS score in this study were similar depending on whether the allergic diseases were diagnosed or not, although not shown in the results. Covaciu’s study reported that allergic diseases, and particularly asthma, significantly affect children’s HRQOL. Frequent wheezing is associated with a high prevalence of problems of ‘pain or discomfort’ and ‘anxiety or depression’ dimensions (26.7 and 23.3%, respectively) . This is similar to our findings, although there is a difference in the quality of life of children assessed by parents using the EQ-5D-3 L. Coviciu et al. reported that allergic comorbidity was associated with the EQ-VAS score . In a Swedish study evaluating the quality of life of patients with asthma with EQ-5D-Y, the problem reporting rates of dimensions of ‘doing usual activities’ was 20%, ‘having pain or discomfort’ was 33.3%, and ‘feeling worried, sad or unhappy’ was 20%, and the VAS average score was 80.7 . Compared with our study, the problem reporting rate is higher in the ‘doing usual activities’ dimension (20% versus 5.9%) . The subjects of the study in Sweden were recruited from the clinics of Stockholm, so the asthma symptom of the subject is likely to be worse than the school-based survey . Our findings are consistent with studies that reported adversely affected quality of life in patients with allergic rhinitis  or atopic dermatitis [3, 30].
Of the total respondents, 2.2% did not answer at least one EQ-5D-Y question, and 13.8% did not respond to the EQ-VAS. A multi-national study  reported that the proportion of missing responses in the EQ-5D-Y dimensions ranged from 0% in Spain and Italy to 2% South Africa, and non-responders of the EQ-VAS ranged from 0% in Italy to 9% in Germany. Our study is a survey done at the student’s home with written instruction, while the multi-national study was mainly conducted at the school, and in Italy, the researchers provided assistance if needed . Our study was not a face-to-face survey, so we recorded the person who wrote the EQ-5D-Y. More than half were written by students with or without assistance. However, 37% of respondents are parents, and the lower the grade level, the higher the percentage of parents answering was. Jelsma also reported more missing responses in the lower grades . Parents’ and children’s thoughts may be different in evaluating the quality of life of a child [11,12,13]. Therefore, the researcher would be wise to assist the elementary school students, especially in the lower grades, to reduce the missing values and improve the completeness of the data.
The strength of our research is that we have covered a large general population. However, there are several limitations of this study. First, our study is a cross-sectional questionnaire survey without clinical examination, so there may be a limitation on the accuracy of diagnosis. However, in previous allergic diseases prevalence studies in elementary school students in Korea, prevalence rates over the past 12 months of symptoms of asthma ranged from 4.8 to 6.5%, allergic rhinitis from 32.9 to 38.5%, and atopic dermatitis from 14.5 to 16.7% [27, 31]. These findings were not much different from our study. Second, our findings include 37% of parents’ responses to EQ-5D-Y. We conducted the same analysis with students only if they wrote their own and with help. The results were not significantly different from those for the whole. In some cases, the p-value slightly increased, but still remained significant. Third, our study could not evaluate the convergent validity, responsiveness and reliability of the EQ-5D-Y due to limited questionnaires and the study design without retest. Further studies are required to examine the other psychometric properties such as test–retest reliability, and responsiveness of the Korean EQ-5D-Y for elementary school students with a wider range of clinical conditions.
In conclusion, the HRQOL was lower in the students with allergic symptoms compared those without, and the HRQOL became lower as the number of symptoms increased. The EQ-5D-Y instrument is acceptable in terms of feasibility and construct validity in assessing the quality of life of elementary school students in Korea. However, it would be beneficial for the researcher to assist the elementary school students, especially in the lower grade, when they fill out the questionnaire to reduce missing values and improve the completeness of the data. EQ-5D-Y would be a useful tool for measuring the quality of life of students with allergic symptoms.
EQ-5D 3 level version
EQ-5D for youth
Health-related quality of life
International Study of Asthma and Allergies in Childhood
Mallol J, Crane J, von Mutius E, Odhiambo J, Keil U, Stewart A. ISAAC phase three study group. The international study of asthma and allergies in childhood (ISAAC) phase three: a global synthesis. Allergol Immunopathol. 2013;41(2):73–85.
Ahn K, Kim J, Kwon H, Chae Y, Hahm M, Lee K, et al. The prevalence of symptoms of asthma, allergic rhinoconjunctivitis, and eczema in Korean children: Nationwide cross-sectional survey using complex sampling design. J Korean Med Assoc. 2011;54(7):769–78.
Lifschitz C. The impact of atopic dermatitis on quality of life. Ann Nutr Metab. 2015;66(Suppl 1):34–40.
Meltzer EO. Quality of life in adults and children with allergic rhinitis. J Allergy Clin Immunol. 2001;108(1 Suppl):S45–53.
Covaciu C, Bergstrom A, Lind T, Svartengren M, Kull I. Childhood allergies affect health-related quality of life. J Asthma. 2013;50(5):522–8.
Bergfors S, Astrom M, Burstrom K, Egmar AC. Measuring health-related quality of life with the EQ-5D-Y instrument in children and adolescents with asthma. Acta Paediatr. 2015;104(2):167–73.
Solans M, Pane S, Estrada MD, Serra-Sutton V, Berra S, Herdman M, et al. Health-related quality of life measurement in children and adolescents: a systematic review of generic and disease-specific instruments. Value Health. 2008;11(4):742–64.
Rajmil L, Roizen M, Psy AU, Hidalgo-Rasmussen C, Fernandez G, Dapueto JJ, et al. Health-related quality of life measurement in children and adolescents in Ibero-american countries, 2000 to 2010. Value Health. 2012;15(2):312–22.
Chen G, Ratcliffe J. A review of the development and application of generic multi-attribute utility instruments for paediatric populations. PharmacoEconomics. 2015;33(10):1013–28.
Peter MF, David M. Quality of life. Assessment, analysis and interpretation. 2nd ed. West Sussex: Whiley; 2007.
Sattoe JN, van Staa A, Moll HA. On your own feet research group. The proxy problem anatomized: child-parent disagreement in health related quality of life reports of chronically ill adolescents. Health Qual Life Outcomes. 2012;10:10.
Jelsma J, Ramma L. How do children at special schools and their parents perceive their HRQoL compared to children at open schools? Health Qual Life Outcomes. 2010;8:72.
Upton P, Lawford J, Eiser C. Parent-child agreement across child health-related quality of life instruments: a review of the literature. Qual Life Res. 2008;17(6):895–913.
Sousa MÁP, Sánchez-Toledo PRO, Fuertea NG. Parent-child discrepancy in the assessment of health-related quality of life using the EQ-5D-Y questionnaire. Arch Argent Pediatr. 2017;115(6):541–6.
Rabin R, Oemar M, Oppe M, Janssen B, Herdman M. EQ-5D-3L user guide: basic information on how to use the EQ-5D-3L instrument, vol. 22. Rotterdam: EuroQol Group; 2011.
van Reenen M, Janssen B, Oppe M, Kreimeier S, Greiner W. EQ-5D-Y user guide. Basic information on how to use the EQ-5D-Y instrument. Version 1; 2014. p. 1.
Wille N, Badia X, Bonsel G, Burstrom K, Cavrini G, Devlin N, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Qual Life Res. 2010;19(6):875–86.
Burstrom K, Svartengren M, Egmar AC. Testing a Swedish child-friendly pilot version of the EQ-5D instrument--initial results. Eur J Pub Health. 2011;21(2):178–83.
Jelsma J. A comparison of the performance of the EQ-5D and the EQ-5D-Y health-related quality of life instruments in south African children. Int J Rehabil Res. 2010;33(2):172–7.
Ravens-Sieberer U, Wille N, Badia X, Bonsel G, Burstrom K, Cavrini G, et al. Feasibility, reliability, and validity of the EQ-5D-Y: results from a multinational study. Qual Life Res. 2010;19(6):887–97.
Scalone L, Tomasetto C, Matteucci MC, Selleri P, Broccoli S, Pacelli B, et al. Assessing quality of life in children and adolescents: development and validation of the Italian version of the EQ-5D-Y. Italian J Public Health. 2012;8(4):331-41.
Robles N, Rajmil L, Rodriguez-Arjona D, Azuara M, Codina F, Raat H, et al. Development of the web-based spanish and Catalan versions of the Euroqol 5D-Y (EQ-5D-Y) and comparison of results with the paper version. Health Qual Life Outcomes. 2015;13:72.
Burstrom K, Bartonek A, Brostrom EW, Sun S, Egmar AC. EQ-5D-Y as a health-related quality of life measure in children and adolescents with functional disability in Sweden: testing feasibility and validity. Acta Paediatr. 2014;103(4):426–35.
Kim SK, Jo MW, Kim SH. A cross sectional survey on health-related quality of life of elementary school students using the Korean version of the EQ-5D-Y. PeerJ. 2017;21(5):e3115.
Eidt-Koch D, Mittendorf T, Greiner W. Cross-sectional validity of the EQ-5D-Y as a generic health outcome instrument in children and adolescents with cystic fibrosis in Germany. BMC Pediatr. 2009;9:55.
Scott D, Ferguson GD, Jelsma J. The use of the EQ-5D-Y health related quality of life outcome measure in children in the western cape, South Africa: psychometric properties, feasibility and usefulness - a longitudinal, analytical study. Health Qual Life Outcomes. 2017;15(1):12.
Suh M, Kim HH, Sohn MH, Kim KE, Kim C, Shin DC. Prevalence of allergic diseases among Korean school-age children: a nationwide cross-sectional questionnaire study. J Korean Med Sci. 2011;26(3):332–8.
Otto C, Barthel D, Klasen F, Nolte S, Rose M, Meyrose A, et al. Predictors of self-reported health-related quality of life according to the EQ-5D-Y in chronically ill children and adolescents with asthma, diabetes, and juvenile arthritis: longitudinal results. Qual Life Res. 2017; https://doi.org/10.1007/s11136-017-1753-8.
Chen G, Flynn T, Stevens K, Brazier J, Huynh E, Sawyer M, et al. Assessing the health-related quality of life of Australian adolescents: an empirical comparison of the child health utility 9D and EQ-5D-Y instruments. Value Health. 2015;18(4):432–8.
Lewis-Jones S. Quality of life and childhood atopic dermatitis: the misery of living with childhood eczema. Int J Clin Pract. 2006;60(8):984–92.
Hong S, Son DK, Lim WR, Kim SH, Kim H, Yum HY, et al. The prevalence of atopic dermatitis, asthma, and allergic rhinitis and the comorbidity of allergic diseases in children. Environ Health Toxicol. 2012;27:e2012006.
This work was supported by the Research Program funded by the Korea Centers for Disease Control and Prevention. Sponsors supported data collection and were not involved in research design, interpretation, or publication.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
This study was approved by the Institutional Review Board of Dongguk University (110757-201701-HR-02-03). Both the parents/guardians and the child consented to participating in the study.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kim, SK., Jo, MW. & Kim, SH. Health-related quality of life by allergy symptoms in elementary school students. Health Qual Life Outcomes 16, 93 (2018). https://doi.org/10.1186/s12955-018-0922-y