Cite as: Ungar W, Santos M. The pediatric economic database evaluation (PEDE) project. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA); 2002. Technology report no 26.
Economic evaluations of health care interventions and services are becoming increasingly frequent to inform clinical and health policy decision-making. In recent years, standard methods have been recommended for conducting health economic studies, especially pharmaceutical evaluations. However, the applicability of standard methods to special populations such as children must be examined. Thus the Pediatric Economic Database Evaluation (PEDE) Project was conceived. The two research objectives were:
- To analyze trends of published economic evaluations in pediatric health care over a 20-year period
- To critically appraise the quality of published pediatric economic evaluations in order to identify gaps and areas for future methodological investigation.
With funding from The Hospital for Sick Children (HSC) Research Institute, a comprehensive pediatric health economic database of 787 publications for January 1, 1980 to December 31, 1999 was established. The database includes key characteristics for each citation, including year, target population, ICD-9-CM disease class, age group, experimental intervention, intervention category, health outcomes and analytical technique and is linked to a bibliographic database containing the full citation information and abstract.
The examination of trends in the literature was accomplished by performing various one-way frequency distributions and two-way cross-tabulations on variables within the database. To conduct the quality appraisal, a draft quality appraisal instrument was developed and subjected to review by a panel of experts. The instrument was pilot tested and the final quality appraisal questionnaire was completed for a random sample of 150 publications by two independent appraisers. Test-retest and inter-rater reliability of the quality appraisal instrument were assessed. Descriptive statistics, correlations and analysis of variance were used to describe the quality of the sample of publications.
A total of 787 papers were published over the study period with an average annual growth rate of 22%. Over half of the papers (440) were published from 1995 to 1999. Whereas infectious diseases and congenital anomalies were most frequently studied in 1980-84, complications of pregnancy and perinatal conditions displayed increasing prominence over time. Infectious diseases commonly studied included hepatitis B, general vaccination strategies, Hemophilus influenzae type B, measles, and varicella. Common pregnancy and perinatal conditions were cardiac abnormalities, low birth weight, prematurity, respiratory conditions, Downs syndrome, and congenital hip dislocation. By 1995-99, infectious diseases, congenital anomalies and complications of pregnancy accounted for 47% of all publications and health prevention, health treatments and detection interventions accounted for 70% of all interventions. Cost-effectiveness analysis (CEA) was the most common technique used, accounting for a majority of evaluations in all time intervals. The proportion of studies using CEA increased by 23% while the proportion of studies using cost benefit analysis (CBA) decreased from 31% in 1980-84 to just 12% in 1995-99. Cost utility analysis (CUA) remained the least common analytic technique at all times.
A 57-item pediatric quality appraisal questionnaire was developed and demonstrated excellent test-retest reliability (ICC = 0.92, 95% CI 0.71-0.98) and very good inter-rater reliability (ICC = 0.75, 95% CI 0.66-0.81). The sampled papers demonstrated good quality for the following domains: Target Population, Economic Evaluation, Discounting, Conclusions and Comparators. The papers were of poor quality with respect to Conflict of Interest, Incremental Analysis, Sensitivity Analysis and Perspective. Over one third (38%) were rated as very good to excellent according to the global rating while 44% were good or fair. Only the Costs & Resource Use, Outcomes and Conclusions domains demonstrated improvement over time but the effect was slight. Analytic technique was a significant predictor of domain score for most of the study design-related domains. Except for Time Horizon and Perspective, studies designated as CUAs demonstrated the highest domain scores and were significantly higher than those of CEAs and most cost minimization analyses (CMA).
The largest proportion of studies represented health prevention interventions rather than treatments of specific diseases. This is consistent with the understanding that the incidence of pediatric disease is rare and the prevention of future pediatric and adult illness is emphasized. The growth in evaluations of diagnostic and treatment strategies for congenital and perinatal conditions is expected to continue with the advent of expensive new technology to support early life and as research advances in genomics are translated into genetic testing and treatment.
Most outcome measures used in the pediatrics economic literature represented intermediate outcomes. Final outcomes such as mortality, life years gained and QALYs are particularly problematic for the pediatric population. Currently no published health economic models exist for measuring life years gained over long time horizons that include periods of maturation, development and rapid physiological change.
The quality appraisal demonstrated a disappointing performance. Incremental analysis and sensitivity analysis were often missing and the overall economic analysis was usually poorly done. Statements regarding funding sources and investigator independence were usually absent. These findings reiterate the importance of adherence to health economic guidelines that promulgate the conduct and publication of high quality economic evaluations. Furthermore, peer reviewers who critique manuscripts for publication in medical journals should become familiar with these guidelines and should insist on high quality manuscripts.
The PEDE Project represents the first attempt to catalogue, analyse and appraise economic evaluations performed in a pediatric population. The findings point to the importance of approaching pediatric health economic research with a child-centered framework. Although this research revealed dramatic growth in the volume of publications over time, there remains much room for improvement in the quality of pediatric economic evaluations. Opportunities for future research relate to the development of statistical models to capture health benefits over time horizons that include development and maturation. The PEDE database, containing 787 citations, will be of value to decision-makers as well as researchers undertaking methods research, a systematic review or meta-analysis of a particular intervention or condition. The conceptual knowledge derived from this project will foster research initiatives that will enhance the quality of economic assessment in children.