Workshops

Systematic reviews synthesize and appraise research results with the aim of providing balanced, unbiased evaluations of evidence on specific research questions. Given the role of systematic reviews in evidence-based medicine and decision-making, and the increasing volume of systematic reviews being made available, further distillations of the results of systematic reviews have a role to play in informing those practitioners interested in using evidence to inform decisions, whether when treating a patient or recommending policy options to government bodies. To support the integration of evidence-based medicine in the day-to-day work of clinicians, policy-makers, and others, meta-syntheses such as overviews of systematic reviews are a way to assemble the evidence so that responses to questions are clear and accessible.

Methods: An introduction to different methods used to systematically evaluate, summarize, and interpret data from systematic reviews will be presented to participants, with a focus on data extraction and analysis. Attendees will have the opportunity to look at and discuss a variety of examples of overview methods and outputs.

Educational Objectives: Participants will gain an understanding of how to conduct an overview of systematic reviews, and the strengths and limitations of using overviews to guide research and support decision-making.

Presenters: Julia Worswick and S. Carolyn Wayne, Cochrane Effective Practice and Organisation of Care (EPOC) Review Group

Clinical guidelines are one of the most highly synthesized forms of knowledge transfer.  Therefore, they are an important means of bringing evidence to practice and, by assisting health care professionals in providing the best and most efficient care to their patients, they also help to maximize the use of health care resources.

The goal of this workshop is to examine the role and value of guidelines in encouraging best practices in medical care, the issues involved in their development, and the best methods of their implementation. 

The workshop will have the following structure:

• Clinical guidelines. This will be an introductory section, reviewing the role of guidelines in knowledge transfer and the important issues in guidelines development.
• Guidelines implementation. This section will examine the evidence for the effectiveness of guidelines and the effectiveness of the various methods of implementing them.
• Guidelines as part of a computerized order entry system. This section will review the work done by the Canadian Association of Radiologists on the implementation of radiology guidelines as part of a computerized order entry system, including evidence on the effectiveness of this methodology and the issues involved in this method of implementation.

The intent is to stimulate discussion and information exchange. To this end, participants will be encouraged to share their knowledge, experience, and questions about guidelines throughout the workshop.

Presenters: Dr. Martin Reed, Head, Department of Diagnostic Imaging, Winnipeg Children's Hospital

Effective search strategies assist researchers in finding high-quality information quickly. This two-part workshop will cover key concepts that will improve your database searching skills and also provide a framework for understanding the role of information specialists in health technology assessment (HTA).

This workshop is geared for HTA producers who want to gain valuable search tips and a better understanding of the search requirements in HTA work. The ten tips presented in this workshop will help make searching for information more effective and efficient. The workshop will provide tips, tricks, and techniques for both PubMed and Ovid MEDLINE searching. Participants will also gain a better understanding of the complex systematic review search strategies created by their information specialist colleagues.

Presenter: David Kaunelis, Information Services Methods Specialist, Canadian Agency for Drugs and Technologies in Health

Pharmaceutical development and commercialization primarily consists of phase I, II, and III randomized controlled trials (RCTs). However, such use of RCTs to guide patient care is limiting, as trials do not allow for exploration of off-label use and examination of treatment effects in a realistic clinical setting (i.e., pragmatic versus explanatory use); most often, these trials evaluate efficacy, not effectiveness. One way pharmaceutical companies can overcome these limitations is by utilizing N-of-1 trials as part of the research and development or R&D process. As they are relatively inexpensive to conduct, N-of-1trials may also offer a cost-effective way for pharmaceutical companies to carry out long-term, post-market surveillance (phase IV evaluation) of drugs. An N-of-1 trial typically refers to a prospectively planned randomized, multiple cross-over evaluation performed in a single subject (i.e., ABAB design).

The purpose of this hands-on workshop is to guide the researcher/clinician in the methods of conduct and reporting of N-of-1 trials using newly developed reporting guidelines for N-of-1 trials — the CONSORT (Consolidated Standards of Reporting Trials) Extension for N-of-1Trials or the CENT Statement. A consensus-based statement, CENT is intended to be a minimum set of items that should be reported in ABAB N-of-1 trials. Attendees will learn how to design and report N-of-1 trials during this interactive workshop.

Presenters:  Dr. Sunita Vohra, Pediatrician/Professor and Ms. Salima Punja, Grad Student, University of Alberta

Decision-makers need reliable affordability information to use in their deliberations on whether to fund an intervention or not. The budget impact analysis framework analyzes the full financial impact of an intervention, including cost add-ons and offsets in the wider health system. This workshop will address the following:

• concepts and techniques applicable to budget impact analyses for both drugs and other types of health technologies
• basic forecasting techniques
• sources of data and other information
• considerations for health programs and services.

Best practices, as outlined in the Patented Medicine Prices Review Board (PMBRB) budget impact guidelines and a helpful list of Dos and Don’ts will also be discussed. To better understand the concepts and principles discussed in this workshop, practical examples relevant to local environments within the Canadian health system will be discussed.

Presenters: Ms. Karen Lee, Director of Health Economics and Mr. Chris Cameron, Health Economist, Canadian Agency for Drugs and Technologies in Health; Mr. Bruce Brady, Health Economist, British Columbia Ministry of Health.

Model-based economic evaluation has been an important tool in supporting evidence-based policy-making. Data inputs to an economic model typically come from various sources. Uncertainty is a key issue that needs to be adequately and appropriately addressed in all model-based economic evaluations. Although probabilistic sensitivity analysis can demonstrate the level of uncertainty, value of information (VOI) analysis is the method that can quantify the uncertainty and then guide future research. However, the application of VOI has been limited since its development, which might be partly due to high computational intensity if nested Monte Carlo simulations are required. This half-day workshop will provide an overview of the basic concept of VOI and will demonstrate how to conduct VOI analysis using nested Monte Carlo simulations. A new method without using nested Monte Carlo simulations will be introduced that may help improve computational efficiency in VOI analysis. 

Educational Objectives:

• Understanding the concept of VOI analysis
• Calculating expected value of perfect information (EVPI) using nested Monte Carlo simulation
• Learning a simpler method for EVPI calculation.

Prerequisite: People who are familiar with economic evaluation and probabilistic sensitivity analysis, and who have some knowledge of basic statistics.

Presenters: Dr. Feng Xie, Assistant Professor, McMaster University; Dr. Nick Bansback, Health Economist and Dr. Mohsen Sadatsafavi, PhD Candidate, University of British Columbia

The goal of health technology assessment is to inform quality decisions that link evidence with health care delivery, leading to a sustainable health care system. In health care, a common scenario includes the introduction of new therapies that may offer an additional health benefit to a patient, but may also pose an added risk. In practice, health systems decisions are often made qualitatively and subjectively due to short timelines for such decisions. Quality decision- making requires comprehensive evaluation of the following comparative evidence:

• expected benefit (i.e., risk reduction of adverse outcomes)
• expected risk (i.e., increased risk of adverse events)
• estimates of uncertainty of expected benefit and expected risk
• estimates for decision-makers’ preferences for benefit/risk trade-off between all relevant comparators.

Making decisions that regard the benefit/risk balance requires preference trade-offs of these potential harms and benefits of one treatment relative to another treatment. Recent recommendations to the United States Food and Drug Administration and the European Medicines Agency include the use of more quantitative, transparent, and explicit benefit/risk analysis method to facilitate regulatory decisions. To enable decision-makers to find the right balance between benefit and risk, this workshop will introduce the use of three different methods of benefit/risk analysis: incremental net benefit, stated choice methods, and multi-criteria decision analysis. Although all methods will be discussed using specific case studies, there will be a focus on the incremental net benefit framework.

Educational Objectives:This workshop will introduce the concept of quantitative benefit/risk analysis, incorporating decision-makers’ risk preferences into clinical and regulatory decision-making using specific drug and non-drug case studies.

Presenters: Dr. Larry Lynd, Associate Professor and Dr. Carlo Marra, Associate Professor, Faculty of Pharmaceutical Sciences, University of British Columbia

Decision-makers who attempt to use technology assessment for health care decisions are often faced with a difficult problem: The best available scientific evidence is often insufficient or inappropriate for making local decisions. Evidence from one specific health care context may not readily apply in another context because of differences in population dynamics and epidemiology, organization of care, availability of resources, local values, and priorities. It has been recognized that evidence must be sufficiently “contextualized” to aid local decisions.

Contextualization in health technology assessment has been defined as a “scientific method that brings forward the different perspectives of key informants on components of a context to construct a multidimensional representation of the issues involved.”[From a presentation by Banken et al., HTAi 2007, Adapted from Mucchielli, A. Dictionnaire des méthodes qualitatives en sciences humaines et sociales. Armand Colin. Paris, 2005.]

The purpose of this workshop is to discuss the various approaches to contextualization that have been applied in Canada and abroad, and the issues surrounding them. Workshop participants will consider how contextualization has been defined, the methods that have been used to achieve it, and the gaps and potential for future improvement. The workshop will end with a discussion of the implications of contextualization for future collaborative opportunities in Canada.
Presenters: Members of the Health Technology Analysis Exchange

 An Adaptive Design Trial is a multistage study design that uses accumulating data to decide how to modify aspects of the study without undermining the validity and integrity of the trial. It requires that the trial be conducted in multiple stages with access to the accumulated data. This workshop will cover adaptive designs for: adaptive allocation rules (dynamically alters the allocation probabilities to reflect the accruing data in the trial); sampling (at each stage determining how many patients will be sampled at the next stage); stopping rules (for reasons of efficacy, harm, futility); and decisions pertaining to design changes (e.g., changing test statistics, redesigning multiple end points, changing the patient population). Adaptive design prospectively specifies and tests the hypotheses that include investigation of the intervention effect in the originally investigated unselected participant population and the pre-specified participant subsets.

Presenters:  Dr. George Wells, Director, Cardiovascular Research Methods Centre, University of Ottawa Heart Institute; Dr. Vijay Shukla, Senior Advisor, Canadian Agency for Drugs and Technologies in Health