Updates to the Therapeutic Review Framework – Issue 2


Stakeholder Feedback on the New Therapeutic Review Framework and Process

Background

On June 25, 2015, CADTH posted a revised Therapeutic Review Framework and Process document for stakeholder feedback. The Therapeutic Review Framework was first developed in 2012, and since this time, various changes have been made to the process.

Stakeholders were given 20 business days to provide comments on the following aspects:

  • Is the information contained within the document complete?
  • Are there any areas that you feel have not been addressed, or is any relevant information missing?
  • Are there any inaccuracies in the document?
  • Do you foresee the document being useful for your work and your organization? If so, how?

CADTH received feedback from 10 groups (representing six patient groups and four from industry).

The feedback has been grouped into the following categories (see Table 1) to reflect themes that were identified through the comments:

  • Inclusion of evidence-based, expanded use of drugs into therapeutic reviews
  • Increased stakeholder engagement in therapeutic review process
  • Process for the identification of therapeutic review topics and the evaluation of evidence
  • Timelines
  • Transparency
  • Components of the therapeutic review team
  • Clarity of process.

CADTH Response

CADTH has made modifications to the Therapeutic Review Framework and Process in order to address concerns expressed by current stakeholder feedback. Modifications include the following:

  • CADTH will typically request stakeholder feedback for therapeutic review procedural changes.
  • A simplified online stakeholder feedback form has been implemented for future projects.
  • CADTH has revised the formal therapeutic review patient group input process to allow for more response time for patient groups (with the exception of feedback on the project scope).

In consideration of stakeholder feedback, additional context has been added to the Therapeutic Review Framework and Process to ensure clarity with regard to the following:

  • When and how CADTH will handle the inclusion of evidence-based expanded use drugs (off-label) within Therapeutic Review reports
  • Incorporation of stakeholder feedback (especially patient input) into the therapeutic review process The point at which observational data are considered for review within therapeutic review projects
  • A definition for “patient group” has been added to Appendix 1.

As part of CADTH’s commitment to address stakeholder concerns, serious and open-minded consideration will also be given to many of the topics raised by current stakeholder feedback. CADTH will explore the following:

  • Ways to build upon the existing work of other health technology assessment agencies
  • Making the topic selection process for therapeutic reviews more transparent in the future
  • How to better prepare all stakeholders for feedback periods
  • The use of qualitative evidence as a means of integrating the patient perspective into CADTH reports
  • Posting key milestone time estimates online for ongoing therapeutic review projects
  • Integrating more plain language into the Therapeutic Review Framework and Process, going forward.

Summary

A detailed summary of stakeholder feedback (organized by theme) with the associated CADTH response is presented in Table 1.

Table 1: Stakeholder Comments (Summarized) and CADTH Responses Grouped by Theme

Source

Summary of Comments

CADTH Response and/or Action Taken

Inclusion of Evidence-Based Expanded Use of Drugs Into Therapeutic Reviews

Industry and Patient Groups

CADTH should not consider the inclusion of an off-label therapy in TRs when there are licensed alternatives available, due to the following concerns:

  • Safety (especially if a governing Health Authority has already made reference to safety concerns regarding the usage of a particular product in a non-approved indication)
  • May prevent access to approved medications
  • May compromise the Canadian drug approval process
  • No international precedent (e.g., NICE).

Additional context has been added to the Therapeutic Review Framework and Process to clarify when and how CADTH will handle the inclusion of drugs with evidence-based expanded use within its reports and recommendations.

Safety concerns:

A key element of all CADTH TRs is the assessment of the evidence regarding the safety of every drug reviewed. The review of off-label drugs is considered only if safety data exist for the indication in question (preferably randomized controlled trial data, but other high-quality studies may be considered).

Access concerns:

TR recommendations do not prevent access to drugs in Canada. Their aim is to provide decision-makers with the best available evidence to make formulary listing decisions and to inform optimization of drug therapy.

Canadian drug approval process concerns:

A key consideration for including drugs with evidence-based expanded use in a CADTH TR is evidence of the drug’s current use for the condition of interest in Canadian clinical practice (e.g., integration of drug in clinical practice guidelines, or consultations with clinical specialists).

International precedent:

NICE in the UK does include drugs that have not received a UK marketing authorization in its guidance reviews when there is good evidence to support this (see peripheral neuropathic pain). Where recommendations have been made by NICE for the use of drugs outside their licensed indications, these drugs are marked with a footnote in the recommendations. Regulatory agencies in France and Italy have similar processes for reimbursing expanded-use products.

Industry Groups

CADTH should halt all current TRs, including off-label drugs, until CADTH performs a thorough evaluation and/or assessment of the impact of reviewing off-label therapies for TRs. This includes reviewing the feedback received on the inclusion of off-label drugs within the revised TR process.

The inclusion of evidence-based expanded use drugs was never stated as an exclusion criterion for TR projects within the original Framework.

In consideration of the current feedback, CADTH has modified its Therapeutic Review Framework and Process to allow for stakeholder feedback on procedural changes. CADTH will continue to consult participating drug plans for procedural changes as per current practice.

Industry and Patient Groups

Whenever off-label drugs are included in a TR, the rationale for inclusion must be clearly documented. In addition, the “exceptional circumstances” in which off-label medications are included should be further defined within the TR process.

The Proposed Project Scope document for the Anti–Vascular Endothelial Growth Factor Therapeutic Review did not make it clear that Avastin is not formulated nor authorized for intravitreal use according to the Canadian product monograph and that serious warnings and precautions exist.

Additional context has been added to the Therapeutic Review Framework and Process to clarify when and how CADTH will handle the inclusion of drugs with evidence-based expanded use within its reports and recommendations.

Patient Group

Any recommendation that includes the use of off-label drugs needs an implementation program to ensure informed patient consent.

Although CADTH does not implement recommendations, when requested, CADTH does provide knowledge mobilization material to the public drug plans to support jurisdictional decisions based upon our TR recommendations.

Patient Groups

Any inclusion of off-label drugs must be driven by the best interests of the patient population (efficacy and safety), not necessarily cost.

Although cost considerations are part of a TR report, cost is not a determining factor for including drugs with evidence-based expanded use for review. A key consideration for including drugs with expanded use is evidence of the drug’s current use for the condition of interest in Canadian clinical practice (e.g., integration of drug in clinical practice guidelines, or consultations with clinical specialists).

Other key considerations for including drugs with evidence-based expanded use within a TR are as follows:

  • Data for the expanded use of a drug exist for the indication in question (preferably randomized controlled trial data, but other high-quality studies may be considered)
  • Evidence that other international health technology agencies or payers have made decisions and/or recommendations to fund a drug despite lack of regulatory approval.

Increased Stakeholder Engagement

Industry and Patient Groups

All stakeholders, including patients and caregivers, should be given the opportunity to provide input during the initial scoping stage of a TR.

The consultation of clinicians in the scoping stages of the review should be expanded from a couple of experts to seeking broad clinician input, as per the NICE and ODPRN models. CADTH should have a process for getting adequate participation from clinician groups. This may include encouraging provinces to notify relevant clinical networks or adapting the ODPRN process.

CADTH has revised the sections on stakeholder feedback within the Therapeutic Review Framework and Process to clarify that stakeholder feedback is not limited to a particular group.

Stakeholder feedback is not limited to a particular group and it occurs throughout the CADTH TR process at the following stages:

  • Stakeholder feedback on Proposed Project Scope
  • Stakeholder feedback on Included Studies List
  • Stakeholder feedback on draft Therapeutic Review Science Report
  • Stakeholder feedback on draft CDEC Recommendations or Advice document

Although anyone (including patients and caregivers) is welcome to give feedback at the above stages, CADTH also seeks focused input from specific patient groups representing the indication being reviewed. This includes an introductory call and a request for specific patient group input, which is included in the draft reports and shared among the members of CDEC. Patient group input is limited to patient groups; however, individual patients, caregivers, and anyone else are welcome to comment on the scope of the review during the general calls for stakeholder feedback phases documented above.

CADTH also seeks engagement during the TR process from the following:

  • Customers (particularly during the topic selection and scoping phase)
  • CADTH Liaison Officers across the country (for linkages to relevant clinical networks and clinical experts)
  • Multiple clinical experts during the TR to ensure the project is relevant to the Canadian context.

Industry and Patient Groups

CADTH should allow for stakeholder feedback on the development of the protocol and methods for a TR.

CADTH has revised sections on stakeholder feedback within the Therapeutic Review Framework and Process to ensure clarity.

CADTH solicits guidance from clinical experts and methodologists when designing the Project Protocol. CADTH also takes into consideration stakeholder feedback on the Proposed Project Scope, which includes the proposed deliverables and aspects of the project design.

Patient Group

There should be a final stakeholder review of the “social acceptability” of the recommendation before finalization.

CADTH consults with clinical experts when draft recommendations are developed for implementation barriers. Patient group input and the CDEC public member may also provide insight on implementation issues.

Patient Group

The process for patient group input is supported and appreciated; however, patient groups may require CADTH expertise or financial help in order to effectively participate in TRs.

CADTH has two Patient Engagement Officers dedicated to supporting the participation of patient groups in the development of CADTH Recommendations.

Industry

The process for notifying manufacturers should be expanded to include manufacturers with a product in late-stage development in the related therapeutic area.

CADTH sends out e-alerts when stakeholder feedback is requested. All organizations are welcome to provide feedback, in particular when the Proposed Project Scope is first posted.

Patient Group

Could CADTH develop a simplified online feedback questionnaire for future input or feedback?

CADTH has implemented a simplified online stakeholder feedback form for projects going forward.

Identification and Treatment of Evidence

Industry and Patient Group

The evidence on which all recommendations are based should be adequately defined, graded, and/or weighted, and presented in a manner that explains the rationale to all stakeholders (including how the various stakeholder contributions and types of evidence have been addressed).

Recommendations made with limited evidence should be given with recognition of limited and/or uncertain data: in these cases, recommendations should be conditional and have a timeframe for re-assessment based upon new evidence.

CADTH incorporates the evidence used to guide the recommendations within the Therapeutic Review Recommendations Report. If CDEC considers the evidence to be insufficient, then a clear recommendation is not issued and CDEC typically provides comments while documenting existing research gaps. Occasionally, if a review identifies ongoing trials that have the potential to affect a recommendation, a future update is suggested.

In all Therapeutic Review Recommendations Reports issued by CDEC, there is a section called “Limitation of the Evidence” that evaluates the evidence available. Whenever there is insufficient evidence, this is made clear and documented in the “Research Gaps” section of the Recommendations Report.

Industry and Patient Groups

Real-world evidence is absent in CADTH’s TR process, compared with that of others. Patient-reported and/or relevant outcomes and real-world data should be considered as primary evidence.

Request greater clarity regarding the use of patient feedback and/or perspectives as a distinct form of evidence with clear guidelines as to how that information is used in the review process and in the development of recommendations and conclusions.

CADTH will continue to participate in and maintain current knowledge of developments and best practices in the methods for HTA. To that end, additional context has been added to the Therapeutic Review Framework and Process to clarify when CADTH considers the inclusion of observational data within TRs. CADTH will also explore the use of qualitative evidence as a means of integrating the patient perspective into CADTH reports.

Industry

Allow unpublished data (evidence) from key stakeholders to be included in the TR. Confidentiality can be handled by redaction (may be “time-limited”) as per the CDR model.

Stakeholders are given the option of identifying and providing unpublished data for consideration in the TR on the condition that, if used, it will be included in publicly available reports and documents related to the TR.

 

CADTH TRs do not allow for redactions at this time, as all information incorporated is required to be publicly available. This ensures an open and transparent review process and accountability for recommendations to patients and the public.

Patient Group

Evidence of inappropriate utilization should not necessarily result in limiting access to drugs; rather, the focus should be on education, coaching, monitoring, and support.

CADTH recommendations are non-binding and it is up to jurisdictions to implement them (or not) as they see fit. Access to drugs is also controlled by jurisdictional drug plans, not CADTH. When requested by customers, however, CADTH may develop educational tools to support the recommendations for optimal use that jurisdictions have chosen to implement.

Industry

More details on how economic evaluations are conducted is needed, including possible collaboration with manufacturers to reuse existing economic models.

CADTH is exploring ways to build upon the existing work of other HTA agencies.

Industry and Patient Groups

Cost-containment should not be the only lens through which a TR is launched or conducted (e.g., opportunity to highlight value in classes or categories not currently funded).

Emphasis on the economic evidence within the TR process is not appropriate for rare diseases, and emphasis should always be on patient health and quality of life.

The ultimate goal of TR projects is to assist decision-makers in benefiting the health of Canadians through improved drug-related health outcomes and quality of life, and by maximizing value of the health care budget.

Generally, a TR is undertaken to address, but is not limited to, the following:

  • Issues regarding effectiveness, either of the class as a whole or of the relative effectiveness of drugs within the class
  • Issues regarding safety, either of the class as a whole or of the relative safety of drugs within the class
  • Issues that affect resource use concerns regarding inappropriate utilization of drugs within a class.

CADTH recognizes the unique challenges of reviewing drugs for rare diseases and will continue to explore ways to address these issues.

Timelines

Industry and Patient Groups

The 10 business-day timeline given for stakeholder feedback is tight, especially for patient and clinician community participation.

CADTH has revised the process so that patient group input occurs at a later stage, allowing for more preparation time. However, feedback from patients, caregivers, patient groups, and other stakeholders, specifically on the scope of a TR, will continue to be requested earlier during the regular call for stakeholder feedback on the Project Scope.

Continued consideration will be given on how to better prepare all stakeholders for feedback periods.

Industry

Recommend that the available information on timelines be enhanced in future iterations of the Framework

CADTH will consider providing key milestone time estimates for ongoing TR projects going forward.

Industry

CADTH should make environmental scans of emerging drug technologies a priority so that TRs required for CDR reviews or pan-Canadian Pharmaceutical Alliance negotiation are identified and completed as soon as possible, to avoid reimbursement delays.

Therapeutic Reviews do not affect timelines for CDR reviews in the current Framework and Process.

Transparency

Industry

CADTH should post the short list of potential topics online for transparency and to allow stakeholders time to prepare for feedback.

Topics on the short list are not refined nor guaranteed for review, and posting them may create false expectations. Despite this issue, CADTH will give consideration to the possibility of posting short-list topics in the future.

Industry and Patient Groups

CADTH should post compete (not summarized) responses to stakeholder feedback online (includes “Discussant Reports”). This will assure stakeholders that their input has been considered and would increase overall trust in the TR process.

Complete patient group input is currently posted and shared on the TR project website. In addition, both unedited and summarized patient group input and stakeholder feedback is provided to CDEC for review before the committee makes recommendations or advice statements.

Confidentiality and disclosure issues will need to be reviewed before any decision is made on posting all stakeholder feedback online.

Industry and Patient Groups

The manner in which topics are selected could be much more transparent and predictable. Patient groups, patients, health care providers, researchers, and industry should be able to identify TR topics for consideration.

The document describes a Framework whose scope references other factors such as “social, legal, ethical, and environment.” As written, the policy, process, and procedures for these aspects are not described in any detail.

Topics selected are based on CADTH’s customers’ needs and requests. Social, legal, ethical, environmental, political, entrepreneurial, and research (innovation) issues may be factors considered in the identification of TR topics.

CADTH will give serious and open-minded consideration to making the topic selection process for TR more transparent in the future.

Industry and Patient Groups

The development of knowledge mobilization tools should reflect the same rigour and consultation process as the review itself (e.g., stakeholder including patient consultation and expert committee validation).

For TRs, specialist experts and the designated advisory committee (CDEC) members for a particular topic always review any key messages, or KM tools.

In order to obtain stakeholder feedback on knowledge mobilization tools, the preferred mechanism is focus group tests. This means a group representing the intended end-user (e.g., clinician, pharmacist, patient) would review the tool before it is finalized.

Changes to Review Team

Industry and Patient Groups

A patient should be added to the review team for TRs wherever possible.

CADTH solicits patient group input for TRs and summarizes it within the review; it is used to inform the Protocol and Recommendations. As one of their roles, the public CDEC member has the responsibility of ensuring patient input is considered.

CADTH has no plans to add a patient to the review team at this time.

Patient Group

The Jurisdictional Working Group should include patient representatives to help identify and define topics from a patient perspective.

Changes to Jurisdictional Working Group membership comprise a decision for the public drug plans. At this point, the membership does not include patients, clinicians, or other stakeholders.

Patient Group

Qualitative researcher(s) should be added to the review team for TRs wherever possible.

CADTH will continue to participate in and maintain current knowledge of best practices in the methods for HTA; however, there are no plans to include qualitative researchers on TR review teams at the moment.

Clarity

Patient Group

There is no definition of patient as customers within the TR process.

CADTH has added a definition for “patient group” to Appendix 1 of the Therapeutic Review Framework and Process. There is no attempt to define “patient” within the Framework, but CADTH encourages stakeholders, including patients, relatives, and caregivers, to provide feedback into TR projects during calls for stakeholder input (see Appendix 2 of the Framework).

Patient Group

Replace technical language in the TR process with plain-language alternatives.

CADTH will explore using plain language in the TR process going forward.

Industry

For clarity, it should be clearly articulated that CDEC receives the full version of all stakeholder responses as part of the Discussant Report.

CADTH ensures that CDEC members receive all stakeholder feedback (both complete and summarized) before they make any recommendation or advice statements. CADTH has revised sections on stakeholder feedback within the Therapeutic Review Framework and Process to ensure clarity.

CDEC = CADTH Canadian Drug Expert Committee; CDR = CADTH Common Drug Review; HTA = health technology assessment; NICE = National Institute for Health and Care Excellence; ODPRN = Ontario Drug Policy Research Network; TR = therapeutic review.