The Use of Real-World Evidence for Medical Device Assessment ─ An Environmental Scan


(March 19, 2020)
Print PDF Opens in a new window

Table of Contents

Abbreviations

  • EHR electronic health record
  • EMR electronic medical record
  • HC Health Canada
  • HF heart failure
  • HTA health technology assessment
  • PCC Pan-Canadian Health Technology Assessment Collaborative
  • RCT randomized controlled trial
  • RWD real-world data
  • RWE real-world evidence

Summary

  • This Environmental Scan was informed by a survey completed by stakeholders from Canadian regulatory groups and health technology assessment (HTA) organizations. The results from the survey indicated that real-world evidence (RWE) has potential value for medical device assessments across a product’s life cycle (i.e., pre and post-market).
  • Although considered to be of lower quality and not consistently submitted by manufacturers, RWE on effectiveness and safety could be used to supplement available randomized controlled trial (RCT) data.
  • Without a consensus definition of RWE and an established framework to collect real-world data (RWD), the acceptability of data sources varies among different Canadian regulatory groups and HTA producers.
  • Respondents suggested that there should be a consistent and systematic method of capturing RWD to help mitigate potential issues such as under-reporting, data privacy, and multiple data sources.
  • If discrepancies exist between RWE and RCT evidence, regulatory groups and HTA producers need to conduct internal and external discussions with key stakeholders and clinical experts to identify the potential sources for the discrepancy.
  • RWE can offer significant added value in situations such as rare conditions or populations not well-studied in RCTs, significant unmet clinical need, identification of safety signals, or innovative/breakthrough technology.

Context

RWD are data pertaining to patient status, health outcomes, or the delivery of health care (Appendix 1), as per the draft guidance document titled “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices” created by the US FDA.1 Reflective of what is observed in routine clinical practice, RWD can be collected from various sources such as electronic health records, patient registries, hospital discharge data, and claims databases.2 RWD collected from these sources can be used for analysis in various study designs such as observational studies (i.e., retrospective or prospective) and randomized trials (e.g., pragmatic clinical trials, large simple trials).1

Considered as the gold standard, RCTs are conducted under controlled conditions and usually consist of homogeneous population samples in order to minimize confounding and selection biases.3 However, RCTs often lack external validity due to the heterogeneity of actual patient populations with various comorbidities and concurrent medications.3 As RCTs are restricted to more narrow time frames, long-term safety data are lacking.3 In light of these uncertainties, regulators and HTA producers in the UK and US are supplementing their decision-making with RWE.4 Through the analysis of RWD, RWE pertaining to the usage, risks, benefits, and resource implications of medical products in practice can be generated.2 With the increasing adoption of electronic medical records and growth of accessible administrative data, there is increasing interest and potential in using RWE to inform HTA and regulatory decision-making.5

Consisting of various stakeholder groups such as HTA producers, regulators, and pharmaceutical companies, the European GetReal Initiative attempts to evaluate methodologies and policies surrounding the use of RWE in drug development.3 RWE is currently applied to aspects of drug development to assess the natural history of diseases, comparative efficacy versus other interventions, and health care utilization and cost-effectiveness6 The FDA has published a framework for evaluating the use of RWE in new drug indication and post-approval requirement processes.7 Currently, regulators in the UK and US are using RWE on medical devices to inform conditional approvals and Coverage with Evidence Development.4 At the time of this Environmental Scan, an FDA framework for medical devices was not available. However, the draft guidance released by the FDA suggests that RWE will be used to supplement their decision-making.1

Health Canada (HC) regulatory groups and Canadian HTA producers have various roles to play in medical device assessments. In the context of pre-market regulatory decision-making, HC evaluates the clinical effectiveness and safety of a medical device before it is authorized for sale.8 In the post-market surveillance space, HC conducts safety and benefit-risk assessments after a medical device has obtained authorization.8 HTA producers provide evidence-based advice through the development of comprehensive reports regarding, but not limited to, the clinical effectiveness and cost-effectiveness of medical devices.9 By working together, HC and the Pan-Canadian HTA Collaborative (PCC) developed  a strategy10 to optimize the use of RWE with a view to improve the accessibility, affordability, and appropriate use of medical devices across the life cycle. Such guidance aims to provide clarity on how to evaluate the strengths and weaknesses of using RWE in decision-making.

In 2018, CADTH conducted an Environmental Scan evaluating the use of RWE in single-drug assessments.11 The results of this previous survey indicated that even though RWE is eligible for consideration in drug assessments, its role in decision-making is dependent on the availability and limitations of RCT evidence, the clinical context, and agency-specific policies.11 This Environmental Scan investigates how RWE is currently being used to assess safety, clinical effectiveness, and cost-effectiveness of medical devices in Canadian regulatory and HTA processes. A description of the criteria being used for inclusion or exclusion of RWE, how organizations use RWE, and the perceived impact of RWE on HTA is provided. To better inform decision-making in an age of uncertainty, stakeholders could benefit from a common framework to collect high-quality RWD and generate reliable RWE.

Objectives

This Environmental Scan aims to determine how RWE is being used in Canada, by both regulatory groups and HTA producers, for medical device technology assessments. The key objectives were as follows:

  • Describe the eligibility criteria for inclusion of RWE to establish device effectiveness and safety for assessments done by HC and HTA organizations.
  • Describe how organizations use RWE of effectiveness and safety in assessments conducted to support regulatory decisions or HTA recommendations.
  • Describe the perceived impact of RWE on assessments.

Methods

This Environmental Scan is based on findings from a survey (Appendix 2), distributed to relevant stakeholders from Canadian regulatory groups and HTA organizations. Follow-up teleconference and email consultations with select survey respondents were also conducted.

Survey

Survey respondents were identified by Health Canada and the PCC, to represent the regulatory groups (within the HC Medical Devices Directorate) and HTA producers, respectively (Appendix 3). From Health Canada, members from pre-market, post-market, and compliance were included. All member organizations of the PCC were included. Representatives who were able to answer questions regarding the use of RWE in technology assessments and decision-making were contacted. Fourteen respondents received an email primer with information about the survey, an invitation to participate, and dates of participation. Within this email, an electronic survey link along with consent to participate was provided. The survey link was disseminated in November of 2019.

The survey included 18 questions and contained various question formats, which required dichotomous (e.g., yes or no), nominal (e.g., list of options), and open-ended free-text responses. The survey was pilot tested by three clinical researchers at CADTH and distributed to the identified stakeholders using the Survey Monkey platform (www.surveymonkey.com).

Consultations

To supplement or clarify the survey data, follow-up teleconference and/or email consultations were conducted with select survey respondents who had noted in their survey response that they were willing to participate. The consultations were led by one author (YL) and were conducted in December 2019 and January 2020. And an initial consultation email containing the question(s) requiring clarification was sent out in December 2019. In cases where teleconference calls were not possible, email consultations with respondents were accepted.

Synthesis Approach

Only feedback from respondents who gave consent to use their survey and consultation information were included in this report. A response was deemed partially complete if one or more questions were left blank by the respondent. Partially complete responses were included in data analysis. Survey responses were excluded if all answers were blank, with the exception of demographic information which did not affect response inclusion. Respondents were sent the draft summary report to verify the information.

Responses were analyzed and organized by the objectives of this Environmental Scan, then by organization. In the case of multiple responses from one organization, all responses were included. Quantitative and multiple choice answers were summarized through tables by organization and presented narratively. Qualitative (or open-ended) answers were presented narratively.

Findings

Survey

All 14 identified stakeholders (nine HC and five PCC), who received the survey invitation, initially gave explicit written consent to use their provided information for the purpose of this report. Eligible respondents included one respondent each from the following Canadian HTA agencies: Institute of Health Economics (IHE) (Alberta), British Columbia Health Technology Assessment Office (BC-HTAO) (British Columbia), Ontario Health (Quality) (Ontario), Institut national d'excellence en santé et en services sociaux (INESSS) (Quebec), CADTH (national), and five respondents from HC. Further details of the participating organizations are presented in Appendix 3. Out of 10 eligible survey responses, eight were complete and two were partially complete.

Consultations

Invitations to participate in the consultation process were sent to eight organizations. Teleconference consultations were conducted with two organizations. Email consultations were conducted with four organizations. Responses from two stakeholders were excluded based on scope discussion.

The findings presented in this report are based on survey responses and are presented by each objective.

Objective 1: Eligibility Criteria for Inclusion of RWE to Assess Device Effectiveness and Safety

Ten survey respondents (5 HC and 5 PCC) provided information on the eligibility criteria for inclusion of RWE in the assessment of device effectiveness and safety. Organized by responses from regulatory groups and HTA producers, a summary of dichotomous and nominal survey responses is presented in Appendix 4.

Health Canada

All five HC survey respondents stated that in their organization, RWE can be included in the assessment of medical devices to answer questions of clinical effectiveness and/or safety. Two respondents highlighted that RWE is an important component in the assessment of effectiveness and safety of a medical device’s life cycle, including pre-market and post-market stages.

With respects to RWE study designs, all five respondents deemed pragmatic trials, case-control, prospective cohort, retrospective cohort, and uncontrolled single-arm studies to be eligible for inclusion in the assessment of effectiveness. Cross-sectional studies were selected by four respondents to be eligible for effectiveness assessment. With respects to the assessment of harms/safety, four respondents selected case-control, prospective cohort, retrospective cohort, and uncontrolled single-arm studies to be eligible. Three respondents deemed pragmatic trials, and two respondents deemed cross-sectional studies to be eligible for harms/safety assessment. One respondent involved in pre-market device evaluation highlighted that while all clinical data are accepted for effectiveness and safety assessments, different RWE is weighted differently, and the evidence is assessed in totality. This notion was echoed by one respondent involved in post-market assessment who stated that any RWE can be considered as part of the body of evidence.

When asked about the acceptability of various data sources, all five respondents indicated that electronic health records (EHRs)/ electronic medical records (EMRs), hospital databases, and data registries can be utilized for the generation of eligible RWE. Four respondents stated that home care database, patient safety and learning system, physician database, all-payer claims database, private health insurance plan claims database, supply chain database, patient-generated data, and electronic/paper-based patient files managed by clinician(s) or health care facility are eligible. One respondent explained that, determined on a case-by-case basis, there are circumstances that allow exceptions to the acceptability of a data source. However, this respondent further stated that regulatory groups need to evaluate the integrity and method of capture of the data. Three respondents indicated that any RWD can be considered as part of the body of evidence to supplement aspects of effectiveness and/or safety assessment. Nonetheless, pre and post-market regulatory groups need to evaluate the quality of RWD to determine its appropriateness and validity.

To complement device assessments, three respondents indicated that their organization requests RWE from manufacturers. Out of these three regulatory groups, one has specific requirements regarding study design and data sources. Specifically, manufacturers are required to provide incident reports and device registry information for assessment. Consequences of non-conformity to these requirements include compliance and enforcement actions such as recalls. One respondent stated that for the purposes of issue analysis in post-market surveillance, RWE submissions from manufacturers are voluntary. Where eligible RWE is accepted, four respondents stated that it does not need to be captured from individuals treated in their jurisdiction.

Three respondents indicated that their organization has plans to change their current approach relative to RWE in the future. Through collaboration with HTA organizations, two respondents stated that their organizations have plans to better understand how to incorporate RWE in post-market surveillance of medical devices. Another respondent provided a link to HC’s webpage outlining plans to strengthen the use of RWE for device assessments. The remaining respondents were uncertain about their organization’s future plans.

Pan-Canadian HTA Collaborative

All five PCC survey respondents stated that in their organization, RWE can be included in the assessment of medical devices to answer questions of clinical effectiveness and/or safety. One respondent explained that their organization uses jurisdiction-specific data that that are readily accessible to supplement peer-reviewed literature in their assessment of medical devices. Another respondent emphasized that their organization regularly incorporates published RWE in their devices assessments but does not conduct any analysis of primary RWD.

With respects to RWE study designs, all five respondents selected pragmatic trials, case-control, prospective cohort, and retrospective cohort studies to be eligible for inclusion in the assessment of effectiveness. Cross-sectional and uncontrolled single-arm studies were selected by three respondents to be eligible for effectiveness assessment. With respect to the assessment of harms and safety, all five respondents selected prospective cohort studies and pragmatic trials to be eligible. Four respondents selected cross-sectional, case-control, and retrospective cohort studies to be eligible. Three respondents deemed uncontrolled single-arm studies to be eligible for harms/safety assessment. One respondent highlighted that for uncontrolled single-arm studies used in effectiveness and safety assessments, the emphasis would be on before-after studies, and not case series.

When asked about the acceptability of various data sources, four respondents indicated that EHR/EMR, hospital database, patient safety and learning system, and data registry can be utilized for the generation of eligible RWE. Three respondents stated that home care database is an eligible source. Two respondents deemed physician database, all-payer claims database, private health insurance plan claims database, supply chain database, and electronic/paper-based patient files managed by clinician(s) or health care facility to be eligible. One respondent stated that patient-generated data are eligible. One respondent explained that there are circumstances which allow exceptions to the acceptability of a data source. However, exceptions would have to be evaluated on a case-by-case basis to assess the appropriateness and validity of the data source. One respondent stated that exceptions may be made to include patient-generated data, but this has not been attempted in their organization. Upon further consultation, another respondent stated that their organization makes eligibility decisions based on study designs, and not data sources. The inclusion of various data sources would be dependent on different evidence needs.

this organization does not have specific requirements regarding study design and data sources. Where eligible RWE is accepted, four respondents stated that it does not need to be captured from individuals treated in their jurisdiction.

Four respondents indicated that they were uncertain if their organization has plans to change their current approach to RWE in the future. One respondent explained that, although no concrete plans have been established, their organization was open to incorporating other data sources into HTAs.

Objective 2: Use of RWE on Effectiveness and Safety in Device Assessments

Ten survey respondents (5 HC and 5 PCC) provided information and case examples on the use of RWE on effectiveness and safety in device assessments. Organized by responses from regulatory groups and HTA producers, a summary of nominal survey responses is presented in Appendix 5 .

Health Canada

For the assessment of medical devices, four respondents indicated that RWE can be used to supplement the RCT evidence on therapy effectiveness and safety. Three respondents stated that RWE can be used to establish the effectiveness and safety of the intervention in the absence or isolation of RCT evidence and validate surrogate outcomes. In post-market stages, two respondents explained that RWE is helpful in identifying increased risks and new issues pertaining to the safety of medical devices.

There are circumstances in which RWE brings added value and can be given more weight, relative to conventional situations where the evidence base consists of RCT data of sufficient quality and quantity. All five respondents indicated that these circumstances can include rare conditions or populations not well-studied in RCTs (few and/or small RCTs). Three respondents stated that RWE can add significant value for circumstances such as a significant unmet clinical need or an innovative/breakthrough technology. One respondent explained that RWE with superior external validity relative to the population of interest should also be given more weight.

All five respondents provided case examples on the use of RWE on effectiveness and safety in device assessments. In the assessment of breast implants, one respondent used RWE generated from patient safety and learning systems (i.e., incident reports) and data registries. The RWE helped inform the safety profile of breast implants. In reviewing diabetes management systems, another respondent indicated, during further consultation, that RWE from RCTs, pragmatic trials, cross-sectional, case-control, prospective cohort, retrospective cohort, and uncontrolled single-arm studies were eligible for inclusion in their assessments. Extracted from data registries and patient-generated data, RWE helped inform effectiveness and safety, as well as how patients interact with their diabetes management devices. In reviewing the CardioMEMS Heart Failure (HF) System used for New York Heart Association Class III HF patients, one respondent included a prospective, multi-centre, single-arm study in their assessment. Generated from EHRs/EMRs (i.e., incidence of HF-related hospital visits, patient hemodynamic data), RWE helped inform the assessment of effectiveness and safety of this device. This respondent stated that RWE can be used in a supportive role to expand indications for existing devices.

In reviewing the AMPLATZER Valvular Plug III used for paravalvular leakage associated with bioprosthetic valves, another respondent indicated that RWE from prospective cohort and uncontrolled single-arm studies were eligible for inclusion in their assessment. Extracted from data registries, physician databases, electronic or paper-based patient files managed by a clinician or health care facility, RWE helped inform the effectiveness and safety of this device. This respondent also stated that RWE can add value to a device review for the purpose of expanding its list of indications. In assessing metal-containing hip implants used for hip arthroplasty, one respondent used Canadian Institute for Health Information (CIHI) registry data in their assessment. Due to the limitations of RWE (e.g., inability to establish causal relationships), this respondent emphasized that this RWE should only be reported as part of the device assessment and should not affect regulatory decisions.

Pan-Canadian HTA Collaborative

For the assessment of medical devices, all five respondents indicated that RWE can be used to supplement the RCT evidence on therapy effectiveness and safety and inform cost-effectiveness and utilization. Three respondents stated that RWE can be used to establish the effectiveness and safety of the intervention in the absence or isolation of RCT evidence. One respondent indicated that RWE can be used to validate surrogate outcomes.

All five respondents indicated that RWE can add significant value for circumstances such as rare conditions or populations not well-studied in RCTs (few and/or small RCTs). Three respondents stated that more weight can be given to RWE if there is an innovative/breakthrough technology, a potentially large budget impact, or for RWE with superior external validity relative to the population of interest. One respondent explained that RWE also brings added value for significant unmet clinical needs.

All five respondents provided case examples on the use of RWE on effectiveness and safety in device assessments. In reviewing multiple devices used in minimally invasive glaucoma surgery for adults with acquired glaucoma, one respondent indicated that RWE from RCT’s, pragmatic trials, case-control, prospective cohort, and retrospective cohort studies were eligible for inclusion in their assessments. Since inclusion of eligible RWE was solely based on study design, this group did not examine the data sources used for each included study. Used to supplement limited RCT information, RWE helped inform effectiveness, safety, adherence to treatment, and utilization data (e.g., resource use, hospitalization data) in this device review. By providing data on resource utilization and actual costs, RWE also helped inform the economic model. In the assessment of hand and arm transplants for patients with limb amputation(s), another respondent used RWE generated from case series. Generated from data registries, RWE helped inform the effectiveness and safety in this assessment. In the assessment of deep brain stimulation for patients with advanced Parkinson’s disease, one respondent used RWE generated from retrospective cohort studies. Generated from hospital databases within a specific province, RWE helped inform utilization data in this assessment.

Two respondents provided partial responses to follow-up questions pertaining to their case example. In reviewing maternal serum screening for triploidy in pregnant women, the respondent did not indicate eligible study types or data sources used in their assessment. Nonetheless, helping to inform the effectiveness, adherence to treatment, and utilization data, RWE provided region-specific context and test accuracy information. In reviewing transcatheter aortic valve implantation, the respondent did not specify which aspects of their review were informed by RWE that was generated from EHR/EMRs and electronic or paper-based patient file managed by a clinician or health care facility. Nonetheless, the respondent indicated that prospective cohort studies were eligible for inclusion in their assessment.

Objective 3: Perceived Impact and Implications of RWE on Device Assessments

Nine of ten total survey respondents (5 HC and 4 PCC) provided information on the perceived impact and implications of using RWE in device assessments. Organized by responses from regulatory groups and HTA producers, a summary of open-ended free-text responses follows.

Health Canada

All five respondents indicated that the use of RWE for device assessments has added benefits in comparison to RCT evidence. They perceived RWE as an essential component in the assessment of a medical device, especially where RCT data are lacking due to low volume or dispersed target populations. In the post-market surveillance space where RCTs are scarce, RWE may help with signal detection. Since the high cost of conducting RCTs may hinder innovation and delay market entry of medical devices, RWE could supplement the body of evidence to help establish effectiveness and safety. RWE may be useful when comparing a new device’s performance to established ones where objective performance criteria exist. By providing information about how a device is used in real-life settings, RWE provides target-specific data on real-life product performance as it is associated with variables such as user training. In the realm of artificial intelligence, the actual effectiveness and safety of an algorithm can only be demonstrated once it has been used in real-life environments. Once deployed under the appropriate regulatory oversight, an algorithm’s performance can help inform ongoing decision-making throughout its life cycle.

However, the respondents acknowledged that there are challenges of using RWE for device assessments. All five respondents indicated that the lack of a consensus definition of RWE and consistent approach to RWD collection are major challenges in incorporating RWE in decisions-making. Data integrity can be affected by variables such as under-reporting, loss to follow-up, and method of data capture. For example, data collected by wearables and smartphones may not represent the Canadian population as a whole. Regulatory groups also need to address other shortcomings of RWE such as the potential for confounding bias, lack of randomization, and inability to establish causality. Some possible solutions were proposed by the survey respondents. By collaborating with international regulators, professional associations and manufacturers, device regulators can develop a consistent framework for implementing RWE in regulatory decisions. One respondent suggested that manufacturer-sponsored device registries may play a role in post-market regulatory assessment, which is an area that is underfunded. However, careful consideration of data ownership and privacy issues would be required. Decision-makers can scan the device landscape to leverage existing strategies of RWD collection and determine device types appropriate for RWE generation.

All five respondents described their decision-making process to reconcile conflicting results between RWE and RCT evidence. Although the RCT study design is more robust and has higher internal validity than RWE, respondents indicated that RWE could be assessed as part of the evidence as it may be more generalizable to the target population. When considering RWE that conflicts with RCT evidence, decision-makers should engage in internal discussions to evaluate strengths and weaknesses of various types of evidence. Upon further consultation, a respondent from the post-market surveillance group stated that regulatory actions can be taken if appropriate safety signals are revealed by RWD (e.g., patient safety data).

Pan-Canadian HTA Collaborative

Four respondents indicated that the use of RWE for device assessments has added benefits in comparison to RCT evidence. One respondent did not provide a response. Three respondents indicated that, in comparison to idealized RCT settings, RWE can provide contextual information on how medical devices perform in real-life scenarios. In non-drug assessments where RCT data are often lacking, especially for rare conditions, RWE can provide additional valuable information pertaining to a device’s effectiveness and safety. Specifically, RWE is potentially a good source of jurisdiction-specific information and safety data.

However, these respondents acknowledged common challenges of using RWE for device assessments. Similar to HC respondents, HTA respondents highlighted potential issues with data quality and weaker study designs associated with RWE. For example, one respondent suggested that the quality of RWD may be low due to incomplete reporting often found in hospital records. Due to multiple sources for RWD and potential issues with data ownership, access to certain data sources may pose a challenge. Some possible solutions were proposed by the survey respondents. To strengthen the quality of RWE, higher quality observational studies (i.e., prospective design) should be used. HTA producers need to establish consistent and systematic methodology in data collection.

These respondents also described their decision-making process to reconcile conflicting results between RWE and RCT evidence. Similar to HC respondents, HTA respondents indicated that RWE could be considered as part of the evidence. In situations where conflicting data exist between RWE and RCT evidence, respondents highlighted the need to examine potential reasons for discrepancy with key stakeholders and clinical experts. Discussions with internal and external stakeholders may help to contextualize and understand conflicting results. For example, decision-makers need to evaluate if the patient population of the RCT match that of the target population in which the device is used. In scenarios where underlying sources of discrepancy cannot be identified, the results from the RCT and RWE need to be highlighted separately while distinguishing differences in data and study quality.

Limitations

This Environmental Scan aims to provide an overview on the eligibility criteria and perceived impact of using RWE in device assessments and is not a comprehensive review. The findings are based on survey results from Canadian regulatory and HTA stakeholders. The case examples provided by survey respondents on the use of RWE to establish effectiveness and safety were based on the experience of stakeholders in their own regulatory or HTA processes and may not be an exhaustive representation of the full potential of RWE.

Due the lack of a clear and consistent definition of RWD or RWE, it may be difficult to compare processes among various organizations. One respondent indicated that it would have been helpful if a definition of RWD and RWE was provided in the survey. Furthermore, given the relatively limited experience and ongoing process reviews of incorporating RWE in device assessments among regulatory groups and HTA organizations, some material summarized in this Environmental Scan may be out-of-date.

The responses provided by eligible respondents were based on unique experiences and perspectives of their own organization. Despite proposed challenges of supplementing device assessment with RWE, respondents explained that their processes are evolving and undergoing review.

Conclusions and Implications for Decision- or Policy-Making

This Environmental Scan aimed to provide a snapshot of how Canadian regulatory groups and HTA producers are implementing RWE in device assessments and decision-making. According to the results of this survey, it is evident that RWE is eligible for inclusion in assessments across the entire life cycle of a medical device (i.e., pre and post-market). However, there is a lack of consensus on the definition of RWE and differences across various agencies in how RWE is used. Since RWE is considered to be of lower quality compared to RCTs, regulatory groups and HTA organizations indicated that RWE on effectiveness and safety should be used to supplement available RCT data. Currently, not all organizations require device manufacturers to submit RWD.

Data integrity was a recurring theme with respect to challenges of using RWE for assessments. Respondents stated that there should be a consistent and systematic method of RWD capture to help overcome potential issues such as under-reporting, data privacy, and multiple data sources. The acceptability of data sources varies among different agencies, but the results indicate that the type of study design was considered to be more important than the data source. The determination of eligible data sources can be made on a case-by-case basis. When conflicting results between RWE and RCT evidence occur, decision-makers can engage in internal and external discussions with key stakeholders and clinical experts to identify the source for the discrepancy.

There are circumstances in which RWE offers significant added value. These situations can include rare conditions or populations not well-studied in RCTs, significant unmet clinical need, or innovative and breakthrough technology. In the post-market space, RWE can be helpful in identifying increased risks and new issues pertaining to the safety of medical devices. RWE with superior external validity to that of RCTs should be given more weight particularly for situations where device performance is closely linked to user training. Due to the high cost of RCTs, RWE can help provide information to support expanding indications for devices already implanted in the health system. RWE can provide jurisdiction-specific and resource utilization information that may not be well-studied in RCTs.

As previously noted in the limitations, the conclusions made in this report are primarily based on responses from Canadian regulatory and HTA stakeholders. Respondents indicated that their processes of incorporating RWE in device assessment are evolving and undergoing review. Further work evaluating the impact of using RWE in device assessment is needed to help create a framework to better collect RWD and generate RWE.

References

  1. Use of real-world evidence to support regulatory decision-making for medical devices. Silver Spring (MD): U.S. Food and Drug Administration; 2017: https://www.fda.gov/media/99447/download. Accessed 2019 Nov 29.
  2. Makady A, de Boer A, Hillege H, Klungel O, Goettsch W. What is real-world data? A review of definitions based on literature and stakeholder interviews. Value Health. 2017;20(7):858-865.
  3. Makady A, Ham RT, de Boer A, et al. Policies for use of real-world data in health technology assessment (HTA): a comparative study of six HTA agencies. Value Health. 2017;20(4):520-532.
  4. Campbell B, Wilkinson J, Marlow M, Sheldon M. Generating evidence for new high-risk medical devices. BMJ Surgery, Interventions, & Health Technologies. 2019;1(1):e000022.
  5. ISPOR. ISPOR 2020 Top 10 HEOR Trends Report. 2020; https://www.ispor.org/heor-resources/about-heor/top-10-heor-trends. Accessed 2020 Feb 4.
  6. Makady A, van Veelen A, Jonsson P, et al. Using real-world data in health technology assessment (HTA) practice: a comparative study of five HTA agencies. Pharmacoeconomics. 2018;36(3):359-368.
  7. Framework for FDA’s real-world evidence program. Silver Spring (MD): U.S. Food and Drug Administration; 2018: https://www.fda.gov/media/120060/download. Accessed 2019 Nov 29.
  8. Directorate HCMD. 2020; https://www.canada.ca/en/health-canada/corporate/about-health-canada/branches-agencies/health-products-food-branch/medical-devices-directorate.html. Accessed Feb 27, 2020.
  9. Assessment BHT.  https://www2.gov.bc.ca/gov/content/health/about-bc-s-health-care-system/partners/health-authorities/bc-health-technology-assessment. Accessed Feb 27, 2020.
  10. Health Canada. A strategy to optimize the use of real-world evidence across the medical device life cycle in Canada. 2020; https://www.canada.ca/en/health-canada/corporate/transparency/regulatory-transparency-and-openness/improving-review-drugs-devices/real-world-evidence-medical-device-strategy.html. Accessed 2020 March 12.
  11. Murphy G, de Léséleuc L, Kaunelis D, Adcock L. Use of real-world evidence in single-drug assessments. (CADTH Environmental scan no.74). Ottawa (ON): CADTH; 2018: https://www.cadth.ca/sites/default/files/pdf/es0323-rwe-in-single-drug-appraisal.pdf. Accessed 2019 Nov 29.
  12. Health Canada. Medical device real-world data and evidence in Canada: An environmental scan highlights. 2020; https://www.canada.ca/en/health-canada/corporate/transparency/regulatory-transparency-and-openness/improving-review-drugs-devices/real-world-evidence-medical-device-strategy/environmental-scan.html. Accessed 2020 March 12.
  13. Goettsh W, Makady A. WP1: Deliverable D1.3. Glossary of definitions of common terms (including comments & replies from consultation rounds). Utrecht (NL): IMI GetReal; 2017: https://www.imi-getreal.eu/Portals/1/Documents/01%20deliverables/D1.3%20-%20GetReal%20Glossary%20of%20Definitions%20of%20Common%20Terms%20%28Including%20Comments%20%26%20Replies%20from%20Consultation%20Rounds%29.pdf. Accessed 2019 Nov 29.

Appendices

Appendices

About this Document

Authors: Yan Li, Melissa Walter

Acknowledgements: Matt Kulka

Cite As: The Use of Real-World Evidence for Medical Device Assessments — An Environmental Scan. CADTH; 2020. (Environmental Scan; no. 91).

Disclaimer: The information in this document is intended to help Canadian health care decision-makers, health care professionals, health systems leaders, and policy-makers make well-informed decisions and thereby improve the quality of health care services. While patients and others may access this document, the document is made available for informational purposes only and no representations or warranties are made with respect to its fitness for any particular purpose. The information in this document should not be used as a substitute for professional medical advice or as a substitute for the application of clinical judgment in respect of the care of a particular patient or other professional judgment in any decision-making process. The Canadian Agency for Drugs and Technologies in Health (CADTH) does not endorse any information, drugs, therapies, treatments, products, processes, or services.

While care has been taken to ensure that the information prepared by CADTH in this document is accurate, complete, and up-to-date as at the applicable date the material was first published by CADTH, CADTH does not make any guarantees to that effect. CADTH does not guarantee and is not responsible for the quality, currency, propriety, accuracy, or reasonableness of any statements, information, or conclusions contained in any third-party materials used in preparing this document. The views and opinions of third parties published in this document do not necessarily state or reflect those of CADTH.

CADTH is not responsible for any errors, omissions, injury, loss, or damage arising from or relating to the use (or misuse) of any information, statements, or conclusions contained in or implied by the contents of this document or any of the source materials.

This document may contain links to third-party websites. CADTH does not have control over the content of such sites. Use of third-party sites is governed by the third-party website owners’ own terms and conditions set out for such sites. CADTH does not make any guarantee with respect to any information contained on such third-party sites and CADTH is not responsible for any injury, loss, or damage suffered as a result of using such third-party sites. CADTH has no responsibility for the collection, use, and disclosure of personal information by third-party sites.

Subject to the aforementioned limitations, the views expressed herein do not necessarily reflect the views of Health Canada, Canada’s provincial or territorial governments, other CADTH funders, or any third-party supplier of information.

This document is prepared and intended for use in the context of the Canadian health care system. The use of this document outside of Canada is done so at the user’s own risk.

This disclaimer and any questions or matters of any nature arising from or relating to the content or use (or misuse) of this document will be governed by and interpreted in accordance with the laws of the Province of Ontario and the laws of Canada applicable therein, and all proceedings shall be subject to the exclusive jurisdiction of the courts of the Province of Ontario, Canada.

The copyright and other intellectual property rights in this document are owned by CADTH and its licensors. These rights are protected by the Canadian Copyright Act and other national and international laws and agreements. Users are permitted to make copies of this document for non-commercial purposes only, provided it is not modified when reproduced and appropriate credit is given to CADTH and its licensors.

About CADTH: CADTH is an independent, not-for-profit organization responsible for providing Canada’s health care decision-makers with objective evidence to help make informed decisions about the optimal use of drugs, medical devices, diagnostics, and procedures in our health care system.

Funding: CADTH receives funding from Canada’s federal, provincial, and territorial governments, with the exception of Quebec.

Contact [email protected] with inquiries about this notice or legal matters relating to CADTH services.