ga

Skip to content

My Files [0]

These are the files you have added to your collection.

  • You don't have any documents yet, feel free to browse the website and add documents.
Top

Interventions Directed to Professionals

Rx for Change Methods for Development: Interventions Directed to Consumers

Searching for Reviews

Searching

Information Specialists have developed search strategies for Medline, EMBASE, The Cochrane Database of Systematic Reviews, DARE (Database of Abstracts of Reviews of Effects) and CINAHL (The Cumulative Index to Nursing and Allied Health Literature). A methodological filter to restrict retrieval to systematic reviews is used as needed (e.g. in databases indexing more than systematic reviews). Search strategies have not remained static over the course of the Rx for Change project. Instead, search terms have changed to reflect emerging terminology and growing understanding of interventions to: modify prescribing habits; influence behaviour of health professionals; change structures and processes within health care organisations and systems. Revised search strategies reflect not only improved conceptual understanding of the subject matter, but also of a more pragmatic concern--precision of the search strategy. Precision is the percentage of relevant citations identified within a group of search results. Updated search strategies for this project have increased precision. Search Strategies Rx for Change - Professional & Organisational Interventions

Screening

Our primary objective was to identify potentially eligible systematic reviews of interventions that evaluate professional, organizational, financial, regulatory, or structural interventions to improve health care delivery or health care systems. Two levels of screening were developed to help identify potential reviews. Each level of screening was conducted by two independent reviewers, and all disagreements were resolved by consensus or consultation with a third reviewer, when necessary.*

Phase 1 — Title and Abstract Screening

This level of screening process was done to ensure that interventions, populations, and outcomes listed in the title or abstract were appropriate for inclusion. 

Articles for further review were included if all of the following criteria were met:

  • The review included professional interventions or the review included organizational, structural, financial, or regulatory interventions and had prescribing as a major outcome focus.
  • The interventions were directed to health care providers or in the case of organizational, structural, financial, or regulatory interventions — to health care providers or health care systems.
  • The article reported on professional (process of care) outcomes, such as test ordering, screening, consultations/referrals, patient-provider communication; or in the case of organizational, structural, financial, or regulatory interventions, reported on prescribing related outcomes, such as concordance, dosage, choice, cost containment, etc.

Non-systematic reviews (primary studies, editorials, commentaries, etc.) were excluded.

Phase 2 - Full-Text Screening

This level of screening was conducted to ensure articles were systematic reviews and to confirm the appropriateness of the interventions.

Full-text articles were considered to be potentially eligible if:

  • The paper had a methods section and explicit selection criteria.
  • Interventions evaluated in the review could be classified as professional, organizational, financial, structural, or regulatory.
  • The review had prescribing as a major focus for organizational, financial, structural, or regulatory interventions. (For professional interventions, the criterion of prescribing as a major focus does not need to be met.)

Reviews were excluded if:

  • It was an overview of reviews.
  • Professional knowledge and/or professional attitudes were the only evaluated outcomes.
  • Consumer-directed interventions only were addressed.

Note: For reviews published in languages other than English, those that were potentially eligible were reviewed, based on the details of the title and/or abstract (published in English). If it could be determined from scanning the full-text article with the help of translation tools that there was a methods section, explicit inclusion criteria, and an appropriate intervention, translating the full-text article for further analysis was considered.

Data Collection and Analysis

Quality Assessment

For each potentially eligible review, the methodological quality was assessed using AMSTAR (A MeaSurement Tool to Assess Reviews) 

http://www.ncbi.nlm.nih.gov/pubmed/17302989?ordinalpos=2&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum ? .

The AMSTAR tool consists of an 11-item checklist where reviews score one point for each criterion met, for a maximum score of 11. Reviews that achieve high scores indicate a higher methodological quality than those that achieve low scores.

Two reviewers independently assessed each AMSTAR checklist item and total for every review. The results for each review were compared and disagreements were resolved by consensus*. Where disagreement or uncertainty prevailed, a senior member of the team was consulted and a final decision was made.*

Reviews rated to be of moderate to high methodological quality (AMSTAR scores of 4 or more) qualified for data abstraction and analysis, and have been summarized and included in the Rx for Change database.

Reviews rated to be of low quality (AMSTAR scores of 3 or less) did not qualify for data abstraction. However, the bibliographic details of these reviews were collected and were made available through the Excluded Reviews page of the Rx for Change database.

Data Abstraction

Two reviewers independently abstracted all moderate- to high-quality reviews identified (AMSTAR scores 4 or more). The data abstraction was conducted in two phases. First, data were extracted from the review characteristics for the individual review summaries and, second, data were extracted from the included studies within each review to help with the analysis.

Phase 1 Individual Review Characteristics

Data on the characteristics of each individual review were abstracted for each of the summary headings used in the database. A description of the type of data captured for each field is presented in the APPENDIX.

Phase 2 Study Characteristics and Results

Data were abstracted about each included study within the review to provide data for the overall analysis of the review.

Studies that reported only patient outcomes or professional attitudes and knowledge were excluded. Studies that evaluated organizational, structural, regulatory, or financial interventions that did not report a prescribing (appropriate use) outcome were also excluded from the analysis.

* For the 2013 April update, a single reviewer screened the literature, two reviewers independently applied the selection criteria to identified reviews, and one reviewer undertook extraction activities, with some assistance from a second reviewer. Quality assessment was also performed by a single reviewer, with assistance from a second reviewer. Reviews scoring AMSTAR 3+ were assessed by a second reviewer and differences in scoring between reviewers were resolved by discussion or third party adjudication if necessary. For the 2014 April update, a single reviewer screened the literature at the title and abstract level only (Phase 1). Citations of review abstracts that met the inclusion criteria are listed on the database as pending further assessment and have been assigned to the intervention categories they address.  

Synthesis, Presentation, and Rating the Evidence

Dual data synthesis and consensus of the results of individual reviews were conducted and intervention summaries were prepared.

Individual Review Summaries

Table of Results

A data synthesis of the evidence from individual reviews were compiled and presented in a Table of Results. For each review, there were typically multiple interventions of interest compared against other interventions, including usual care; these comparisons were listed individually in the table.

For each comparison, all outcomes were tabulated under “appropriate care” unless there were only prescribing-related outcomes, in which case they were tabulated according to the “appropriate use” outcome categories. “Change in behaviour” was also used in regulatory or financial intervention comparisons.

Reviews varied greatly in the type and amount of data provided. For consistent presentation of results, vote counting was most often used for data synthesis. Analyses by vote counting were reported as the number of studies favouring the intervention out of the total number of studies for that comparison. When reviews provided additional data, such as a meta-analysis, this was included as well.

The following decision rules were used to create standardized statements of the results regarding the effectiveness of each intervention:

Vote Counting (percentage of studies favouring the intervention), where:

  • 0% of studies favour intervention = no effect
  • 1% to 33% of studies favour intervention = generally ineffective
  • 34% to 66% studies favour intervention = mixed effects
  • 67% or more of studies favour intervention = generally effective.

Absolute (or adjusted) risk difference (RD); individual, median, or pooled value, where:

  • less than 0% = deterioration in effect
  • 0% = no statement of effect size
  • 1% to 5% = small effect size
  • 6% to 15%  = medium effect size
  • more than 16%  = large effect size.

Results and Conclusions

The Results and Conclusions sections for each review were created, based on the data presented in the Table of Results, along with other considerations.

The methodological limitations, results of the comparisons, and any other items of note are described for the overall results. This can include:

  • the number of RCTs included in the analysis
  • the combined results for appropriate care and prescribing outcomes (this was reported here as “appropriate care”
  • the number of studies and the overall effectiveness of each intervention (insufficient evidence was reported for interventions with only one comparison)
  • the number of studies that were outside the scope of the review.

If results were presented in the overall results, the methodological limitations were not restated for the prescribing results. The results of the impact of interventions on appropriate use prescribing-related outcomes only (dosage, cost containment, concordance, choice, etc.) — were described. For each intervention, the overall effectiveness and the number of studies for each outcome were reported. For interventions with only one comparison, it was stated that there was insufficient evidence. If there were no prescribing-related outcomes in the review, a statement to that effect was made.

The conclusions restate the methodological quality of the review. The following criteria were used to make statements about the overall effectiveness of the interventions on appropriate care and prescribing-related outcomes:

  • If five or more studies reported outcomes for an intervention comparison, the effectiveness statement was restated in the Results section.
  • If four or fewer studies reported outcomes for an intervention comparison, it was reported that it was difficult to make conclusions based on the small number of studies that were located.
  • Any studies that had “insufficient evidence” in the Results section were not mentioned in the Conclusions section.

Intervention Summaries

Intervention summaries are based on our analysis of high-quality and highly relevant (or key) reviews. For each intervention category, overall summary statements of the evidence are presented.

Effectiveness statements describe the number of reviews that were found to be effective, ineffective, or have mixed results for appropriate care outcomes; and, where possible, these reviews are identified. Also reported are reviews with prescribing outcomes and those with insufficient evidence. If no high-quality or key reviews were located for the intervention, a statement to that effect was reported. If there were an insufficient number of studies (n ≤ 4) within the high-quality or key reviews from which to draw conclusions about the effectiveness of the intervention, a statement to that effect was made.

In the Summary of Overall Findings from Reviews and the Summary of Findings Related to Prescribing sections, the effectiveness of the intervention is stated.

Structure of the Database

Systematic methods were employed to construct the database to maximize transparency and minimize bias. Summaries of the individual reviews have been structured to present the overall characteristics of the reviews and the findings as they relate to change in professional behaviour.

For each individual review summary within the database, the number of included studies, as noted in the review, is indicated in the summary of the review. Also indicated is the number of studies that evaluated professional interventions or organizational, structural, financial, or regulatory interventions with prescribing as a main outcome focus (and, therefore, relevant to the database). A link to the list of individual primary studies included in the analyses of each systematic review is provided in each review summary. At times, the number of studies listed and the number of links to studies in the list are not equal. Possible reasons for this discrepancy include:

  • One study can be published in multiple articles (all citations were included in the list).
  • Multiple studies/comparison arms can be described in one article.
  • Citations were not provided or made available or accessible in major databases such as PubMed, EMBASE, CINAHL, or PsycINFO.
  • Not all studies in the analyses were clearly identified or cited in the review.

Database users are encouraged to consult the full text of reviews, in addition to the summaries listed on the database, for information that may have been beyond the scope of this project.

APPENDIX 

The following are abbreviations used in the database:

BA   before-and-after (study)
CBA   controlled before-and-after (study)
CI   confidence interval
CT or CCT   controlled trial
ITS   interrupted time series (analysis)
RCT   randomized controlled trial
RD   risk difference
RM   repeated measures
SS   statistical significance

The following are explanations and descriptions of the fields used in the database to present summary data and results for each review.

Individual Review Summary — Fields

Review Question

This section describes what question the review addresses, based on the information in the abstract, introduction, and/or methods of the review. The question may include the populations, interventions, and outcomes investigated.

Background

This section describes the intervention(s) evaluated in the review. It includes a general description; a more detailed description is provided when it is made available by the authors of the review. Background information about the condition, state, or problem is not provided. For reviews that use the EPOC taxonomy, this can be the statement for this background section. If a review does not provide this information in the introduction or methods sections, a statement to the effect of “not described by the author” is made.

Search for Evidence

This section describes the most recent date for searching that is provided. If there are multiple databases with different search dates used, the most recent date that a large electronic database such as MEDLINE or EMBASE was searched is noted.

Included Studies

This section describes the number of studies that met the inclusion criteria of the review and provided data about the effects of the interventions. Not included are references or studies cited elsewhere in the review that were not part of the analysis.

Studies Related to Prescribing

This section describes the number of included studies that evaluated interventions for evidence-based prescribing and drug use. This can be a subset of the included studies or it can comprise all of the included studies. If the number of studies included was unclear, this field was left blank.

Study Design

This section describes the types of study designs included in the review. The following categories were used:

  • BA (before-and-after study)
  • CBA (controlled before-and-after study)
  • CT or CCT (controlled trial — including quasi-randomized controlled trials)
  • ITS (interrupted time series analysis control may or may not be present)
  • Meta-analysis
  • RCT (randomized controlled trial)
  • Other — study designs other than those just listed; i.e., RM (repeated measures)
  • Unclear/not specified — used when authors did not indicate study designs.

Interventions

The interventions evaluated in the included studies are listed, with additional details, as appropriate. The interventions of interest were categorized according to the EPOC taxonomy, and multi-faceted interventions were indicated separately. 

Target Population

This is the population to which interventions were directed in the included studies. Populations can be grouped into categories such as health care professionals, patients, or caregivers. Health care professionals have been grouped into broad categories:

  • physicians
  • nurses
  • psychologists
  • pharmacists
  • dentists
  • other (list types in brackets)
  • any kind of professional
  • not specified.

Other target populations may be listed (e.g., midwives). For the professionals section, the types of patients indirectly targeted were listed under “Patients Reviewed.”

Setting

This section describes the settings in which the interventions took place in the included studies. Settings were broadly categorized by location, ownership, and institution or organization type (if applicable):

Location

Primary care practices
Outpatient clinics
Nursing homes
Hospitals
Pharmacies
Communities
Other
Not specified/unclear

Ownership

Ownership — private
Ownership — public
Ownership — other
Ownership —
not  specified/unclear

Institution

Academic — teaching
Academic — non-teaching
Academic — not specified/unclear
Any type of organization

Patients Reviewed

This section describes the patients who are indirectly affected by interventions that directly target health care professionals. Patient populations among the included studies were grouped into broad categories according to the information provided by the authors and may not include comprehensive reporting for age, residence, or diagnosis for each study.

Age

Children (0- to 16-years-old)
Adults (17- to 54-years-old)
Elderly (65 and older)
All ages

Other

Specific diagnosis or clinical condition
Any type of patient
Other
Not specified

Drug-Related Outcomes Category

Prescribing-related outcomes reported in the review were grouped into the following categories:

  • Concordance
  • Cost containment
  • Appropriate use — dosage
  • Appropriate use — choice
  • Appropriate use — route of administration
  • Appropriate use — (other prescribing-related outcomes)
  • Other (includes drug safety and any patient outcomes that do not fit into the above categories).

Other Outcomes Categories

Outcomes that are not related to prescribing or drug use, but are of relevance to professional behaviour in the process of care include the following:

  • procedures (follow guidelines or protocols)
  • professional-patient communications (counselling)
  • health promotion and preventive services (vaccination)
  • screening (mammograms, or for HIV)
  • test ordering
  • referrals
  • diagnoses
  • record-keeping
  • other resource use
  • discharge planning
  • not specified or unclear.

All behaviours were considered as part of the review.

Authors’ Analyses of Data

This section describes the methods used by the authors of the review to analyze the data. More than one type of analysis may be used. Categories of analyses include:

  • descriptive only (if no statistical result is given for single studies or results are given but no vote counting)
  • vote counting based on direction of effect (where studies are not pooled)
  • vote counting based on statistical significance (where studies are not pooled)
  • range of effect sizes (when authors give the range but no data from included studies)
  • meta-analysis (used to pool results)
  • meta-regression (used to analyze pooled results)
  • time series analysis (used for single study)
  • effect size (used for single study)
  • not applicable (no data or no studies identified)
  • other.

Funding

Both internal and external sources of funding are listed, if applicable. If nothing is reported by the authors regarding conflict of interest with funding, “not stated” is used. If the authors report no conflict of interest, “no perceived conflict of interest” is stated.

Table of Results 

Comparison

For each review, there are typically multiple interventions of interest that are compared against other interventions, including usual care; these comparisons are listed individually in the table. Different multi-faceted interventions may appear together as one comparison, depending on the authors’ original analyses. 

Outcome

For each comparison, all outcomes are tabulated under “appropriate care” unless there were only prescribing-related outcomes, in which case they would be tabulated according to the “appropriate use” outcome categories listed previously. “Change in behaviour” is also used in regulatory or financial intervention comparisons.

Number of Studies

This section describes the number of studies providing data for a given comparison and outcome. 

Note: The number of studies in this column may not add up to the total number of included studies or studies related to prescribing for various reasons, such as the following:

  • A study evaluated more than one comparison or outcome category.
  • For the Professionals section, studies were excluded when not meeting inclusion criteria (e.g., professional knowledge was a sole outcome).
  • Outcomes data were not available.

Analysis

This section describes the analysis method(s) used by the reviewers. Cases exist where these analyses differ from the authors’ analyses; comparisons between this field and the “Authors Analysis of Data” field in the summary can be made to identify such situations. The following are standardized statements used to describe the analysis method used by the reviewers:

  • vote counting based on direction of effect. Median effect size (range) across (or between) studies
  • vote counting based on direction of effect. Effect size)
  • vote counting based on direction of effect
  • vote counting based on statistical significance
  • vote counting based on direction of effect or statistical significance (SS). Effect size
  • vote counting based on direction of effect. Risk difference (RD) (95% CI) from meta-analysis
  • vote counting based on unclear method
  • vote counting. Method unclear due to authors' presentation of results
  • descriptive
  • vote counting based on direction of effect (non-interrupted time series analysis [ITS]) or statistical significance (SS). Median effect size (RD) (range) among comparisons
  • unable to assess.

Results

A variety of methods may have been used to report the data provided by the authors and include but are not limited to:

  • vote counting by direction plus reporting of absolute or relative effect measures (with or without measures of variability)
  • effect measures (whether for a single study or for meta-analyses)
  • vote counting alone by direction or statistical significance, descriptive, or authors’ reporting of data where analyses or data were unclear. 

Where possible, absolute effect measures were used rather than relative measures, as absolute measures provide a relatively unbiased measure of effect. Where pooled measures were not available or appropriate, a range or median, plus a range of measures, was provided. Analyses by vote counting were always reported as the number of studies favouring the intervention out of the total number of studies.

The following decision rules were used by the reviewers to create standardized statements of the results regarding the effectiveness of each intervention:

Vote counting (percentage of studies favouring the intervention) where:

  • 0% of studies favour intervention = no effect
  • 1% to 33% of studies favour intervention = generally ineffective
  • 34% to 66% of studies favour intervention = mixed effects
  • 67% or more studies favour intervention = generally effective.

Absolute (or adjusted) risk difference (RD); individual, median, or pooled value, where:

  • < 0 = deterioration in effect
  • 0 = no statement of effect size
  • 1% to 5% = small effect size
  • 6% to15%  = medium effect size
  • more than 16% = large effect size.