Appendix 5 : ADP Questionnaire & Follow-Up

** Comments and notes from interviews indicated in italics **

Needs Assessment and Key Messages

1. Who is responsible for AD topic selection (e.g. committee of practitioners, board of directors, etc.)?

AD Working Group (program director, drug evaluation pharmacist, senior ADer)
AD Advisory Committee ( family physicians)
DEANS – Drug evaluation Alliance of Nova Scotia

Question: Describe the government influence on Needs Assessment

Government represented by DEANS group. Besides this representation the government has very little direct influence. They may be consulted if Mike et al. cannot think of a topic, or once a topic is selected, go to them with clarification.

Question: How can you see topics from COMPUS fitting in with this process.

Try to take the topics developed by COMPUS and discuss with working group.
Potentially bring COMPUS material to family docs (will have to agree with the material)
COMPUS material may influence priorities of DEANS.

2. What process is used to select AD topics? In other words what, if any, formal “Needs Assessment” is conducted to determine an AD topic?

Family physicians who use the AD service are asked for suggestions on evaluation forms
“Hot” issue
New evidence
We conducted a survey about three years ago.

Note: Copy of Survey provided as well as the general topic ratings and for diabetes specifically.

3. Once a topic is chosen, who is responsible for development of “Key Message”?

AD Working Group in consultation with Advisory Committee and clinical content expert
The drug evaluation pharmacist plays a very important role in development of the handout material and key messages as well. The literature search is preformed by the drug evaluation pharmacist who then directs the Working group to the background literature that must be read and understood. Initial draft reports on specific topic issues are often prepared by the drug evaluation pharmacist.

4. What process is utilized, or what considerations are made, when developing “Key Messages”? (E.g. does each key message have to result in a measurable outcome?)

Key message does not have to be a measurable outcome.
Key message should help the physician incorporate the evidence into decision making in clinical practice.
Results of needs assessment on a particular topic would be utilized in developing the key message.
Clarification of process. The general topics are chosen through surveys or evaluation forms. Then research is conducted and key messages are proposed to the Physician Advisory board and then finals are chosen. As drafts are being conducted, clinical content experts are contacted to help in assessment of what is being done well by general practitioners.

5. Do you have a set number of key messages for each topic, or can any number of messages be chosen?

We limit it to 3 or 4.

Academic Detailer Training and Background Material

1. Who is responsible for training Academic Detailers on an AD topic?

AD Working Group
Clinical content experts are sometimes involved in Face-to-Face training in Halifax, sometimes through TC. It depends if they are required. Involved with varying degrees depending on need and how busy they were. They would be utilized as much as possible, it is useful to have the content experts throughout the process.

2. How is Detailer training conducted (e.g. group tutorials, teleconference training, etc.)? Where is the training held and how long does training usually take?

We hold teleconference group tutorials usually based on a draft of the handout material. Detailers comment on the draft and suggest changes.
Clarification of Process: During the TC there is discussion of their understanding based on what is written. By that time, they have read a lot of material. Used as sound-board for materials. They also help develop the “script” for the AD session.
We hold teleconference group meetings with members of the Advisory Committee.
Clarification of Process: The Teleconference includes the detailers, they get a feel for the issues in their area of the province. It gives the detailers an idea of where their local physicians are coming from.
We hold a face-to-face meeting in Halifax, sometimes attended by the clinical content expert if necessary.
Clarification of Process: The above always occurs. Purpose is to give an opportunity to meet in person to discuss more thoroughly to assess understanding of topic. Important for their level of confidence to have a face-to-face meeting
Number of teleconferences is dependent on the scope of the topic – there is no set number.
Question about videoconferencing, Mike indicates that it is in general too expensive and less convenient (detailers would have to go to the hospitals to do VC where they can do TC from home)

3. What materials are routinely involved in the training of Academic Detailers (e.g. power point presentations on topics)? Please provide examples of training materials employed by your organization in Detailer training.

Draft of handout material
Draft of the AD session script. Examples of AD scripts provide.
Follow-up question, what are the scripts. They expand on the 3 issues being brought to the physician and what each detailer is expected to talk about in each session. It is a 5-6 page script worked on in conjunction with the detailers. It ensures that the message is being delivered in a similar way with similar and specific terminology. A lot of thought is put into the wording of the script.

4. What background materials or support materials on an AD topic are provided to detailers to ensure they are well enough versed in the topic to conduct physician visits (e.g. detailed summary documents, primary literature, etc.)? Please provide examples of materials developed by your program that act as background or support materials for detailers.

Canadian and other guidelines if applicable
Primary literature with relevant RCTs, meta-analyses, etc
Commentaries on trials
Draft documents from previous literature reviews
Working drafts of handouts
Script for detailing session developed and rehearsed
“Back-pocket” information on potential issues/questions that may arise in the session

5. Do you conduct any type of knowledge assessment or testing of detailers to assess the effectiveness of training for AD topics?

No formal assessment
Get general assessment of detailer’s grasp of topic from teleconferences.
Assessment of their interpretation of the scripted material
Clarification of Process: The detailers are so involved throughout the process that formal evaluation would not be fruitful as well since the training is very progressive, there would be little benefit in evaluating the detailer “training”

6. Are there any mechanisms put into place within your organization to provide ongoing support to Detailers on a specific AD topic (e.g. access to clinical experts for follow-up questions, etc.)?

The drug evaluation pharmacist plays a vital role in providing support to the individual detailers. Each detailer contacts the drug evaluation pharmacist for assistance in responding to questions. This is usually done by email but we have a teleconference 1 to 2 weeks after starting a new topic to discuss progress and problems.
Clarification of Process: If an assessment by the content expert is required then they are referred to by the drug evaluation pharmacists. Network used.
ADer has access to clinical content expert only through the Working Group.
Follow-up question, How would this work for COMPUS topics.

NS envisions that there will still need local content expert. They expect to utilize the COMPUS document as one more source of evidence based medicine. But would still go through the same process.

Academic Detailing Material for Primary Care Practitioners

1. Who is responsible for the production of AD materials in your organization (e.g. internal staff, contract experts, etc.)?

Internal staff of CME office and Working Group
Clarification of Process: Clinical content experts see the pen-ultimate draft and suggest changes but are not involved in preliminary development..

2. Briefly describe the types of AD materials produced by your organization, and provide examples of the materials. If possible, please categorize the AD materials you produce into one of the following groups:

Newsletters
Prescribing Aids (e.g. titration guides, quick reference cards, reference charts)
Patient Materials

(Note: as this material is of primary interest to COMPUS, please provide as many examples as possible)

Detailed handout (25 to 50 pages long) that contains a thorough review of the evidence. Approximately 4 pages of summary statements at the beginning summarize key points of the evidence. More examples provided through URL.
Two-sided 81/2 x 11 double-sided laminate summarizes key messages and acts as ready reference for physicians.
Note: Patient-related materials are not produced – may go to volunteer organization (ex osteoarthritis society, Lung association) and utilize their material.
Note: It is too early to tell if the COMPUS final report could take the place of their detailed reports.

3. Is there any standard formatting or branding used in the AD materials your organization produces?

We use a standard border and logo on our materials and if colors are needed, usually blue and yellow are used.
Binders supplied to physicians to keep all AD material in one place.

4. How are AD materials utilized during an AD intervention (e.g. are all materials brought by detailer or are some materials mailed out ahead of time)?

All materials are brought by the detailer at time of visit.
Handout is mailed out in advance if requested by individual physician. This does not happen often.

5. Are there AD materials you would like to produce or have access to that you currently do not? What barriers are there to the production of these materials?

Would like to have PDA format but takes a lot of time to develop properly.
Patient information sheets. By the time things are produced for detailers and doctors, need to get on the road. No time/energy/brain power. Concentrate a lot in getting stuff out. To produce patient information properly, would have to be vetted by patients to see if they understand.
“Non-prescription” pads
Barriers to producing eye-catching materials : time and money for expertise, NS sees a lot of potential vale in COMPUS producing this type of material.

Impact Evaluation

1. Describe the types of Impact Evaluation conducted by your organization to assess the effectiveness of an AD intervention. If reports of Impact Evaluation are available, please provide. Please also include any examples of informal evaluation, surveys or evaluation efforts that are still in process.

Evaluation form is left with physician at time of visit. This can be mailed (self-addressed stamped envelope provided) or faxed in. This is subjective. Note: Examples provided of both the evaluation and summary results.
Note: The typical return rate is about 50%.
We have asked physicians to rate the usefulness of our handout material and enclose the results.
Physicians are sometimes left with a form to fax to Pharmacare to get information on patients that might be impacted by our message. We are now doing a project to study the effectiveness of this approach. Note: The study is being done on the Statin topic, they want to follow-up prescribing patterns. Project done in August.
DEANS has performed at least one evaluation on the osteoarthritis AD
These project results were presented by Steve Graham in Alberta. The presentation focused on propensity scores, it was very complicated. Results indicated that there was change in prescribing that lasted about 3 months but was not sustained at one year. Like CURE

2. What data sources, if any, must be accessed to conduct the Evaluation (e.g. provincial drug databases, “linked” healthcare databases)?

Pharmacare database linked to databases at Population Health Research Unit.

3. Who conducts the Impact Evaluation (e.g. conducted by persons within the organization or by external evaluators such as government)?

Tabulation of subjective evaluation is performed within the organization.
Project to evaluate effectiveness of fax forms being done by our office, Drug Evaluation Alliance of NS, and Population Health Research Unit.
As above DEANS evaluation of osteoarthritis by researchers on the committee

Some general comments about evaluation of AD:

The complexity of evaluation depends on level of evaluation. The 1st level, satisfaction is quite easy; the 2^nd level, impact on prescribing is somewhat easier to access through basic prescription databases; the 3^rd level is most difficult, impact on patient outcomes which requires linked. The evaluations become more difficult as time goes buy. Our best chance is utilizing Malcolm’s delayed crossover trials.

Questions about application of COMPUS material:

What types of material are you expecting from COMPUS?

NS would like COMPUS to do a needs assessment of CADC before they start on a project. Example, for HTN, what is the absolute incremental benefit (change in outcomes) of reaching new targets. If you get below a target BP of <130/80 what is the NNT to get solid endpoint outcomes compared to BP of 140/90
It might help if they asked us the questions ahead of time about a topic as it will make easier to incorporate into the material
Another example, diabetes, new guidelines says everyone should be on ACE Inhibitors, is this true, contentious.

How are you envisioning these materials to be utilized in your particular AD organization? Would materials have to be reviewed by local advisors?

Yes.

Would you want to be able to “brand” the materials with your own logo, as well as the COMPUS logo?

NS would likely want to be able to brand topics. For programs that are fully established, it is more useful to have the materials branded with local and COMPUS. As COMPUS continues to work out, then they could stand alone. Possibly.