Appendix 1 : CDUP Questionnaire & Follow-Up

CDUP Interview Questions

** Notes and comments from interview process in italics **

Needs Assessment and Key Messages

1. Who is responsible for AD topic selection (e.g. committee of practitioners, board of directors, etc.)?

the program coordinator (Anne)
we do not have a committee or board of directors
The Pharmacy Department manager at Lions Gate hospital is the administrative director of the program. This person is responsible for negotiation with the B.C. Pharmacare department and general program goals, but is very “hands-off” in terms of programmatic descisions.

2. What process is used to select AD topics? In other words what, if any, formal “Needs Assessment” is conducted to determine an AD topic?

we use a variety of methods to choose a topic:
- physician surveys (q2yr they fill out a questionnaire). The survey is of the physicians associated with the Lion’s Gate Hospital.
- Aside: utilization of technology was a part of the most recent survey. Doctors as a whole were not interested in the use of PDA technology, although some groups were very interested.
- current topics and/or topics of controversy
- topics for which new guidelines are released
- new drugs on the market
- in the past, topics were chosen based on $ spent (and potential to be saved) and drug utilization in our health region, but we don’t do this anymore (difficult to get timely information now, and we are now trying to cater to physicians’ needs). CDUP used to have a contact who worked with BC Pharmacare who could get retrospective DU data on specific areas which were used to assist in decision-making. When RBP came into effect, it substantially eliminated the need for AD on agent choice for cost savings and wiped out the ability to detect changes in prescribing process based on AD. When this connection was lost, there was less direct access to Pharmacare. CDUP also does not want to appear as a cost-savings organization or an agent of the government directly.
- Aside: some regional drug use data provided by CSPR Drug Atlas. Available on their website, current to December 2005.
we try to balance topics which can be measured and those which cannot be measured using prescription databases

3. Once a topic is chosen, who is responsible for development of “Key Message”?

program coordinator (Anne)

4. What process is utilized, or what considerations are made, when developing “Key Messages”? (E.g. does each key message have to result in a measurable outcome?)

relevance to family physicians
not all key messages are measurable

5. Do you have a set number of key messages for each topic, or can any number of messages be chosen?

we do not have a set # of key messages for each topic
key messages are usually determined after the newsletter is written, however during the research for the newsletter it becomes which area to focus in on key messages

Academic Detailer Training and Background Material

1. Who is responsible for training Academic Detailers on an AD topic?

the person who wrote the newsletter; usually this is the program coordinator, however there have been instances where others have written the newsletter and thus they train the detailers

2. How is Detailer training conducted (e.g. group tutorials, teleconference training, etc.)? Where is the training held and how long does training usually take?

Shawn/Kyle: I will be talking both about my regular detailing newsletters & a diabetes project where there are more detailers involved. This diabetes project is new and will last for 18 months. Some of the materials in the diabetes project is new, but some have been adopted by the way we do things at BC CDUP.
most of the topics thus far have been generated and delivered by the program coordinator, thus no training was required; however we are working on a project and in the past 6 months have expanded our detailing capacity to include detailers from all over the province. The first time these detailers were trained on the topics we conducted a live lecture with simultaneous teleconferencing; we also used a tool similar to Elluminate to change the slides on the participants’ screen (for those who teleconferenced in). The next time we do this we will continue to train in person but use videoconferencing instead. We need the tele/videoconferencing because there are detailers who are geographically separated from the training site.

New Program (technology driven AD)

Funded by a grant with faculty of medicine
18 month study on technology supported AD, 9-10 new detailers. Part-time job besides their regional jobs.
1^st - Trained on how to detail using workshop and role playing. Done live and VTC with northern B.C. to train how to do detailing. Note, this training was more successful when done live vs. by teleconference
2^nd - Trained on how to utilize the technology to do AD. Done on computer lab. Technical support provided for technology and drug information support available.
3^rd - Training on the topic (diabetes), clinical trials etc. The next topic is diabetics on dyslipidemia, pharmacists will only receive training on topic.

Describe the benefits and drawbacks of teleconferencing as a trainer education strategy.

Live preferred, VC for detailer training, content and technology training done by teleconference. More difficult. For upcoming topics, VC is going to be used for content training. TC is too disconnected to have good educational interaction.

How would you envision COMPUS providing training on the materials that they produce that is to be taken out to physician community?

1. Ideally, live training would be preferred
2. Other option would be using technology like “theheart.org”, videotaped lectures online. They are easy to use and can be viewed on ones own time.
3. They could potentially use self directed learning, but people should be provided with options to cater to different people’s learning styles. The verbal presentation for Anne are better to generate understanding.
4. Self assessment. Should be provided.

I don’t know how long it took them to do the required prereadings, but the lecture was about half a day.

3. What materials are routinely involved in the training of Academic Detailers (e.g. power point presentations on topics)? Please provide examples of training materials employed by your organization in Detailer training.

powerpoint presentation & handout [attachment provided; the author has requested that it not be distributed without his permission. I had his permission to distribute it to COMPUS. annedmpresentationFINAL (da 08feb2006).pdf]
are you planning to ask about how detailers are trained? Or will you be using Frank May’s materials? _ I think this is beyond the scope of our project.
obviously this does not apply when I create & deliver the materials myself for other topics

4. What background materials or support materials on an AD topic are provided to detailers to ensure they are well enough versed in the topic to conduct physician visits (e.g. detailed summary documents, primary literature, etc.)? Please provide examples of materials developed by your program that act as background or support materials for detailers.

required pre-reading: newsletter & drug chart [diabetes newsletter on mytead (24feb2006).pdf, diabetes drugchart on mytead (24feb2006).pdf
other pre-readings: primary literature from trials that were going to be discussed [not attached because I don’t think it is necessary to illustrate this point. I think it will be obvious to you which ones he provided based on the presentation handout]
For the Technology Assisted Academic detailing, all detailers were given access to the electronic library of all primary literature through the “Virtual online community practice”. All the people on the project can meet online on this project. The website has threads, the ability to posted questions to anyone in the community (detailers or physicians, diabetologists and clinical experts). Each AD has their own space so that the physicians you detail can visit your section. Your section also included the ppt version of the drug chart which you used whichever ones you want. Through website, can set up elluminate meetings with the physicians and communicate with research personnel. Working on getting limited access for web-shots and better understanding.
these were posted on a website similar to Shirwin (www.mytead.ca) which is only accessible by participants in the project. Participants can post messages from the website, and can upload information as needed. It goes beyond being a regular web page.
there were other articles which were of interest to the detailers and have been posted on the website by myself, or others, as needed
obviously this does not apply when I create & deliver the materials myself for other topics

5. Do you conduct any type of knowledge assessment or testing of detailers to assess the effectiveness of training for AD topics?

no formal test, but to ensure that pre-readings were performed, they were asked to bring 3 questions into the training session
there was also a quiz at the end of the session (based on cases)

Was this quiz assessed – self-assessed or submitted? What it detailer not up to par?

The quiz not assessed, not submitted. Might be considered in the future, but since all pharmacists, no summary evaluation.
COMPUS could consider an evaluation or self-evaluation component of their material to insure detailers understand. It could be structured as a self-assessment (multiple choice) with descriptive answer book or like paper CE questionnaires.
for the next session, we may do a pre-lecture quiz & a post-lecture one too
obviously this does not apply when I create & deliver the materials myself for other topics

6. Are there any mechanisms put into place within your organization to provide ongoing support to Detailers on a specific AD topic (e.g. access to clinical experts for follow-up questions, etc.)?

for drug related questions: Drug & Poison Information Centre
for clinical practice related questions: pharmacists field the questions from physicians, and if they cannot answer the question it is posted on a site similar to Shirwin and anyone in the community of practice (pharmacists, physicians, research assistants, researchers) can answer it. However only our endocrinologist (who is working on this project) has been answering the clinical questions.
Topic specific questions can be posed to the clinical expert that assisted in material production.
All DI questions will go through Anne before going to the clinical expert.

Academic Detailing Material for Primary Care Practitioners

1. Who is responsible for the production of AD materials in your organization (e.g. internal staff, contract experts, etc.)?

program coordinator
materials are reviewed by a LOCAL medical expert on the topic and also by another pharmacist. I stress LOCAL because it is important for physicians to recognize someone they consider an “expert” in the materials who knows what practice is really like, and that it didn’t come from the ivory towers of the University.
With the new AD project, working in the 5 health regions in B.C. Trying to send the newsletter to one doctor in each health region to act as reviewers. So, physicians from Prince George will see a familiar doctor.
Anne is not sure how COMPUS is going to get “around” the need for local expert “buy-in” in the material they produce
sometimes we contract experts (pharmacists) to write the newsletter

2. Briefly describe the types of AD materials produced by your organization, and provide examples of the materials. If possible, please categorize the AD materials you produce into one of the following groups:

Newsletters
- yes we use these [diabetes newsletter on mytead (24feb2006).pdf]
- www.cdup.org for more
- The preferred length of document is 1 page double-sided, but a double page spread of 11x7 folded is often required. The shorter document is preferred by G.P.’s.
- The Drug Card is sometimes incorporated into the newsletter, sometimes delivered separately
- The references list is created, but not necessarily provided directly on the newsletter, but kept on the website material for reference.

Prescribing Aids (e.g. titration guides, quick reference cards, reference charts)
- we create drug charts & try to put as much information as we can on them (but not as much as Rxfiles) [diabetes drug chart on mytead (24feb2006).pdf]
- The most popular component of the drug card are the drug cost comparisons as well as the pharmacare covereage information.
- www.cdup.org for more
Patient Materials
- we did these once long ago for antibiotics (antibiotics are not for viruses). There were posters & prescription pads too. I was not involved in these but examples can be downloaded from
- CDUP would like to produce more patient-related material however, they are limited by time and expertise. The patient information has to be written in a different way, that Anne is not comfortable creating. Instead of creating material directly, sometimes detailers help find useful patient information for physicians (not regularily)
- http://www.cdup.org/pamphlets.html. Currently we don’t develop anything for patients, however some physicians will photocopy our newsletter for patients to read.

3. Is there any standard formatting or branding used in the AD materials your organization produces?

see our masthead, logo

paper = cream coloured, masthead = blue
we try to keep a similar layout, but now will consider the information provided to by Stan & Jorge
we try to have 1 page double sided or at most a 4 pager (1 double page spread printed on both sides)
summaries (=Key Messages) are at the bottom/end of the newsletter
placement of the key messages at the back is historical in origin, that is the way it has been done since the start, that is what is expected, lots of doctors go right to the back to review key messages first.
detailed drug chart
we are slowly introducing numbers needed to treat.

4. How are AD materials utilized during an AD intervention (e.g. are all materials brought by detailer or are some materials mailed out ahead of time)?

we only provide the newsletter & drug chart
these are provided ahead of time (snail mail or hospital mailbox), but also extra copies are brought into the office just in case
The newsletters and charts are sent to all GP and pharmacists related to Lion’s Gate. In addition there is a “snail mail” listing which anyone can sign up for. Finally there is an electronic mailing that links to the newsletter on the website.

Note: we also seem to need to re-supply frequently after mailed out.

We should also ask about the web-site a resource – are physicians using it to access their material after visits. Is this internet access to documents an important feature of detailing efforts? Not actively pursued. Activity is higher when a notice is sent

in our diabetes study, for physicians enrolled in the technology arm, the newsletters are emailed to the physicians (or alternatively they can download it from our Shirwin-like website)
during the intervention, the detailer goes over the materials in the newsletter & drug chart and points out the information on the page(s) where necessary
B.C. does not produce a detail document (like NS) to back-up their newsletter and chart material.
in our diabetes study, for physicians enrolled in the technology arm, we tried to put the newsletter on the screen, but it was too difficult to read. Thus we developed a PowerPoint version of the newsletter highlighting the important points. The drug chart could not be incorporated into PowerPoint, thus the physician has to print this out and follow along with the detailer.
These ppt were not used for group presentations. In the technology arm, since some of the AD was done online over an elluminate link, the detailers required the chart information in ppt slides so they could be easily presented on screen.

5. Are there AD materials you would like to produce or have access to that you currently do not? What barriers are there to the production of these materials?

patient materials
materials for the PDA that go beyond pdfing a file. These are hard to read, and time/expertise is needed to develop the proper interface.
In one of the tech-enabled arms of the new AD project, Dr. provided with PDA however PDA programmed with more patient tracking software vs. educational. The software allowed the creation of pt flowsheets and the AD will focus around the use of this technology. The newsletter and charts will not go into the PDA, no decision making tools provide, maybe in future. They wanted to convert the newsletter, but to costly and not enough time.
we would also like to do what Rxfiles is doing and provide email alerts of important information.
Anne is not sure if COMPUS could facilitate a role in “on the ball” alerting of AD groups and doctors of important/new information.

Impact Evaluation

1. Describe the types of Impact Evaluation conducted by your organization to assess the effectiveness of an AD intervention. If reports of Impact Evaluation are available, please provide. Please also include any examples of informal evaluation, surveys or evaluation efforts that are still in process.

subjective evaluations: surveys sent to physicians q2 years. I have the questions from Survey 2004 attached [survey – 2004.pdf], but I don’t have the results in a nice format; some of the data from Survey 2000 is in a chart for you. [Kyle MacNair cdup summary (28apr2006).doc]
objective evaluation: older impact evaluations compared our region to another health region of similar size/age. This was just in-house data because the analyses were crude and is in the second part of the attachment. [Kyle MacNair cdup summary (28apr2006).doc]
objective evaluation: rigorous evaluation. A draft, which was used for the BPCP final report, is attached. I am still working on the poster, so this draft version will be changed. However, most of the methods should be there. I can get the CAPT 2006 poster to you after the conference. I will be back on April 15^th for one day. Can you wait this long? [CAPT 2006 poster for rmorrow (26apr2006).pdf]. Anne has supplied.

2. What data sources, if any, must be accessed to conduct the Evaluation (e.g. provincial drug databases, “linked” healthcare databases)?

see CAPT 2006 poster [see note above]
patience (in BC it took +++ years to get clean data. Prior to this, you have to go through ethics & other committees for the proper approval). Noted
expertise to extract and make sense of the data

3. Who conducts the Impact Evaluation (e.g. conducted by persons within the organization or by external evaluators such as government)?

the team is listed on the CAPT 2006 poster. They are not employees of BC CDUP, and we paid them through funds from grants. They are people who have worked closely with Malcolm Maclure in the past.

Looking at the survey data already provided, there are “economic impacts” associated with many of the interventions, how were those impacts derived?

Derived by previous person, who had access to the database. There were 50 health regions, could compare similar sized regions, to utilization on specific topic.

Questions about application of COMPUS material:

What types of material are you expecting from COMPUS?

Newsletter.
Support document
Presentation, similar to diabetes presentation. Live or taped.
Access to content support.
Process for networking with other detailers (informal), access to discussion board.
Patient handouts to provide to physicians.
F A Q section this post production (based on feedback from field material)

How are you envisioning these materials to be utilized in your particular AD organization?

Once we are trained, present short newsletter to Dr. Supplant the materials you currently use.

Would you want to be able to “brand” the materials with your own logo, as well as the COMPUS logo?

Depends what they produce, don’t know. Easier if the CDUP log is on it, for recognition, but didn’t produce. If it was material we liked, then yes, if not, no. Be careful as being seen as agent of the government, cautious of autonomy view.