Appendix 7 : DATIS Questionnaire & Follow-Up
DATIS
Section I: Needs Assessment and Key Messages
1. Describe how Academic Detailing topics are chosen within or for your organization
Generally speaking this selection is done pragmatically and adventitiously. (ie. if there is resourcing available for a particular topic, we will certainly consider it!) However where we have a genuine choice amongst topics the following five points guide our thinking:
- The topic must have high current contemporary interest amongst our GP practitioner clients.
- The further average ‘actual’ practice is from ‘ideal’ practice, the more likely we are to adopt it.
- Where value for money can be reasonably readily improved in a topic, we are more likely to adopt it.
- Where uncertainty about ‘correct’ or ‘ideal’ practice is greatest, we usually find such topics the most rewarding.
- Where it is possible to reasonably readily measure outcomes, then we are perhaps more likely to want to adopt such a topic.
2. Does your organization employ any formal “Needs Assessment” when choosing a topic or to identify issues within at topic that could be detailed? Please describe your process.
We do not generally use a formal ‘needs assessment’ to choose topics or component messages. For our service programmes, (as distinct from research topics where we are formally testing aspects of AD effectiveness) prior to the advent of the National Prescribing Service in Australia, we would gain most ‘needs’ information from previous visits to our client GPs. As relationships develop between GPs and DATIS visitors, GPs will volunteer topics which they believe are important to them which they would like DATIS to elaborate for them. In addition, telephone calls from GPs who are seeking therapeutic advice between visits also adds to the range of possible topics for consideration.
Then from this field, visitors together with DATIS managers rank order such suggestions, and then present them to our Project Advisory Board for decision. The Board, (which includes a number of privately practicing GPs) would then make a final decision which topics should be covered.
3. Describe the process used for development and refinement of “Key Messages” used in your Academic Detailing material.
At the outset, a comprehensive review of the primary literature is undertaken by one or more of the DATIS visitors themselves. (For this period, visitors are generally taken ‘off-line’ from their visiting load so that they can intellectually focus on the new topic area in their reading.) In addition to individual RCTs studied in the particular field, reviews of all kinds, (systematic and those more qualitative in nature), grey literature and guidelines are scanned and relevant hardcopies collected. This process generally results in a large bibliography which customarily includes between 500 and 1500 articles on any one topic. After absorption of this material the authoring visitor(s) map out some scoping of the specific boundaries of the area which will be covered. This discussion occurs ideally, (and generally in practice) with all current visitors and DATIS managers. The purpose is to define both outer boundaries of the topic, and also to identify possible ‘chapter’ headings which it is believed are needed to cover the interests and concerns of our primary care doctors with regards material to be covered and which will be considered ‘in scope’.
After this, a comprehensive review is synthesized and committed to writing including an associated bibliography which covers effectively the entire ‘in scope’ field. Most importantly the review is not written to suit the interests of the professional individual(s) writing the material, but rather the client GPs. (A stylishly produced publication drawn from this review will be provided as a credibility-building gift at future office visits.) As an example, whilst pharmacokinetic considerations may be of serious interest to the authors, such abstractions are of little interest to GPs whose concerns are more associated with caring for their patients and the comparative value of different therapeutic approaches to problems where considerable uncertainty is generally a feature of presentations. The clear purpose of authoring this review is to ensure that the DATIS team has a peerless understanding of the evidence and experience in the chosen field.
Once the comprehensive summary of the field has been written, it is then simultaneously reviewed by as many of the on-service DATIS visitors as possible, external medical specialists in the particular field, selected interested GPs, and DATIS managers. Detailed suggestions for textual adjustments are generally not sought from these reviewers as might be the case for formal publication of a book or major monograph in the therapeutic area. Rather after the reviewer has carefully read the material and perhaps made margin-notes, detailed face-to-face discussions are held with the reviewers. The purpose of this interaction is to ensure that textual subtleties and meaning can be clarified and where necessary, adjusted in light of knowledge of the needs of our GPs. This review process has been found to be critically important as a first step to ensuring that the knowledge, beliefs, views (and even prejudices) of individuals who might be considered ‘opinion leaders’ in the specific field become well known to DATIS. Where possible in light of primary evidence, our text will be adjusted to account for such new ‘local-ecology’ information.
Most often, this material will be commercially printed in an A5 loose-leaf three-ring folder in an attractive and easy-to-use format. The purpose of the printed version is to provide a gift-of-value to be left with each GP at the conclusion of each visit. (Its physical location at subsequent visits is an excellent passive evaluation tool.)
After this review process is complete, the ideas for key messages which have generally crystallized to a considerable degree in the minds of the authors are then finally committed to writing.
At this time, the points 2, 3 and 5 about the chosen topic listed at response point 1 above are then taken into account, and key messages are finalized with very considerable further discussion amongst authors and DATIS managers. This final step is often protracted and involves very great care in ensuring that textual expression of the messages (and any subsidiary points) is most carefully fashioned.
Perhaps the only other aspect which comes into play here is the balance between behavioural and knowledge-imparting elements in the messages. ‘Knowledge’ elements in the KMs are often selected where some particular prior knowledge of a point is a fundamental pre-requisite before it would be reasonable to expect clinical practice behaviour change from those other KMs targeting potentially improvable clinical practices or decision-making.
After this KM selection process, a study of barriers and enablers to the behaviour change messages is undertaken by the review authors and DATIS managers. From this detailed study and discussions are drawn up a draft detailing card, plus subsidiary materials, (often hard copy) which may then be used in circumstances where delivery of certain KMs meets barriers during individual visitor-GP interactions. This final process is somewhat iterative, and extends right into the time when the program is actively being delivered.
The detailing card is never intended as a mechanism to impose linear structure on the AD encounter in a temporal sense, but rather to serve as an aide-memoire and support for the detailer. Not uncommonly, after an individual detailer’s first twenty encounters or so on a particular topic, the detailing card is unlikely to be closely followed, particularly where a solid trusting relationship has already evolved between detailer and the individual GP through multiple prior visits.
Section II: Academic Detailer Training and Background Material
1. Who is responsible for training Academic Detailers on an AD topic and how is this training conducted?
For those detailers who did not have the privilege of initially developing topic materials and KM’s a selection of between about 50-75 key papers/reviews is made by the author(s) of the DATIS review. Before any visits are undertaken by anyone, the newly authored DATIS review is thoroughly read and discussed by the visiting-team, and the key paper selections are read and tutorially journal-clubbed sequentially often together with the review author(s) and sometimes DATIS managers. This process overlaps with the study of proposed barriers and enablers to KMs, thereby gaining maximum wisdom from the field from different academic detailer knowledge and experience during development of print materials and the detailing card.
This training generally takes 4-6 weeks and is generally done at high intensity.
Towards the end of the topic-upskilling process, role-played visits are carried out with other DATIS staff (including the materials author(s) and DATIS managers) acting as role-played GPs. In addition to this, it is also customary for detailers, before their first visit on any new topic, to try out their skills with a specifically ‘friendly’ GP who understands their level of preparedness and also their lack of practical experience of visiting on the new topic area.
Before this final aspect of preparation for visiting, we strongly encourage each detailer to develop a ‘script’ in hardcopy which puts into textual form an imagined conversation with a hypothetical GP, persuasively delivering KM’s in light of the, by-then well characterized barriers to behaviour-change messages. This scripting exercise has been found to be most valuable to ensure that complex or difficult ideas can be placed into succinct and terse verbal forms for possible use in early visits on the topic. This discipline is most helpful for the initial visits on each specific topic. After the first twenty or so visits, the detailer will have evolved their most comfortable means of eliciting of GP needs in specific areas where behaviour change is targeted: at that point they will have also explored a range of different approaches to translate features of KMs into possible benefits for the individual GP in light of individual elicited needs.
2. What materials are routinely involved in the training of Academic Detailers (e.g. power point presentations on topics)? Please provide examples of training materials employed by your organization in detailer training.
We completely separate the process of AD persuasive-communication skills training and development from upskilling in a particular topic area. Skills training for new detailers is done in a well-tested and now finely adjusted intensive 24-contact hour structured educational program over three calendar days. Subsequent continuing professional skills development of detailers is undertaken throughout their time with the DATIS service and is not covered in these questionnaire responses.
The topic of upskilling on specific topics is described above in previous sections of this response, and as far as possible, class-room learning of the materials is avoided. Interactive workshopping is the method we have found to be of far greater value than attempts at passive absorption of materials delivered from stand-up lectures. The speed and intensity with which learning needs to occur makes such ‘class-room’ learning impractical in our experience.
Examples of recent US DATIS print materials have been provided under separate cover. These materials have been derived from prior Australian materials, although considerable time and effort from prime authors of the US material was spent in ‘americanizing’ text and of course researching the basis of the very extensive bibliography.
In addition to the textual materials for upskilling on a particular topic, we always ensure that visitors who were not prime authors of the DATIS manual materials also have the opportunity to sit with, and discuss key issues in the topic area with medical and other specialists working in the area surrounding the topic field. This often will include non-medical people in laboratory sciences, nursing, allied health, social aspects of the area in question etc etc. These interactions are key to breaking up the intense reading and discussions which are occurring every day during periods of topic upskilling.
3. What background materials or support materials on an AD topic are provided to detailers to ensure they are well enough versed in the topic to conduct physician visits (e.g. detailed summary documents, primary literature, etc.)? Please provide examples of materials developed by your program that act as background or support materials for detailers.
See above
4. Do you conduct any type of knowledge assessment or testing of detailers to assess their understanding of an AD topics?
No formal assessment of this kind is made. However the process is very interactive and collegial, bringing each detailer to a high level of confidence in the soundness of their knowledge in the light of their peers progress in learning. All members of the team undergoing the topic upskilling readily become aware of dimensions where individual visitors remain insecure, and group efforts are made to remedy any such situations when they arise.
5. Do you conduct any evaluation or assessment of the training that detailers are provided by your organization (e.x. a training evaluation survey)?
No. Attention to ongoing professional skills development and coaching does occur however for each detailer as they upskill and deliver successive programs.
Section III: Academic Detailing Material for Primary Care Practitioners
1. How are Academic Detailing materials (newsletters, patient information, treatment algorithms, etc.) produced by your organization?
See Section I question 3 above
2. Describe the types of Academic Detailing materials produced by your organization, and provide examples of the materials. Some possible categories include:
• Newsletters
• Prescribing Aids (e.g. titration guides, quick reference cards, reference charts)
• Patient Materials
Please feel free to include other categories of materials provided by your organization.
Apart from in-house enabler print-materials - (eg. as a Cockcroft-Gault calculation card in a type two diabetes program which has a key message for more metformin use) - we only try to deliver print or textually-based materials to the hands of GPs when they themselves will feel it will be of use to them. The one exception to this rule, is our DATIS manual (and often also our detailing card). The purpose of leaving these materials is to be able to demonstrate our commitment to deliver to them comprehensive knowledge and understanding of the clinical area in question: viz. as a means help the visitor to build trust, and particularly credibility.)
Customarily, most GPs we visit are swamped with printed materials from myriad sources: we therefore try not to add anything to that flood unless it will meet specific generalized needs in areas where behaviour change messages in our programs call for it.
Wherever possible in such circumstances where further printed aids will possibly meet specific GP needs we try to use materials from reputable independent sources, (ie. independent of commercial or third-party payor interests). This adds to the sense of our linkage with reputable and sound sources which are trusted to create printed materials of genuine usefulness for GPs.
In general terms, it should be said that we believe our prime delivery mechanism for behaviour changing messages is the skill of our visitors persuasive communication methods. Where print materials add something to this potency, we will adopt them, but otherwise we would prefer to leave to guideline producers and expert authors the business of production of text based educational materials.
3. Estimate the length of time required to produce each type of Academic Detailing material (including research, production and peer-review if applicable).
Depending on the skills and pre-existing interests of the visitor-author(s), our methods for producing broadly scoped AD programs usually have engaged one EFT professional for about six months. Thus up to about A$60,000 is needed to produce the materials and complete the topic upskilling processes outlined above.
4. Is there any standard formatting or branding used in the AD materials your organization produces?
Yes, we believe that this form of styling and branding is a crucial component of building confidence in our corporate and professional services to primary care practitioners through our visits and therapeutic advice services between visits.
5. Are the Academic Detailing materials you produce pre-viewed by a test audience to determine acceptability and clarity to the end-user? Please describe your process.
Yes, to an extent - see above for details.
6. How are AD materials utilized during an AD intervention (e.g. are all materials brought by detailer or are some materials mailed out ahead of time)?
For a comprehensive description of the way in which we use the print materials to enhance our relationships and usefulness/potency with our GPs, see above.
Before the personal individual visit, we try to avoid ever letting our materials into the hands of GPs or other interested parties such as commercial detailers. Hence they are not generally distributed, (or available) other than from the hands of our DATIS visitors.
Section IV: Impact Evaluation
1. Describe the types of Impact Evaluation conducted by your organization to assess the effectiveness of your Academic Detailing intervention. If reports or publications of your impact evaluation are available, please provide copies. Please also include any examples of informal evaluation, surveys or evaluation efforts that are in process.
In years prior to the advent of the National Prescribing Service in Australia, we were funded to develop evaluation methods and assess outcomes for our work. Over the past eight years since then, evaluation has progressed only within research studies of academic detailing funded separately from our service work for the National Prescribing Service in the State of South Australia.
Two papers are attached for information describing the sorts of evaluation we have performed in such circumstances. In the case of the 1999 Medical Journal of Australia article, this work was done when funding was available to us for evaluation, and in the case of the 2003 Family Practice article, within the framework of a clinical trial of our methods.
As a rule we never ask GPs to fill in evaluation forms or other forms of survey for us. Placing an imposition on the time of GPs we have found (particularly at the commencement of our visitor-GP relationship) to be an unnecessary barrier to rapid and harmonious development of relationships.
One key feature of our programs has always been the attempt to deliver skilled AD-propelled services to the generality of all primary care practitioners working within a defined geographic area. In this way we have been trying to pursue a specific mission for those GPs who customarily pay little attention to, or perhaps spend sparse time in CME activities.
The urge to obtain confirmatory evaluations of the value of our service through inevitably unimodal, or at worst, bimodal spontaneous questionnaire responses has been felt by successive management Boards to be of little value. This has been felt to be particularly true because other more subtle gaugings of the perceived value of our services are readily available.
We have always evaluated the usefulness of our service through indirect means, usually relying on service retention by GPs who must give up their practice time to spend time with our DATIS visitors. Key systematically recorded observations of visitors immediately after visits, and anecdotal information gathered by visitors at the time of visits have also proven to be revealing particularly when considered sequentially over time in relation to each individual client GP.
2. Who conducts the Impact Evaluation (e.g. conducted by persons within the organization or by external evaluators such as government/insurance agencies)?
As discussed above, for the past seven years, the National Prescribing Service has evaluated our Australian DATIS services throughout South Australia.