Appendix 10b : ADUP Internal Evaluation (Academic Detailing) : (Behavioural Change) Initiative - Osteoporosis

ALBERTA MANAGEMENT COMMITTEE ON DRUG UTILIZATION / ALBERTA DRUG UTILIZATION PROGRAM

INTERNAL EVALUATION - ACADEMIC DETAILING (BEHAVIORAL CHANGE) INITIATIVE – OSTEOPOROSIS

Prepared by Harold Lopatka

EXECUTIVE SUMMARY

An internal evaluation of the behavioral change (academic detailing) initiative was conducted based on the osteoporosis educational topic provided in David Thompson Health Region over the period October 2003 to April 2004. The evaluation consisted of four components (individual reviews): 1) review of physician feedback, 2) review of academic detailer feedback, 3) review of Alberta Blue Cross prescription claims data, and 4) review of lessons learned. The following were the results. Physician feedback was consistent with that received from other topics and very positive. Academic detailer feedback was congruent with physician comments and indicated continued high levels of physician interest. The review of prescription claims data revealed a small discernable impact on drug utilization. Also, there is an indication that the intervention may have had an impact on drug special authorizations for high risk osteoporosis patients with fractures. In terms of lessons learned, it takes time and effort for relationships to be established between the detailer and front line physicians. The initiative is still maturing with continued growth occurring and this is an important point in terms of the evaluation context. Further evaluation of the impact on drug utilization will occur with additional claims data requested from Alberta Health and Wellness.

INTRODUCTION

In Canada, approximately 1 in 4 women and 1 in 8 men have osteoporosis. Osteoporosis is presumed to cause 24 000 hip fractures in Canada each year, and these figures are projected to double by 2040 as the population at highest risk, women aged 75 years and older increases. The risk of death within one year of hip fracture is 17% in women, and 32% in men. As well, post-fracture ramifications for patients include 50% losing their independence, and 33% never returning to pre-fracture health.

It has been estimated that in 1993, the total acute care cost for osteoporosis (inpatient, outpatient care and drug therapy) was over $1.3 billion CAD. Without effective action on osteoporosis prevention and treatment strategies, it is estimated that by 2018 Canada will spend at least $32.5 billion treating osteoporotic fractures. Given the increasing proportion of older people in the population, these costs will likely rise.

In 2002 a national guideline was published, 2002 Clinical Practice Guidelines for the Diagnosis and Management of Osteoporosis in Canada.

There are a number of pharmacologic agents available for the prevention and treatment of osteoporosis: bisphosphonates (etidronate, alendronate, and risedronate); calcitonin, hormone replacement therapy (estrogen and/or progesterone), and selective estrogen-receptor modulators (raloxifene). Figure 1 indicates the increased number of persons receiving various osteoporosis therapy over seven years. Only etidronate and hormone replacement therapy have open listing on the Alberta Health and Wellness Drug Benefit List (AHWDBL). Appendix 1 contains the special authorization criteria for the other agents (based on the April 2004 AHWDBL).

Figure 1 - Number of Patients Taking Different Osteoporosis Therapies in Alberta * Over the Past 7 Years Ending March 31, 2003

*Includes claimants covered under the Alberta Health & Wellness drug programs: Group 1 and Group 66/66A.

Literature reviews were conducted for studies related to the assessment and treatment of osteoporosis in patients with fragility fractures, and in patients on long term glucocorticoid use. Reported medication prescribing adherence to clinical practice guideline rates vary from 4-86%. The following table summarizes studies with documented rates of clinical practice guideline adherence.

A recent systematic review by Grimshaw et al. showed the effect of various educational interventions for guideline dissemination and implementation. It is important to note that where evidence exists, the interventions produced modest to moderate improvements. The evidence suggests that single-patient directed interventions and reminders are more effective than other single interventions or multifaceted interventions (with educational outreach).

One study was located about an educational intervention to improve osteoporosis clinical practice guideline adherence. Majumdar et al. reported on a multifaceted intervention designed to improve primary care physician adoption of evidence based guidelines in patients with wrist fractures. The intervention consisted of physician reminders (personalized, patient-specific and faxed), treatment guidelines generated and endorsed by opinion leaders and patient education (provision of written materials and telephone counselling). The intervention resulted in 30% higher treatment rates in treatment compared to control patients.

PURPOSE

The purpose of this report is to evaluate the impact of the Osteoporosis academic detailing (behavioral change) intervention in the David Thompson Health Region.

The following were anticipated apriori outputs or outcomes from the intervention.

• Equivalent levels of physician satisfaction and interest as compared to earlier topics
• Equivalent or increased usage of academic detailer compared to earlier topics
• Increased utilization of first choice agents (and reduced use of editronate)
• Increased requests for the newer special authorization agents
• Increased claim costs for physicians receiving intervention

METHODS

Description of intervention

The multifaceted educational intervention consisted of passive clinical practice guideline dissemination, multi-disciplinary continuing education, academic detailing (educational outreach), printed educational materials (see appendix 2), opinion leader consultation and comparative prescribing feedback reports. The intervention was conducted in the David Thompson Health Region (RHA #4) over the period October- December 2003 and January-March 2004.

¹Dr. Don Morrish (endocrinologist) plenary speaker. Telehealthed to 10 sites in David Thompson Health Region – 61 (25 physicians) attended.

²51 detailing visits completed (detailing plus opinion leader sessions).

³Two reports sent – April and June 2004 (see appendix 3 for template).

Four evaluation strategies were employed; review of physician evaluations, review of academic detailer evaluations, review of Alberta Blue Cross prescription claims data and documentation of lessons learned. A description of the specific methods used for each evaluation strategy follows.

Review of physician evaluations

Physician feedback was collected from feedback from continuing medical education and post visit questionnaires. Copies of the data collection forms are attached in the appendix (see appendix 4 – CME evaluation and appendix 5 – post visit). Mean scores were determined and tabulated.

Physician feedback about the continuing education was collected after the initial continuing education session and evaluated the presentation and materials provided. Physician feedback about the educational visit and printed materials was collected after the academic detailing visit and evaluated the visit and the printed materials provided.

Review of academic detailer evaluations

Academic detailer written reports from visits documenting perceived success and accomplishments were examined. A copy of the data collection form is attached in the appendix (see appendix 6). Key data reviewed included wait length (minutes), visit length (minutes), key messages discussed (number), written material left (number of documents), follow-up items (number), perceived physician interest in visit (rating out of 5), perceived physician interest in written materials (rating out of 5), perceived physician interest in CPG (rating out of 5), and overall impressions about the guideline topic and physicians’ reactions. Information was tabulated.

Review of Alberta Blue Cross prescription claims data

Data was requested from Alberta Blue Cross (ABC) for the government sponsored plans (Groups 1, 66, 66A) for special authorization and drug claim data for the period April 1, 2003 to September 30, 2004 (see appendix 7). This provided data for 6 months preceding the intervention and 6 months post intervention. Data requested is specific to the requests received for special authorization of etidronate, alendronate, raloxifene, risedronate and synthetic calcitonin salmon. The data requested is for two health regions: David Thompson Health Region (#4), test group, and Peace Country Region (#8) for a control group. A number of physicians in David Thompson Health Region participated in individual components of the intervention but not the full intervention. They were not included in the test group. Agents were grouped into first and second choice agents for treatment. When there are no vasomotor symptoms or when fragility fractures occur, the clinical practice guideline lists alendronate, risedronate and raloxifence as first choice agents and calcitonin and etidronate as second choice agents. An inter- and intra- health region analysis was conducted with the above data. Tabulations and cross tabulations were performed.

Lessons learned

Academic detailer and management correspondence and reports were reviewed to determine lessons learned. Common implementation and operational themes were identified.

RESULTS

Physician evaluation

Table 4 summarizes physician evaluations and compares the osteoporosis visits to data from previous topics. The mean scores for physician’s evaluations of the osteoporosis detailing visit was 4.5 (out of 5) or higher except for the one rating for the value of the visit compared to traditional CME (mean score was 4.2 out of 5). Mean scores are similar to scores attained for other topics. Visit times were the highest for the osteoporosis topic while physicians indicated the time was just right.

Table 5 shows that the mean score of physicians was 4.2 out of 5 in agreement to the three major behavioral messages.

Academic detailer evaluation

Table 6 summarizes data and compares the osteoporosis visits to data from previous topics (upper respiratory infections, gastro-intestinal conditions). The average visit time was the highest at 29.6 minutes. An average score of 4.5 was rated for the three measures; perceived physician interest in the visit, written materials and clinical practice guidelines. This score was similar to that from the previous topic GI and slightly higher than the average scores for URTI. The average number of follow-up items was 0.7, which was similar to the URTI, but less than the GI. Overall, the detailer’s impression was that the osteoporosis clinical practice guidelines were frustrating to physicians because provincial payment guidelines do not cover all bisphosphonates for first line use and many were not aware of situations where Fosomax® and Actonel® could be used as first line agents. Physicians found summary treatment charts (printed educational materials) very helpful.

Drug utilization evaluation

Prescription claims

Table 7 summarizes the distribution of prescription claims (total, individual agents, first and second choice) for RHA #4 overall, RHA #4 test group physicians, and RHA #8 physicians. Overall, RHA #4 had approximately 5 times the claims of RHA #8. In terms of individual agents, Risedronate claims were more common in RHA #4 compared to RHA #8 (7.3 vs 1.7%). Use of 1^st choice agents was slightly greater in RHA #4 compared to RHA #8. RHA #4 test group physicians had more risedronate claims and fewer etidronate claims.

Figure 3 – Pre and post intervention comparison of clinical guideline adherence between RHA #4 test physicians and RHA #8 control physicians

Table 8 and Figure 3 show the results from pre (Q2 & Q3 2003) and post (Q2 & Q3 2004) intervention comparisons of agent selection. In the comparison between RHA #4 test physicians and RHA #8 control physicians, guideline adherence increased in both regions, but the increase was higher in RHA #4 test physicians (9.5% compared to 4.3%).

Table 9 shows the results from two pre and post intervention comparisons of average claim costs. In comparing within RHA #4, test physicians average claim costs increased but were lower than control physicians before and after the intervention. In comparing between RHAs, RHA #4 overall average costs were lower than RHA #8 physicians. RHA #4 test group physicians average costs were lower than RHA #8 control physicians.

Special authorization requests

Table 10 summarizes total special authorization requests, approval and denial information for the test group physicians, RHA #4 overall and RHA #8 physicians (number and percent special authorization approvals for the agents alendronate, raloxifene, residronate and synthetic calcitonin salmon). Approvals for the indication osteoporosis and fracture are nearly two times higher in the test group (24.5%) compared to the control group (14%).

Lessons learned

The osteoporosis topic was the third educational topic offered to physicians in the region over a two year period. Physician participation changed from 10 physicians for the first topic to over 50 for the osteoporosis topic and the centers (cities and towns) covered increased from 2 to 15. Approximately 30% of eligible family physicians have participated in programs.

Overall, we have achieved successes because of the attention given to building and maintaining relationships at the “system” and “front line” levels. At the “systems” level relationships were established with our key provincial organizations to assist us in a number of support and production processes. As general operating principles, we work on the premise that activities should focus on front line delivery of services and that we should not duplicate activities performed by other organizations. The following illustrates some of the specific “system” relationships.

• The clinical practice guidelines we use are produced and validated by the provincial clinical practice guidelines program, Towards Optimal Practice (TOP). Local experts are organized to develop new guidelines, validate national guidelines or update older guidelines. When the guidelines are validated, they are disseminated provincially. The guidelines are made available in written and PDA formats. In addition TOP produces 1 page guideline summaries for use in academic detailer and opinion leader visits. After our visits and internal evaluations, physician feedback from our guideline dissemination activities is provided to TOP.
• Prescription claims and physician billing data is provided from Alberta Health and Wellness, Alberta Blue Cross and Health Canada (First Nations and Inuit Health Branch). The data is used to determine potential care gaps and educational needs (before the intervention) and to evaluate the impact of the educational intervention (after the intervention).
• Alberta Blue Cross produces the comparative prescribing feedback reports for us. The reports are based on recent prescription claims data and compare consenting physicians with an average physician in the region. Physicians are very interested in this information.
• The University of Alberta continuing medical education department assists in the organization and delivery of rural multidisciplinary continuing medical education programs. They coordinate promotion activities, registrations, produce handouts, arrange for telehealth broadcasts to rural sites, produce attendance certificates and collate evaluations.
• Assistance in promoting and marketing of the initiative occurs through articles in bulletins and other publications of the David Thompson Health Region. In addition, the region facilitates attendance at medical staff meetings to discuss the initiative and its progress.

At the “front-line” level our success occurred because of our ability to build relationships with increasing numbers of front line physicians and more in-depth relationships with physicians already participating in the initiative. The following illustrate some of these relationships.

• Local physician champions have been recruited to increase acceptance of our initiative by their peers. As participation in the program is voluntary for physicians, the involvement of a highly supportive local physician champion has made a significant difference in the physician recruitment and participation. In centres (towns) where a local champion is present local physician participation in topics has been as high as 100%. In other centres where there is no local champion or the centre is too large for a local champion to have an effect, participation is approximately 10%. Unfortunately, not all centres have an identifiable local champion.
• Local community pharmacists were hired to improve efficiency and increase physician participation. The David Thompson Health Region is very large in geographic size and significant time was spent traveling from Edmonton and between centers in the region. Two local community pharmacists were hired to provide contract detailing services and coordination for distant centers. The local community pharmacists reduced travel time for our Edmonton detailer and improved physician participation. Because of the pharmacist’s local familiarity and established relationships, the pharmacist gained access to physicians who had previously resisted participation in the project. One pharmacist increased physician participation by greater than 100% in one centre.
• Opinion leaders help to improve the credibility of the initiative. Our initial plan was to co-opt local physicians to serve as an opinion leaders and to have the physician consult with their peers about the guideline being addressed. Difficulties were encountered finding local opinion leaders. The approach was changed and an opinion leader was recruited from Edmonton who along with the detailer meet physicians in small groups and discuss cases. The addition of our opinion leader, an Edmonton internal medicine specialist, is a popular learning approach with participating physicians. Our physician opinion leaders provide advice to the academic detailers about interpreting critical evidence and assist the detailers prepare for a topic by allowing them to attend outpatient clinics.
• Innovative marketing concepts were employed to increase physician interest and participation. For example, physicians are offered an opportunity to “trial” or “sample” the program (i.e., newly recruited physicians can receive detailing and / or opinion leader consultation but not continuing education or comparative prescribing feedback report). Based on experience in more recent topics (e.g., osteoporosis and COPD) approximately 10% of physicians receive sample or trial detailing visits.
• The nature of relationships between the pharmacist academic detailer and physician changes. Our academic detailer received formal training by experts from Australia on methods of creating and maintaining relationships. This helped our detailer create relationships so that basic information exchange occurred. As the relationships matured and comfort levels increased, the discussions between the detailer and individual physicians became more meaningful. It enabled the detailer to identify common uncertainties experienced in family medicine practice and specific educational needs relating to the clinical practice guideline. For example, our detailer was able to identify early in the process that the following were common areas of uncertainty amongst family medicine physicians for osteoporosis: 1) payment guidelines regarding patients with fractures and on long terms steroids, 2) effectiveness of bone density scanning for females 30-50, and 3) incidence of osteoporosis in males.

DISCUSSION

Average scores evaluating the continuing medical event were similar for the osteoporosis compared to upper respiratory tract infections. Reports from other projects and programs indicate that continuing education topics about anti-infective agents are more popular with physicians compared to other topics such as osteoporosis.

Physicians ranked the academic detailing visits similarly to previous topics, however, the visit time increased significantly compared to earlier topics. One explanation for this is that the academic detailer conducted only single visit for osteoporosis, while multiple visits (3 for URTI and 2 for GI) were conducted for the other topics. Another explanation is that physicians were more comfortable with the detailer and willing to spend more time. Physicians indicated they had a strong intention (4.2 out of 5) to follow the three key guideline messages provided.

Detailer ratings confirmed physician reports about increased time and similar interest in the osteoporosis topic compared to other topics. Physicians shared their frustration about the non alignment of the osteoporosis clinical practice guideline and the provincial payment guideline adding ambiguity to treatment decisions.

Attribution of effect is difficult with most behavioral change interventions because of the amount and type of data available, small size of effect, and the amount of system noise affecting measurements. The small effect of the intervention on drug utilization was associated with the type of data available for the analysis. Only non-linked Alberta Blue Cross prescription claims data was available. Linkable data will be available from Alberta Health and Wellness in 2005 or 2006. Information was not available about the numbers of patients with fractures or who were on long term steroids. Also, the claims data did not include other therapies; hormone replacements, vitamin D and calcium. Grimshaw et al suggested a median absolute improvement of 7% was attained from multi-faceted interventions.

The difference in the percent of special authorizations between RHA #4 and RHA #8 is notable as a major educational message of the intervention was to clarify ambiguity relating the clinical and payment guidelines. This difference reflects a directional change in congruence with the message provided as part of the educational intervention. Some caution is required in interpreting this difference as the time period for this observational data was a wider interval than for the claims data. Attribution of the intervention to this change would have to be verified through a review of additional data from Alberta Blue Cross.

The program is gaining operational maturity, but is has not reached a steady state in terms of growth. Important lessons have been learned about building relationships, recruiting physicians, improving credibility and improving efficiency. This context is important in assessing the impact of the initiative.

CONCLUSION

The ADUP osteoporosis educational intervention was delivered well and accepted by physicians, had a small effect on clinical practice guideline adherence, likely contributed to increased special authorization requests, and had a small impact on increasing average claim costs.

As our initiative continues to mature and expand, it is expected that additional relationship challenges will be identified and need to be addressed at both the “system” and “front-line” levels. It should be recognized that considerable time and effort is required to create and maintain relationships and this is critical for the success of our early knowledge translation activities.

The effect on the utilization of other resources (e.g., hospitals) and other health outcomes (e.g., health related quality of life) could not be measured.

A more extensive evaluation of the impact on drug utilization from the educational intervention is planned when linkable claims data is received from Alberta Health and Wellness.

REFERENCES

Grimshaw, J. M., Thomas, R. E., MacLennan, G., Fraser, C., Ramsay, C. R., Vale, L., Whitty, P., Eccles, M. P., Matowe, L., Shirran, L., Wensing, M., Dijkstra, R., Donaldson, C. (2004). Effectiveness and efficiency of guideline dissemination strategies, Health Technology Assessment, NHS R&D HTA program.

Kluchky, J. (2004). A drug utilization review research proposal – The prevention and treatment of osteoporosis related to fragility fractures and long term glucocorticoid therapy, Alberta Drug Utilization Program, Edmonton, AB.

Majumdar, S. R., Rowe, B. H., Folk, D., Johnson, J. A., Holroyd, B. H., Morrish, D. W., Maksymowych, W. P., Stelner, I. P., Harley, C. H., Wirzba, B. J., Hanley, D. A., Blitz, S., Russell, A. S. (2004). A controlled trial to increase detection and treatment of osteoporosis in older patients with a wrist fracture, Ann Intern Med, 141, 366-373.

Rees, S. (2005). Summary of files – Thoughts and perceptions on academic detailing experiences, Alberta Drug Utilization Program Report, Edmonton, AB.

Lopatka, H., Rees, S. (2005). Perceptions and experiences of an academic detailer, Poster presented at 2005CAPT and AFPRN conferences.

APPENDIX 1

Special Authorization Criteria

Alendronate Sodium

• For the treatment of osteoporosis in patients who have documented hip, vertebral or other fractures.
• For the treatment of osteoporosis in patients with documented evidence of intolerance or lack of response to etidronate (i.e. demonstrated as a >2% loss in bone mineral density in one year).

• For the treatment of osteoporosis in patients who have documented hip, vertebral or other fractures.
• For the treatment of osteoporosis in patients with documented evidence of intolerance or lack of response to etidronate (i.e. demonstrated as a >2% loss in bone mineral density in one year).

• For the treatment of osteoporosis in patients who have documented hip, vertebral or other fractures.
• For the treatment of osteoporosis in patients with documented evidence of intolerance or lack of response to etidronate (i.e. demonstrated as a >2% loss in bone mineral density in one year).

Synthetic Calcitonin Salmon (Salcatonin)

• For the treatment of osteoporosis in patients with documented evidence of intolerance or lack of response to etidronate (i.e. demonstrated as a >2% loss in bone mineral density in one year).

*** INSERT APPENDICES 2-7 (IMAGES, TABLES, ETC) ***