Prescribing Aids - Evaluation

Introduction

Prescribing aids represent a variety of tools that primary care prescribers may utilize to enhance or improve efficiency in direct patient management. Academic detailing organizations may develop prescribing aids as value-added products to complement detailing visits. The prescribing aids were sub-grouped to allow for cataloguing and evaluation. The materials were grouped as follows:

Drug/Disease References
Risk Calculators
Case Studies

Methods

Summary descriptive information was collected from each of the representative prescribing aids evaluated. This summary information included number of pages, use of colour and branding within the document and the document’s key information.

Since there is no existing tool available to evaluate academic detailing prescribing aids, a pragmatic evaluation was developed. This evaluation was comprised of five composite elements. A written evaluation/commentary was provided on each of the elements. A less structured evaluation of the prescribing aids was conducted as compared to the evaluation of the newsletters. The newsletter evaluation involved scoring of 20 individual elements, which was feasible because of the general consistency between newsletters. The extreme heterogeneity of prescribing aids does not allow for this degree of structure. For example, a drug reference usually involves a chart of varied information relating to the drug, where a case study may be an entirely text-based document. Individually applying all 20 elements of the newsletter to the heterogeneous prescribing aids would therefore result in relatively meaningless valuation of quality. Instead the most applicable elements of the newsletter evaluation have been combined into composite quality elements for prescribing aids. This allows the flexibility to assess only the relevant quality elements and apply to a particular prescribing aid format. A graphical representation of how the 20 elements of the newsletter evaluation were applied to the prescribing aids is included in Appendix 25.

One researcher conducted the evaluation of each of the prescribing aids. A second researcher then reviewed these evaluations and provided additional comments. There was no formal scoring system used in the prescribing aid evaluation (as was conducted with the newsletters).

The final version of the evaluation questionnaire for the prescribing aids is available in Appendix 28. The rationale for each element is discussed below:

Element #1

Readability – Evaluation of the prescribing aids’ readability considered a number of design elements, many of which were highlighted in the evaluation of the newsletters.

In documents where prose was utilized, the appropriate division of text into readable “chunks” was evaluated and the document was monitored for overly long sections of text.
In documents where prose was utilized, the column width was assessed recognizing that, to enhance readability, columns should have 7 to 12 words per line.
The justification of text was assessed recognizing that un-justified (left flush) text provides a variety in line length that helps readers locate the next line when reading. The over-use of hyphenation was also monitored.
The consistent and appropriate use of colouring and highlighting (including use of bold and underlined text) was assessed recognizing that appropriate use can improve readability and inappropriate use can cause confusion.
The font size was assessed to ensure ease of reading.
The general editorial consistency within charts (similar format, colour patterns and structure) and within text (similar heading formats and use of bullet points) was assessed.

Element #2

Decision Support – Evaluation of the decision support provided by the prescribing aids considered a number of content elements, many of which were highlighted in the evaluation of the newsletters:

The presence of relevant therapeutic information including alternative agents, dosing considerations and any other information required to support prescribing behaviours.
The presence of comparative cost information provided in appropriate clinical context is highly relevant to the decision making process of practitioners.
The presence or absence of referencing was evaluated. Referencing provides the reader with the ability to assess the source of the material and is an important aspect of an evidence-based prescribing resource.
The presence of mechanisms for storage and retrieval were assessed. These mechanisms could include dating and/or assigning edition numbers to the documents and providing a web-based repository. The ability to easily store documents in paper or electronic form (website) is considered an asset.

Element #3

Impact – Evaluation of the prescribing aids’ impact considered a number of content and design elements, many of which were highlighted in the evaluation of the newsletters.

Both the presence of key points or main messages and whether they were highlighted within the document was assessed. Readers may only remember one or two points and insuring these standout and are easily retrieval may be an effective way to enhance retention. However, as these documents are generally designed for reference, it is not as crucial a consideration as it would be within the newsletter.
The presence or absence of specific prescribing behaviour targets was assessed. While many prescribing aids may be designed to simply, convey a specific set of information. The presence of behaviour targets within the document may enhance the usefulness of the documents to prescribing practitioners.

Element #4

Strengths – Highlight of the best qualities of the prescribing aid, including consideration of any positive aspects that were not evaluated through elements 1-3.

Element #5

Weaknesses – Highlight of the worst qualities of the prescribing aid, including consideration of any negative aspects that were not evaluated through elements 1-3.

Of note there were two elements (statistics and peer review) that were considered in the evaluation of the newsletters, but not in the evaluation of prescribing aids. In order to understand why these elements were not considered, one must consider the purpose of the prescribing aids. They are tools that primary care prescribers may utilize to assist in patient management, as such their content is generally accepted (ex. dosing or side effect) information that is formatted into a useful document.

Comparative statistical information is less applicable to fact-based content. Likewise peer review may be less important for fact-based tools that do not address controversial issues. This is not to say that a prescribing aid could not contain statistical information or be peer reviewed, the inclusion of these elements may greatly improve a specific prescribing aid. However, because the lack of universal applicability, these elements were not considered in this formal evaluation.

Results

A total of 20 prescribing aids from six different academic detailing organizations (two of the prescribing aids were from organizations that were not part of the survey) were evaluated. The prescribing aids had an average length of 2.7 pages and ranged from one to six pages in length. Additional reference pages skewed the average document length. On average, the content of the prescribing aids were two pages in length (varying from one to four) with the addition of between one to four pages of references.

Seventy five percent of the documents were full colour with the other 25% being either black and white or partial colour (i.e. only the logo was coloured).

The catalogue of individual prescribing aids and their associated evaluation is included in Appendix 29.

Limitations

There are significant limitations to this evaluation. The elements used to evaluate the material were highly subjective in nature. Although recognized design and content elements were considered in the development of the evaluation, no attempt was made to quantify or validate this scale. The points considered in each element may not provide appropriate emphasis in relation to impact or outcomes of the prescribing aid. The ideal tool would validate prescribing aid characteristics against outcomes or some surrogate of behavioural change outcomes. This complex task is beyond the scope of the current project. The evaluation scale does attempt to descriptively analyse multiple dimensions of different documents to provide useful template recommendations.

Within the spectrum of academic detailing materials, the category of prescribing aids is quite heterogeneous. Any tool used by a practitioner during the prescribing process could be considered within this category. As such evaluated elements may not universally apply to each type of prescribing aid equally. For example, the need for drug pricing information within a case study may be unnecessary depending on nuances of the cases being considered. Overall assessment of individual prescribing aids must be interpreted with these limitations in mind.

Discussion

Importance

Prescribing aids are an important offering used by many academic detailing organizations. Prescribing aids are not produced as commonly as newsletters as certain topics don’t lend themselves for the provision of these resources. However, for one of the Canadian organisation (RxFiles) prescribing aids are the key resource they produced. These resources are catalogued and distributed to practitioners throughout the province and heavily utilized during the academic detailing process. A copy of the entire “book” of RxFiles prescribing aids is included in Appendix 30 (note: hard copy of RxFiles prescribing aids, no electronic copies of are available), and six of the aids have been chosen for evaluation within this section. Prescribing aids are important also because they provide an avenue for providing specific information in a concise document that would be difficult to provide within the structure of a newsletter. Although these prescribing aids can be provided as stand-alone documents, a majority are produced as a supplementary resource or tool in conjunction with an organization’s newsletter. They are routinely used as a value-added addition provided to practitioners during an academic detailing visit.

As a potential toolkit item, prescribing aids represent an actual hands-on, utilitarian, material provided by COMPUS that practitioners could use in their day-to-day practice. Although they would likely be provided in the context of a topic (with a newsletter) they would ideally be the component that is incorporated into practice. Prescribing aids, when provided in context with a newsletter, may be a component of a toolkit that could be used in areas of the country where no academic detailing programs exist.

Content and Format

The concepts of content and format are discussed in a single section as they are highly dependent on one another in the context of these documents. The prescribing aids reviewed varied from one to six pages with an average length of 2.7 pages. However, the preferred length of content within the document tended to be approximately two pages with additional pages for references as needed. With three distinct categories of prescribing aid, there is understandable variety in the content of the documents. The Drug/Disease references tend to rely heavily on charts of information. Of the 14 Drug/Disease reference documents, only two were primarily text-based, the rest were chart-based. The Risk Calculators were entirely chart-based. The calculators that employed graphics and colour coding to convey risk were both aesthetically preferred and more functional then comparators. The two Case Studies that were identified and evaluated in this analysis were both narrative-based owing to the need to convey complex clinical scenarios. Two of the prescribing aids combined the Drug/Disease Reference information with the Risk Calculator; this strategy improved the overall utility of the document by providing clinical utilization information corresponding to the risk calculations.

One content component that would be a challenge to centrally produced COMPUS material would be cost and drug coverage information. Both formulary cost and coverage status may vary between provinces, capturing that information in a prescribing aid would require the production of different prescribing aids for each province. This degree of customization is likely impractical, therefore use of approximate wholesale costs without direct discussion of provincial coverage may be required. The additional province-specific information should be provided during the detailing visit.

A variety of document formats are included in our evaluated prescribing aids. Most organisations have attempted to develop a relatively uniform style and approach with their materials. Over time, this consistency likely helps to enhance recognition by physicians and “brand” the product as a useful source of non-commercial drug information.

Process

Most of the Canadian academic detailing organizations produce prescribing aids on an as-needed basis as part of their product offering to prescribers. The various organizations employ different distribution strategies. For example, PrISM sends all of its prescribing aids (associated with newsletters) out to all general practitioners in the province. The Nova Scotia group provides their materials only to physicians who request their service. Any prescribing aids produced by COMPUS would likely be incorporated into existing distribution strategies. Well-established organisations with established “branded” materials may resist disseminating centrally (COMPUS) produced newsletters (in favour of disseminating their own). But, they are generally more receptive to the concept of using additional tools produced by COMPUS (ex. Prescribing Aids and Patient Information Materials) that complements the topic being detailed. There may be less of a need to facilitate the branding of COMPUS material by local organizations as they would rarely be distributed in isolation. These materials are likely to be disseminated in conjunction with newsletters that may be co-branded. All academic detailing organizations and other organizations (health regions) should be able to make ready use of the prescribing aids as supporting tools on given topics.

Based on interviews with the international and domestic academic detailing groups, it is estimated to take anywhere from a few weeks to several months to produce a newsletter. The additional time spend on developing prescribing aids is generally minimal as the background material is already collated from newsletter development. The key factor is not the time required for development, but determining what (if any) prescribing aids would complement the product offering within of a given topic. Receiving centrally produced prescribing aids from COMPUS could be a considerable advantage. Given the planning horizons of academic detailing organisations, it will be important to establish delivery dates and provide information on early drafts of the materials. This will allow organization to adjust their schedules to take advantage of the materials produced by COMPUS.