'); //]]>
Assessment of risk of bias was not conducted because the heterogeneity in the quantitative and qualitative study designs, reviews, and reports did not allow for a comprehensive and consistent evaluation of bias.
We started the study with 3160 articles from 5 different literary sources (). After the removal of duplicates (725 duplicates), 2435 articles remained for title and abstract review. Eventually, 186 articles were included for full text review, of which 60 were included in this review. The interrater reliability between LL and SVL was calculated using Cohen’s kappa. An interrater agreement of 90.9% with a reliability of Κ=.787 (95% CI 0.686-0.888) was reached between the two reviewers. Additionally, 9 articles were added based on expert opinion, resulting in a total selection of 69 articles. Although these 9 articles were not retrieved by the search query, they reported on aspects relevant for this review, including information related to the coding of allergy information, reporting of DAs, and strategies for improving CPOE alerts. These articles were identified by PC from automatic weekly updates on PubMed (My NCBI). These weekly updates were based on separate queries with individual key words including “Computerized Physician Order Entry,” “Clinical Decision Support,” “Medication error,” and “Drug allergy.” PC collected and indexed relevant articles from this weekly list using reference manager software over the years. Articles included in the library of the reference software that dealt with relevant topics but were not discovered via the search queries were included.
The names of the authors, year of publication, study design, aim of the study, and key findings are shown in . Papers in are organized by topic [,,-,,-]. Papers belonging to different topics are categorized in all corresponding topics. In , a table with the number of subjects included in the study can be found. Most studies (56/69, 81%) were published after 2005. The 69 studies consisted of 28 observational studies (23 retrospective studies, 2 cross-sectional studies, 1 prospective time series analysis, 1 prospective study with interviews, and 1 cohort study), 13 review articles, 8 practice experiences, 6 before-after studies, 3 surveys, 3 controlled trials (1 randomized controlled trial, 1 randomized crossover study, and 1 nonrandomized controlled trial), 2 economic evaluations, 2 focus group studies, 1 scenario-based simulation study, 1 study describing a draft for an algorithm to classify information automatically, 1 study discussing a workshop, and 1 study describing a comparative study on standards in the DA field.
For a functional CDSS, accurate and consistent documentation of patient allergy information is necessary. Currently, there is no agreement about what needs to be recorded and how to do so []. In 1964, Mills addressed the importance of capturing information on allergies and drugs, proposing a new checklist that served as a guideline in hospitals []. A similar initiative was taken more recently by Burrell et al [], who introduced a pharmacist-driven protocol in a hospital to improve the completeness of DA or intolerance documentation. A review of medical notes in a general district hospital demonstrated poor documentation practices where 97.4% (114/117) of drug allergy boxes were only partially completed and 2.6% (3/117) had nothing documented []. In another analysis comparing two oncology wards, one ward showed 100% consistency, while the other ward demonstrated drug charts with allergy entries in 82.4% of cases, of which only 68.8% corresponded to information in the medical notes []. Failure to accurately document DAs may lead to prescribing and administering medications that could be harmful to the patient. Besides accurate documentation, correct patient identification with linking of medication information with patient DA information, for example wristband barcoding, is required for real-time CDS [].
Lopez-Gonzalez et al [] reviewed factors for not reporting ADRs and found that the most prominent factor associated with underreporting was ignorance based on the fact that physicians often think that only severe ADRs need to be reported. Secondary factors, such as the hierarchical nature of hospital culture combined with stressful working conditions, also contributed to prescribing errors []. Moreover, at the time of documenting, there is often no clear distinction between a real allergy-related ADR and other minor reactions [].
Inaccurate or outdated DA information can also be a factor that influences the functioning of a CPOE system with CDS. Rimawi et al [] observed a small use case (150 patients) that was documented as intolerant to penicillin even though a negative penicillin skin test was observed. Of these, 36% (20/55) patients who revisited the medical center within the year were redocumented as having a penicillin allergy without proper indication.
Porter et al [] demonstrated that without healthcare personnel re-asking and validating information, there is a significant risk of error at the decision step of ordering or prescribing medications. Additionally, side effects related to the drug’s primary pharmacological effect are sometimes misinterpreted and documented as DAs, resulting in inaccurate DA information.
The more complete and accurate DAs recorded in patients’ EHRs, the greater the potential of CPOE systems with CDS to improve patient safety and reduce medication-related costs [].
Information on how allergy data are structured or coded in EHRs is scarce. Slight et al [] recently stated that the US government has not yet specified what standard terminologies should be used to structure allergy information.
A first approach is to enter information concerning DAs in free text. In order to use free text information for CDSS, natural language processing is applied taking the context into consideration []. Currently, this technique is not widely used because of the difficult nature of natural language processing.
A review of the available literature indicates that different coding systems are used for documenting DA information in EHRs, including International Classification of Diseases (ICD) [,], Systematized Nomenclature of Medicine, Clinical Terms (SNOMED CT) [], RxNorm, and National Drug File-Reference Terminology (NDF-RT) []. Sometimes mapped coding schemes are used to add functionality. Benkhaial et al [] used the Anatomical Therapeutic Chemical classification for mapping all drugs belonging to a specific ICD group. However, mapping might not always be this simple. Bernstein [] recently indicated in a Danish use case that there is not yet a clear consensus regarding the alert information concept (eg, drug alerts) and how drug allergy or other allergy subtypes are linked to that concept.
Goss et al [] performed a comparative study of the SNOMED CT, NDF-RT, Medication Dictionary for Regulatory Activities, unique ingredient identifier, and RxNorm standards for encoding allergy information. The qualitative part of their study demonstrated that SNOMED CT had the most desirable characteristics, including concept coverage, subset capabilities, and vocabulary structure. The quantitative part showed that RxNorm had the highest concept coverage to represent drug allergens, followed by unique ingredient identifier, SNOMED CT, NDF-RT, and Medication Dictionary for Regulatory Activities. SNOMED CT was the only coding system capable of representing unique concepts to encode ignorance of allergies.
The option to have an entry to indicate the ignorance of allergies is important for patient care because not documenting any DA information does not necessarily mean that there are no known DAs. Abookire et al [] stated that “every hospitalized patient should have DAs entered by the admitting physician (this is a forced entry; 'no allergies' may be entered).”
In the literature, we observed underreporting of the rule bases used to support CDSS. Most CDSS for DA screening are knowledge-based systems supported by evidence-based rule databases. Some organizations use internally developed rule bases [], while others use vendor-supplied rule bases [].
We observed two types of CDSS in the literature: basic and complex. Basic CDSS provide alerts when a prescribed drug is listed in the patient’s DA list. In these systems, rules are implemented to screen for cross-reactivity within and between drug classes []. More complex CDSS use an inference mechanism to generate recommendations specific to a patient by integrating contextual information from the patient’s EHR (eg, previously tolerated administrations of the drug and results from DA tests) [].
As reported by Kuperman et al [], healthcare provider organizations ideally use a combination of vendor-supplied rule sets, which are developed by other organizations, and internally developed rules, which are derived from the literature and national and local consensus on what constitutes best practice. In any system, as medicine evolves and clinical knowledge grows, a timely review of the rule bases is warranted, for example using a Delphi approach to analyze what rules are useful [].
Alert fatigue has been defined as “declining physician responsiveness to a particular type of alert as the physician is repeatedly exposed to that alert over a period of time, gradually becoming ‘fatigued’ or desensitized to it” []. Alert fatigue caused by one type of alert may also lead to declined responsiveness to other types of CDS alerts (eg, drug-drug interaction, DDI, alerts). The state of the art in CDS for DA is such that alerts are not specific enough, resulting in high override rates [,]. Current systems, which generate an alert at the moment of prescribing, have very high override rates of over 90% [,]. The first concern regarding increasing DA alerting rates and overrides was raised by Abookire et al []. This problem has since been investigated in several other studies. Bryant et al [] retrospectively analyzed physician responses to DDI and DA interaction alerts in two university hospitals and reported high override rates in all categories, ie, 92.87% (2280/2455 alerts) in general and 90.86% (1183/1302 alerts) for DAs. No significant difference in override rates was observed between hospitals or between physicians-in-training and residents. Topaz et al [] demonstrated a significant increase in DA alert overrides from 83.3% in 2004 to 87.6% (P<.001) in 2013 in a retrospective longitudinal study of two large academic medical centers. Similarly, Lin et al [] demonstrated an increase in drug allergy override rates from 72 overrides out of 105 alerts (68.6%) in 2001 to 341 overrides out of 420 alerts (81.2%) in 2006.
In a recent observational study, Slight et al [] evaluated DA alerts generated over a 3-year period in a tertiary care teaching hospital and 36 primary care practices and found that in total, 81.10% (128,157/158,023) DA alerts in both settings were overridden. In a retrospective 5-month chart review study conducted by Genco et al [], a similar override rate of 87.4% (153/175) DA alerts was observed and the overall override rate for all types of alerts was 93.51% (12,829/13,719).
When an alert is overridden, analyzing the override reasons can help to understand the specific context. Several studies have reported the reasons frequently given for DA alert nonadherence, including (1) “medication was previously tolerated”; (2) “known DA for which only monitoring is needed”; (3) “the benefit outweighs the risk”; and (4) “alert considered not clinically important” [,,].
Besides evaluating override reasons, the underlying causes of high override rates should be investigated. Hsieh et al [] cited two important causes. The first cause is highly inclusive drug class and drug cross-reactivity mapping, which generates a large number of DA alerts for drugs with only minor potential to cause an allergic reaction. Alert acceptance is more likely when the warning is infrequently encountered [] and when the physician encounters an exact drug match instead of a drug class warning []. The second cause is difficulty maintaining accurate allergy lists because there may not be a clear distinction between immune-meditated allergies and nonimmune-mediated sensitivities, and there is no general consensus on whether both should be included in allergy lists []. The likelihood that alerts will be ignored is related to the low predictive value for allergic drug reactions and inaccurate alerts because of inconsistent information in medical records [,].
Strategies for improving alert specificity and acceptance have been proposed. Horsky et al [] stated that the specificity and contextual relevance of alerts can be increased by periodically reviewing trigger rules, a thorough analysis of performance logs, and maintenance of accurate allergy, problem, and medication lists in EHRs. Additionally, Brodowy et al [] demonstrated a reduction in DA alerts by simply eliminating alerts resulting from inactive ingredients.
The possibility of customizing CDSS to increase alert specificity and alert acceptance has also been reported []. CDSS, where the severity levels for drug or disease interactions can be modified by the physician to exclude alerts at a level considered not relevant, or the use of an on-demand approach that provides decision support only when a physician considers it relevant, could improve alert acceptance []. This may be related to the caregiver status of the person using CDSS. Knight et al [] demonstrated, for example, that nurses are nearly twice as likely to accept an alert compared with a resident (odds ratio [OR] 1.92, 95% CI 1.44-2.57). The usability of the alerts can also be improved by applying human factors design principles [,]. For example, a tabular format for presenting multiple alerts and the grouping of similar information aid in making prescribing decisions []. Designing a workflow with minimum disruptions by only showing critical to high-severity alerts, as suggested by Shah et al [], could also be effective.
In the literature, we found limited results related to the outcomes of CDSS for DA. We did find studies that investigated the number of prescribing errors (PEs) and studies discussing patient outcomes for different types of ADEs, including ADEs originating from DAs.
CPOE systems can help in making fewer PEs, although not all studies quantify this improvement. Benkhaial et al [] did not find a significant difference in the risk of being prescribed a drug potentially inducing an allergy using electronic recording of the allergy via ICD‑10 codes compared with paper records. Oliven et al [] compared the number of PEs between a department using handwritten drug orders with that of a department using CPOE systems and found a reduced number of PEs in the department using CPOE. No significant difference was found between the two departments for DAs. On the contrary, Evans et al [] demonstrated a reduction from 2.46% (28/1136) patients to 0.07% (4/545) patients with adverse events caused by anti-infective agents due to the introduction of a computerized anti-infective management program. Likewise Mahoney et al [] demonstrated a reduction in PE rates related to DAs from 833 in the preimplementation phase to 109 in the postimplementation phase (OR 0.14, 95% CI 0.11-0.17).
When looking at the outcomes for ADEs, Bates et al [] demonstrated a reduction of 55%, from 10.7 events per 1000 patient-days to 4.86 events per 1000 patient-days for nonintercepted serious medication errors, defined as those either resulting in or with the potential to result in ADEs. In a follow-up study [], this rate decreased to 1.1 events per 1000 patient-days after additional refinements of the system. This objective was reached by including a dose selection menu, simple DA and DDI checking, and the requirement that clinicians indicate the route and frequency of drug administration. Hsieh et al [] found that ADEs resulting from overridden DA alerts do not occur frequently (19/320, 5.9%). In this study, none of the ADEs were considered preventable because the overrides were deemed clinically justifiable. There is limited evidence that systems, mostly electronic systems combining CPOE with CDS, for preventing represcription after the occurrence of an ADE (including DAs) are effective [].
The implementation of CDSS can also influence economic outcomes, eg, by decreasing costs related to medication errors [,]. However, information on cost-effectiveness, specifically of CDS for DAs, was not found.
To the best of our knowledge, this is the first systematic review focusing on CDSS in the field of DA. We included 69 articles in our review. The main findings are the problem of incomplete and inaccurate recording of patients’ DA information, the absence of an appropriate standard terminology that guides the rule bases within a CDSS, problems with rule bases, and the low specificity of DA alerts resulting in alert fatigue.
The first key finding was the incomplete or inaccurate documenting of patients’ DA information in medical records. Accurate and comprehensive recording of DA information in EHRs is essential for the proper functioning of CDSS for DA screening. A recurrent problem described in the literature is the absence of documented information on patients’ allergies, which can be interpreted in two ways: (1) the patient has no known allergies or (2) the patient has an allergy to a certain substance that has not yet been documented in the patient record []. Therefore, the absence of any known DA should also be documented in EHR. Besides underreporting of DAs, outdated or inaccurate DA information in EHRs also poses an important problem.
The second key finding was the absence of a generally accepted standard terminology for the structured documentation of allergy information. The use of Anatomical Therapeutic Chemical, ICD, NDF-RT, RxNorm, and SNOMED CT was described in the literature, but limited information was provided about the exact manner of implementation or integration of these coding systems. An evaluation of terminologies by Goss et al [] showed that currently, a combination of RxNorm and SNOMED CT satisfies most criteria for encoding allergies. The use of free text for documenting DA information in EHRs should be discouraged because of the difficulties for CDS. The use of a standard terminology is required for coded exchange of DA information between institutions on a national and international level and for creating exchangeable decision rules based on standard terminologies. Governments have an important role in providing standardized terminologies in the official national languages. Policies and regulations may be required to support the effective use of coding standards in clinical practice.
The third key finding was that all reported CDSS for DA screening were knowledge-based systems requiring timely review of the rule bases to keep CDSS up to date. Ideally, end users and program developers should work together to regularly review the alerts logs and decision rules to reduce the risk of alert fatigue []. This is a continuous process and not a “one and done” step. Both in-house curated knowledge bases and vendor-based rule bases were reported in the literature, and both have their advantages and disadvantages. In an in-house curated knowledge base, flexibility is guaranteed, leading to potentially higher alert specificity, but it requires substantial effort to develop and maintain the rules base. A vendor-based rule base is easily purchased, but it has less flexibility when it comes to changing decision rules. The end user is dependent on the vendor for updates, but the maintenance burden lies with the vendor. A third possibility is the implementation of a hybrid system combining a commercial rule base with internally defined content refinements or decision rules.
The last key finding is the consistently low specificity of DA alerts. This remains an important problem as it causes high override rates, resulting in alert fatigue. Researchers have tried to tackle the problem of alert fatigue by providing on-demand decision support or customizable computer-triggered decision support. Another option is to turn off certain alerts, for example, by looking at the personal preferences of the healthcare provider who can decide to no longer receive a particular type of warning []. It remains difficult to find a good balance between reducing alert overload and keeping patient safety at a high level. A fixed rule base may therefore not always be appropriate; rather, an adaptive CDSS supported by a predictive risk model may be more useful []. Taking contextual factors into consideration as part of the CDS rules may help in increasing the specificity of DA alerts and lowering the rate of alert overrides. This strategy has been successfully applied for increasing the specificity of DDI alerts. Duke et al [,] and Cornu et al [] have developed context-aware DDI alerts based on relevant patient-specific information, resulting in improved alert acceptance.
Future policies should focus on the implementation of standard terminologies to allow standardized coded exchange of DA information on a national and international level and to create exchangeable decision rules.
Information on the effect of CDSS for DAs on patient outcomes was very limited. Thus, future research should focus on evaluating patient outcomes. Hsieh et al [] demonstrated that after overriding DA alerts, none of the resulting ADEs were preventable. However, in their study, only overridden DA alerts were evaluated. It would be interesting to know the number of ADEs that was effectively prevented by CDSS.
It is assumed that implementing DA checking in a CPOE system also has a beneficial financial impact. We did not find any studies specifically related to the economic outcomes of CDS for DAs, but general conclusions about the economic benefits of implementing CPOE systems for the hospitals were documented. At the start, the implementation of a CPOE system requires a large investment, but soon the costs are outweighed by the benefits and result in savings []. However, the cost-effectiveness of CDSS for DA should be further investigated.
Current systems often warn about all possible cross-reactions, although the substance-specific risk should be estimated and the severity of the alert may change as a function of the possibility of cross-reaction (eg, likely, possible, or unlikely). Future research should explore strategies for optimizing cross-reactivity rules and enhancing alert specificity.
This study has several limitations. First, because of the heterogeneity across the study objectives, study designs, study populations, and reported results, a meta-analysis could not be performed. Second, different study designs require a different methodological framework for assessing bias. The heterogeneity in quantitative and qualitative study designs, reviews, and reports did not allow for a comprehensive and consistent evaluation of bias. This may limit the generalization of the results, but it allowed us to take a broader view of all relevant research in the field of CDS for DAs. Third, we excluded non-English papers, which may constitute selection bias. Additionally, 9 papers were added based on expert opinion because they included information relevant to this review. These extra articles were not retrieved with the query because they included keywords other than those included in the search query. Adjusting the query was not feasible because the keywords were often too general (eg, medication safety), which would result in a high number of irrelevant articles. Finally, publication bias cannot be excluded. We observed a high number of studies published in the US setting, which may lower the international relevance of the results. However, we believe that the findings of our review are relevant in an international context.
This review shows that CPOE systems with CDS for DA screening are perceived as useful in clinical practice. Nevertheless, there are some important problems associated with their use. First, it is not yet clear how and where to document DA information in patients’ EHRs. Second, there is a lack of proper coding terminology for documenting allergies. A major problem with current systems is that alerts are not specific enough, resulting in high override rates and consecutive alert fatigue. Future research should focus on strategies to improve alert specificity and evaluating patient and economic outcomes.
LL, SVL, and PC participated in the design of the study. LL and SVL conducted the search. LL, SVL, and PC were responsible for critical evaluation of the search results. LL, SVL, and PC drafted the manuscript. MN, SS, AGD, and PC provided expert opinion on the topic. All authors critically evaluated the article and gave their final approval before submission.
None declared.
Table with author names, year of publication, study design, aim of the study, and key findings of articles included in the review.
PDF File (Adobe PDF File), 334KB
| ADE: adverse drug event |
| ADR: adverse drug reaction |
| CDS: clinical decision support |
| CDSS: clinical decision support system(s) |
| CINAHL: Cumulative Index to Nursing and Allied Health Literature |
| CPOE: Computerized Physician Order Entry |
| DA: drug allergy |
| DDI: drug-drug interaction |
| EHR: electronic health record |
| ICD: International Classification of Diseases |
| NDF-RT: National Drug File-Reference Terminology |
| NLP: natural language processing |
| PE: prescribing error |
| OR: odds ratio |
| PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
| SNOMED CT: Systematized Nomenclature of Medicine, Clinical Terms |
| UNII: unique ingredient identifier |
Edited by G Eysenbach; submitted 15.06.17; peer-reviewed by H Seidling, R Robinson; comments to author 09.09.17; revised version received 25.05.18; accepted 21.06.18; published 07.09.18
©Laura Légat, Sven Van Laere, Marc Nyssen, Stephane Steurbaut, Alain G Dupont, Pieter Cornu. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 07.09.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
0 
Star