Original Article

Potential Drug Interactions in Adults Living in Manaus: A Real-World Comparison of Two Databases, 2019

10.4274/tjps.galenos.2021.96603

  • Tayanny Margarida Menezes Almeida BIASE
  • Giulia Sartori BRUNIÉRI
  • Marcus Tolentino SILVA
  • Taís Freire GALVÃO

Received Date: 24.06.2021 Accepted Date: 25.10.2021 Turk J Pharm Sci 2022;19(5):543-551

Objectives:

Drug information systems are commonly used by professionals to assist in the identification of drug interactions and to ensure the safe use of medications. Real-world evidence about the comparison of different drug interaction sources is scarce. We aimed to compare two drug interaction databases to identify interactions in a population-based survey.

Materials and Methods:

This is a cross-sectional study based on a previous survey performed in the city of Manaus, Brazil, in 2019. We included adults aged 18 years and over, who used two or more medicines 15 days before the interview. To assess potential drug interactions, we searched Micromedex and UpToDate databases. The primary outcome was the prevalence of potential drug interactions in each database. Weighted Kappa statistics were calculated to assess agreement on the presence of drug interaction, documentation and severity.

Results:

A total of 752 participants were included in the study. The prevalence of drug interactions was 43.8% [95% confidence interval (CI): 40.2, 47.3%] in UpToDate and 30.2% (95% CI: 26.9, 33.5%), in Micromedex. The agreement related to drug interactions between the two databases was fair (Kappa: 0.631). For severity (Kappa: 0.398) and documentation (Kappa: 0.311), the agreement was poor.

Conclusion:

Agreement among compared databases was sub-optimal. Better quality and transparency of evidence available in drug interaction sources are needed to support informed healthcare professionals’ decision.

Keywords: Drug interactions, software, documentation, classification, population health

INTRODUCTION

Proper identification of drug interactions is essential to ensure the safety and effective use of medications.1 Pharmacists play a significant role in guiding drug therapy and the rational use of medicines in different levels of health care.2,3,4 Clinical decision support software systems are commonly used in hospitals and in the community to assist pharmacists in identifying drug interactions of clinical significance.5 These systems that are used by pharmacists and other health professionals to identify interactions have evolved to integrate computerized screening banks for drug interactions, clinical information, and other drug-related problems.6,7,8

Although these software tools can increase the ability of pharmacists to detect clinically significant interactions, these systems are far from fail-safe.5,9 Optimal clinical decision support software should have a balance between low and high-risk alerts.10,11 Excessive warnings can cause tiredness and suppression of clinically significant interactions, while the warning shortage can increase the risk of ignoring possible damage and decrease the user’s perception in relation to the reliability and usefulness of the system.12

Searching for drug interaction is not a trivial step, as there is a wide variety of search sources, from package inserts to medicines, scientific literature and various databases and websites. This diversification of sources makes the search difficult, when looking for reliable information about drug interactions and ensuring patients receive safe drug therapy. Assessments of software performance to identify potential drug interactions mainly focus on hospital environment or are based on theoretical scenarios,13,14,15,16,17 involving patients with multimorbidity, in polypharmacy and old age.18 This highly selected population, usually from hospitals19 may not reflect the reality of multiple drug use and possible interactions by the general population. A limited number of studies have investigated the prevalence of potential drug interactions in the general population. Further assessments and comparisons of sources for assessing potential drug interactions in the community can add valuable information, especially in less developed settings.18,20 We compared two systems of drug interaction for a population-based survey.


MATERIALS AND METHODS


Study design

This is a cross-sectional study based on a previous survey performed in the city of Manaus (Brazil) from April to June 2019.21


Setting

The study setting was Manaus, the city capital of the State of Amazonas, with an estimated population of 2.219.580 people in 2020.22

Participants who had taken two or more medications in the previous 15 days were assessed for the presence of potential drug interactions.


Participants

The adults, who were included in the study were those at the age and over 18 years who self-reported using two or more medicines 15 days before the interview. In the original survey, the participants were selected by probabilistic sampling carried out in three stages: (1) Draw of the census tracks of the city, (2) systematic selection of households, and (3) random selection of the individual interviewed based on sex and age quotas.21 The sample size was calculated as 2.300 participants for the main study - unrestricted to individuals who took at least two medicines - considering 20% of healthcare usage, confidence level of 95%, absolute precision of 2%, design effect of 1.5, and 2.106.355 in habitants aged ≥18 years living in Manaus.21 Based on the results in the present analysis, post-hoc sample size was calculated.


Variables

The primary outcome was the prevalence of potential drug interactions. For clarity, in this study, we use “drug interactions” as a synonym of “potential drug interaction”. The independent variables were sex (men, women), age (in years, categorized as: 18-24, 25-34, 35-44, 45-59, ≥60), economic classification (A/B, C, D/E, according to the 2018 Brazilian Economic Classification Criteria, in which A is the wealthiest and E is the poorest class),23 education (higher education or beyond, high school, elementary school, below elementary school), health status (good, fair, poor), chronic diseases (yes, no), number of drugs used in the last 15 days (2, 3-4, ≥5).


Data sources and measurements

Experienced interviewers visited the participants’ households in this study. The interviews were georeferenced, and the data collected were stored in e-devices. The use of medicines was assessed by the question: “Have you taken any medications in the last 15 days (two weeks)?” and its possible answers: “Yes” or “No”. If yes, the name of the medication was registered as informed by the participant and could be confirmed by checking the medication packages and/or available medical prescriptions. The data were compiled in the Microsoft Excel® 2010 software and the drugs were coded according to the Brazilian Common Denomination and, subsequently, according to the World Health Organization’s (WHO) Anatomical Therapeutic Classification System (ATC). Ineligible drugs or without an ATC code were classified as “uncoded”.24 From February to March 2021, we searched IBM Micromedex® Drug Interaction Checking25 and Lexicomp® Drug Interaction from UpToDate®26 to identify the drug interactions. These databases are commonly used to investigate drug interactions in clinical practice and subscription was available for research team, allowing present investigation.

All ATC-coded drugs were assessed in each database to verify drug interactions. If positive for drug interactions, the combination of drugs, severity and documentation was recorded according to the classification of the database used. Commercial combinations of drugs unavailable as an association in the database were searched by including each substance separately and interaction was recorded if occurred between the association and the other medicine. Both databases classify drug interactions according to severity and documentation. Micromedex classify severity of drug interactions as: contraindicated (medications are contraindicated for concomitant use), major (the interaction may be life-threatening and/or require medical intervention to reduce or avoid serious adverse effects), moderate (the interaction may result in the health problem exacerbation and/or require treatment change), and minor (the interaction would result in limited clinical effects).25 In this database, documentation is categorized into the following: excellent (interaction confirmed from controlled studies), good (the interaction exists, but there is absence of properly controlled studies), and fair (the available documentation is unsatisfactory, but pharmacological considerations lead clinicians to suspect the existence of the interaction).

UpToDate database defines severity as: major (effects may result in death, hospitalization, permanent injury, or therapeutic failure), moderate (medical intervention needed to treat effects, effects do not meet criteria as major), and minor (effects would be considered tolerable in most cases, no need for medical intervention). Documentation reliability is defined as excellent, good, fair, and poor. It also assigns a risk rating, which is a rapid indicator regarding how to respond to the interaction: A (unknown interaction), B (minor, no action required), C (moderate, monitor therapy), D (main, consideration to modify therapy) or X (contraindicated, avoid combination).26

To allow comparability of the databases, “contraindicated” severity category from Micromedex was regrouped in “major”; “poor” documentation from UpToDate were rated “fair”; and interactions of risk “A” from UpToDate were disregarded (considered as no drug interaction).


Bias

The data were collected by a team of experienced and trained interviewers.21 The participant could optionally present the medicine package mentioned in the interview to confirm the data and avoid misclassifications. To ensure the encoding of all medicines according to the ATC, herbal, and homeopathic products were excluded from the research.


Ethics approval

This study was approved by the Ethics Committee of the Federal University of Amazonas (opinion no: 3,102,942), on December 28, 2018 (Certificate of Presentation for Ethical Appreciation 04728918.0.0000.502020). All participants signed a term of free and informed consent.


Statistical analysis

Participants were described statistically according to independent variables. Frequency of drug interactions, severity, and documentation classifications in each database were described, as well as more relevant disagreements on interactions between them (major severity or excellent documentation in one database was not considered a drug interaction in the other).

Weighted Kappa statistics were calculated to assess agreement on drug interaction, documentation, and severity classifications between both databases. Kappa values >0.75 were considered excellent agreement beyond chance, between <0.75-0.40 represented fair agreement, and values <0.40 denoted poor agreement beyond chance.27


RESULTS

From 2.321 interviewed, 752 participants were taking two or more medicines and were included in the study. Most participants were women (58.6%), aged 45-59 years (27.3%), belonged to economic classification C (low middle class, 54.5%), had higher (49.2%), self-reported good health status (49.7%), had chronic diseases (76.2%) and used only two drugs (49.3%; Table 1). The prevalence of drug interactions in UpToDate was 43.8% [95% confidence interval (CI): 40.2, 47.3%] and in Micromedex, 30.2% (95% CI: 26.9, 33.5%).

A total of 344 unique participants was reported with the presence of drug interactions in one or in both databases. More patients had drug interactions according to UpToDate (n: 329) Micromedex (n: 227); and 212 patients with drug interactions were identified by both databases (Figure 1).

The agreement on drug interactions between the two databases was fair (Kappa: 0.631). Using UpToDate, over half of the interactions were classified as moderate severity (61.2%), while Micromedex classified most as major severity (62.6%). Between the databases, the agreement on the severity classification was evaluated as poorly with a Kappa value of 0.398. In both databases, more than half of the interactions were based on fair documentation (UpToDate: 70.6%; Micromedex: 61.4%) and documentation agreement was poor (kappa: 0.311) (Table 2). The post-hoc minimum sample size based on this agreement would be 94 patients.

Among the more relevant classification disagreements identified between the databases, 27 different discordant drug interactions were reported with major severity or with excellent documentation in one database and not detected in the other (Table 3). Out of these discrepant classifications, 20 were present only in UpToDate (13 with major severity, 7 with excellent documentation), and seven present only in Micromedex (6 with major severity, 1 with excellent documentation). Most frequent drug interactions shown in UpToDate were related to major severity interactions: Carisoprodol-orphenadrine (n: 10), chlorpheniramine-orphenadrine (n: 8), and ciprofloxacin-ibuprofen (n: 3); and in Micromedex, acetylsalicylic acid-hydrochlorothiazide (n: 3). The paracetamol-tramadol interaction (n: 3) presented excellent documentation and minor severity in UpToDate. All major severity drug interactions were related to fair documentation, according to UpToDate and, based on Micromedex, the major severity interactions were fair (n: 4) and good documentation (n: 2). The drug interaction of excellent documentation (minor severity) was amitriptyline-estradiol (n: 1), according to Micromedex and not present in UpToDate. Drug interactions with excellent documentation ranged from minor (n: 7) to moderate (n: 1) severities, in UpToDate and not present in Micromedex. Most major severity interactions in UpToDate belonged to X risk classification (9 of 13), and minor severity interactions were classified as B risk (6 of 7) (Data not shown in Tables).

In UpToDate, orphenadrine appeared in seven different drug interactions that were not similarly regarded in Micromedex. Moreover, it was the most frequent drug involved in these discordant interactions (Table 3). Non-steroidal anti-inflammatory drugs were the main ones in the drug interactions, present in nine different drug interactions, and additive effects between medicines were the main mechanism of the interactions (n: 10).


DISCUSSION

Drug interactions were present in 3 to 4 people among 10 adults living in Manaus, according to the consulted databases, showing a higher frequency in UpToDate than Micromedex. Agreement on the identification of drug interactions between the databases was considered fair, while severity and documentation classifications of these interactions were poor agreements. Depending on the source used, a lot of work may result from screening drug interaction in the population setting.

Due to the cross-sectional nature of this study, participants were not monitored over time to confirm the occurrence of adverse events due to drug interactions. Based on a list of self-reported medicines used by the participants 15 days before the interview, we assessed drug interactions and did not clinically investigate these interactions. This limitation can make our results prone to memory and information biases. The databases are periodically updated and may have undergone changes during or after the study, also potentially affecting our results.

In agreement with our findings, a higher prevalence of drug interactions was observed, when UpToDate was the reference for interactions. In the United States, an assessment performed in 2012 by screening 240 patients’ medication profiles showed almost twice as many drug interactions using Micromedex.28 In Türkiye, a study with 80 renal transplant recipients observed similar results, presenting almost twice the drug interactions identified in UpToDate in compared to Micromedex.29 The use of different databases shows the lack of agreement on the number of possible drug interactions in different investigations, including ours, which raises concerns about the clinical relevance of checking multiple sources. Excessive alerts in clinical practice can lead to high workloads for healthcare professionals and mask important alerts.30,31

Micromedex and UpToDate had a fair agreement on the identification of drug interactions. Similar results were observed in previous studies that investigated agreement on multiple sources of drug interactions in clinical practice, including drugs for metabolic disorders, antiretrovirals, antimicrobials, and psychiatric drugs.13,16,32,33 A study involving common therapeutic combinations of drugs for bipolar disorder tested 125 pairs of drug interactions in six databases in 2019, showing low agreement among the databases assessed.16 Assessment of drug interactions in an Indian hospital using Epocrates and Medscape presented a significant discrepancy between the severity categories of drug interactions in 2015.34 A retrospective analysis in an intensive care unit in Germany, including prescriptions for transplant patients, used five databases to identify drug interactions and only 9% interactions were identified by all of them, showing discrepancies in the overall performance of these tools.35

When comparing the documentation and the severity classifications, the agreement between Micromedex and UpToDate was poor. Based on Micromedex, the interactions identified were more frequently rated major severity, whereas, based on UpToDate, they were more frequently rated minor or moderate. Most drug interactions relied on fair documentation in both databases. The assessment of drug interactions involving 78 patients from an Australian hospital in 2018 was compared using three databases: Stockley’s Drug Interactions, Micromedex and YouScript. The results were low agreement on the severity classification of the consulted interactions.36 Cross-sectional systematic comparative study using drug pairs, conducted in the United Arab Emirates in 2020, identified disagreements on the severity and documentation of drug interactions between eight databases: Micromedex reported a greater number of interactions related to major severity compared to other databases (Portable Electronic Physician Information Database, UpToDate, Medscape, Drugs.com, Stockley’s Drug Interactions, Drug Interactions Analysis & Management: Facts and comparisons and British National Formulary).37

Most of the drugs involved in discordant drug interactions were over-the-counter, such as ibuprofen, diclofenac, paracetamol, and dipyrone. Drugs for treatment of chronic diseases, such as hypertension, heart disease, and diabetes were also frequent. Among the discordant drug interactions between the two databases analyzed, most were identified from UpToDate. More frequent management showed that simultaneous use should be avoided, and the potential result of the interactions consisted mainly of enhancing or decreasing therapeutic effects with mostly unknown mechanisms of action. Mostly, the alerts were based on minor severity and fair documentation, promoting alerts that were not considered clinically relevant by the health team.

Healthcare professionals are under constant pressure to provide appropriate care by making clinical decisions daily, with the help of drug information databases. The choice of the database can impact patient care and its outcomes.38 Such sources, usually provided on a subscription-basis, should be periodically reviewed to improve relevant information based on high-quality evidence from real-world data.

Investments on well-designed studies to determine the incidence, outcomes, and risk factors related to the patients affected by drug interactions are needed to support the provided recommendations. Algorithms to define systematic and clear evidence assessment processes to assess the risk and severity of drugs should ideally be integrated into these electronic systems.39 This low quality of evidence potentially overestimates the severity of drug interactions and leads to overriding warnings when they are considered less serious, which can gradually neglect serious drug interactions.33 These disagreements disadvantage healthcare professionals when making clinical decisions in cases of drug interactions in which the patient’s condition justifies the use of both drugs that interact with each other, especially when there are no alternatives available.33,37

We also observed that the search for drugs available as commercial combinations may interfere with the result of drug interactions in the database, such as those including dipyrone and orphenadrine, commonly used combined in Brazil. Since these sources are based on developed settings, these fixed combinations are usually not included in the databases and may represent a higher burden in searching for interaction. Professionals should also be aware, when searching for the active ingredients separately, because it is possible to find interactions between active ingredients contained in a combination.


CONCLUSION

As for the identification of drug interactions, slight agreement was observed between UpToDate and Micromedex in this real-world analysis, indicating poor agreement on severity and documentation of drug interactions. Consulting multiple databases to identify drug interactions may increase healthcare professionals’ workload as well as undetermined clinical outcomes for patients. Better-qualified sources for obtaining drug information are in need so that they can provide better support for health professionals and patients.


Ethics

Ethics Committee Approval: This study was approved by the Ethics Committee of the Federal University of Amazonas (Opinion No. 3,102,942), on December 28, 2018 (Certificate of Presentation for Ethical Appreciation 04728918.0.0000.502020).

Informed Consent: All participants signed a term of free and informed consent.

Peer-review: Externally peer-reviewed.

Authorship Contributions

Concept: T.F.G., M.T.S., Design: T.F.G., M.T.S., Data Collection or Processing: T.M.M.A.B., G.S.B., Analysis or Interpretation: T.M.M.A.B., G.S.B., T.F.G., M.T.S., Literature Search: T.M.M.A.B., G.S.B., Writing: T.M.M.A.B., G.S.B., T.F.G., M.T.S.

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: This work was supported by the National Council for Scientific and Technological Development (grants no: 404990/2013-4 and 448093/2014-6). TFG receives a productivity scholarship from CNPq (grant no: 310238/2020-0).

Images

  1. Nguyen T, Liu X, Abuhashem W, Bussing R, Winterstein AG. Quality of evidence supporting major psychotropic drug-drug interaction warnings: a systematic literature review. Pharmacotherapy. 2020;40:455-468.
  2. Zaal RJ, den Haak EW, Andrinopoulou ER, van Gelder T, Vulto AG, van den Bemt PMLA. Physicians’ acceptance of pharmacists’ interventions in daily hospital practice. Int J Clin Pharm. 2020;42:141-149.
  3. Ravn-Nielsen LV, Duckert ML, Lund ML, Henriksen JP, Nielsen ML, Eriksen CS, Buck TC, Pottegård A, Hansen MR, Hallas J. Effect of an in-hospital multifaceted clinical pharmacist intervention on the risk of readmission: a randomized clinical trial. JAMA Intern Med. 2018;178:375-382.
  4. Ylä-Rautio H, Siissalo S, Leikola S. Drug-related problems and pharmacy interventions in non-prescription medication, with a focus on high-risk over-the-counter medications. Int J Clin Pharm. 2020;42:786-795.
  5. Bagri H, Dahri K, Legal M. Hospital pharmacists’ perceptions and decision-making related to drug-drug interactions. Can J Hosp Pharm. 2019;72:288-294.
  6. Peabody J, Acelajado MC, Robert T, Hild C, Schrecker J, Paculdo D, Tran M, Jeter E. Drug-drug interaction assessment and identification in the primary care setting. J Clin Med Res. 2018;10:806-814.
  7. Peabody J, Tran M, Paculdo D, Schrecker J, Valdenor C, Jeter E. Clinical utility of definitive drug-drug interaction testing in primary care. J Clin Med. 2018;7:384.
  8. Warholak TL, Hines LE, Saverno KR, Grizzle AJ, Malone DC. Assessment tool for pharmacy drug-drug interaction software. J Am Pharm Assoc. (2003). 2011;51:418-424.
  9. Saverno KR, Hines LE, Warholak TL, Grizzle AJ, Babits L, Clark C, Taylor AM, Malone DC. Ability of pharmacy clinical decision-support software to alert users about clinically important drug-drug interactions. J Am Med Inform Assoc. 2011;18:32-37.
  10. Coleman JJ, van der Sijs H, Haefeli WE, Slight SP, McDowell SE, Seidling HM, Eiermann B, Aarts J, Ammenwerth E, Slee A, Ferner RE. On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop. BMC Med Inform Decis Mak. 2013;13:111. Erratum in: BMC Med Inform Decis Mak. 2013;13:122. Ferner, Robin E [corrected to Slee, Ann]; Slee, Ann [corrected to Ferner, Robin E].
  11. Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood). 2010;29:655-663.
  12. Hedna K, Andersson ML, Gyllensten H, Hägg S, Böttiger Y. Clinical relevance of alerts from a decision support system, PHARAO, for drug safety assessment in the older adults. BMC Geriatr. 2019;19:164.
  13. Suriyapakorn B, Chairat P, Boonyoprakarn S, Rojanarattanangkul P, Pisetcheep W, Hunsakunachai N, Vivithanaporn P, Wongwiwatthananukit S, Khemawoot P. Comparison of potential drug-drug interactions with metabolic syndrome medications detected by two databases. PLoS One. 2019;14:e0225239.
  14. Riu-Viladoms G, Carcelero San Martín E, Martín-Conde MT, Creus N. Drug interactions with oral antineoplastic drugs: the role of the pharmacist. Eur J Cancer Care (Engl). 2019;28:e12944.
  15. Ramos GV, Guaraldo L, Japiassú AM, Bozza FA. Comparison of two databases to detect potential drug-drug interactions between prescriptions of HIV/AIDS patients in critical care. J Clin Pharm Ther. 2015;40:63-67.
  16. Monteith S, Glenn T, Gitlin M, Bauer M. Potential drug interactions with drugs used for bipolar disorder: a comparison of 6 drug interaction database programs. Pharmacopsychiatry. 2020;53:220-227.
  17. Fung KW, Kapusnik-Uner J, Cunningham J, Higby-Baker S, Bodenreider O. Comparison of three commercial knowledge bases for detection of drug-drug interactions in clinical decision support. J Am Med Inform Assoc. 2017;24:806-812.
  18. Kardas P, Urbański F, Lichwierowicz A, Chudzyńska E, Czech M, Makowska K, Kardas G. The prevalence of selected potential drug-drug interactions of analgesic drugs and possible methods of preventing them: lessons learned from the analysis of the real-world national database of 38 million citizens of Poland. Front Pharmacol. 2020;11:607852.
  19. Mousavi S, Ghanbari G. Potential drug-drug interactions among hospitalized patients in a developing country. Caspian J Intern Med. 2017;8:282-288.
  20. Burato S, Leonardi L, Antonazzo IC, Raschi E, Ajolfi C, Baraghini M, Chiarello A, Delmonte V, Di Castri L, Donati M, Fadda A, Fedele D, Ferretti A, Gabrielli L, Gobbi S, Lughi S, Mazzari M, Pieraccini F, Renzetti A, Russi E, Scanelli C, Zanetti B, Poluzzi E. Comparing the prevalence of polypharmacy and potential drug-drug interactions in nursing homes and in the community dwelling elderly of Emilia Romagna Region. Front Pharmacol. 2021;11:624888.
  21. Silva MT, Nunes BP, Galvao TF. Use of health services by adults in Manaus, 2019: protocol of a population-based survey. Medicine (Baltimore). 2019;98:e15769.
  22. Instituto Brasileiro de Geografia e Estatística. Cidades@. Manaus. IBGE. Accessed August 25, 2021. Available from: https://cidades.ibge.gov.br/brasil/am/manaus/panorama
  23. Brazilian Association of Research Companies. [Brazil’s Economic Classification Criteria 2018]. ABEP. Accessed May 11, 2019. Available from: http://www.abep.org/criterio-brasil
  24. WHO Collaborating Centre for Drug Statistics Methodology (WHOCC). ATC/DDD Index. Norwegian Institute of Public Health. Accessed 13 December 2019, 2019. Available from: https://www.whocc.no/atc_ddd_index/?code=J&showdescript
  25. Micromedex. Drug Interactions. Truven Health Analytics. Available from: https://www.micromedexsolutions.com/micromedex2/librarian
  26. Uptodate. Lexicomp. Drugs & Drug Interaction. Available from: https://www.uptodate.com/home/drugs-drug-interaction
  27. Fleiss JL, Levin B, Paik MC. The Measurement of interrater agreement. Statistical Methods for Rates and Proportions. 3 ed. Wiley; 2003:598-626.
  28. Smithburger PL, Kane-Gill SL, Seybert AL. Drug-drug interactions in the medical intensive care unit: an assessment of frequency, severity and the medications involved. Int J Pharm Pract. 2012;20:402-408.
  29. Tecen-Yucel K, Bayraktar-Ekincioglu A, Yildirim T, Yilmaz SR, Demirkan K, Erdem Y. Assessment of clinically relevant drug interactions by online programs in renal transplant recipients. J Manag Care Spec Pharm. 2020;26:1291-1296.
  30. Nanji KC, Seger DL, Slight SP, Amato MG, Beeler PE, Her QL, Dalleur O, Eguale T, Wong A, Silvers ER, Swerdloff M, Hussain ST, Maniam N, Fiskio JM, Dykes PC, Bates DW. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc. 2018;25:476-481.
  31. Muhič N, Mrhar A, Brvar M. Comparative analysis of three drug-drug interaction screening systems against probable clinically relevant drug-drug interactions: a prospective cohort study. Eur J Clin Pharmacol. 2017;73:875-882.
  32. Vivithanaporn P, Kongratanapasert T, Suriyapakorn B, Songkunlertchai P, Mongkonariyawong P, Limpikirati PK, Khemawoot P. Potential drug-drug interactions of antiretrovirals and antimicrobials detected by three databases. Sci Rep. 2021;11:6089.
  33. Liu X, Hatton RC, Zhu Y, Hincapie-Castillo JM, Bussing R, Barnicoat M, Winterstein AG. Consistency of psychotropic drug-drug interactions listed in drug monographs. J Am Pharm Assoc (2003). 2017;57:698-703.e2.
  34. Kannan B, Nagella AB, Sathia Prabhu A, Sasidharan GM, Ramesh AS, Madhugiri V. Incidence of potential drug-drug interactions in a limited and stereotyped prescription setting - comparison of two free online pharmacopoeias. Cureus. 2016;8:e886.
  35. Amkreutz J, Koch A, Buendgens L, Trautwein C, Eisert A. Clinical decision support systems differ in their ability to identify clinically relevant drug interactions of immunosuppressants in kidney transplant patients. J Clin Pharm Ther. 2017;42:276-285.
  36. Meslin SMM, Zheng WY, Day RO, Tay EMY, Baysari MT. Evaluation of clinical relevance of drug-drug interaction alerts prior to implementation. Appl Clin Inform. 2018;9:849-855.
  37. Shariff A, Belagodu Sridhar S, Abdullah Basha NF, Bin Taleth Alshemeil SSH, Ahmed Aljallaf Alzaabi NA 4th. Assessing consistency of drug-drug interaction-related information across various drug information resources. Cureus. 2021;13:e13766.
  38. Clauson KA, Marsh WA, Polen HH, Seamon MJ, Ortiz BI. Clinical decision support tools: analysis of online drug information databases. BMC Med Inform Decis Mak. 2007;7:7.
  39. Hines LE, Malone DC, Murphy JE. Recommendations for generating, evaluating, and implementing drug-drug interaction evidence. Pharmacotherapy. 2012;32:304-313.