Original Article

Validation of a Knowledge Test in Turkish Patients on Warfarin Therapy at an Ambulatory Anticoagulation Clinic


  • Meltem TÜRKER
  • Mesut SANCAR
  • Nazlıcan UÇAR
  • Osman UZMAN
  • Pınar AY
  • Ömer KOZAN
  • Betul OKUYAN

Received Date: 03.09.2020 Accepted Date: 21.10.2020 Turk J Pharm Sci 2021;18(4):445-451 PMID: 34496551


This study aimed to evaluate the validity and reliability of an oral anticoagulation knowledge (OAK) test in Turkish patients on warfarin therapy at an anticoagulant outpatient clinic.

Materials and Methods:

This study was conducted at an ambulatory anticoagulation clinic and included patients older than 18 years who had been using warfarin for at least six months. Patients’ demographic and clinical data were collected. Internal consistency was calculated using the Kuder-Richardson 20 (KR-20) coefficient, and the test-retest reliability of the Turkish version of the OAK test was assessed.


Patients’ mean age was 59.83±11.93 (26-90) years (n=240; 133 women). The mean score of the OAK test was 14.19±3.01. The test-retest reliability of the scale (n=30) was moderate for the total score (p<0.001). The KR-20 value, a measure of internal consistency, was 0.671. Patients of a younger age and higher educational level were more likely to have higher levels of anticoagulation knowledge than patients of an older age and lower education level (p<0.05 for both comparisons).


The Turkish version of the OAK test can be used to determine the patients’ knowledge on oral anticoagulation.

Keywords: Warfarin, anticoagulant, knowledge, pharmacist


Warfarin is mostly used as an oral anticoagulant for the prophylaxis and management of primary and secondary thromboembolism.1,2,3 Although, recently, new oral anticoagulants have been developed, warfarin remains the most commonly prescribed vitamin K antagonist in the clinical settings.2,3,4 However, patients are subject to risks when on warfarin.1,2 Bleeding is a common adverse event that occurs at an annual rate of 7-8% among patients on warfarin. Additionally, it was reported that the most common cause of drug-induced emergency department admissions was related to warfarin use.5 Warfarin’s narrow therapeutic range and wide dose-response variability should be considered when managing its use in treatment.1 Patients treated with warfarin should be closely monitored to ensure their adherence to warfarin therapy and detect and prevent adverse events.6 Serious problems may occur if patients adhered poorly to medications, especially those with a narrow therapeutic range, such as warfarin. Missed doses decrease the efficacy of the medication, and overdoses cause various adverse events.7

Wang et al.8 showed a relationship between the level of medication adherence and level of medication knowledge among patients using warfarin. In the study conducted in Turkey, poor medication adherence of patients receiving anticoagulant therapy was associated with a poor time in therapeutic range (TTR), poor warfarin knowledge, and higher bleeding score.9 Based on the findings of a multicenter study conducted in Turkey, poor knowledge of potential warfarin-food interactions was more common in older patients.10

To our best knowledge, there is no valid and reliable anticoagulation knowledge test in Turkish. Few validated anticoagulation knowledge tests exist for this purpose.11,12,13 The oral anticoagulation knowledge (OAK) test used in our study was short compared with other questionnaires11,12 and is commonly used as a reliable and valid tool in the United States,14 Malaysia,15 and Brazil1 to identify and evaluate patients’ knowledge of anticoagulation. This study aimed to evaluate the validity and reliability of a knowledge test in Turkish patients on warfarin therapy at an ambulatory anticoagulation clinic.


Participants and setting

A previous research suggested that the number of test items should be multiplied by 10 to obtain the number of participants, so, for this methodological study, 200 patients were required for an adequate sample size.16 On allowing for a 20% loss due to missing data or participants discontinuing the medication during the study, the necessary sample size was 240 patients.

This methodological study was conducted at the outpatient anticoagulation clinic of a university hospital located in Istanbul between 15 April 2017 and 15 October 2017. The clinic provided a service that adjusted the patients’ warfarin dose according to their international normalized ratio (INR). Patients older than 18 years who had been using warfarin for at least the past six months and had at least four INR measures in their medical records were eligible for this study. Patients who could not read were excluded from the study.

The study was approved by the Ethical Committee of Marmara University, Institute of Health Sciences (03.04.2017-121). Informed consent was obtained from all participants.

Data collection

Patients’ demographic and clinical data, including their age, gender, educational level, total number of medications used, indication for oral anticoagulant therapy, and previous INR measurements, were collected using individual interviews and patients’ medical charts. Individual interviews were conducted by a single researcher (MT). At least four consecutive INR measurements taken at least a month and no more than two months apart were retrospectively recorded from patients’ medical charts. Patients’ TTR was calculated using the method developed by Rosendaal et al.17

Translation and cross-cultural adaptation of the OAK test

Permission to use the OAK test for this study was obtained from Zeolla et al.14 This knowledge test includes a total of 20 questions about follow-up, drug-drug interactions, fundamental medication information, adverse effects, and nutritional problems. Higher test scores indicate a better level of oral anticoagulant knowledge.14 Patients were classified into three groups according to their total score in the OAK test based on previous studies.1,13,18 Patients with total OAK test scores of less than 10 (<50%) had low anticoagulation knowledge; patients with total OAK test scores between 10 and 15 (50-75%) had moderate anticoagulation knowledge; patients with total OAK test scores of more than 15 (>75%) had good anticoagulation knowledge.

The original English test was translated into Turkish independently by two native Turkish speakers, who were also fluent in English. Then, two researchers (MS and RD) reviewed the translations and reconciled them into one Turkish version. This Turkish version was back translated into English independently by two native English speakers, who were also fluent in Turkish. Differences between this draft English version and the original English version were evaluated by two researchers (MT and BO). After the translation process, the draft Turkish version was evaluated for grammar, conceptual equivalence, and cultural compatibility by a group of experts (two clinical pharmacists, an internal medicine specialist, a cardiologist, a nurse, and a Turkish literature lecturer). A pilot study was conducted on a group of patients (n=20) for cultural adaptation. It took approximately 10-12 minutes to complete the test. To assess the test-retest reliability, the knowledge test was completed by 30 patients from the study population within two weeks. To analyze the construct validity, demographic and clinical data in each group were evaluated.

Statistical analysis

Categorical data were presented as numbers and percentages. Continuous data were presented as mean ± standard deviation or median and interquartile range (IQR). The Kolmogorov-Smirnov test was used for the normality of distribution. The test-retest reliability was assessed using Spearman’s correlation test. To measure the internal consistency, a Kuder-Richardson 20 (KR-20) coefficient value was calculated for the OAK test. Continuous data between two or more groups were analyzed using the Mann-Whitney U test or Kruskal-Wallis test, respectively. Categorical data were analyzed using the chi-square test. In this study, p values <0.05 were statistically significant.


This study included 240 patients (133 women) who were on warfarin. Patients’ demographic and clinical data are shown in Table 1. Participants’ mean age was 59.83±11.93 (26-90) years. The mean score of OAK test was 14.19±3.01. The most frequent wrong responses were related to drug-drug and drug-food interactions. Less than half of the patients did not know the correct way to distinguish between different strengths of warfarin. The correct answer to this question varied from that in the original scale developed by Zeolla et al.14, due to differences between the national health systems in each country. This was emphasized in the validation study of the Brazilian version of the OAK test.1 The right answer in Turkish version was “size” rather than “color”, and this was taken into consideration during scoring. The percentage of correct answers, corrected item-total correlation, and KR-20 coefficients, if each item was deleted, is shown in Table 2.

The KR-20 coefficient was 0.671. There was a strong correlation between the test-retest results of patients’ total score in the OAK test at baseline and two weeks later (r=0.739; p<0.001; data not shown).

Patients of a younger age and higher educational level were more likely to have higher OAK test scores than those of an older age and lower educational level (p<0.05; Table 3). There was no significant difference in TTR scores or the number of medications used between patients with low, moderate, and high anticoagulant knowledge (p>0.05; Table 3). Evaluation of related factors (demographic and clinical characteristics of the patients) in groups according to patients’ anticoagulant knowledge is shown in Table 3.


In this study, the validity and reliability of the Turkish version of the OAK test were evaluated. The Turkish version of the OAK test was valid and reliable. The KR-20 value in the study conducted in Brazil was 0.818.1 In the United States, the KR-20 value was 0.76.14 Although the value obtained in our study was acceptable, it was lower than the values obtained in the previously mentioned studies. The rate of the right responses in the present study was similar to that obtained by da Silva Praxedes et al.1 The total number of correct responses in studies conducted in the United States,14 Brazil,1 India,18 and Saudi Arabia13 was also similar to our study.

According to a study conducted in India, 50% of patients using oral anticoagulants had OAK test of less than 10, 37% had scores between 10 and 15, and 13% had scores greater than 15.18 According to a study conducted in Denmark, patients had a low knowledge of vitamin K antagonist.19 In a study conducted in Toronto, more than half of the participants had insufficient knowledge of vitamin K antagonists.20 Patients had higher knowledge levels in our study compared with these previous studies. In a study conducted in Brazil, similar to the one conducted in Turkey, 71% had insufficient knowledge.1 In a study conducted in Malaysia, 11.2% had insufficient knowledge.15 When the OAK test score in a study conducted in Singapore was evaluated, patients had a moderate knowledge level.8 However, another study reported that more than half of the participants had a poor knowledge level.13 Consistent with this study, other studies reported similar results, ranging from 61.2% to 70%.21,22 These results highlight the need for intensive training and awareness programs to increase patients’ knowledge of such serious issues.

In this study, women had better knowledge of oral anticoagulants than men. Similar results were obtained in a study conducted in Saudi Arabia.13,21 In other studies, contrasting results were obtained.20,22,23 Studies conducted in Toronto20 and in North India2 indicated that the total OAK test score was higher when participants had a higher educational level. These results were similar to our findings and those from other studies, which showed that OAK test scores were lower in older patients.2,14,24,25

In this study, no significant correlation was found between TTR and OAK test scores, which suggested that there was no correlation between good anticoagulant knowledge and good INR control. A similar result was obtained in studies conducted in Saudi Arabia.13,23 Similar to our study, using the Rosendaal method, some international studies concluded that there was no significant relationship between patients’ knowledge of oral anticoagulants and anticoagulation controls15,26,27 In another study conducted in the United States, it was concluded that there was no significant correlation between patients’ knowledge of warfarin and their INR control.28 In a study conducted in China, a different result was obtained.29 Shilbayeh et al.13 stated that incompatible results could be attributed to differences in test items, languages, settings, INR control measures, and different sample sizes.

When examining the OAK tests developed so far, the scale we used in terms of both content and validity was like the scale developed by Briggs et al.11 However, while the scale we used was valid only for vitamin K antagonists, the scale developed by Obamiro et al.12 can be used for vitamin K antagonists and direct-acting oral anticoagulants. While the scale, which is developed by Obamiro et al.12 includes multiple-choice and open-ended questions, the scale we used included only multiple-choice questions. Based on our best knowledge, no existing validated tests assess patients’ knowledge of oral anticoagulants in our country.

In future studies, the potential impact of duration of warfarin therapy on patients’ anticoagulant knowledge test result should be evaluated. Besides assessing the patients’ anticoagulant knowledge level, the problems related with medication administration (such as dose adjustment difficulty, which is, particularly, due to limited strengths of warfarin tablets in Turkey) might be also assessed. It is necessary to provide comprehensive patient education for patients’ receiving warfarin therapy. Using the OAK test in a clinical setting could provide an opportunity for healthcare providers to identify and resolve patients’ misunderstandings and/or correct any misinformation they may have encountered. This brief test could easily be conducted in outpatient clinics, as it only requires a short time to complete (10-12 minutes).

Study limitations

The generalizability of the results to different patient groups in Turkey may be limited in this study, because it was conducted in a single outpatient clinic in Turkey. Another limitation of the study was that the OAK test designed by Zeolla et al.14 was to be self-administered by individuals with an educational level of at least the seventh grade. However, in most studies, including this study, which used the OAK test, individuals with an educational level lower than the seventh grade took the test.


The Turkish version of OAK test can be used to determine the patients’ knowledge of oral anticoagulation. This test would be helpful for identifying patients who need education and counseling regarding warfarin therapy. Additionally, it can be used to assess changes in patients’ knowledge after receiving education and/or counseling. The test can be used to identify and resolve patients’ misunderstandings of anticoagulant therapy and/or correct misinformation to which they may have been exposed. Test items may remind patients and providers of key points to consider during warfarin therapy.

Conflict of interest: No conflict of interest was declared by the authors. The authors are solely responsible for the content and writing of this paper.


  1. da Silva Praxedes MF, de Abreu MHNG, Paiva SM, de Melo Mambrini JV, Marcolino MS, Martins MAP. Assessment of psychometric properties of the Brazilian version of the oral anticoagulation knowledge test. Health Qual Life Outcomes. 2016;14:96.
  2. Joshua JK, Kakkar N. Lacunae in patient knowledge about oral anticoagulant treatment: results of a questionnaire survey. Indian J Hematol Blood Transfus. 2015;31:275-280.
  3. Dumas S, Rouleau-Maillous E, Bouchama N, Lahcene H, Talajic M, Tardif JC, Gualin MJ, Provost S, Dubé MP, Perreault S. Pillbox use and INR stability in a prospective cohort of new warfarin users. J Manag Care Spec Pharm. 2016;22:676-684.
  4. Gallagher J, Mc Carthy S, Woods N, Ryan F, O’Shea S, Byrne S. Economic evaluation of a randomized controlled trial of pharmacist-supervized patient self-testing of warfarin therapy. J Clin Pharm Ther. 2015;40:14-19.
  5. de Terline MD, Hejblum G, Fernandez C, Cohen A, Antignac M. Discrepancies between patients’ preferences and educational programs on oral anticoagulant therapy: a survey in community pharmacies and hospital consultations. PloS One. 2016;11:e0146927.
  6. Barnes GD, Nallamothu BK, Sales AE, Froehlich JB. Reimagining anticoagulation clinics in the era of direct oral anticoagulants. Circ Cardiovasc Qual Outcomes. 2016;9:182-185.
  7. Kimmel SE, Troxel AB, French B, Loewenstein G, Doshi JA, Hecht TE, Laskin M, Brensinger CM, Meussner C, Volpp K. A randomized trial of lottery-based incentives and reminders to improve warfarin adherence: the warfarin incentives (WIN2) trial. Pharmacoepidemiol Drug Saf. 2016;25:1219-1227.
  8. Wang Y, Kong MC, Lee LH, Ng HJ, Ko Y. Knowledge, satisfaction, and concerns regarding warfarin therapy and their association with warfarin adherence and anticoagulation control. Thromb Res. 2014;133:550-554.
  9. Baykız D, Akyuz A, Alpsoy S, Fidan C. The influence of warfarin adherence on time in therapeutic range among patients with mechanical heart valves. J Heart Valve Dis. 2018;27:55-64.
  10. Çelik A, İzci S, Kobat MA, Ateş AH, Çakmak A, Çakıllı Y, Yılmaz MB, Zoghi M. The awareness, efficacy, safety, and time in therapeutic range of warfarin in the Turkish population: WARFARIN-TR. Anatol J Cardiol. 2016;16:595.
  11. Briggs AL, Jackson TR, Bruce S, Shapiro NL. The development and performance validation of a tool to assess patient anticoagulation knowledge. Res Social Adm Pharm. 2005;1:40-59.
  12. Obamiro KO, Chalmers L, Bereznicki LR. Development and validation of an oral anticoagulation knowledge tool (AKT). PloS One. 2016;11:e0158071.
  13. Shilbayeh SAR, Almutairi WA, Alyahya SA, Alshammari NH, Shaheen E, Adam A. Validation of knowledge and adherence assessment tools among patients on warfarin therapy in a Saudi hospital anticoagulant clinic. Int J Clin Pharm. 2018;40:56-66.
  14. Zeolla MM, Brodeur MR, Dominelli A, Haines ST, Allie ND. Development and validation of an instrument to determine patient knowledge: the oral anticoagulation knowledge test. Ann Pharmacother. 2006;40:633-638.
  15. Matalqah L, Radaideh K, Sulaiman S, Hassali MA, Abdul Kader MAS. Relationship between patients’ warfarin knowledge and anticoagulation control: results of a validated tool in Malaysia. J Pharm Biomed Sci. 2013;30:967-974.
  16. Streiner DL, Norman GR, Cairney J, Health measurement scales: a practical guide to their development and use. 3rd ed. New York; Oxford University Press; 2003.
  17. Rosendaal F, Cannegieter S, Van der Meer F, Briet E. A method to determine the optimal intensity of oral anticoagulant therapy. Thromb Haemost. 1993;70:236-239.
  18. Alphonsa A, Sharma KK, Sharma G, Bhatia R. Knowledge regarding oral anticoagulation therapy among patients with stroke and those at high risk of thromboembolic events. J Stroke Cerebrovasc Dis. 2015;24:668-672.
  19. Nybo M, Skov J. Patient knowledge of anticoagulant treatment does not correlate with treatment quality. Public Health. 2016;141:17-22.
  20. Hu A, Chow CM, Dao D, Errett L, Keith M. Factors influencing patient knowledge of warfarin therapy after mechanical heart valve replacement. J Cardiovasc Nurs. 2006;21:169-175.
  21. Al-Omair SF, Musallam NA, Al-Deghaither NY, Al-Sadoun NA, Bayoumy NM. Compliance with and awareness about long-term oral anticoagulant therapy among Saudi patients in a University Hospital Riyadh, Saudi Arabia. J Appl Hematol. 2016,7;10.
  22. Elbur AI, Albarraq A, Maugrabi M, Alharthi S. Knowledge of, satisfaction with and adherence to oral anticoagulant drugs among patients in King Fasial Hospital: Taif, Kingdom Saudi Arabia. Int J Pharm Sci Res. 2015;31:274-280.
  23. Mayet AY. Association between oral anticoagulation knowledge, anticoagulation control, and demographic characteristics of patients attending an anticoagulation clinic in Saudi Arabia: A cross-sectional prospective evaluation. Trop J Pharm Res. 2015;14:1285-1291.
  24. Davis NJ, Billett HH, Cohen HW, Arnsten JH. Impact of adherence, knowledge, and quality of life on anticoagulation control. Ann Pharmacother. 2005;39:632-636.
  25. Tang EOY, Lai CS, Lee KK, Wong RS, Cheng G, Chan TY. Relationship between patients’ warfarin knowledge and anticoagulation control. Ann Pharmacother. 2003;7:34-39.
  26. Khudair I, Hanssens Y. Evaluation of patients’ knowledge on warfarin in outpatient anticoagluant clinics in a teaching hosiptal in Qatar. Saudi Med J. 2010;31:672-677.
  27. Barcellona D, Contu P, Marongiu F. Patient education and oral anticoagulant therapy. Haematologica. 2002;87:1081-1086.
  28. Ryals CA, Pierce KL, Baker JW. INR goal attainment and oral anticoagulation knowledge of patients enrolled in an anticoagulation clinic in a Veterans Affairs medical center. J Manag Care Pharm. 2011;17:133-142.
  29. Li X, Sun S, Wang Q, Chen B, Zhao Z, Xu X. Assessment of patients’ warfarin knowledge and anticoagulation control at a joint physician-and pharmacist-managed clinic in China. Patient Prefer Adherence. 2018;12:783.