Original Article

Online Evidence-Based Medicine Learning During the COVID-19 Era: Personal Experience in Two Online Autumn Schools

Please cite this article as follows: Sani A, Khalichi R, Beheshti R, Dehghanzadeh S, Kabir-Saber N, Kiani-Nasab S, et al. Online evidence-based medicine learning during the covid-19 era: personal experience in two online autumn schools. Int J Drug Res Clin. 2024; 2: e17. doi: 10.34172/ijdrc.2024.e17

Introduction

The COVID-19 outbreak was declared a global pandemic by the World Health Organization on March 11, 2020.¹ On February 19, 2020, Iran reported its first confirmed cases of COVID-19 infection, followed by public limitations in March.² The pandemic situation affected the educational system similarly to other sectors. The initial decision of the government was to cancel in-person learning in educational institutions for a few weeks, but the closure was extended. The closure of schools and universities posed serious challenges to health professional educators and class designers as well.

Distance educational programs have emerged as an alternative method of learning to continue medical education during the closure of universities related to the COVID-19 pandemic.^3,4

Over recent decades, medical education systems have had limited experiences in the transition from traditional forms of teaching to alternative media that employ distance, online, or electronic learning.⁵ Electronic or online learning can be described as using electronic technology and media to deliver and improve both teaching and learning and involves communication between teachers and students using online content.⁶ As educational technology advances, medical education is utilizing digital mediums and exploring instructional methods more and more every day. Multiple models of virtual learning activities with different purposes are available, including recorded classes, massive open online courses, online live interaction, and conferences. Internet-based learning can provide students with easier and more effective access to a more extensive variety and greater quantity of information. E-learning activities offer a range of possibilities, such as convenience and flexible learning; a decrease in travel time and expenses for trainees, and accessibility for a large audience, especially when face-to-face training resources and opportunities are limited ⁷. These factors make distance learning a pragmatic strategy for health-related education during COVID-19 pandemic restrictions. There is evidence demonstrating distance e-learning as an efficient education modality.^8-10 A recent systematic review indicates that blended learning (lectures alongside online supplements) is effective for examination preparation, concept clarification, and a potential strategy to decrease problems in medical student performance.¹¹ However, another systematic review suggests that evidence is insufficient regarding the effectiveness of online education programs on healthcare professional behavior or patient outcomes and asks for further high-quality research in this area.¹² Nevertheless, it is undeniable that adapting to such abrupt and fundamental changes in the educational process is challenging for both students and teachers.

Evidence-based medicine (EBM) is a systematic approach to clinical questions that allows the combination of the current best research evidence with clinical experience and patient values.¹³ EBM has been introduced as a reliable model in medical sciences, and EBM education helps promote using the serious, clear, and rational top common evidence for decision-making.¹⁴ EBM is an essential skill for healthcare providers, and EBM education is planned in the curriculum of many medical schools as an essential requirement.¹⁵ Different EBM teaching methods, such as problem-based learning, clinically integrated training, and e-learning, can enhance knowledge, perceptions, and skills in undergraduate medical students. Nonetheless, there is insufficient evidence that these strategies can influence clinical practice directly.¹⁶ The Iranian Students’ Center for EBM (ISCEBM) started up in 2011 to connect students interested in EBM. ISCEBM has been following its research and educational goals ever since by organizing annual “EBM summer schools”, workshops, journal clubs, student research projects, and weekly reviews of COVID-19 article bulletin.¹⁷ However, this center had no previous experience of holding online EBM schools. During the COVID-19-related lockdowns, ISCEBM organized an online EBM school in two parts in collaboration with the Iranian EBM Center of Excellence.

Objectives

This study reports our experience at this pilot online EBM school for medical and other healthcare students. In addition, the study evaluates the effectiveness of the course and participants’ satisfaction and discusses the possible limitations and challenges.

Methods

Program Description

The first phase of the online EBM autumn school was held from October 17 to December 13, 2020, on an online platform. This course was an introduction to EBM basics and consisted of six 2-hour lectures followed by assignments. The lectures covered topics such as an introduction to EBM, critical thinking, study designs and level of evidence, PICO, searching databases, and critical appraisal of studies. The second phase took place from October 26to November 4, 2021, on another online platform. This phase was designed to provide a general overview of systematic review studies and comprised 10 one-hour lectures, four complementary educational videos, and practical assignments. The lectures included an overview of different review studies, PICO, advanced database search, study selection and data extraction, proposal writing, protocol registration, and an introduction to meta-analysis, as well as writing a systematic review article and submitting a systematic review article. All classes were interactive lectures, and participants were encouraged to share their opinions and questions. Complementary videos were tutorials on using Embase, Cochrane, and Web of Science databases and Endnote reference manager software. As for the assignments, participants were divided into groups and practiced the stages of conducting a systematic review of an already-known topic and then comparing their answers to the original article.

Participants

Two phases of the autumn school were announced through posters on ISEBM’s social media platforms, and the posters were shared with university groups and research clubs. Participants signed up for the program by filling out a Google Form and joining the groups on social media. They received a certificate of attendance at the end of the course if they were active during the program.

Pre-test and Post-test

Participants were asked to voluntarily complete a pre-test before the first lecture and a post-test after the last one, through each autumn school. There was no limitation on the duration of the online tests. This study sought to develop a simple, comprehensive questionnaire to measure learners’ competence in EBM and systematic reviews by consulting experts in EBM. Based on the data in Table 1, this test consisted of open-ended questions and short answers. Short answer questions require the participant to formulate a focused clinical question (PICO question), show an understanding of research designs and identify the appropriate design for answering clinical questions, demonstrate knowledge of online database searching, and explain the importance of certain steps in conducting a systemic review.

Open-ended test questions reflected the objectives of this online course on general knowledge of EBM in addition to critical thinking and evidence appraisal. The open-ended questions require learners to show higher order and creative thinking, unlike true-false or multiple-choice questions.¹⁸

Each question was assigned 20 points, and the total score was calculated out of 100. Personal identifiers were removed from the tests, and two researchers scored all of them independently. Scoring criteria were developed based on predicted responses from reliable sources and expert opinions about the elements of an ideal answer. Grading strategies were discussed and revised to minimize ambiguity, and inconsistencies were discussed and solved within the group.

Opinion Poll

After the last class, participants were asked to share their opinions about the strengths and weaknesses of the course and their satisfaction with it anonymously. The participants’ feedback helped us to understand students’ attitudes toward an online EBM school. Open-ended questions and satisfaction questionnaires (a 5-point Likert-type scale) were used to collect students’ opinions.

Statistical Analysis

The obtained data were encoded and analyzed using the Statistical Package for the Social Sciences (SPSS), version 21 (IBM SPSS Corp.; Armonk, NY, USA). The normal distribution of the data was determined by the Kolmogorov–Smirnov test. The Mann–Whitney U test was used to compare the groups on baseline characteristics and to compare the outcome measures, and the Wilcoxon rank-sum test was utilized where appropriate.

The data are presented as means, standard deviations (SD), or medians (interquartile range: IQR). Statistical significance was defined as P < 0.05.

Results

Baseline Characteristics

Twenty-nine students, including 10 males and 19 females, participated in the first phase of EBM online autumn school, and 51 students, including 24 males and 27 females, participated in the second phase. Participants’ mean (SD) age in the first phase was 21.10 (1.11), with a range of 20–25 years, and in the second phase, it was 21.11 (1.80), with a range of 19–27 years old. Participants were students from different university fields, including medicine, nursing, pharmacy, dentistry, surgical technology, and occupational therapy. The majority of them were medical students from Tabriz University of Medical Sciences. Participants from Tehran, Iran, Mashhad, Hamedan, Babol, Urmia, Yazd, Isfahan, Shahid, Shahid Beheshti, and Azad University of Medical Sciences were also present. They entered the university between 2014 and 2019, mainly in 2018. Table 2 provides the baseline characteristics of students.

In the first phase, 29 and 18 participants completed the pre-test and post-test with mean scores of 28.10 and 64.27, respectively. In the second phase, 33 and 38 students completed the pre-test and post-test with mean scores of 46.97 and 81.89, respectively. In both phases, the post-test scores improved significantly according to the results of the Wilcoxon nonparametric test (P < 0.001).

Among 29 students in the first phase, one-third were male, and students in the second phase were distributed almost evenly by gender. Considering the gender of the student, the pre-test score in females was lower than that in males in both phases, though the difference was not statistically significant in phase two (P = 0.031 and P = 0.957). Overall, 5 male and 13 female students completed the first post-test. The minimum score in the post-test was 45 for males and 40 for females, and the maximum score was 95 and 82, respectively. Based on analysis according to the gender of the students, the post-test score improved significantly, and there was no statistical difference between the two groups. In the second phase, 19 males and 19 females completed the post-test, and their mean scores were not significantly different based on the Mann-Whitney U test (P = 0.207). The data on the association between gender and the study results are presented in Table 3.

Opinions

Overall, 14 participants commented on the anonymous Google form. They were generally satisfied with the classes and the teachers. The follow-up assignments and social media groups for discussing later questions were mentioned as the strong points. The internet problems and the subsequent disorderliness in the first phase and shortness and compactness in the second phase were pointed out as the weaknesses of the course.

Discussion

The COVID-19 pandemic has affected medical education worldwide for more than two years, and different medical students and groups have their individual stories of how this pandemic has changed their learning systems. This article presented the experience of ISEBM with an online EBM autumn school during the pandemic and reviewed some aspects of digital education.

The COVID-19 pandemic impacted educational systems, including university education at an extensive level. However, medical education systems have been disrupted in a particular way since not all kinds of education can be successfully implemented by online learning. As for medical and health education, only theoretical lessons can be taken part via online learning and teaching clinics, and hospitals have postponed practical parts of students’ education. Organizing the practical part of medical education is specifically challenging in times of pandemic.¹⁹ Moreover, neglecting the problems in health-related education in challenging times may jeopardize the future care of patients. The lack of direct patient visits and replacement of on-site educational settings with online equivalents are matters of concern for medical students.^20,21 Telehealth is a cost-effective solution for managing COVID-19 outbreaks, providing vital services such as disease management, research, and education. It allows for rapid diagnosis and immediate action, facilitating direct-to-patient consultations in hard-to-reach regions. Its benefits include limited admission capacity, increased infection risk, remote assistance, and improved investigations. However, adoption may be difficult due to costs, technology challenges, lack of awareness, and technical support issues. Telehealth could connect healthcare personnel worldwide, making it a reliable strategy for ineffective classical healthcare systems.²² Telemedicine can help manage non-critical patients remotely. The Ministry of Health and Medical Education has issued guidelines for diagnosis and treatment. Restrictions on gatherings, ceremonies, and sports events are in effect, and prioritizing primary healthcare programs can help improve patient outcomes.²³ Many students have also lost the opportunity for educational development through off-curriculum courses and activities. A study evaluated the effectiveness of electronic learning in improving cardiopulmonary resuscitation knowledge among 84 interns in the Department of Emergency Medicine. The results revealed that the average answer score for 21 questions before education was 7.5 ± 2.6, but it increased to 11.0 ± 3.9 after education. However, the electronic learning method was not associated with a significant increase in knowledge compared to the lecture-based group. Therefore, further research is needed to determine the effectiveness of electronic learning in improving cardiopulmonary resuscitation knowledge.²⁴ Another study compared the effectiveness of e-learning and discussion-based learning models on emergency department interns’ sedative knowledge acquisition. A total of 129 interns were divided into two groups. Both groups received pre-tests and electronic software, while group II received discussion-based training for three weeks. Post-tests were conducted after a month. The results showed significant improvements in learning rates for all aspects of sedation knowledge, including narcotics, intravenous anesthetics, clinical judgments, and neuromuscular blocking agents. However, the discussion-based learning group had better results.²⁵ These activities outside the regular classes are beneficial experiences both for learning and building connections, including the EBM summer schools held for six continuous years at Tabriz University of Medical Sciences. A prior study by ISCEBM reported the results of a one-week EBM summer school that used innovative learning methods. This school included team-based learning, seminar-style presentations, and acting-based EBM journal clubs. The comparison between pre-course and post-course questionnaires demonstrated that the school was effective, and medical students found it useful and satisfying.¹⁷

Despite the explosion of research concerning COVID-19, a few studies have discussed the radical transition of medical and health sciences education, including EBM learning, from traditional methods to electronic education.

A cross-sectional study investigated differences in performance and knowledge of 62 postgraduate students who learned EBM either in face-to-face or online classes.²⁶ In general, participants showed high performances in different assessment exams, and the results were not significantly different between the two learning groups. In addition, students of both groups were similarly satisfied with the lecturers, course contents, and the teaching materials. However, one-third of students in face-to-face classes felt non-equality among students in instructors’ teaching practice. The findings of this study confirmed that online EBM learning may be as effective as a traditional face-to-face class. Moreover, it was found that distance class learning is associated with fair practices and equality between participants and leads to better satisfaction among students.

Another recent study evaluated the impact and engagement of online EBM and research methods learning over three months. The participants in the online course reported improvements in their knowledge, skills, confidence, and practice of EBM after the course.²⁷

While using technology is a convenient and innovative solution to help students adapt to the pandemic situation, there are certain drawbacks.

One inevitable downside to online classes is the technical issues such as internet connection problems and a lack of basic knowledge about digital learning procedures. The abrupt transition of educational systems to using new technologies might accompany a negative attitude toward it from teachers and students. One possible reason is that learning new teaching and learning techniques that are compatible with online education is challenging, particularly when facing technical issues. A study conducted in Pakistan about e-learning in a medical college post-COVID-19 outbreak revealed that students did not prefer e-learning over face-to-face learning because of poor internet access and stability.²⁸ As for our experience, in the first phase of the online course, the students were not highly familiar with online classes, but the number of participants doubled after a year in the second phase. Additionally, due to complaints about technical issues with the online platform during phase one, the medium was changed to a different online platform that was more user-friendly. Asynchronous learning environments such as recorded lectures are another way to overcome this problem, though student-teacher interaction is extremely low in this method.

Another problem for many students is staying focused in an online classroom due to the surrounding distractions. This makes it difficult for students to actively interact with their teachers, and motivating them with regular class assessments is also challenging for teachers. A recent study reviewing online medical education during the COVID-19 pandemic suggests keeping participants interested as a potential solution for increasing student engagement in internet-based classes.²⁹ One possible suggestion in this article is to deliver the curriculum in synchronous and asynchronous modes. The synchronous method, such as e-lectures, allows learners to give feedback and have instant interaction with other classmates and the teacher through a discussion. The asynchronous mode, such as online groups with chat rooms, gives students time to think through issues and then take part in the group discussion over the Internet. To increase participation and engagement, online lectures should have certain features, including creative and constructive questions during lectures, mini assignments after online classes, and certain video game tools, are some examples. These complementary education materials can enhance participants’ engagement and feedback and help them in better understanding, problem-solving, and analysis skills.

During online learning, it has been found that recorded video will be more powerful than directly switching to live online classes. During this time, continuing mentoring and self-directed learning will be more effective.²⁷ Transcript of the slide, on-screen caption, and limiting the texts in the slide will also motivate the students for better engagement.

During the two online EBM schools, we attempted to keep the relationship between students and teachers interactive, and students could share their opinions and ask questions by raising their hands in online classes. However, some educational contents require in-person interactions between teachers and students in order to be fully conveyed. For example, in our experience in the first phase of the course, the ‘searching database’ virtual class was not as efficient as previous physical classes since participants could not practice searching alongside the teacher and discuss their problems. To overcome this problem, the researchers of this study prepared supplementary videos of database tutorials so that the students could control the video playback setting and practice at the same time. Moreover, for better learning, each group designed a search strategy for a certain topic and compared it with the original systematic review article. Increasing learner engagement in remote classes is an effortful challenge. In this online course, it was tried to give assignments and encourage the participants to be actively involved in class discussions as a method to increase interactions, which received positive feedback from the participants. Considering that e-learning is more independent, appropriate instructions concerning assignments must be provided to keep participants motivated.

Another downside of online classes is that they limit the education method options, and choosing a proper method to teach EBM to students is highly important. A recent study compared EBM education outcomes through peers and TBL workshops in Iran University of Medical Sciences. The findings are consistent with those of prior studies, indicating that teaching EBM by expert teachers can be effective and peers can participate in classes as facilitators.^30,31 This online EBM school was mostly taught by expert teachers in this field, and peers organized the classes and groups and provided supplementary tutorial videos.

Nevertheless, there are certain advantages to virtual classes. From a personalized approach, remote education is more flexible in time, space, opportunities, and resources; therefore, it has a broader reach than traditional settings.^32,33 Students can access classes from everywhere, as in this online course, the participants enrolled in the program from twelve different universities.

Based on the results of pre-test and post-test analyses, this online course was significantly effective in improving students’ knowledge about EBM (P < 0.001) and systematic review studies (P < 0.001). Some studies also compared the effectiveness of online learning with the traditional format, suggesting that students learn well or better from e-learning compared to in-class settings.^34,35 A recent study explored the effect of the course delivery method (online vs. in-class lecture formats) on the long-term outcomes in graduate degrees of 4,978 students.³⁶ The results demonstrated that the proportion of graduates with any doctoral degree was not different between the two modalities, indicating that online education had no adverse effect on the student’s academic success.

Based on the findings of the analysis, female students participated in the course more than males. However, there was no statistical difference between the two genders in terms of improvements in the test. Since the COVID-19 outbreak and before, there have been studies about gender differences in digital education. A large-scale study conducted based on data from 39 countries in 2006 revealed that men use the internet and computers for educational purposes more than women in all of the investigated countries.³⁷ The results of another study regarding digital education after the pandemic showed higher intrinsic values in education and learning engagement among female students than males. However, there were no significant gender differences in competence beliefs about e-learning.³⁸ The inconsistencies between studies about gender differences in digital education may be due to differences in the context of studies (e.g., cultural differences of the countries in which the studies were conducted³⁹ and constant changes in the medical educational system).

The main limitations of this study were the small study population and the lack of a control group to compare the results. Future research may consider large-scale designs that compare different modalities of digital education.

The findings of this study confirmed that despite the challenges facing digital education during the COVID-19 outbreak, this online EBM school was effective for the participants. Two key concerns regarding online courses were technical problems and the student engagement level, which can be controlled by modifying the online platform and giving group assignments to students, respectively. The other possible ways to overcome the low student engagement that has been encountered in this study include:

1. Interactive content: Incorporating more interactive elements, such as polls, quizzes, and breakout discussions during sessions, can help maintain participant interest and encourage active participation. Furthermore, supplementary videos and practical group assignments fostered engagement and collaborative learning among participants.

2. Regular feedback: Providing timely and constructive feedback on assignments and assessments can motivate students to engage more deeply with the material and feel more connected to the learning process. The use of pre- and post-tests provides measurable outcomes that demonstrate significant knowledge gain.

3. Peer learning opportunities: Facilitating peer-to-peer interactions, such as study groups or discussion forums, can foster a sense of community and encourage students to take an active role in their learning.

4. Flexible learning options: Offering recorded sessions or supplementary materials for review at participants’ convenience may help accommodate different learning styles and schedules, thereby increasing overall engagement. The online format allows participants from various locations to join without travel constraints.

5. Incentives for participation: Implementing a reward system for active participation, such as certificates or recognition, could also incentivize students to engage more fully with the course content.

By incorporating these strategies into future iterations of the course, student engagement enhances and creates a more dynamic learning environment.

Conclusion

The EBM online autumn school revealed significant improvements in student learning outcomes, with post-test scores increasing significantly in both phases. The course’s effectiveness in enhancing students’ understanding of EBM principles was evident. The diverse demographic of participants, including medicine, nursing, pharmacy, and occupational therapy, reflects the interdisciplinary nature of EBM education. Although male participants initially scored higher, post-test results demonstrated no significant gender differences in performance. Feedback indicated satisfaction with the course structure and teaching quality, emphasizing the value of follow-up assignments and social media groups. However, challenges such as internet connectivity issues and course content compactness were noted, highlighting areas for improvements in future programs. Hence, it is recommended that online EBM education beyond the pandemic be continued for sustainability and that hybrid models combining online and in-person learning be used to enhance engagement for future iterations of the course.

Ethics statement

Regional ethics committee of Tabriz University of Medical Sciences approved this study (IR.TBZMED.REC.1402.275).

Conflict of interests declaration

None.

Acknowledgments

We would like to thank the Research Center for EBM.

Data availability statement

The authors confirm that the data supporting the findings of this study are available within the article.

Author contributions

Conceptualization: Anis Sani; Hanieh Salehi-Pourmehr, Sakineh Hajebrahimi.

Data curation: Rozhin Khalichi, Rasa Beheshti, Saba Dehghanzadeh, Nadia Kabir-Saber, Sepideh Kiani-Nasab, Maryam Taraghikhah, Ali Adabi.

Formal analysis: Hanieh Salehi-Pourmehr, Sakineh Hajebrahimi.

Funding acquisition: Hanieh Salehi-Pourmehr, Sakineh Hajebrahimi.

Investigation: Robab Hassanzadeh.

Methodology: Hanieh Salehi-Pourmehr, Sakineh Hajebrahimi, Robab Hassanzadeh.

Project administration: Hanieh Salehi-Pourmehr.

Resources: Hanieh Salehi-Pourmehr.

Software: Hanieh Salehi-Pourmehr.

Supervision: Sakineh Hajebrahimi, Robab Hassanzadeh.

Validation: Sakineh Hajebrahimi.

Visualization: Sakineh Hajebrahimi.

Writing–original draft: Anis Sani, Rozhin Khalichi, Rasa Beheshti, Saba Dehghanzadeh, Nadia Kabir-Saber, Sepideh Kiani-Nasab, Maryam Taraghikhah, Ali Adabi.

Writing–review & editing: Hanieh Salehi-Pourmehr, Sakineh Hajebrahimi, Robab Hassanzadeh.

Consent for publication

Not applicable.

References

1. World Health Organization (WHO). WHO Director-General’s Opening Remarks at the Media Briefing on COVID-19 - 11 March 2020. Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020. Accessed March 11, 2020.
2. Yavarian J, Shafiei-Jandaghi NZ, Sadeghi K, Shatizadeh Malekshahi S, Salimi V, Nejati A. First cases of SARS-CoV-2 in Iran, 2020: case series report. Iran J Public Health 2020; 49(8):1564-8. doi: 10.18502/ijph.v49i8.3903 [Crossref] [ Google Scholar]
3. Taylor D, Grant J, Hamdy H, Grant L, Marei H, Venkatramana M. Transformation to learning from a distance. MedEdPublish (2016) 2020; 9:76. doi: 10.15694/mep.2020.000076.1 [Crossref] [ Google Scholar]
4. Rose S. Medical student education in the time of COVID-19. JAMA 2020; 323(21):2131-2. doi: 10.1001/jama.2020.5227 [Crossref] [ Google Scholar]
5. Shachar M, Neumann Y. Differences between traditional and distance education academic performances: a meta-analytic approach. International Review of Research in Open and Distributed Learning 2003; 4(2):1-20. doi: 10.19173/irrodl.v4i2 [Crossref] [ Google Scholar]
6. Howlett D, Vincent T, Gainsborough N, Fairclough J, Taylor N, Cohen J. Integration of a case-based online module into an undergraduate curriculum: what is involved and is it effective?. E-Learning and Digital Media 2009; 6(4):372-84. doi: 10.2304/elea.2009.6.4.372 [Crossref] [ Google Scholar]
7. Richmond H, Copsey B, Hall AM, Davies D, Lamb SE. A systematic review and meta-analysis of online versus alternative methods for training licensed health care professionals to deliver clinical interventions. BMC Med Educ 2017; 17(1):227. doi: 10.1186/s12909-017-1047-4 [Crossref] [ Google Scholar]
8. Letterie GS. Medical education as a science: the quality of evidence for computer-assisted instruction. Am J Obstet Gynecol 2003; 188(3):849-53. doi: 10.1067/mob.2003.168 [Crossref] [ Google Scholar]
9. Rotimi O, Orah N, Shaaban A, Daramola AO, Abdulkareem FB. Remote teaching of histopathology using scanned slides via Skype between the United Kingdom and Nigeria. Arch Pathol Lab Med 2017; 141(2):298-300. doi: 10.5858/arpa.2016-0111-EP [Crossref] [ Google Scholar]
10. Bernard RM, Abrami PC, Lou Y, Borokhovski E, Wade A, Wozney L. How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Rev Educ Res 2004; 74(3):379-439. doi: 10.3102/003465430740033 [Crossref] [ Google Scholar]
11. Ahmady S, Khajeali N, Sharifi F, Mirmoghtadaei ZS. Factors related to academic failure in preclinical medical education: a systematic review. J Adv Med Educ Prof 2019; 7(2):74-85. doi: 10.30476/jamp.2019.44711 [Crossref] [ Google Scholar]
12. Sinclair PM, Kable A, Levett-Jones T, Booth D. The effectiveness of Internet-based e-learning on clinician behaviour and patient outcomes: a systematic review. Int J Nurs Stud 2016; 57:70-81. doi: 10.1016/j.ijnurstu.2016.01.011 [Crossref] [ Google Scholar]
13. Akobeng AK. Principles of evidence-based medicine. Arch Dis Child 2005; 90(8):837-40. doi: 10.1136/adc.2005.071761 [Crossref] [ Google Scholar]
14. Kim YS, Kim J, Park MM. Factors influencing competency in evidence-based practice among clinical nurses. J Korean Acad Nurs Adm 2015; 21(2):143-53. doi: 10.11111/jkana.2015.21.2.143 [Crossref] [ Google Scholar]
15. Glasziou P, Burls A, Gilbert R. Evidence based medicine and the medical curriculum. BMJ 2008; 337:a1253. doi: 10.1136/bmj.a1253 [Crossref] [ Google Scholar]
16. Ahmadi SF, Baradaran HR, Ahmadi E. Effectiveness of teaching evidence-based medicine to undergraduate medical students: a BEME systematic review. Med Teach 2015; 37(1):21-30. doi: 10.3109/0142159x.2014.971724 [Crossref] [ Google Scholar]
17. Farhadi F, Mostafaie H, Taleschian-Tabrizi N, Hajebrahimi S, Madani N, Mohammadali H. Evidence based medicine summer school for undergraduate medical students using innovative methods. Med Sci 2018; 22(92):377-84. [ Google Scholar]
18. Bloom BS, Englehart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co; 1956.
19. Eva KW. Strange days. Med Educ 2020; 54(6):492-3. doi: 10.1111/medu.14164 [Crossref] [ Google Scholar]
20. Ferrel MN, Ryan JJ. The impact of COVID-19 on medical education. Cureus 2020; 12(3):e7492. doi: 10.7759/cureus.7492 [Crossref] [ Google Scholar]
21. Theoret C, Ming X. Our education, our concerns: the impact on medical student education of COVID-19. Med Educ 2020; 54(7):591-2. doi: 10.1111/medu.14181 [Crossref] [ Google Scholar]
22. Mahmoodpoor A, Akbarzadeh MA, Sanaie S, Hosseini MS. Role of telehealth in outbreaks-where the classical healthcare systems fail. Infect Control Hosp Epidemiol 2020; 41(8):992-4. doi: 10.1017/ice.2020.120 [Crossref] [ Google Scholar]
23. Mahmoodpoor A, Sanaie S, Hosseini MS. COVID-19 outbreak in Iran: lessons to learn, measures to take. Disaster Med Public Health Prep 2022; 16(4):1309-10. doi: 10.1017/dmp.2021.37 [Crossref] [ Google Scholar]
24. Khoshbaten M, Soleimanpour H, Ala A, Shams Vahdati S, Ebrahimian K, Safari S. Which form of medical training is the best in improving interns’ knowledge related to advanced cardiac life support drugs pharmacology? An educational analytical intervention study between electronic learning and lecture-based education. Anesth Pain Med 2014; 4(1):e15546. doi: 10.5812/aapm.15546 [Crossref] [ Google Scholar]
25. Hills AP, Dengel DR, Lubans DR. Supporting public health priorities: recommendations for physical education and physical activity promotion in schools. Prog Cardiovasc Dis 2015; 57(4):368-74. doi: 10.1016/j.pcad.2014.09.010 [Crossref] [ Google Scholar]
26. El Sayed I, Abdelmonem S. A cross-sectional study to assess evidence-based medicine teaching method: online or face-to-face?. BMJ Evid Based Med 2019; 24(2):59-62. doi: 10.1136/bmjebm-2018-111117 [Crossref] [ Google Scholar]
27. Rahimi-Ardabili H, Spooner C, Harris MF, Magin P, Tam CW, Liaw ST. Online training in evidence-based medicine and research methods for GP registrars: a mixed-methods evaluation of engagement and impact. BMC Med Educ 2021; 21(1):492. doi: 10.1186/s12909-021-02916-0 [Crossref] [ Google Scholar]
28. Abbasi S, Ayoob T, Malik A, Memon SI. Perceptions of students regarding E-learning during COVID-19 at a private medical college. Pak J Med Sci 2020; 36(Covid19-S4):S57-61. doi: 10.12669/pjms.36.COVID19-S4.2766 [Crossref] [ Google Scholar]
29. Nimavat N, Singh S, Fichadiya N, Sharma P, Patel N, Kumar M. Online medical education in India - different challenges and probable solutions in the age of COVID-19. Adv Med Educ Pract 2021; 12:237-43. doi: 10.2147/amep.s295728 [Crossref] [ Google Scholar]
30. Ramezani G, Norouzi A, Moradi E, Pourbairamian G, Aalaa M, Alizadeh S. Comparing peer education with TBL workshop in (EBM) teaching. Med J Islam Repub Iran 2020; 34:70. doi: 10.34171/mjiri.34.70 [Crossref] [ Google Scholar]
31. Sastre EA, Denny JC, McCoy JA, McCoy AB, Spickard A, 3rd 3rd. Teaching evidence-based medicine: impact on students’ literature use and inpatient clinical documentation. Med Teach 2011; 33(6):e306-12. doi: 10.3109/0142159x.2011.565827 [Crossref] [ Google Scholar]
32. Vásquez D. Benefits, disadvantages and eight recommendations for virtual medical education in times of COVID-19. CES Med 2020; 34:14-27. doi: 10.21615/cesmedicina.34.COVID-19.3 [Crossref] [ Google Scholar]
33. Gomez E, Azadi J, Magid D. Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic. Acad Radiol 2020; 27(9):1285-90. doi: 10.1016/j.acra.2020.06.001 [Crossref] [ Google Scholar]
34. Nguyen T. The effectiveness of online learning: beyond no significant difference and future horizons. MERLOT J Online Learn Teach 2015; 11(2):309-19. [ Google Scholar]
35. Dell CA, Low C, Wilker JF. Comparing student achievement in online and face-to-face class formats. MERLOT J Online Learn Teach 2010; 6(1):30-42. [ Google Scholar]
36. Ardissone AN, Drew JC, Triplett EW. Online and in-person delivery of upper division lecture courses in undergraduate life sciences degree programs leads to equivalent post-graduate degree outcomes. J STEM Educ Res 2020; 3(3):403-12. doi: 10.1007/s41979-020-00043-x [Crossref] [ Google Scholar]
37. Drabowicz T. Gender and digital usage inequality among adolescents: a comparative study of 39 countries. Comput Educ 2014; 74:98-111. doi: 10.1016/j.compedu.2014.01.016 [Crossref] [ Google Scholar]
38. Korlat S, Kollmayer M, Holzer J, Lüftenegger M, Pelikan ER, Schober B. Gender differences in digital learning during COVID-19: competence beliefs, intrinsic value, learning engagement, and perceived teacher support. Front Psychol 2021; 12:637776. doi: 10.3389/fpsyg.2021.637776 [Crossref] [ Google Scholar]
39. Li N, Kirkup G. Gender and cultural differences in Internet use: a study of China and the UK. Comput Educ 2007; 48(2):301-17. doi: 10.1016/j.compedu.2005.01.007 [Crossref] [ Google Scholar]