Accreditation As An Indicator Of The Specialists Training Quality During Covid-19 Pandemic

Abstract

Over the past 5 years, a new procedure for admitting a specialist to professional activities has been introduced. The paper analyzes the results of the primary accreditation of specialists from the NEFU Medical Institute during the pandemic and for 2017–2019. The results of the survey of 2019 are also presented. The monitoring revealed that there is a tendency towards the increase in the share of accredited persons up to 100 % in the last 2 years compared to the first years of the introduction of the accreditation procedure. 2020–2021 are characterized by an emergency transition to the distance learning format of medical universities, due to the high risk of the spread of the COVID-19 and the introduction of anti-epidemic restrictive measures. The problems and difficulties during the primary accreditation and solutions are indicated. The analysis of the survey of specialists in 2019 on the primary accreditation showed that the 3rd stage of accreditation, which was first conducted in 2019 in a new online format, caused more difficulty for those being accredited.

Keywords: Competencies, medical school graduates, medicine, primary accreditation, practical skills, quality of training

Introduction

The Republic of Sakha (Yakutia) is the region with the largest and sparsely populated territory of the Russian Federation. An underdeveloped transport network, remoteness and a small number of populated areas cause difficulties in the provision of health care to population. Medical Institute of North-Eastern Federal University named after Ammosov provides medical organizations of the republic with medical personnel. Thus, the employment of graduates of this Institute is 100% annually. Currently, after graduation, a graduate has the right to start independent medical practice or enter clinical residency in the chosen profession.

The Order of the Ministry of Health of the Russian Federation No. 334n of June 2, 2016 “On approval of the Regulations on the accreditation of specialists” approved the Regulations on the primary accreditation of specialists, according to which doctors who have a certificate of specialist accreditation are allowed to independently carry out medical activities. The Ministry of Health of the Russian Federation approved the procedure for accreditation, in the form of a three-stage exam. In order to conduct accreditation, the Ministry of Health of the Republic has created an accreditation commission, consisting of representatives of the Ministry of Health and state medical organizations.

The procedure for primary accreditation of a specialist is a three-stage exam – testing, assessing practical skills in simulated conditions and solution of situational problems or mini-cases. The unified requirements under the Regulations for conducting primary accreditation of specialist are the mandatory provision of classrooms and premises with telecommunications: audio and video communications and recording. The preparatory stage for primary accreditation of the future specialist consisted of training for situational tasks and testing with the help of questions posted on the website of the Methodological Center for Accreditation of the Ministry of Health of Russia. To pass the 2nd stage, a simulation center was created, where future graduates could prepare for the exam (see Table 1, Figure 1).

See Full Size >

Problem Statement

COVID-19 pandemic has posed a global challenge to modern healthcare. In this regard, higher medical education institutions occupy a leading position in the training of specialists. Despite certain difficulties, the transition to online learning remains quite effective in training future specialists. In this paper, we analyzed the results of the primary accreditation of a specialist in the field of medicine.

Research Questions

The research question is to find out the level of training of a specialist in the direction of training General Medicine, including when switching to distance learning.

The first issue is the comparison of the results of primary accreditation shown by specialists in the period 2020–2021 with the results of graduates who were accredited earlier (2017–2019).

The second issue is the identification of the most common mistakes when passing the 2^nd stage of accreditationtesting practical skills in simulated conditions.

Purpose of the Study

The purpose of this study is to analyze and evaluate the results of the primary accreditation of specialists from Medical Institute for 2 years of the pandemic (2020–2021) and the first 3 years of the introduction of the primary accreditation procedure for a specialist.

Research Methods

In order to assess the quality of training of specialists, we analyzed the results of primary accreditation for five years (2017 – 2021) and evaluation sheets (checklists) of accredited persons filled out by experts of the accreditation subcommittee in the specialty "General Medicine" in 6 stations (Basic cardiopulmonary resuscitation, Emergency medical care, Physical examination of the patient, Hypoglycemia, Medical examination, Medical history and intake of complaints). We also analyzed the results of an anonymous survey of specialists after the initial accreditation.

Findings

As we see from the table in 2017, out of 103 accredited persons, 79 people passed at the first try, 11 passed at the second try, 8 passed at the third try. Unfortunately, 5 graduates did not pass the 1^st stage of accreditation. In 2018, out of 130 people, 2 accredited people did not pass the 1^st stage. Further, all 128 admitted successfully passed the second and third stages. In 2019, out of 81 admitted, 2 failed to pass the first stage and 2 people did not pass the third stage.

Technical failures (of fmza.ru official site) were noted in 2017 (1) and in 2018 (6). The failures were "freeze ups" of the site, which led to data loss and testing restart. It is also possible to note the presence of a considerable number of mistakes and typos in individual test questions of the 1^st stage in the first year of accreditation.

In order to understand what mistakes are most often observed when passing the stations of the 2^nd stage, we analyzed the checklists of experts. The analysis showed that specialists most often made mistakes in the form of a violation of the sequence of actions. It is worth emphasizing that most of the mistakes were insignificant.

As we see from the figure, graduates most often make mistakes at the station "Emergency Medical Assistance", at the rest they made mistakes with approximately the same frequency. At the station "Emergency Medical Care" several scenarios for the provision of emergency care are presented – acute coronary syndrome 1, acute coronary syndrome 2, anaphylactic shock, gastrointestinal bleeding, broncho-obstructive syndrome, pulmonary embolism, pneumothorax, hypoglycemia, hyperglycemia, acute cerebrovascular accident. In our opinion, this determined the presence of a greater number of mistakes. Our data are consistent with the results of other researchers (Gostimsky et al., 2018; Vakhitov et al., 2019).

Solving situational problems, graduates had difficulties with the choice of diagnostic tactics for rare diseases, patient routing, and outpatient rehabilitation of patients. In 2019, there was a change in the format of the exam – the accredited person had to solve 3 problems in 60 minutes, and not just one as it was in previous years. Apparently, this affected the exam results, which is also emphasized by other authors (Davydova et al., 2018).

In general, graduates showed a good percentage of passing the exam in practical skills and in solving situational problems, which can be explained by the subjectivity of the assessment during the interview and the features of the assessment scale for testing practical skills. Thus, only serious mistakes when passing the station can lead to the assessment “Not Passed.” Our results do not contradict the data shown by graduates from other universities (Kulikovsky et al., 2018; Pavlenko et al., 2018).

Despite the fact that in 2019 4 accredited students failed to pass the exams, we believe that the pre-accreditation preparation was better than in previous years. Since two people who did not pass accreditation were among those who recovered from academic leave and two were graduates of other universities. The teachers of the department began to pay more attention to both test tasks and practical skills. Educational work was organized in the simulation center of the teachers of the department, where each teacher, according to the schedule, carried out advisory work, explained the features of the stations to students.

The improvement in the results of PAS was facilitated by the fact that since 2018 a large number of students began to take part in the All-Russian Olympiad “I am a professional.” The structure of the Olympiad includes testing and verification of practical skills in simulated conditions. In 2019, 16 students took part in the selection of the Olympiad and one graduate reached the final full-time stage. In 2020, there were 5 diplomats of the All-Russian Olympiad and 82 participants.

In 2020, due to the COVID – 19 pandemic, the initial accreditation was not carried out on time, it was postponed to autumn 2020 and spring 2021. The transition to online learning required the solution of organizational issues, including changes in curricula and the introduction of distance technologies for the preparation for accreditation.

Only 47 primary care physicians chose to pass PAS between October 22 and November 9, 2020. This was explained by the fact that due to the pandemic, medical organizations released only a part of the doctors due to production needs. In addition, clinical residents received an exemption from undergoing the PAS procedure. It is necessary to note that out of 47 successfully passed graduates, 9 people were graduates of other universities. This time, all graduates passed all stages of accreditation at the first try.

109 graduates have chosen to undergo primary accreditation from June 28 to July 15, 2021. Another 11 people completed PAS in October 2021. Of the 109 persons, only one graduate did not pass the PAS because he did not come to retake the exam. All others were accredited. 108 people passed the first stage of the PAS, there was one failure to appear for the second retake of the test. At the second stage all graduates passed exam successfully. All 108 accredited people passed the 3^rd stage. This year's graduates found it difficult to pass an exam in the form of an oral interview.

In 2019, for the first time, a survey was conducted of 79 graduates undergoing primary accreditation to assess the preparation for the accreditation procedure of 6^th year students and in connection with the adjustment of tests of the 1^st stage, and the introduction of cases from tasks at the 3^rd stage of accreditation this year. The survey included questions on the amount of time spent on preparation, the external interface and the quality of the questions.

During the survey at the first stage of accreditation, it turned out that trial testing as a stage of preparation for accreditation was passed 20–30 times in average by 100 % of graduates. At the same time, a trial online exam on fmza.ru were held under the supervision of teachers. 62 % of graduates were completely satisfied with the trial testing interface, 38 % were not entirely satisfied with the format of online testing. The first accreditation stage turned out to be accessible in terms of difficulty level for the majority of graduates (87 % of 70 who answered this question), while 13 % had difficulties. Moreover, for the majority of accredited students (97 %), the allotted time within 1 hour was enough to solve all the tests. For 20 % of graduates, the wording of the test during accreditation was not entirely clear, 37 % noted the fuzzy choice of answers, 35 % saw typos, and 8 % of respondents noted the presence of obvious mistakes.

It was very interesting to know the opinion of the graduates on the modified third stage, preparation for which took place in the second semester of the 6^th year. 71% of respondents answered that they solved tasks in the form of trial testing 1–3 times, 25 % prepared from 4 to 10 times and only 4 % spent more time preparing (from 11 to 20 times). A half of the graduates had difficulties in the solution of cases. All graduates noted that not 30, but 60 minutes would be enough to solve cases in time.

According to the analysis of the survey, the following conclusion can be drawn: the 1^st stage of accreditation did not cause problems for the majority of graduates, which meant good pre-accreditation preparation. The 3^rd stage caused more difficulties due to the fact that for the first time in 2020 an online solution of situational problems was carried out, especially in a short period of time, which was unusual for graduates against the backdrop of a stressful examination situation. However, it is necessary to note that, according to teachers and experts, the introduced new level of the identification of the quality of graduate training in the form of mini-cases was more objective and encouraged students to better self-training.

Moreover, we should note that the technical problems on the fmza.ru have disappeared. Thus, in 2017–2018 there were 8 technical failures. In 2019–2021 there were no such failures. There was a significant decrease in the number of mistakes and typos in the test tasks of the 1^st stage. Although there is still work to be done to correct test items and describe scenarios for the 2^nd stage in particular and the content of accreditation as such to improve the quality of specialist training in general (Prasad et al., 2020; Sizova et al., 2017).

Conclusion

Due to the COVID-19 pandemic higher medical education has faced an urgent transition in the training of medical students to distance learning and this has caused difficulties for both teaching staff and students. The most important negative factor of the pandemic for the professional training of medical personnel is the impossibility for students to obtain the necessary professional competencies and practical skills in the absence of access to clinical bases because their infectious diseases departments were repurposed into “red zones”. It was a challenge for those graduate students who were to have a state final certification, state exam and subsequent primary accreditation of specialists. The organization of planned training of graduate students by teachers of the medical institute with analysis of educational videos, independent work of graduates with a convenient mobile application of test tasks and in a simulation center for practicing practical skills according to an approved schedule in compliance with anti-epidemic measures to prevent the spread of a new coronavirus infection turned out to be effective. This affected the results of the primary accreditation. The results and evaluation of the 1^st stage showed that, despite the distance learning format, the accredited students successfully passed the test. Therefore, online learning needs further development (Hussain et al., 2020).

The analysis of the results of 2^nd stage showed that the mistakes were non-systemic. These mistakes can be explained by the insufficient practical training of individual specialists specifically in clinical situations approved by the Methodological Center for Accreditation of Specialists. In this part, the students recovering from academic leave and students working in medical organizations who do not have time for additional self-training have difficulties. The results of the primary accreditation of NEFU General Medicine specialists showed that despite the epidemic difficulties of the transition to online education of medical students, the quality of training of medical specialists remained at a good level. We would like to note that the contribution to practical training was made by the participation of NEFU medical students since the beginning of the pandemic in the volunteer movement, practical assistance to healthcare at the level of outpatient clinics and emergency medical care.

The results of the 3^rd stage of the online solution of situational cases also showed that the majority of those accredited who passed the exam at the 1^st try. The rest of the students except for 2 people passed at the regulated allowable 3 tries. Thus, in the end 99 % accredited students passed the exam.

According to the analysis of this survey, the following conclusion can be drawn: accreditation did not cause problems for the majority of those accredited, which meant good pre-accreditation preparation and the accumulated experience of teachers in monitoring the self-training of graduates.

References

• Davydova, N. S., Dyachenko, E. V., Samoilenko, N. V., Shubina, L. B., Serkina, A. V., & Bottaev, N. A. (2018). Federal pilot: assessment of communication skills during the initial accreditation of a specialist – 2018: results, results. Virtual technologies in medicine, 2(20), 61–62. https://rosomed.ru/theses/606

• Gostimsky, A. V., Lisovsky, O. V., & Lisitsa, I. A. (2018). Primary accreditation of specialists. Experience, problems and possible solutions. Virtual technologies in medicine, 2(20), 35.

• Hussain, F. N., Al-Mannai, R., & Agouni, A. (2020). An Emergency Switch to Distance Learning in Response to the COVID-19 Pandemic: Experience from an Internationally Accredited Undergraduate Pharmacy Program at Qatar University. Medical Science Educator, 30, 1393–1397. https://link.springer.com/article/

  Google Scholar

  Crossref

• Kulikovsky, V. F., Khoschenko, Y. A., Nachetova, T. A., & Nagorny, A. V. (2018). Analysis of the results of the primary accreditation of graduates of the Medical Institute of Belgorod State National Research University, specialty “General Medicine” on the basis of the center of simulation training and assessment of professional qualifications. Scientific statements. Ser. Medicine. Pharmacy, 14(4), 584–590. https://cyberleninka.ru/article/n/analiz-itogov-provedeniya-pervichnoy-akkreditatsii-vypusknikov-meditsinskogo-instituta-niu-belgu-spetsialnosti-lechebnoe-delo-na-baze

• Pavlenko, V. I., Zhukovets, I. V., & Loskutova, N. V. (2018). Primary accreditation of a specialist as an indicator of the quality of the organization of the educational process in a medical university. Amur Medical Journal, 1–2(20–21), 68–70. https://cyberleninka.ru/article/n/pervichnaya-akkreditatsiya-spetsialista-kak-indikator-kachestva-organizatsii-obrazovatelnogo-protsessa-v-meditsinskom-vuze

• Prasad, H. L. K., Prasad, H. V. K., Sajitha, K., Bhat, S., & Shetty, K. J. (2020). Comparison of Objective Structured Practical Examination (OSPE) Versus Conventional Pathology Practical Examination Methods Among the Second-Year Medical Students-a Cross-sectional Study. Medical Science Educator, 30, 1131–1135. DOI:

  Google Scholar

  Crossref

• Sizova, J. M., Semenova, T. M., & Chelyshkova, M. B. (2017). Assessment of the professional readiness of healthcare professionals during accreditation. Medical Bulletin of the North Caucasus, 12(4), 459–463. https://medvestnik.stgmu.ru/files/articles/650.pdf

• Vakhitov, M. S., Vlasov, T. D., Kameneva, E. G., Avramenko, E. A., Orlova, S. A., Aleksandrin, V. A., & Belash, V. A. (2019). Primary accreditation of specialists. Errors and omissions, their causes. Virtual technologies in medicine, 1(21), 24–26. https://www.elibrary.ru/item.asp?id=39169109