Population Assessment Of Health Care Quality: The Problems Of Research Methodology

Abstract

The paper raises the problem of the population’s assessment of the quality of services provided by medical organizations in the Russian Federation. As a foreign experience, current methods of research on the quality of services in medical institutions in the United States and Western Europe are mentioned. In the United States, the methodology for examining a patient’s satisfaction in inpatient care is standardized and mandatory. In the Russian Federation, the system of independent evaluation of the quality of the provision of services by medical organizations was introduced by normative documents in 2014. The results are public and are posted on a specially created official portal on the Internet. In addition, there are other methodological approaches to assessing the quality of services provided by medical organizations in Russian regions. The article confirms the diversity of existing methods and procedures for assessing the quality of services provided by medical organizations in Russia. The authors have been established that most of the methods used in the questionnaires focus on ascertaining a patient’s satisfaction with the medical care. The results of the assessment of patient satisfaction with medical care in Russian subjects (the Samara Region, the Tula Region, the Tyumen Region and the Chuvash Republic) were presented. Problems of independent assessment in the Russian Federation related to the organization of interactive interviewing of patients and the quality of the methodological support were identified.

Keywords: Independent evaluationmedical organizationpatient surveysquality of health services

Introduction

The modernization of health care involves the active participation of the population in the assessment of the quality of health care provided along with the modification and improvement of medical technologies and the organization of the health system. Traditional health indicators (morbidity, mortality, etc.) do not fully reflect the level of health care provided to the population and cannot serve as unambiguous assessments of health care. Socio-medical health indicators, which can be considered as a variety of «social indicators» characterizing health outcomes, are widely used (WHO/OECD/World Bank, 2018). The legislation of the Russian Federation defines the quality of medical assistance as a set of characteristics reflecting the timeliness of the provision of medical assistance and the correctness of the choice of methods of prevention, diagnosis, treatment and rehabilitation in the provision of medical assistance, as well as the extent to which the planned output has been achieved (Federal Law of 21.11.2011 No 323-FZ). Health care quality indicators recommended by the WHO European Office include: the level of professionalism of health personnel; the efficient use of resources; the safety of health care - minimum risk of injury and disease due to medical intervention, observance of medical technologies; a patient’s satisfaction with medical care (WHO Documentation Centre in Russia, 1993). A patient’s satisfaction with health care is an estimate of the objective state of the health care system on the one hand and its emotional perception of by an individual on the other. Satisfaction with health care, which depends directly on the level of patients’ expectations, comprises a number of factors, most of which are subjective (Anderson et al., 2020). The researchers involved note that the results of the satisfaction assessment of a quality provided by an organization depend on how the concept itself is interpreted and operationalized and what kind of survey methodology is used (Kondratova, 2016).

Problem Statement

Systematic research on the social efficiency of health care, economic and political transformations, and the development and construction of a social monitoring system have been actively carried out since the late 1940s in the United States and Western European countries. Systems to monitor patient satisfaction with quality of care have been set up in these countries. In the United States, the methodology for examining patient satisfaction in in-patient care is standardized and mandatory. Since 2002, hospitals have been using the HCAHPS (Hospital Consumer Assessment of Health Care Providers and Systems) system. The questionnaire consists of the following units: medical care provided by nurses and doctors - assessment of courtesy, respect, attention and communication of medical personnel; quality of the hospital environment; experience of the hospital assistance from the staff for the care and control of a patient’s pain; understanding the medical care after discharge from the hospital and the data on the patient (HCAHPS, 2020). An important fact is that patients fill in the questionnaire about a month after their discharge from the hospital and not in the hospital itself, under the supervision of the medical staff and the patient’s dependence on the hospital. Currently, HCAHPS annually provides data on the quality of medical performance of clinics in the United States, which underlie the ranking of medical institutions in the country. The public presentation of results creates incentives for clinics to improve the quality of care.

In Europe, the PPE-15 (Picker patient experience) questionnaire is widely used at the state level and includes the following modules: awareness; psychological and physical condition; respect for the patient; involvement of family and friends; and continuity of care (Jenkinson et al., 2002). In the Russian Federation, an independent assessment of the quality of service delivery by medical organizations was introduced in 2014 as a form of public control being carried out for the purpose of providing citizens with information on the quality of the provision of services by medical organizations, as well as for the purpose of improving the quality of their activities (Federal Law of 21.07.2014 No 256-FZ).

An independent assessment of the quality of service delivery by healthcare organizations is carried out by public boards of independent quality assessment not more than once a year and not less than once every three years for the same health-care organization. In order to realize these goals, a portal for the independent assessment of the quality of health-care service delivery by organizations has been created. Patients make their assessments by filling in an interactive questionnaire on the official websites of public health-care authorities and medical organizations participating in the implementation of the Programme of state guarantees of free medical care for citizens. The same form can be filled out in a medical organization on paper. Questionnaires for all types of medical care were developed by the Ministry of Health of the Russian Federation in 2013, revised in 2015 and in 2017. They include questions that meet the established criteria for assessment: openness and accessibility of information about the medical organization; comfort in the conditions of the provision of medical services, including waiting times for the provision of medical services; courtesy of the staff of the medical organization; satisfaction with the conditions of service provision, accessibility of health care services for the disabled (Federal Law of 05.12.2017 N 392-FZ). All this raises the issue of the representativeness of the data received when completing online questionnaires on official websites, as not all patients are aware of the procedure and do not have access to new information and communication technologies (Kurnikova et al., 2020). There are also traditional forms of viewing patients' opinions on the medical services provided through social surveys at their place of residence or in medical institutions. In this case, it is difficult to compare medical facilities both within the Russian regions and across the country (Butova et al., 2018; Karpov & Makhnev, 2009). A significant factor is that such research involves considerable material costs. Thus, the challenge now remains to improve the assessment of the quality of care provided by health-care organizations in a manner that is scientifically sound and efficient.

Research Questions

The research questions for this paper were:

Is the diversity of existing methods and procedures for assessing the quality of the services provided by medical organizations in Russia confirmed? What are the results of the assessment of patient satisfaction with medical care in Russian subjects? How differentiated are they? Does the assessment of a patient’s satisfaction with health care take into account aspects of the quality of service delivery by health-care providers? What are the main shortcomings of the methods used to assess the quality of service delivery by health-care providers? These questions set the goal and the research methods.

Purpose of the Study

The research is aimed at identifying problems in the application of existing methods for assessing the quality of services provided by medical organizations in Russia. The main focus of the study is a patient’s satisfaction with health care. The research into the methods used in the study of the population’s assessment of the quality of care in medical institutions is aimed at developing a uniform methodology. Given the significant regional differences in Russia, its application will increase the representativeness of the data. It will be possible to build rankings of medical institutions not only in each individual region, but also across the whole Russia. The application of the results of the population’s assessment of the quality of service delivery in health-care organizations is extremely broad. They go beyond the healthcare issues and provide research opportunities on the standard and quality of life of the population.

Research Methods

The research involved general scientific methods of logical, comparative analysis and sociological methods. The study included the analysis of: 1) the results of sociological research in Russian subjects implemented as part of the initiative programme «Problems and trends of sociocultural evolution of Russia and its regions» in 2015-2016 (Lastochkina, 2015). The sample included four regions: the Samara Region, the Tula Region, the Tyumen Region and the Chuvash Republic. The survey was conducted at the place of residence. In each region, 1,000 respondents were interviewed; 2) procedure and results of opinion polls of patients on the portal of independent evaluation of the quality of services in medical institutions of in-patient care in 2017-2019. In this case, the sample included three Russian regions: the Samara Region, the Vologda Region, the Chuvash Republic.

Findings

The generalized results of sociological surveys carried out in four Russian subjects (the Samara Region, the Tula Region, the Tyumen Region and the Chuvash Republic) are characterized by the following provisions. The level of satisfaction of the population with healthcare services can be assessed as generally insufficient, and the evaluation for this indicator fluctuates between 14 and 30 per cent. At the same time, the lowest estimates of the population’s satisfaction with medical care (14 per cent) were obtained the Samara Region (Isupov et al., 2016).

Figure 1 presents the main shortcomings of medical care in public health institutions noted by respondents from the Samara region in 2015. These deficiencies can be considered as reasons for the dissatisfaction of patients.

Figure 1: Figure 1. Residents of the Samara Region on deficiencies in public health services
Figure 1. Residents of the Samara Region on deficiencies in public health services
See Full Size >

Source: authors.

The problem of long waiting lists is among the main shortcomings in the provision of public health services. 58 per cent of respondents in the Samara Region, 47.4 per cent the Tyumen Region and 72.3 per cent in the Tula Region indicated the importance of this problem. On average across the regions, one in five respondents complained about impolite, neglectful treatment of doctors, medical personnel with patients. In the Samara Region, the problem was indicated by 20.6 per cent of respondents.

The population’s claims on health care are systemic. the process of providing all types of health care. For example, lack of urgency in the provision of medical care is a matter of particular concern. Almost 30 per cent of respondents referred to this fact. The examination of the procedure and results of voting (opinions) of patients on the portal of the independent evaluation of the quality of services in medical institutions of in-patient care in three Russian subjects (the Samara Region, the Vologda Region, the Chuvash Republic) led to the following conclusions.

In all three regions, since 2015, the level of satisfaction and the assessment of the quality of service delivery in health-care organizations has been steadily increasing. For specific health-care facilities, grades have been 100 since 2018. Every year, the rating of health-care facilities changes significantly. The difference can be up to 30 positions, both upwards and downwards. In the former case, the facilities and staff in the institutions concerned may have undergone substantial modernization, while in the latter case the equipment and staff have become obsolete. That aspect goes beyond the scope of the paper and requires a separate study.

When completing the questionnaire online, the problem of the representativeness of the results of the survey arises. In essence, it is the view of a certain audience that is aware of the existence of an independent quality assessment portal for health-care providers and has digital Internet skills. Note also that part of the questionnaire (its size cannot be determined) is filled in on paper, the information from which is then transferred to the portal.

Shortcomings were also identified in the form of the questionnaires posted on the portal:

  • no data on patients: age, sex, social status, etc. is presented;

  • the questionnaire does not allow for a separate assessment of doctors and nurses;

- in the answer scale only two options «yes» and «no» are used, which significantly reduces the sensitivity of the scale.

Conclusion

The modernization of social institutions is very different from technological modernization. Social institutions change not only slowly, but often in ways that are not defined as targets for modernization. Social institutions are sufficiently influenced by the interests of the participants in the process and by well-established sociocultural norms of behaviour. This is also true, to a large extent, for the development of an assessment of the quality of service delivery in healthcare organizations. The assessment of the quality of service delivery at health-care facilities, introduced as a form of public monitoring and information on the quality of service delivery by health-care organizations due to poor formality and opacity of procedures could become another campaign and a simulation of the process of health care system modernization. In this connection, it is necessary to develop a uniform methodology for assessing the quality of the services provided by medical organizations in our country, drawing on the best foreign practices. For adaptation purposes, it is advisable to perform pre-verification in order to take into account different regional contexts (Tradori et al., 2019).

We believe that the most effective method (if the results are sufficiently representative) would be to interview patients in health-care institutions after a visit or discharge from a health-care institution. The expert community should develop a standardized questionnaire that evaluates the medical services provided by nurses and doctors including the courtesy, respect, care and communication of medical personnel; quality of the hospital environment; experience in the hospital - care and pain service by management staff; understanding of post-discharge care and data on patients. Patients could receive the questionnaire directly at the facility during the visit or discharge, then fill it out at home and return it to the facility.

References

  1. Anderson, P.M., Krallman, R., Montgomery, D., & Kline-Rogers, E. (2020). The relationship between patient satisfaction with hospitalization and outcomes up to 6 months post-discharge cardiac patients. Journal of Patient Experience.
  2. Butova, T. G., Yakovleva, Ye. Yu., Danilina, Ye. P., & Zhlnikova, M. Yu. (2018). Assessment of medical service quality in the context of changing industrial statutory and legal regulations. Service Plus, 12, 88-99.
  3. Federal Law of 05.12.2017 N 392-FZ "Amendments to selected legislative acts of the Russian Federation on independent evaluation of the quality of services provided by organizations in the areas of culture, social services, health and education". http://base.garant.ru/71825214/
  4. Federal Law of 21.07.2014 No 256-FZ "Amendments to selected legislative acts of the Russian Federation on independent evaluation of the quality of services provided by organizations in the areas of culture, social services, health and education". http://base.garant.ru/70701066/
  5. Federal Law of 21.11.2011 No 323-FZ "On the basics of protecting the health of citizens in the Russian Federation". http://www.consultant.ru/document/cons_doc_LAW_121895/
  6. HCAHPS (2020). About the survey. https://hcahpsonline.org
  7. Isupov, A. M., Karpov, I. V., Martyshkin, S. A., Prokhorov, D. V., Zlaf, V. M., & Shabunin, D. M. (2016). Sociocultural portrait of the Samara region: Evolution and modernization of the region (1989-2015). Publishing House "GLAGOL".
  8. Jenkinson, C., Coulter, A., & Bruster, S. (2002). The Picker patient experience questionnaire: Development and validation using data from in-patient surveys in five countries. International Journal for Quality in Health Care, 14(5), 353-358.
  9. Karpov, O. E., & Makhnev, D. A. (2009). The methodology for organizing patients’ surveys to assess their satisfaction with the quality of health care. Bulletin of the National Medical and Surgical Center named after N.I. Pirogov, 4(1), 68-72.
  10. Kondratova, N. V. (2016). Methodological approaches for patient satisfaction survey in medical organization. In the World of Scientific Discoveries, 5(77), 23-37.
  11. Kurnikova, M., Koroleva, E., & Chirkunova, E. (2020). Digital competences of rural population: Theory, estimation methodology and research experience. European Proceedings of Social and Behavioural Sciences, 82, 545-553.
  12. Lastochkina, M. A. (2015). Scientific life: Research in socio-cultural modernization of Russian regions. Economic and Social Changes: Facts, Trends, Forecast, 5(41), 211-222.
  13. Tradori, V., Brescia, V., & Biancone, P. (2019). Dal concetto di qualitа multidisciplinare in sanitа a un modello di verifica. Economia Aziendale Online - Business and Management Sciences International Quarterly Review, 10(4), 607-625.
  14. WHO Documentation Centre in Russia (1993). Continuous quality development: A proposed national policy. https://whodc.mednet.ru/en/main-publications/kachestvo-mediczinskoj-pomoshhi/118.html
  15. WHO/OECD/World Bank (2018). Delivering quality health services: A global imperative for universal health coverage. http://documents1.worldbank.org/curated/en/482771530290792652/pdf/127816-REVISED-quality-joint-publication-July2018-Complete-vignettes-ebook-L.pdf

Copyright information

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

About this article

Publication Date

30 April 2021

eBook ISBN

978-1-80296-105-8

Publisher

European Publisher

Volume

106

Print ISBN (optional)

-

Edition Number

1st Edition

Pages

1-1875

Subjects

Socio-economic development, digital economy, management, public administration

Cite this article as:

Koroleva, E. N., Shabunin, D. M., & Martyshkin, S. A. (2021). Population Assessment Of Health Care Quality: The Problems Of Research Methodology. In S. I. Ashmarina, V. V. Mantulenko, M. I. Inozemtsev, & E. L. Sidorenko (Eds.), Global Challenges and Prospects of The Modern Economic Development, vol 106. European Proceedings of Social and Behavioural Sciences (pp. 1518-1524). European Publisher. https://doi.org/10.15405/epsbs.2021.04.02.181