Comparative Analysis Of Examination Materials In Geography In Russian And Foreign Schools

Abstract

The paper reports on the comparison of school geography examination materials of Great Britain and Russia (Basic State Exam). In both countries the examination materials include specification and sample examination paper with the system of evaluation of students’ responses; the examination papers assess all studied geographical content. Unlike Russia in Great Britain school geography examination paper consists of three parts; the examination paper includes questions related to students’ field research materials that are carried out during the school year. In Great Britain each part of the examination work includes mini-tests consisting of a different number of items. In both countries there are many items based on the different sources of information. The structural unit in Russian examination paper is an item. Comparison of item types showed that they are similar in many ways, but similar types of items have different instructions for recording responses. The paper compares open-ended items of examinational materials in the countries. The comparison of school geography examination materials of Great Britain and Russia revealed that in both countries students are asked to explain geographical processes or phenomena in some examination papers items, however, these items assess different skills. Unlike Russia in Great Britain the test is mainly aimed at the assessment of ability to describe or explain the cause-and-effect relationships. Unlike Russia in Great Britain the criteria for some items include assessment of written communication level. The authors suggest some questions on possible ways of further improvement of examination work of geography BSE in Russia.

Keywords: Assessmentschoolgeographytestexamstandard

Introduction

The problem of evaluation the school education quality is widely discussed in the professional community now. Education is being modernized in many countries, learning objectives are being changed and curricula are being adjusted. Informatization of society, globalization, development of technologies force educators to take steps leading school education meet modern challenges (National Research Council, 2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas, Jones, Wheeler & Centurino, 2015). The development of the system of educational achievements of students evaluation, including examination materials occurs (Barabanov, Ambartsumova & Dyukova, 2015; Kovaleva, Krasnyanskaya, 2016; Kautz, 2014; Bijsterbosch, van der Schee, Kuiper, 2017; Race, 2014). The objective assessment of the level of achievement of educational results in many school subjects, including geography are discussed (Ambartsumova, 2014; Ambartsumova, Barabanov & Dyukova, 2018; Bijsterbosch, van der Schee, Kuiper, 2017; Race, 2014; Rangvid, 2015; Bourke & Lane, 2017). Requirements to ensure the qualitative and objective assessment of educational achievements of a large number of graduates make it necessary to create standardized examination materials (Efremova, 2017). In this work, under examination materials, we understand the variants of examination papers, and documents that define their content and structure. By the content of the examination paper, we mean the content to be tested in an external examination.

As a structure of examination paper we mean all parts it includes, the number and types of items. As the item we mean the test task, which clearly defines what is considered to be the correct answer, and in case we analyze an answer constructed by a student the item has got the description of answers possible levels of completeness. Each test task has a standard execution instruction, the standard of the correct answer on the evaluation of the results of its execution (Zvonnikov & Chelyshkova, 2012). It should be noted that instructions for the test tasks implementation are also parts of the test tasks. The instructions contain information what the student should do, how to perform the task, how and in what form write the answer. Instruction, providing accessibility of items and understanding the modalities for all subjects, is an important condition for the successful performing of examination work items.

Standardization of examination papers assumes a uniformity procedure of the exam, including a unified form of presentation written examination wok, a certain time to perform the examination work, instructions for carrying out the work as a whole and its parts (Chelyshkova, 2002).

The comparative analysis of examination materials used in different countries allows to find out possible ways of improvement of examination materials used for the purpose of the state final certification of graduates.

In this research, we analyzed the Russian examination materials of the Basic State Exam (BSE) in geography, which is passed by the 9th grade students at the end of basic secondary school. It is not an obligatory exam, students can choose it among other subjects. We compare examination materials of BSE in Russia and geography exam of Great Britain.

Problem Statement

For an objective assessment it is important to have measuring instruments that provides for controlling the level of geographical knowledge and skills, competencies fixed in the regulatory documents that sets aims and content of geographical education.

The issues that defines the necessity to find out ways that can improve the geography examination materials in Russia, are connected with the necessity to assess the requirements of Federal State Educational Standard of General Secondary Education (Federalny gosudarstvenny obrazovatelny standart osnovnogo obshchego obrazovaniya, 2012). This document has begun to define school education in Russia and the Basic State Exams will be developed in accordance with this document in 2020.

The existing examination materials in geography do not assessed the achievement of some substantive learning outcomes defined in the Federal State Educational Standard of General Secondary Education. These results include the ability to use instruments and tools to determine the quantitative and qualitative characteristics of the components of the geographical environment, the skills of finding, using and presenting geographic information. The achievement of these results can be evaluated on the basis of performance of students of educational projects, but a single criterion of their estimation is not currently developed. It makes impossible to implement external objective evaluation of the implementation by students of such educational projects.

Comparison of examination materials of BSE in geography in Russia and the geography exam of UK allows us to identify common features and differences of examination materials. The identified features will allow to analyze approaches to assessing those learning outcomes that are not currently being assessed in Russia, and to analyze the possibility of using elements of this experience in the further improvement of examination materials. Comparison of the types of items, checking the same content, will allow to evaluate assess the existing approaches and see the similarity and difference in the forms of objective assessment and criteria identified as indicator of the level of achievement of the verifiable elements of the educational progress of students.

Research Questions

In order to determine the elements of foreign experience in assessing the educational achievements of students in the final assessment, which is appropriate to use to improve the examination materials on geography in Russia, we formulated the following research questions:

• Compare the content of examination materials, highlight the elements of the content of school geographical education, the mastering of which should demonstrate students in both countries.

• Compare the structure of examination materials, the types of items, and the instructions for their performing, used in the measuring materials of the two countries to check the same content and the same skills.

• To study experience of development of the examination materials which are aimed at measuring the abilities to use devices and tools for determination of quantitative and qualitative characteristics of components of the geographical environment, finding, use and presentation of geographical information.

Purpose of the Study

The aim of this work is to improve the geography examination materials of the Basic State Exam in the modernization of education in Russia on the basis of comparison and study of the experience of the development of examination materials in the UK.

Research Methods

For comparison with the Russian examination materials, we have chosen the examination materials of UK as the country whose experience in assessing the educational achievements of students is used in many countries. Besides the examination materials on the geography of this country includes questions related to the use of the results of field studies of students and the presentation of information obtained in the course of field studies. The possibility and methods of assessing these skills is currently being discussed in our country due to the need to improve the measuring materials of the geography BSE in accordance with the new Federal State Educational Standard of the Secondary Education.

During the research, the analysis of normative documents regulating the aims of the assessment, the content of measuring materials, instructions for evaluation was carried out. The analysis of literature, reporting materials, as well as publications in scientific journals reflecting the problems of the chosen topic were carried out.

The structure of examination papers, forms and content of the items of examination papers of the two countries were compared. We selected and compared the ways of presentation of several elements of the geography content in the examination papers. These several elements of content are related to the study of natural processes on Earth – earthquakes and volcanoes, weather and climate.

We did not compare the content of the school geographical education of the two countries, as it did not meet the purpose of our study. We did not take into account the procedure of the exam, namely – the time of the work.

Findings

The content and structure of examination materials on geography in Russia and the UK are determined as by historical differences as school geographical education, as by existent systems of assessment of educational achievements of students.

At the same time, we can find the grounds for comparison these materials to identify the most effective ways to assess the educational achievements of students in geography, corresponding to the global trends in the development of school geographical education.

The content of the geography examination papers for a secondary school course in the UK is determined according to the school geography education specification. Such specifications are developed by several organizations, and each school has the right to choose its own specification (Shemelina, 2016).

For comparison with the examination materials of Russia, we have chosen Specification in Geography and Specimen (Assessment Materials for Geography) developed by the Council for Curriculum, Examinations and Assessment of the UK. In the specification (programme) selected meaningful sections that are divided by topics and intended learning outcomes for each of the topics. Thus, the specification of examination materials serves the same role that approximate programmes in subjects perform in Russia. The UK specification also defines the structure of the examination paper and the proportion of each part of the examination paper to the maximum score for the performance of all work assignments and the time assigned to each part of the examination work.

The examination materials also include a demonstration version of the examination paper with instructions on to evaluation each of the item and examples of items on each topic.

In Russia examination materials for BSE are unified for the country. They are published annually on the website of the Federal Institute of Pedagogical Measurements initially as a project that is under discussion, and after the discussion as approved materials. They include three documents: 1) specification, 2) so-called codifier (list) of the elements of content and requirements for the level of students’ skills for the BSE in geography, 3) demonstration version, at the end of which the system of assessment of examination work and criteria for evaluation of open-ended answer (OGE. Demoversii, spetsifikatsii, kodifikatory. Geografiya, 2018).

The Specification of measuring materials for conducting the Basic State Exam in geography in 2018 includes:

• designation of BSE;

• documents defining the content of measuring materials;

• approaches to the selection of content, the development of the measuring materials structure;

• characteristics of the structure and content of measuring materials;

• distribution of measuring materials items in terms of content, skills and ways of activity, items complexity;

• additional materials and equipment;

• system for assessing the performance of items and examination work in general and the duration of the exam;

• generalized plan of the BSE examination work.

The content of the BSE 2018 is determined on the basis of the Federal Component of the State Standard of The Basic General Education In Geography (the order of the Ministry of Education of Russia No. 1089 of 05.03.2004).

Selection of the content to be checked in the examination work for the BSE 2018 is carried out in accordance with the obligatory minimum content of the main educational programmes and the requirements for the level of graduates’ knowledge and skills of the Federal Component of the State Standard of Basic General Education in Geography.

In each variant of the BSE measuring materials 2018 items checking the level of knowledge of the contents of all the main sections of the geography course for the main school and the achieving of the basic learning outcomes for the level of graduate training are included.

In BSE measuring materials much attention is paid to requirements for learning outcomes aimed at the practical usage of geographical knowledge and skills. It is also important to check the ability to extract and analyze data from various sources of geographic information (maps of atlases, statistical materials, diagrams, media texts). It is allowed to use rulers, non-programmable calculators and geographic atlases for grades 7, 8 and 9 during the geography exam.

The generalized plan of examination work is a part of Specification of BSE for 9 grade graduates.

The generalized plan of examination work is also presented in the examination materials of Great Britain. The difference between the two generalized plans can be defined. The generalized plan of BSE shows the exact number of items in the work, and only the number of blocks of items consisting of several mini-tests on each of the topics in the CCEA GCSE examination paper plan are shown. At the same time, the number of items (in our understanding) that make up each of the mini-tests can be different and differ from the number of items in the corresponding mini-tests presented in the demonstration version.

Geography examination materials of the two countries are aimed at assessing the students’ knowledge of all content of geography school courses. CCEA GCSE examination paper includes two of the large content sections of school geography, fixed in the Specification: section “Understanding Our Natural World”, which includes the questions of physical geography and the interaction of nature and humane society, and the section “Living in Our World”, devoted to the questions of social and economic geography. The third section of the examination paper examines developing of field research, conducted by the students during the school year.

Measuring materials of BSE include items, assessing the content of all sections of geography courses of the secondary school:

• The Sources of Geographic Information.

• World’s Nature and Humanity.

• Continents, Oceans, Nations and States.

• Environmental Management and Geoecology.

• Geography of Russia.

Comparison of the examination papers of the two countries shows that they have got a number of common features determined by their common purposes, and they have significant differences.

Structure of the examination papers

BSE examination paper consists of 30 items which are not divided into any parts, while the items, assessing the content of different sections of the school geography courses alternate regardless of their content.

CCEA GCSE examination paper consists of 3 units. Unit 1 contains questions on the section “Understanding Our Natural World”. Unit 2 – questions on the section “Living in Our World”. Unit 3 contains questions answering which students base their answers on their knowledge and experience of the fieldwork skills.

Each of the units 1 and 2 includes four multi-part questions, one on each of the themes studied in each section The number of items in each of the first two units may vary in different variants of the examination paper, but the maximum score for all the questions in each part is the same and is equal to 100.

An important distinctive feature of the CCEA GCSE geography examination paper is the presence in it of the third unit, assessing the results of the student's performance in the fieldwork (educational project). The research itself can be carried out individually, in a group or by the whole class, but answers questions during the exam each individually. The maximum score for answering all the questions of this unit of the work is 40.

Fieldwork is designed not only to enable the student to apply their geographical knowledge to investigate a real geographic process or phenomenon, but also to contribute to understanding of the process of geographical enquiry;- collecting primary data, processing, presenting, analyzing and interpreting their fieldwork data, drawing conclusions, identifying limitations of the data collected and understanding the limitations of the conclusions drawn (CCEA GCSE Specification in Geography).

Learning outcomes of the fieldwork suppose, that the students would be able to:

• demonstrate understanding of the process of geographical enquiry;

• plan their enquiry by;

• select and use appropriate methods of enquiry,

• collect and present their fieldwork data;

• analyse and interpret their fieldwork data;

• draw evidenced conclusions;

• evaluate their fieldwork.

Assessment for the fieldwork is a written examination. Students must identify one aim and a minimum of two hypotheses, identify sources and methods for collecting data, explain the choice of the hypothesis and research methods, prove that by the methods that they used, are appropriate for collecting the data necessary for the enquiry, describe the obtained dependencies, and indicate the risks. Students must submit a word-processed fieldwork statement and table of data examiners, so that the examiners could mark the answers to questions on field research consistently and fairly the students together with examination paper submit a word-processed fieldwork statement and table of data.

Mark schemes for these tasks and questions which require candidates to respond in extended written form are marked on the basis of levels of response which take account of the quality of written communication.

Types of test items used in the examination papers

The same types of testing items are used both in CCEA GCSE and BSE geography examination papers. These are multiple-choice questions, questions for matching up the elements of two lists, questions for arranging the elements of the list in the correct order, short response questions, and extended writing questions.

At the same time the items of the same type may have different form. For example multiple-choice questions used in BSE paper have four choices, while questions of the same type in CCEA GCSE papers have only three or even two choices.

Answers to the questions for arranging the elements of the list in the correct order and to the questions for matching up the elements of two lists in BSE papers must be written as a sequence of digits in the provided table:

Example 1 (OGE. Demoversii, spetsifikatsii, kodifikatory. Geografiya)

Arrange the following regions of Russia in order of the sequence in which the people living them celebrate the New Year. Write the resultant sequence of digits in table:

• The Republic of Karelia

• Sverdlovsk region

• Transbaikal region

Example 2 (OGE. Demoversii, spetsifikatsii, kodifikatory. Geografiya)

Travel agencies of various regions of Russia have developed slogans to attract tourists to their regions. Set the correspondence between slogans and regions: match each element of the left column, with the corresponding element from the right column.

SLOGANSREGIONS

• Welcome to the coast of Teletskoye Lake - a mountain pearl with a unique beauty!

• Welcome to the Valdai National Park - the "pearl" of the central Russia! Admire the beauty of Lake Ilmen!

• The Republic of Karelia

• Krasnodar region

• The Republic of Altai

• Novgorod region

Write the chosen digits in the table under the corresponding letters.

In CCEA GCSE paper students answering such questions should write elements of the list in the correct order or draw arrows to match up the elements of the two lists.

Example 3 (a question for arranging the elements of the list in the correct order) (CCEA GCSE Specimen. Assessment Materials for Geography)

Complete the below by arranging the following settlements in order of size beginning with the largest. One has been completed for you.

• Durham

• Sherburn

• Edmondsley

• Sacriston

Example 4 (a question for matching up the elements of two lists) (CCEA GCSE Specimen. Assessment Materials for Geography)

Complete the by drawing an arrow to match up each air mass with its correct description.

It should be noted that the proportion of multiple-choice questions and short response questions in the total number of questions and in the maximum score for answering all the questions in the BSE paper is much higher, and the proportion of open-ended questions is lower than in the CCEA GCSE paper.

Both BSE and CCEA GCSE Assessment Materials include mark schemes, for marking responses to the particular questions and the whole paper – indications of marks which should be awarded for a particular response.

Mark schemes for questions which require students to respond in extended written form

Approaches to the construction of criteria for marking responses to the extended writing questions in the two countries are significantly different. Let's consider it on an example of questions assessing the ability to explain geographical processes and phenomena. Both the completeness and correctness of the explanation of a particular response are taken into consideration. But when the correctness of the explanation is understood in the same way, the criteria of completeness of the response is different in BSE and CCEA GCSE Assessment Materials.

In CCEA GCSE Assessment Materials, a complete explanation should include a general statement, consequence and elaboration. For example, in a task where it is required to explain the prevalence of dry weather in an anticyclone, the complete answer implies an indication that there is no rain, due to with the absence of cloudiness, and it should be explained, that the clouds are not formed due to the fact that the air is sinking in an anticyclone, as it sinks the air warms up so condensation cannot occur. That is, a complete correct answer should include a complete chain of cause-effect relationships. In the above example 3 points are awarded for the full correct response.

In BSE examination paper there are also questions that require an explanation of certain geographical phenomena, the complete correct response to which must contain description of the chain of cause-effect relationships. For example, a question requiring to explain how breezes are formed on the sea coasts suggests an answer pointing out that the surface of the land is warming up faster in the daytime than the surface of the ocean, which results in a difference in atmospheric pressure over the land and over the sea, which leads to the formation of winds blowing in the daytime from land to sea. But in this case 2 points are awarded for the full correct response (Ambartsumova, Barabanov, & Dyukova, 2018).

It may be noted, that neither the criteria for assessing the validity of answers responses to such questions nor the sample answers in BSE paper require using the words like "therefore", "consequently", "thus", which are markers denoting the cause-effect relationships.

In most cases, the questions for explanation in the examination papers of the BSE does not require an explanation of the connections between the phenomena. Thus, for example, the explanation of the high seismic activity of the territory implies an indication that this territory is in the region of the boundary of lithospheric plates (Ambartsumova , Barabanov & Dyukova, 2018).

The implementation of the Federal State Educational Standard, which defines the necessity of students’ ability to explain the studied geographic objects and phenomena, will probably require some change in the approaches to assessing the responses to the items, where skills to explain are evaluated. A study of the UK experience can be useful for developing criteria for evaluating students’ responses.

Another important feature of the CCEA GCSE Assessment Materials is that the quality of written communication is taken into account in assessing candidates’ responses to all tasks and questions that require them to respond in extended written form. Quality of written communication is distinguished within levels depending on the accuracy spelling, punctuation and correct using of grammar rules, on students’ ability to present and organize effectively relevant information and to use a style of writing which suits its purpose and the ability the ability to use correctly the appropriate geographical terminology. Using similar criteria for assessing extended answers in BSE papers may contribute to enhancing the graduates’ level of literacy.

Conclusion

The differences between the multiple-choice items and short response items of the same types used in the examination papers of the two countries are due to the fact that in Russia, unlike in the UK, a blank technology is used, which involves the use of scanner and automatic evaluation of students' answers to such types of items. The technology used in Russia is the only one possible for our country with its vast territory and a large number of students taking exams.

At the same time, some features of the UK examination materials can be considered in the developing of the examination materials of the BSE, constructed to conduct the exam according to the Federal State Educational Standards.

It is advisable to study the possibility of including a part aimed at assessing the performance of students' skills to use instruments and tools to find, use and present geographical information in the examination work. It is interesting to see the UK's experience in identifying the list of skills needed to verify how students conduct field studies. The question of the relationship of assessing these skills with educational projects in the examination requires further study. It seems that it is possible to consider the experience of the UK in this area in two aspects: 1) to develop objective criteria for assessing educational projects in geography, 2) to develop models of items for examination papers that verify the formation of these skills, but not directly related to the results of field studies of the examinees.

The study allows us to formulate some conclusions on approaches to the development of criteria for answer the open-ended items. The revealed difference to the approaches used in the development of criteria for assessing the tasks that assess the ability to explain geographical features and phenomena, allows to raise the question of the typology of tasks with a detailed answer used in the BSE to verify the formation of this skill. On the one hand, measuring materials uses items that assess the students’ ability to explain the features of nature or the economy of certain areas. On the other hand, items that test the ability to explain or describe how a natural phenomenon is formed or the process takes place are used. It seems that the tasks requiring establishment and tracing of causal links between geographical phenomena and processes in the existing examination materials of the BSE may not be enough. It is advisable to clarify the concept of "explanation" and "establishment of cause-effect relationships" of geographical processes or phenomena in relation to the construction of BSE items.

It may be advisable to adjust the form a question for arranging the elements of the list in the correct order in the examination papers of BSE, taking into account the fact that graduates write down the answer figures in the reverse sequence (Ambartsumova, 2014) – the form of tasks used in the UK measuring materials prevents this error.

The possibility of assessing the skills of written communication in the framework of the examination paper of the BSE requires a separate research.

Acknowledgments

The study is conducted within the State Assignment of the Institute for Strategy of Education Development of the Russian Academy of Education for 2017-2019 on the subject matter: “Updating the content of general education and teaching methods in the modern educational environment” (27.6122.2017/BP).

References

1. Ambartsumova, E. M. (2014). Osnovnye rezultaty itogovoy attestatsii za osnovnuyu shkolu po geografii i rekomendatsii dlya podgotovki k GIA [Main results of the Basic State Exam for the secondary school on geography and recommendations for the preparation for the BSE]. Geografiya v shkole, no 4, pp.37-48. [in Rus]
2. Ambartsumova, E.M., Barabanov, V. V., Dyukova, S. Ye. (2018). OGE. Geografiya: tipovye ekzamenatsionnye varianty: 25 variantov [BSE. Geography: typical examination papers: 25 examination papers]. Moskva: Natsionalnoye obrazovaniye. [in Rus]
3. Barabanov, V. V., Ambartsumova, E. M., Dyukova, S. Ye. (2015). Uchebno-metodicheskiye materialy dlya predsedateley i chlenov regionalnykh predmetnykh komissy po proverke vypolneniya zadany s razvernutym otvetom ekzamenatsionnykh rabot YeGE 2015 goda [Educational materials for chairmen and members of regional substantive commissions for the assessment of open-response items of the examination papers for the Unified State Examination for Geography 2015]. Retrieved from http://fipi.ru/ege-i-gve-11/dlya-predmetnyh-komissiy-subektov-rf. [in Rus]
4. Bijsterbosch, E., van der Schee, J., & Kuiper, W. (2017). Meaningful learning and summative assessment in geography education: an analysis in secondary education in the Netherlands. International Research in Geographical and Environmental Education, V. 26, no 1, pp. 17-35.
5. Bourke, T., & Lane, R. (2017). The inclusion of geography in TIMSS: can consensus be reached? International Research in Geographical and Environmental Education, V. 26, no 2, pp. 166-176. Retrieved 28 February, 2017 from http://eprints.qut.edu.au/91936/9/91936.pdf.
6. Chelyshkova, M.B. (2002) Teoriya i praktika konstruirovaniya pedagogicheskikh testov». Moskva: Logos. [in Rus]
7. Efremova, N. (2017) Testovy kontrol v obrazovanii. Moskva: Litres. [in Rus]
8. Federalny gosudarstvenny obrazovatelny standart osnovnogo obshchego obrazovaniya (2012). Retrieved 10 February, 2018 from https://минобрнауки.рф/%D0%B4%D0%BE%D0%BA%D1%83%D0%BC%D0%B5%D0%BD%D1%82%D1%8B/543. [in Rus]
9. Jones, L. R., Wheeler, G., & Centurino, V. A. (2015). TIMSS 2015 science framework. TIMSS, pp. 29-59.
10. Kautz, T., Heckman, J. J., Diris, R., Ter Weel, B., & Borghans, L. (2014). Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success (No. w20749). National Bureau of Economic Research. Retrieved 07 February, 2017 from http://www.nber.org/papers/w20749.pdf.
11. Kovaleva, G. S., Krasnyanskaya, K. A. (2016). Analiz sistem otsenki realizatsii obrazovatelnykh programm v Singapure, Yuzhnoy Koreye i Kitaye. [Analysis of assessment systems of educational programmes in Singapore, South Korea and China]. Nauka i shkola, no 1. [in Rus]
12. Kovaleva, G. S., Krasnovsky, E.A., Krasnyanskaya, K. A., Loginova, O. B., Tatur, A. O. Model sistemy otsenki rezultatov osvoyeniya obshcheobrazovatelnykh programm [Model of the system for assessing the results of educational programmes]. Retrieved 15 February, 2018 from www.orenedu.ru/files/koncept/federal/aprobaz/2_11.doc. [in Rus]
13. National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press.
14. OGE. Demoversii, spetsifikatsii, kodifikatory. Geografiya. (2018). [BSE. Sample examination papers, specifications, codifiers. Geography]. Retrieved 05 January, 2018 from http://fipi.ru/oge-i-gve-9/demoversii-specifikacii-kodifikatory. [in Rus]
15. Race, P. (2014). The lecturer's toolkit: a practical guide to assessment, learning and teaching. Routledge.
16. Rangvid, B. S. (2015). Systematic differences across evaluation schemes and educational choice. Economics of Education Review, 48, pp. 41-55.
17. Shemelina, Ye.Yu. (2016). Izucheniye geografii v shkolakh Velikboritanii [Studying geography in schools in the UK]. Geografiya v shkole, 4, pp. 38-42. [in Rus]
18. Zvonnikov, V. I., Chelyshkova M.B. (2012). Otsenka kachestva rezultatov obucheniya pri attestatsii: (kompetentnostny podkhod) [Assessment of the quality of learning outcomes in attestation: competence approach]. Moskva: Logos. [in Rus]