Quality Assessment Of Educational Achievements Of Students On Chemistry In Russian Schools

Abstract

The article covers key methods of monitoring students’ performance in chemistry; those methods make part of Russia’s nationwide education quality assessment system. In particular, the article indicates international monitoring studies investigating knowledge of Russian schoolchildren in natural sciences. The “education quality” concept is brought up, and Russian scholars who have made a major contribution to the establishment of an education quality assessment system are mentioned. Peculiarities of the research of the matters currently studied within the framework of the school course in chemistry are outlined, and skills and abilities built in that course are specified. In addition, principal requirements for test-and-assessment materials used in the final certifying chemistry examination are listed. Emphasis is made on the importance of a combination of a competency-based and subject-based approach to the monitoring of graduates’ performance. Sample assignments are given; those assignments are included in various standardized monitoring assessment procedures. When analyzed, the contents of the assignments helped stress the significance of universal learning activities in the course in chemistry, which is dictated by the specifics of the chemistry-related contents that includes a sign system, images and models of objects, as well as real-life objects. The concluding part of the article underlines the importance of using element-by-element analysis in developing an assignment assessment scale.

Keywords: School chemical education»quality assessment»educational achievements of students

Introduction

Improvement of a system of monitoring and assessment of efficiency of the educational process is a priority at all levels of Russian education. High importance of getting objective results of test-and-assessment procedures is dictated above all by the need to provide reliable information to all those involved in the pedagogical process: students, teachers, parents, administration of educational institutions, employees of education management bodies; that information concerns the education system as a whole and its individual components.

Depending on the targets of the test-and-assessment procedure, a certain assessment toolbox must be used; it must have a monitoring foundation and a screening orientation as its main characteristics. It means that, on the one hand, there must be a system of measures that permits multifaceted monitoring studies of the quality of the educational process, and, on the other, we need a package of assessment materials, which must both reflect the current condition of the education system and identify issues in the system’s operation. Above all, it is important for subsequent decision-making, which must be of remedial nature, i.e. must enhance the quality of the education. The “quality of the education” concept means “the ability of an educational product or service to comply with the rules of the state standard and commissioning of social services.”

The process of building a quality assessment system in Russia began more than a decade ago. It is now called Russia’s nationwide education quality assessment system (Russian acronym: OSOKO) and includes both federal-level procedures and international monitoring studies:

• final certifying examination of school graduates conducted in the form of a unified state examination (for 11th graders) and a basic final certifying examination (for 9th graders);

• a nationwide test (NT) conducted at various school cycles in different subjects;

• national education quality studies conducted on the basis of a representative sample of educational institutions of various regions of Russia;

• a program for international student assessment of scholastic performance in mathematics, science, and reading (Programme for International Student Assessment, or PISA) ( Carnoy, Khavenson, Loyalka, Schmidt, Zakharov, 2016);

• international assessments of the mathematics and science knowledge of students (Trends in Mathematics and Science Study, or TIMSS), and assessment of student learning in reading (Progress in International Reading Literacy Study, or PIRLS) ( Polivanova, 2015), ( Zuckerman, Kovaleva, & Kuznetsova, 2013);

• an international comparative study of the faculty (Teaching and Learning International Survey, or TALIS, and Programme for the International Assessment of Adult Competencies, or PIAAC) run under the auspices of the Organization for Economic Co-operation and Development (OECD).

Problem Statement

What are the tendencies in the development of the assessing quality system for education considering the implementation of standards and Russian participation in international monitoring surveys of the educational quality?

As can be seen from the above list, the studies included in the OSOKO are diverse in their focus, which gives a sufficiently complete picture of development of the Russian education system both on the nationwide scale and in individual areas of knowledge or subjects of the curriculum ( Polivanova, 2015; Bolotov & Valdman, 2013 ; Kovaleva & Loginova, 2011; Chelyshkova, 2002; Yefremova, 2007) and other scholars played a major role in the establishment and development of the OSOKO as a whole and of its individual components. Works of those scholars expound the philosophy underlying the construction of its elements, both in terms of organizational and managerial approaches, and from the standpoint of practice of designing pedagogical tests, processing of their results, international and domestic practices of using assignments of varied types, forms and degree of difficulty or complexity. The methodological substantiation of the theory and practice of research in the assessment of education quality helped significantly improve the quality of assessment materials and the efficiency of the use of the findings.

The practice of organizing monitoring and test-and-assessment procedures shows that their findings can be used for the design and implementation of a wide range of activities at various levels of the education system, from enhancement of the learning process in a specific educational institution to large-scale measurements in the country’s education system: adoption and improvement of state educational standards, update of curricula and study materials in various subjects, modification of programs for basic and advanced training of teachers, development of measures to aid underperforming schools.

All of the above-mentioned opportunities of using the findings of the OSOKO procedures are immediately relevant for school chemistry education. It stems from the fact that chemistry, as a fundamental science determining the process of building an integral scientific worldview in students, has not been in the forefront of the school education system in the past few decades. For instance, in recent surveys among school graduates, the importance of chemistry as a school subject was mentioned only by 9% of those polled, and 29% listed it among their top three least favorite school subjects. Giving arguments explaining their opinion, the respondents cited “overcomplexity,” “fundamental nature” and “theoretical slant” of the school course in chemistry, which makes it “disconnected from real life” ( Dobrotin, 2016).

Research Questions

What components of the schoolchildren preparation in chemistry are directed to the assessing quality system of education? What are the prospects for the Russian assessing quality system of chemical education?

Purpose of the Study

Identify main elements of the system for assessing the quality of school chemical education and formulate the main trends in its development, taking into account the introduction of state educational standards and Russia's participation in international comparative studies of the quality of education.

Research Methods

In the process of work theoretical and experimental methods of research are used:

Findings

It must be noted that one requirement for the test-and-assessment materials is their strict correspondence to the contents and list of contemplated results, which must be achieved by students at a certain stage of learning according to requirements of the standard. Taking into account the fact that the contents of the school course in chemistry can be studied at various levels, the test-and-assessment materials of the unified state exam in chemistry must help differentiate students with varied levels of skills and knowledge. For instance, for the graduates who plan further studies in natural science colleges or medical schools, assignments of test-and-assessment materials must be both of fundamental and applied nature.

Students who have no plans of studying chemistry in college generally study chemistry at the basic level and within a limited scope, and are not highly motivated for learning. In that case, a question arises about the substantive foundation for the development of assignments, which, on one hand, must purport to test the knowledge of basic concepts, and, on the other, must showcase the significance of chemistry knowledge in everyday life.

Russia’s participation in international monitoring studies of education quality has played a major role in addressing the above challenges. In particular, PISA and TIMSS, two studies assessing maturity of students’ science-related knowledge and skills, use a competency-based approach when designing their assignments ( Dobrotin, 2018c). The following are the principal tested items in the above-mentioned studies: understanding, by schoolchildren, of the boundaries of the use of scientific cognition methods, an ability to characterize phenomena of the world around us, to formulate conclusions and to cite them to support scientific facts and proofs, to demonstrate understanding of the role of natural sciences in society’s development, and to show one’s civic stance in addressing social, environmental and other issues ( Kaverina , Molchanova, Sviridenkova, & Snastina, 2017).

Thus, Soviet and Russian practices of designing means of monitoring and assessment of the quality of students’ knowledge in chemistry, and best practices of international specialists in the assessment of the quality of students’ science knowledge have been taken into account when developing test-and-assessment materials for the basic final certifying examination (summarizing middle-school studies) and the unified state examination (summarizing high-school studies), and also when developing assignments for Russia’s nationwide test (NT) conducted in the form of a final test for 11th-grade students of non-majoring classes ( Dobrotin, 2018b).

Below we give samples of the assignments used within the framework of final test-and-assessment procedures.

Example of assignment 7 ( 2017 Unified State Exam, advanced level of complexity)

Alkali solution Y was added to a test tube with a solution of substance X. As a result, a reaction occurred, which is described by the following shortened ionic equation: HS ^– + OH ^– = S ^2– + H ₂O

From the list below, select substances X and Y that can undergo the above reaction.

• Hydrosulfuric acid

• Potassium hydrosulfide

• Ammonium hydrosulfide

• Potassium hydroxide

• Strontium hydroxide

Select the appropriate numbers and write them down in the table below under corresponding letters.

In order to complete the assignment, students need to analyze the information represented in the form of names of compounds and of the shortened ionic equation, to convert the names using a sign system, i.e. the language of chemical formulas, and to recall chemical properties of the compounds on the list. They also must use the solubility chart, in order to juxtapose the properties of selected compounds with the proposed shortened ionic equation of the reaction. Thus, the assignment tests not only the chemistry-related but also the competency-based component of students’ knowledge, which must include the ability to work with symbolic information ( Dobrotin, 2017a).

The next group of assignments, which are united by a common context, also purports information analysis and processing. However, considering the non-chemistry major of the classes taking part in the NT, the definition of the assignment tests, to a greater extent, the ability to reason, to extract information from a chemical text and to convert it into a new form (schematic format). Let us give examples of assignments 5 to 7 of the 2018 NT ( Dobrotin, 2018d);

Read the text and complete assignments 5 to 7.

Carbon dioxide (СО ₂) is used in large quantities in the food industry to manufacture fizzy drinks, sodium carbonate and urea. In the industry, carbon dioxide is produced in various fermentation processes and in limestone calcination (CaCO ₃). Additionally, it is produced in large quantities in combustion of coal or carbonaceous substances — for instance, when carbon monoxide (СО) interacts with oxygen.

To measure the percentage of carbon dioxide in the air, its reaction with calcium hydroxide (Са(ОН) ₂) is used.

Another area where carbon dioxide is used is fire-fighting. Carbon dioxide is produced in foam extinguishers when sulfuric acid (H ₂SO ₄) interacts with sodium carbonate (Na ₂CO ₃) or sodium bicarbonate (hydrogen carbonate) (NaHCO ₃), to which a foaming agent is added. Foam extinguishers have one downside: the produced СО ₂ carries drops of sulfuric acid. That drawback is not present in extinguishers filled with dry ice, which is the solid form of carbon dioxide.

Assignment 5

Inorganic compounds can be conditionally divided in (or classified by) four groups, as the diagram shows. For each of the four groups on the diagram, insert one chemical formula of the compounds out of those mentioned in the above text.

See Full Size >

Assignment 6

• Compose a molecular equation of the reaction of carbon oxide (II) with oxygen (the reaction was mentioned in the text).

• Specify which thermal effect (heat absorption or heat liberation) this reaction is accompanied with.

Assignment 7

• Compose a molecular equation of the reaction between carbon dioxide and calcium hydroxide, which was mentioned in the text.

• Describe indicators of a reaction proceeding between carbon dioxide and calcium hydroxide.

One distinct peculiarity of the assignments used in the above-mentioned formats of final tests is availability of clear-cut assessment criteria. In order to achieve maximum objectivity in the assessment, the criteria proposed for the assignments must include a scale that reflects a step-by-step opportunity of getting points for each correct element of the solution (answer). In that case, one can say that assessment criteria are built on an element-by-element analysis of the solution. Generally, every element of the solution (answer) is underpinned by a certain intellectual operation, which, in turn, is an element of the skill or ability that needs to be tested, according to requirements of the standard ( Dobrotin, 2018a). Thus, assignments included in test materials of the unified state examination, the basic final certifying examination or NT can be called “standardized”, and the approach used in the development of assessment criteria can be called “element-by-element analysis.” It is a combination of computer technologies and element-by-element analysis that can ultimately become the foundation for the development of computer-based procedures for the assessment of model assignments that require a detailed answer, in which a solution is to be written as a text ( Reshetnikova & Demidova, 2015).

Conclusion

In the process of determination the direction of assessing quality system development should take into consideration the experience accumulated in Russian school chemical education and the approaches using in international monitoring studies.

The Russian assessing quality system of school chemical education should be built on the principles of systemic, general educational importance and subject-competence orientation of the tested skills.

Improving the Russian assessing quality system of school chemical education, extremely important thing is not only the knowledge and practical skills, but also the competence component of training. This implementation into the chemistry teaching is impossible without the experience gained in the framework of international monitoring studies.

References

1. Bolotov, V.A., Valdman I.A. (2013). Types and purposes of schoolchildren assessment programs». Pedagogics. 8, 15–26.
2. Carnoy, M., Khavenson, T., Loyalka, P., Schmidt, W. H., Zakharov, A. (2016). Revisiting the Relationship between International Assessment Outcomes and Educational Production: Evidence from a Longitudinal PISA-TIMSS Sample. American Educational Research Journal. 4 (53). 1054–1085.
3. Chelyshkova, M.B. (2002). Theory and practice of constructing pedagogical tests: A manual. Moscow: Logos, 432 p.
4. Dobrotin D.Yu. (2017a). Context- and competency-based assignments in the test-and-assessment activities in chemist. Proceedings of the 3rd international research and practical conference “Contemporary education: Global trends and regional aspects.” Mogilev, Republic of Belarus: MGOIRO, 16–19.
5. Dobrotin D.Yu. (2017b). The monitoring function of chemical experiment. Chemistry at School. 3, 47–51.
6. Dobrotin D.Yu. (2018a). Operationalization of requirements of the standard as a systemic factor of test-and-assessment activities in school chemistry education. Standards and Monitoring in Education. 3 (60), 47–51.
7. Dobrotin D.Yu. (2018b). Peculiarities of standardized monitoring of students’ performance in chemistry. Science and School. 2, 21–27.
8. Dobrotin, D.Yu. (2016). Most relevant issues of the school chemistry education and ways of addressing them. Bulletin of the Moscow City Pedagogical University. “Natural Sciences” series. 2, 116–122.
9. Dobrotin, D.Yu. (2018c). Competency-based approach in test-and-assessment activities in science education. Biology in School. 2, 20–26.
10. Dobrotin, D.Yu. (2018d). Russia’s nationwide test in chemistry as a new format of test-and-assessment activities of graduates. Pedagogical Assessments. 1, 46–69.
11. Kaverina, A.A., Molchanova, G.N., Sviridenkova, N.V., & Snastina, M.G. (2017). From the practice of developing assignments to test scientific literacy of schoolchildren in teaching chemistry. Pedagogical Assessments. 2, 91–96.
12. Kovaleva G.S., & Loginova, O.B. (2011). The second-generation standard of general education: New ideas in assessment of educational performanc. Pedagogical Diagnostics. 1, 33–47.
13. Mullis, I.V.S., Martin, M.O., Goh, S., & Cotter, K. (Eds.) (2016). TIMSS 2015 Encyclopedia: Education Policy and Curriculum in Mathematics and Science. October 2016.
14. Polivanova, K. (2015). Obrazovatelnye rezultaty osnovnoy shkoly v kontekste mezhdunarodnykh issledovaniy [Educational Outcomes of Middle School in the Context of International Research] Psychological Science and Education. 4 (20). 19–30. doi: 10.17759/pse.2015200402
15. Reshetnikova O.A., Demidova M.Yu. (2015). New approaches to the development of test-and-assessment materials. People’s Education. 9, 82–87.
16. Yefremova, N.F. (2007). Test-based monitoring in education. Moscow: University Book Publishers: Logos, 263 p.
17. Zuckerman, G.A., Kovaleva, G.S., Kuznetsova, M.I. (2013). Between PIRLS and PISA: The advancement of reading literacy in a 10–15-year-old cohort. Learning and Individual Differences 26, 64–73.