Assessment Of Success In Learning Mathematics: Cross-Cultural Analysis

Abstract

The article presents the results of a cross-cultural analysis of the correlation of various success indicators in learning mathematics – teacher assessments, computerized tests and the results of state examinations. The sample consisted of 304 high school students studying in Grade 11 of general education institutions in Russia and Kyrgyzstan with similar educational conditions. The Russian sample included 120 students (mean age = 17.4, SD = 0.4, 41.9% male); the Kyrgyz sample included 184 students (mean age = 17.1, SD = 0.5, 36.4% male). In the course of the cross-cultural analysis, both similarities and differences in the interrelationships between different indicators of success in learning mathematics at high school age were found. On both cross-cultural samples moderate significant associations among the indicators of success in learning mathematics confirm the hypothesis about the use of various cognitive, motivational and emotional resources in solving mathematical tasks of different types - with and without time limit, in written or oral form. Cross-cultural differences are more closely related to the specific relationship between the results of the state examinations and other analyzed indicators of mathematical success. It was shown that on the Kyrgyz sample the results of the state examination are more associated with the computerized test, while on the Russian sample the results are more associated with the teacher's annual assessment. The results are interpreted in the context of the specifics of the national educational systems, in particular, related to the evaluation of success in learning the school disciplines;

Keywords: Success in learning mathematicsState examAnnual gradesMath test scoreNational educational systemHigh school age

Introduction

The problem of measuring success in learning is relevant both for educational practice and for research in education sciences. It has been shown that success in school education is associated with a whole spectrum of events in later life - further education, career achievements, and mental and physical well-being (Goodman et al., 2011; Power et al., 2013, etc.). During the school-age period, success in learning is the most important criterion for children's social status, an indicator of their psychological comfort and the basis of relationships with peers and educators (Tikhomirova, Malykh, 2017; Nisbett et al., 2012).

In studies aimed at finding the factors of individual differences in academic success, this psychological trait is analyzed on the basis of teachers' assessments (Tikhomirova et al., 2015a; Tikhomirova et al., 2015b), test scores (Tikhomirova, 2017; Rodic et al., 2015) and the results of state examinations (Verbitskaya et al., 2017a; Verbitskaya et al., 2017b). At the same time, the analysis of academic success based on each of these indicators has a number of advantages and disadvantages. Grades given by teachers vary only from 2 to 5, and annual grades, as a rule, vary from 3 to 5. This circumstance makes it difficult to conduct statistical procedures in the studies. At the same time, school assessment is an indicator of the success in learning, which can be used throughout the period of schooling. Tests are easy to use, but their content does not usually correlate with the state educational standards. The results of the exams – the Basic State Exam and the Unified State Examination – can only be applied in studies involving high school students.

Problem Statement

The observed differences in the results of studies of cognitive predictors of academic success are associated, among other things, with the specifics of expert, test and exam success indicators. For example, the study of the success in learning the Russian language showed differences in the relationship between cognitive characteristics and the results of the two exams (Verbitskaya et al., 2015). In the study of cognitive predictors of success in learning mathematics, differences were revealed for two mathematical tests – with and without the time limit (for example, Tikhomirova et al., 2014).

At the same time, the results of studies of mathematical success indicate the existence of stable cross-cultural differences (see more Tikhomirova, Malykh, 2017). Thus, according to the results of a large-scale cross-cultural rating of the Program for International Student Assessment (PISA, URL: http://www.oecd.org/pisa/aboutpisa), within mathematical achievement among 15-year-olds the first ten positions are usually occupied by the countries of the Asian-Pacific region and Finland, Great Britain holds middle positions, Russia shows results slightly below average, and schoolchildren from Kyrgyzstan take the last place (OECD, 2010). These cross-cultural differences in the mathematical success are usually associated with the effect of macro-environment factors reflecting the specifics of national education systems, for example, the orientation of the education system on mathematical achievement, the duration of schooling and others (Malykh et al., 2012; Rodic et al., 2015; Brinch & Galloway, 2012; Nisbett et al., 2012). It is concluded that there is a significant influence not only on the indicators of success in learning, but also on the structure of their interrelations (Tikhomirova, Malykh, 2017).

Thus, the study of the correlation of different indicators of success in learning mathematics is important in the context of the different conditions of the educational environment.

Research Questions

The main issue addressed in this study is related to the understanding of cross-cultural similarities and / or differences in the structure of the interrelationships between different types of tasks that measure success in learning mathematics; and to what extent teachers' assessments will match the scores of computerized with and without the time limit, as well as the results of the state exam in mathematics.

Purpose of the Study

The aim of the current work is to study the relation of various success indicators in learning mathematics – teacher assessments, indicators of computerized tests and the results of the state exam – in different national educational systems. The study will be conducted with the participation of Russian and Kyrgyz schoolchildren of Grade 11, at the time when it is possible to analyze all three types of indicators of success in learning.

Research Methods

Findings

Conclusion

In general, cross-cultural analysis revealed both similarities and differences in the relationship between different indicators of success in learning mathematics at high school age.

Thus, we found moderate, and in some cases, weak interrelations between teacher's, test and exam results on mathematics on both samples. These statistically significant moderate interrelationships between success indicators support the hypothesis of the possible use of various cognitive, motivational and emotional resources in solving problems of different types – with and without the time limit, in written or oral form (Tikhomirova et al., 2015a; Tikhomirova, 2017). This result obtained on the two cross-cultural samples indicates the need for increased attention to the analyzed indicators of mathematical success, the development of standardized tasks that could be used throughout the schooling and ethical principles in education practice.

Cross-cultural differences are more closely related to the specific relationship between the results of the state examinations and other analyzed indicators of mathematical success. In particular, it was shown that on the Kyrgyz sample the results of the state examination (ART) are more associated with the results of a computerized test task aimed at understanding mathematical operations and their relations, formulated in the form of mathematical and logical tasks. At the same time, on the Russian sample, the successful performance at the state exam (USE) was more correlated with the annual grade given by a teacher. These differences can be explained by the content and objectives of examinations in Russia and Kyrgyzstan. Thus, the purpose of the USE is to certify students who are learning to master basic general education programs of secondary general education using sets of tasks of a standardized form for school disciplines. At the same time, ART is not aimed at revealing the actual knowledge, but on researching the analytic abilities of a student, their ability for further education. In addition, higher correlations can be a consequence of the subject orientation of the USE and ART. In particular, the USE is conducted both in mathematics and the Russian language separately, and ART only includes sections on mathematics and the Russian language.

It should be noted that the ratio of teachers', test and examination evaluations of success in learning mathematics was analyzed on the data collected at one point in time. At the same time, in order to assess the temporal stability of the relationship between success indicators in the learning mathematics it is required to conduct longitudinal studies.

Acknowledgments

This study was supported by the grant from the Russian Science Foundation №17-78-30028

References

1. Brinch, C.N., & Galloway, T.A. (2012). Schooling in adolescence raises IQ scores. Proceedings of the National Academy of Sciences. 109(2), 425–430.
2. Malykh, S.B., Tikhomirova, T.N., Zhou, X.,  Wei, W.,  Rodic, M., Misozhnikova, E.B., Davidova, Y., Кovas, Y.V.  (2012). The structure of the relationship of cognitive characteristics and success in arithmetic in preschool children: cross-cultural analysis. Voprosy psihologii. 5, 133–143.
3. Nisbett, R., Aronson, J., Blair, C., Dickens, W., Flynn, J., Halpern, D., & Turkheimer, E. (2012). Intelligence: New findings and theoretical developments. American Psychologist. 67, 130–159.
4. OECD, (2010). Education at a Glance 2011: OECD Indicators. OECD Publishing. Retrieved from http://www.oecd.org/edu/skills-beyond-school/48631582.pdf
5. Power, C., Kuh, D., Morton, S. (2013). From developmental origins of adult disease to life course research on adult disease and aging: insights from birth cohort studies. Public Health. 34(1), 7–28.
6. Rodic, M., Zhou, X., Tikhomirova, T., Wei, W., Malykh, S., Ismatulina, V., Sabirova, E., Davidova, Y., Tosto, M., Lemelin, J-P., & Kovas, Y. (2015). Cross-Cultural Investigation into Cognitive Underpinnings of Individual Differences in Early Arithmetics. Developmental Science. 18(1), 165–174.
7. Tikhomirova, T. (2017). Spatial thinking and memory in Russian high school students with different levels of mathematical fluency. Procedia – Social and Behavioral Sciences. 237, 1260 – 1264.
8. Tikhomirova, T.N., Modyaev, A.D., Leonova, N.M., Malykh, S.B. (2015a). Factors of academic achievement at primary school level: sex differences. Psihologicheskij zhurnal. 36(5), 43–54.
9. Tikhomirova, T.N., Voronin, I.A., Misozhnikova, E.B., Malykh, S.B. (2015b). The structure of the interrelation of the cognitive characteristics and academic achievement in school age. Teoreticheskaya i ehksperimental'naya psihologiya. 8(8), 55–68.
10. Tikhomirova, T.N., Кovas, Y.V. (2013). Interrelation between cognitive characteristics and mathematical achievements (by the example of high school pupils). Psihologicheskij zhurnal. 34(1), 63–73.
11. Tikhomirova, T.N., Malykh, S.B. (2017). Cognitive bases of individual differences in academic achievement. M.; SPb .: Nestor-History.
12. Tikhomirova, T.N., Malykh, S.B., Tosto, M.G., Kovas, Y.V. (2014). Cognitive characteristics and mathеmatical achievement in high school students: cross-cultural analyses. Psihologicheskij zhurnal. 35(1), 41–53.
13. Tosto, M.G., Tikhomirova, T., Galajinsky, E., Akimova, K., Kovas, Y. (2013). Development and Validation of a Mathematics-number sense Web-based Test Battery. Procedia – Social and Behavioral Sciences. 86, 423–428.
14. Verbitskaya, L.A., Malykh, S.B., Tikhomirova, T.N. (2017a). Cognitive predictors of success in learning Russian in native and non-native speakers at high school age. Procedia – Social and Behavioral Sciences. 237, 1236 –1241.
15. Verbitskaya, L.A., Zinchenko, Y.P., Malykh, S.B., Tikhomirova, T.N. (2017b). Cognitive foundations of successful teaching of the Russian language: a cross-cultural study. Voprosy psihologii. 1, 26–40.
16. Verbitskaya, L.A., Malykh, S.B., Zinchenko, Y.P., Tikhomirova, T.N. (2015). Cognitive predictors of success in learning Russian. Psychology in Russia. State of the Art. 8(4), 91–100.