Assessment Of Success In Learning Mathematics: Cross-Cultural Analysis

Abstract

The article presents the results of a cross-cultural analysis of the correlation of various success indicators in learning mathematics teacher assessments, computerized tests and the results of state examinations. The sample consisted of 304 high school students studying in Grade 11 of general education institutions in Russia and Kyrgyzstan with similar educational conditions. The Russian sample included 120 students (mean age = 17.4, SD = 0.4, 41.9% male); the Kyrgyz sample included 184 students (mean age = 17.1, SD = 0.5, 36.4% male). In the course of the cross-cultural analysis, both similarities and differences in the interrelationships between different indicators of success in learning mathematics at high school age were found. On both cross-cultural samples moderate significant associations among the indicators of success in learning mathematics confirm the hypothesis about the use of various cognitive, motivational and emotional resources in solving mathematical tasks of different types - with and without time limit, in written or oral form. Cross-cultural differences are more closely related to the specific relationship between the results of the state examinations and other analyzed indicators of mathematical success. It was shown that on the Kyrgyz sample the results of the state examination are more associated with the computerized test, while on the Russian sample the results are more associated with the teacher's annual assessment. The results are interpreted in the context of the specifics of the national educational systems, in particular, related to the evaluation of success in learning the school disciplines;

Keywords: Success in learning mathematicsState examAnnual gradesMath test scoreNational educational systemHigh school age

Introduction

The problem of measuring success in learning is relevant both for educational practice and for research in education sciences. It has been shown that success in school education is associated with a whole spectrum of events in later life - further education, career achievements, and mental and physical well-being (Goodman et al., 2011; Power et al., 2013, etc.). During the school-age period, success in learning is the most important criterion for children's social status, an indicator of their psychological comfort and the basis of relationships with peers and educators (Tikhomirova, Malykh, 2017; Nisbett et al., 2012).

In studies aimed at finding the factors of individual differences in academic success, this psychological trait is analyzed on the basis of teachers' assessments (Tikhomirova et al., 2015a; Tikhomirova et al., 2015b), test scores (Tikhomirova, 2017; Rodic et al., 2015) and the results of state examinations (Verbitskaya et al., 2017a; Verbitskaya et al., 2017b). At the same time, the analysis of academic success based on each of these indicators has a number of advantages and disadvantages. Grades given by teachers vary only from 2 to 5, and annual grades, as a rule, vary from 3 to 5. This circumstance makes it difficult to conduct statistical procedures in the studies. At the same time, school assessment is an indicator of the success in learning, which can be used throughout the period of schooling. Tests are easy to use, but their content does not usually correlate with the state educational standards. The results of the exams the Basic State Exam and the Unified State Examination can only be applied in studies involving high school students.

Problem Statement

The observed differences in the results of studies of cognitive predictors of academic success are associated, among other things, with the specifics of expert, test and exam success indicators. For example, the study of the success in learning the Russian language showed differences in the relationship between cognitive characteristics and the results of the two exams (Verbitskaya et al., 2015). In the study of cognitive predictors of success in learning mathematics, differences were revealed for two mathematical tests – with and without the time limit (for example, Tikhomirova et al., 2014).

At the same time, the results of studies of mathematical success indicate the existence of stable cross-cultural differences (see more Tikhomirova, Malykh, 2017). Thus, according to the results of a large-scale cross-cultural rating of the Program for International Student Assessment (PISA, URL: http://www.oecd.org/pisa/aboutpisa), within mathematical achievement among 15-year-olds the first ten positions are usually occupied by the countries of the Asian-Pacific region and Finland, Great Britain holds middle positions, Russia shows results slightly below average, and schoolchildren from Kyrgyzstan take the last place (OECD, 2010). These cross-cultural differences in the mathematical success are usually associated with the effect of macro-environment factors reflecting the specifics of national education systems, for example, the orientation of the education system on mathematical achievement, the duration of schooling and others (Malykh et al., 2012; Rodic et al., 2015; Brinch & Galloway, 2012; Nisbett et al., 2012). It is concluded that there is a significant influence not only on the indicators of success in learning, but also on the structure of their interrelations (Tikhomirova, Malykh, 2017).

Thus, the study of the correlation of different indicators of success in learning mathematics is important in the context of the different conditions of the educational environment.

Research Questions

The main issue addressed in this study is related to the understanding of cross-cultural similarities and / or differences in the structure of the interrelationships between different types of tasks that measure success in learning mathematics; and to what extent teachers' assessments will match the scores of computerized with and without the time limit, as well as the results of the state exam in mathematics.

Purpose of the Study

The aim of the current work is to study the relation of various success indicators in learning mathematics – teacher assessments, indicators of computerized tests and the results of the state exam – in different national educational systems. The study will be conducted with the participation of Russian and Kyrgyz schoolchildren of Grade 11, at the time when it is possible to analyze all three types of indicators of success in learning.

Research Methods

Sample

The study included 304 students studying in Grade 11 from two general education institutions in Russia and Kyrgyzstan with similar educational conditions in terms of the quality of education, qualifications and structure of the teaching staff, departmental affiliation and educational programs implemented at school. The Russian sample is represented by 120 students (mean age = 17.4, SD = 0.4, 41.9% male); the Kyrgyz sample included 184 students (mean age = 17.1, SD = 0.5, 36.4% male).

Assessment of success in learning mathematics

In the current study the success in learning mathematics was analyzed on the basis of three types of indicators.

  • Expert teacher assessment

As an expert assessment of the mathematical success of students in Grade 11, annual grades for algebra and geometry given by the teacher of mathematics were analyzed. The average grade was calculated.

  • Computerized tests

«Understanding numbers», success in solving mathematical tasks without time limit

The test includes 18 mathematical tasks the successful solution of which requires an understanding of mathematical operations and their relations formulated in the form of mathematical tasks and logical problems. The mathematical tasks are organized into 3 levels of different complexity, each consisting of 6 tasks. The test represents an hierarchical system: the starting point is the same task for all participants, but the order in which further tasks are presented is determined by student's performance in solving mathematical tasks. The program registers the number of correct answers. Examples of tasks and their detailed description are presented in the works of Tikhomirova, Kovas, 2013; Tikhomirova et al., 2014; Tosto et al., 2013.

«Problem Verification», success in solving mathematical tasks within limited time

The test consists of 48 already solved mathematical tasks with numbers and ratios at the top of the screen and "keys" "True", "Wrong" and "I do not know" at the bottom of the screen. The participants are asked to decide whether each mathematical example is true or false, and within 10 seconds press the corresponding key on the keyboard. The time indicator is located at the top of the screen to show the remaining time for the decision. If the answer is not given at the designated time, the program automatically proceeds to the next task. The program registers the number of correct answers. Examples of the tasks of this mathematical test and their detailed description are presented in the works of Tikhomirova et al., 2014; Tosto et al., 2013;

  • State exams

The Russian State Exam in Mathematics the Unified State Exam (USE) is compulsory for all Russian students at the end of the 11th grade. The USE consists of two sections: 1) tasks with a short answer in the form of an integer or a finite decimal fraction; 2) assignments of increased complexity with a short answer in the form of an integer or a finite decimal fraction and with a detailed solution with justification of the performed actions. The total score for the USE was analyzed.

Kyrgyz State Examination All-Republican Testing (ART) is mandatory only for those students who plan to enter Kyrgyzstan's higher professional education institutions. ART consists of three subject sections: 1) Mathematics, 2) Verbal Logic Test and 3) Grammar of the Native Language. The section Mathematics includes questions on arithmetic, algebra and geometry within the requirements of the school curriculum aimed to test procedural knowledge, conceptual understanding and ability to solve non-standard problems and the tasks. In this paper, the total score for ART was analyzed.

Findings

Descriptive statistics

Table 01 presents the descriptive statistics on the analyzed indicators of the success in learning mathematics on the samples of Russian and Kyrgyz school students.

Table 1 -
See Full Size >

Table 01 presents the mean values and standard deviations (in parentheses) of the analyzed indicators. The minimum and maximum values are: for the annual assessment from 2 to 5; for Problem Verification task from 0 to 48, Understanding numbers from 0 to 18; for the exams: USE from 0 to 100, ART from 0 to 231. According to Table 01 , Russian children show higher mathematical success in comparison with the Kyrgyz students. At the same time, the range of variation is wider on the Kyrgyz than on the Russian sample.

Cross-cultural differences in test indicators of mathematical success

In the course of a one-way analysis of variance (ANOVA) on the test indicators of mathematical success, where the categorical factor was the belonging to Russia or Kyrgyzstan, cross-cultural similarities and differences in the indicators of mathematical success were studied. Table 02 shows the results of the ANOVA.

Table 2 -
See Full Size >

According to Table 02 , statistically significant differences were found for both test scores of mathematical success with and without the time limit. At the same time, a slightly larger effect size of the country effect was obtained for the indicator of mathematical fluency (ƞ2 = 0.05, p < 0.01). Thus, Russian students show better performance on the computerized tests in mathematics than their Kyrgyz peers. For example, the average value for Problem verification task on a sample of Russian schoolchildren was 40.68, and on a sample of Kyrgyz schoolchildren 37.62. The average values are shown in Table 01 .

Cross-cultural analysis of interrelationships of success indicators in learning mathematics

Table 03 presents the results of the correlation analysis of expert, exam and test indicators of academic success in mathematics on the Russian (upper line) and Kyrgyz (lower line) samples ( p < 0.01).

Table 3 -
See Full Size >

According to Table 03 , on the Russian sample the annual grade in mathematics is most closely related to the result of the state examination (r = 0.76, p <0.01) compared to the Kyrgyz (r = 0.46, p  < 0.01). Similar results were obtained for indicators of the success of learning Russian (Verbitskaya et al., 2017a; Verbitskaya et al., 2017b). In particular, it is reported that the results of the Russian state exam are more closely related to the annual assessment of the Russian language (r = 0.71, p < 0.001) than the results of the Kyrgyz exam (r = 0.25, p < 0.01).

For computerized test tasks, the teacher's assessment is almost equally related on the samples of Russian and Kyrgyz schoolchildren (for mathematical fluency, respectively, r = 0.40 and r = 0.30, p  < 0.01). The test results are closely and almost to the same extent related on both cultural samples (0.52 < r < 0.61, p < 0.01). On the contrary, cross-cultural differences are observed for the interrelations between the test scores and the exam grades. On the Kyrgyz sample, the Understanding numbers test and the exam grade show stronger association than on the sample of Russian high school students (0.60 vs. 0.37, p < 0.01). The cross-cultural differences in the relationship of the second test indicator with the exam grade were less evident (0.52 vs. 0.41, p < 0.01).

Conclusion

In general, cross-cultural analysis revealed both similarities and differences in the relationship between different indicators of success in learning mathematics at high school age.

Thus, we found moderate, and in some cases, weak interrelations between teacher's, test and exam results on mathematics on both samples. These statistically significant moderate interrelationships between success indicators support the hypothesis of the possible use of various cognitive, motivational and emotional resources in solving problems of different types – with and without the time limit, in written or oral form (Tikhomirova et al., 2015a; Tikhomirova, 2017). This result obtained on the two cross-cultural samples indicates the need for increased attention to the analyzed indicators of mathematical success, the development of standardized tasks that could be used throughout the schooling and ethical principles in education practice.

Cross-cultural differences are more closely related to the specific relationship between the results of the state examinations and other analyzed indicators of mathematical success. In particular, it was shown that on the Kyrgyz sample the results of the state examination (ART) are more associated with the results of a computerized test task aimed at understanding mathematical operations and their relations, formulated in the form of mathematical and logical tasks. At the same time, on the Russian sample, the successful performance at the state exam (USE) was more correlated with the annual grade given by a teacher. These differences can be explained by the content and objectives of examinations in Russia and Kyrgyzstan. Thus, the purpose of the USE is to certify students who are learning to master basic general education programs of secondary general education using sets of tasks of a standardized form for school disciplines. At the same time, ART is not aimed at revealing the actual knowledge, but on researching the analytic abilities of a student, their ability for further education. In addition, higher correlations can be a consequence of the subject orientation of the USE and ART. In particular, the USE is conducted both in mathematics and the Russian language separately, and ART only includes sections on mathematics and the Russian language.

It should be noted that the ratio of teachers', test and examination evaluations of success in learning mathematics was analyzed on the data collected at one point in time. At the same time, in order to assess the temporal stability of the relationship between success indicators in the learning mathematics it is required to conduct longitudinal studies.

Acknowledgments

This study was supported by the grant from the Russian Science Foundation №17-78-30028

References

  1. Brinch, C.N., & Galloway, T.A. (2012). Schooling in adolescence raises IQ scores. Proceedings of the National Academy of Sciences. 109(2), 425–430.
  2. Malykh, S.B., Tikhomirova, T.N., Zhou, X.,  Wei, W.,  Rodic, M., Misozhnikova, E.B., Davidova, Y., Кovas, Y.V.  (2012). The structure of the relationship of cognitive characteristics and success in arithmetic in preschool children: cross-cultural analysis. Voprosy psihologii. 5, 133–143.
  3. Nisbett, R., Aronson, J., Blair, C., Dickens, W., Flynn, J., Halpern, D., & Turkheimer, E. (2012). Intelligence: New findings and theoretical developments. American Psychologist. 67, 130–159.
  4. OECD, (2010). Education at a Glance 2011: OECD Indicators. OECD Publishing. Retrieved from http://www.oecd.org/edu/skills-beyond-school/48631582.pdf
  5. Power, C., Kuh, D., Morton, S. (2013). From developmental origins of adult disease to life course research on adult disease and aging: insights from birth cohort studies. Public Health. 34(1), 7–28.
  6. Rodic, M., Zhou, X., Tikhomirova, T., Wei, W., Malykh, S., Ismatulina, V., Sabirova, E., Davidova, Y., Tosto, M., Lemelin, J-P., & Kovas, Y. (2015). Cross-Cultural Investigation into Cognitive Underpinnings of Individual Differences in Early Arithmetics. Developmental Science. 18(1), 165–174.
  7. Tikhomirova, T. (2017). Spatial thinking and memory in Russian high school students with different levels of mathematical fluency. Procedia – Social and Behavioral Sciences. 237, 1260 – 1264.
  8. Tikhomirova, T.N., Modyaev, A.D., Leonova, N.M., Malykh, S.B. (2015a). Factors of academic achievement at primary school level: sex differences. Psihologicheskij zhurnal. 36(5), 43–54.
  9. Tikhomirova, T.N., Voronin, I.A., Misozhnikova, E.B., Malykh, S.B. (2015b). The structure of the interrelation of the cognitive characteristics and academic achievement in school age. Teoreticheskaya i ehksperimental'naya psihologiya. 8(8), 55–68.
  10. Tikhomirova, T.N., Кovas, Y.V. (2013). Interrelation between cognitive characteristics and mathematical achievements (by the example of high school pupils). Psihologicheskij zhurnal. 34(1), 63–73.
  11. Tikhomirova, T.N., Malykh, S.B. (2017). Cognitive bases of individual differences in academic achievement. M.; SPb .: Nestor-History.
  12. Tikhomirova, T.N., Malykh, S.B., Tosto, M.G., Kovas, Y.V. (2014). Cognitive characteristics and mathеmatical achievement in high school students: cross-cultural analyses. Psihologicheskij zhurnal. 35(1), 41–53.
  13. Tosto, M.G., Tikhomirova, T., Galajinsky, E., Akimova, K., Kovas, Y. (2013). Development and Validation of a Mathematics-number sense Web-based Test Battery. Procedia – Social and Behavioral Sciences. 86, 423–428.
  14. Verbitskaya, L.A., Malykh, S.B., Tikhomirova, T.N. (2017a). Cognitive predictors of success in learning Russian in native and non-native speakers at high school age. Procedia – Social and Behavioral Sciences. 237, 1236 –1241.
  15. Verbitskaya, L.A., Zinchenko, Y.P., Malykh, S.B., Tikhomirova, T.N. (2017b). Cognitive foundations of successful teaching of the Russian language: a cross-cultural study. Voprosy psihologii. 1, 26–40.
  16. Verbitskaya, L.A., Malykh, S.B., Zinchenko, Y.P., Tikhomirova, T.N. (2015). Cognitive predictors of success in learning Russian. Psychology in Russia. State of the Art. 8(4), 91–100.

Copyright information

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

About this article

Publication Date

18 December 2019

eBook ISBN

978-1-80296-032-7

Publisher

Future Academy

Volume

33

Print ISBN (optional)

-

Edition Number

1st Edition

Pages

1-481

Subjects

Cognitive theory, educational equipment, educational technology, computer-aided learning (CAL), psycholinguistics

Cite this article as:

Tikhomirova, T., Gaydamashko, I., Malykh, A., Lysenkova, I., Khusnutdinova, E., & Malykh, S. (2019). Assessment Of Success In Learning Mathematics: Cross-Cultural Analysis. In S. B. Malykh, & E. V. Nikulchev (Eds.), Psychology and Education - ICPE 2017, vol 33. European Proceedings of Social and Behavioural Sciences (pp. 345-352). Future Academy. https://doi.org/10.15405/epsbs.2017.12.37