Meta-Subject Olympiad For Development And Diagnostics Of Students’ Global Competencies

Abstract

In a dynamically developing society, scientists and practical teachers focus on the issues of the formation and development of global competency in students as new educational results. Among them, one of the main issues is conditions for the development of global competencies, in particular, such universal skills as awareness of global problems, understanding of their impact on problems at the regional and local levels; ability to interact effectively with other people; communication skills; ability to think critically. In this respect, it is required to search for new tools, conditions, and tools for diagnosing students’ global competencies. The article defines the potential of one of the forms of extracurricular activities, the Meta-Subject Olympiad, as conditions for the development and diagnostics of students’ global competency. Criteria of the global competence formation are defined. An integrated method for assessing the results of the global competencies formation of students, based on a combination of expert assessments and self-assessments is proposed. The approach to the project tasks setting of Meta-Subject content as means of diagnosing global competencies, in the conditions of the Olympiad, is described. The article presents an experimental confirmation of the effectiveness of the presented Meta-Subject Olympiad as a condition for the development and diagnostics of global competencies of students. On the basis of the data obtained, the practicability of using Meta-Subject Olympiads for the development of global competencies of students is disclosed.

Keywords: Formation criteriaprojectsmeta-subject content

Introduction

The globalization of the modern world, as a continuous process of expanding and deepening the interrelationships and interdependence of the economic, social, cultural, and other spheres, requires a person to possess new competencies. Such competencies are now referred to as global competencies. Having these competencies ensures human competitiveness in the twenty-first century. In this regard, the essence of global competencies, their composition, structure, conditions of formation and development, are the subject of research of many modern scientists and practitioners.

The essential characteristics of the notion of global competence as the ability to understand and interact with the global world have been studied in many works ( Byker & Putman, 2019; Deardorff, 2006; Harshman, 2016; Koval, & Dukova, 2019a; Schleicher & Ramos, 2018; etc.).

Various researches have revealed the core of global competencies as critical thinking ( Andrews & Higson, 2010; Avdeenko et al., 2018; Dorozyński et al., 2016), communicative skills, ability to cooperate ( Dorozyński et al., 2016; Vogler et al., 2018), awareness and understanding of global issues ( Byker & Thomas, 2018; Cook et al., 2016).

The researchers focus on the methodological aspects of the formation and development of global competencies of students. The aspects are mainly private, oriented towards the formation of specific competencies. The work of Avdeenko et al. ( 2018) presents an approach to the development of tasks aimed at shaping the students’ creativity during collaborative problem solving. The specifics of the development of collaborative work skills were studied by Koval and Dukova ( 2019b), Vogler et al. ( 2018), etc. Possibilities of enriching curriculum with content contributing to the development of global competencies were considered in the works of Yashin et al. ( 2018) and Yildiz and Palak ( 2016).

Methodological problems of the formation and development of a set of global competencies, as a holistic personal quality of students, have been studied to a lesser extent. Implementation of such an approach to solving the problem of development of global students competencies will allow obtaining a synergistic effect in levels of development of each global competence and the whole set of these competencies. Today, we know the first results of this approach through student interaction ( Beetham et al., 2009; Harshman, 2016; Kang et al., 2018).

Problem Statement

One of the main stages of the formation and development of global competencies is the diagnostics of their results, which are levels of formation. Currently, testing is the most developed method of diagnosing and assessing the level of formation of student’s global competencies. The development of tests as tools for diagnostics of global competencies was studied by Händel et al. ( 2013), Koval and Dukova ( 2019b), Shkerina et al. ( 2019). The testing method is implemented by the Organisation for Economic Cooperation and Development (OECD) as part of the Programme for International Student Assessment (PISA) to evaluate students’ educational outcomes.

On the other hand, the composition of global competencies and the specifics of their display in the relevant activities indicate that the methods of their diagnostics, based on expert observation of student activities in real or specially created conditions, will be more effective.

Nowadays, there is a problem of identifying conditions, developing and testing new interactive methods and models of formation and diagnostics of students’ global competencies.

Research Questions

The article explores the following issues:

• extracurricular activities as a condition for the development of students’ global competencies;

• diagnostics of students' global competencies during their extracurricular activities.

Purpose of the Study

The purpose of the study: development and testing of the methodology of organization and carrying out the Meta-Subject Olympiad for students, as their global competencies development and diagnostics conditions.

Research Methods

The research was based on the works of authors engaged in studying the composition of global competencies, developing methods of forming global competencies, and algorithms of their diagnostics. Works that present the experience of developing students' global competencies in the context of real educational practice were also taken as a basis ( Avdeenko et al., 2018; Harshman, 2016; Kang et al., 2018).

As theoretical methods of research, we have used the analysis of scientific literature on the problem of research, generalization and systematization of scientific search results. On the basis of these methods, we have identified the composition of global competencies, the formation and diagnostics of which are possible in the conditions of the modern Russian school. The following competencies are identified as the main ones: awareness of global problems, understanding of their impact on problems at the regional and local levels; ability to interact effectively with others; communicative abilities; ability to think critically. The study attempts to determine the potential of students’ extracurricular activities in the conditions of Meta-Subject Olympiad for the development and diagnostics of selected competencies.

The use of empirical methods at various stages of research allowed us not only to collect up-to-date information about the problem under study but also to test the validity of theoretical conclusions. During the study, we applied questionnaires, observation, pedagogical experiment, method of expert assessment, self-assessment, and statistical methods: F -test, analysis, and interpretation of results.

At the initial stage of the study, we conducted a questionnaire survey to obtain a true picture of the perception of global competencies by students of secondary schools and their understanding of the need to possess these competencies in modern society. 782 pupils of 10th and 11th forms of secondary schools took part in the questionnaire survey (2018). The questionnaire fragment is shown in Table 01 .

The survey revealed students' self-assessment of their awareness of global problems, understanding of their impact on local (regional) problems, willingness and ability to interact with other people openly and effectively, possession of communication skills, ability to critically consider problems.

The special feature of global competencies is that they reflect the content of not one particular training discipline, but are integrative. Individual components of global competencies are presented fragmentally in various school subjects (PISA 2018 Assessment and Analytical Framework, 2019). It is possible to combine individual fragments and create conditions for the manifestation of global competencies in the process of extracurricular activities. Extracurricular activities are activities of students in the field of additional education, which are organized both by the school and other educational institutions, and implemented in forms other than curriculum. This activity allows creating conditions for students’ individual development, to include them in various types of activities. Extracurricular activities provide the integrative nature of activity content, not limited to one subject matter. This circumstance was the basis for the development and further testing of one of the forms of extracurricular activity - Meta-Subject Olympiad, as conditions for the development and diagnostics of students’ global competencies.

Meta-Subject Olympiad is a team competition in project task performance. The project assignment is based on one of the pressing issues of regional (local) scale, which can be interpolated to the global problem. This task is not connected to any specific subject of the school curriculum and allows students to select different strategies for its execution.

During the Meta-Subject Olympiad 2020, the participating teams were asked to complete the task: "Design of metro lines."

Here is a brief description of this project task. It consists of several parts.

• 1. Introduction data. Here is a brief description of the metro as an urban off-street railway.

• 2. Historical background. The task presents a brief description of the metro design history in Krasnoyarsk, starting in 1970, and the need to improve the existing project in accordance with modern conditions.

• 3. Aims and objectives of the project. The aim of the project: to improve the project of the Krasnoyarsk metro in accordance with modern conditions. Project objectives.

  • 3.1. Develop a new metro map.

  • 3.2. Create a design sketch for one of the metro stations.

  • 3.3. Develop a site plan of the area adjacent to this station.

  • 3.4. Create a logo model of the Krasnoyarsk Metro.

• 4. Additional Task Comments.

Task 3.1. Considering that in recent years Krasnoyarsk has been transformed (new residential areas, new attractions and much more), it is necessary:

• to determine the optimal route and places, where you think metro stations should be built;

• to propose a name for each station;

• to present the route of the Krasnoyarsk Metro (with stations) in the form of a diagram.

Task 3.2. New metro - new stations! In most countries, stations are utilitarian and indistinguishable from one another. Moscow Metro is one of the most beautiful in the world. In solving this task, make a design sketch of one of the metro stations you proposed. What does it look like outside? What does it look like from the inside? How should the station be equipped technically?

Task 3.3. The finished appearance and aesthetic attractiveness the new metro station gains through the improvement of the adjacent territory. Offer a site plan of the area adjacent to the metro station. At the same time, remember the convenience of its functional use.

Task 3.4. The logo is a compact visual image that reflects the most important element of the object (signature style). It is very important that the logo leaves the right impression. The metro logo is also used to indicate the location of stations. Offer your visual image of Krasnoyarsk Metro.

The task was considered complete if all the assigned tasks were solved and a presentation of the developed project was prepared.

The presentation of the project was conducted using both traditional and innovative digital means and was accompanied by a computer presentation.

The mandatory condition of the Olympiad was the requirement of socially heterogeneous teams (4-5 people) and the exclusion of any contacts before the Olympiad. Teams are given 90 minutes to complete the task, during which they must solve all tasks assigned to them and prepare a presentation of the result of command work.

A characteristic feature of the Meta-Subject Olympiad is the conditions created during its implementation for students to exhibit global competencies and diagnose them ( Glazunova & Gromyko, 2019). This is important from the perspective of many years of cognitive laboratories research. Their results confirm that only direct observation of the activity, its analysis, and analysis of its results make it possible to draw objective conclusions about the formation or lack of formation of individual skills ( Schwarz, 2007).

Assessment of the formation of students’ global competencies in the process of collaborative execution of the proposed task was carried out according to the following criteria:

• knows local problems;

• knows global problems and their influence on solving local problems;

• is able to critically evaluate both their own and others' proposals to solve the local problem;

• is able to find alternative solutions;

• is able to choose the correct linguistic form to express their viewpoint depending on the situation;

• is able to negotiate in case of conflict of interests;

• exhibits business communication skills;

• shows respect for members of the group, regardless of gender, nationality, views expressed, etc.;

• demonstrates the breadth of views, suggesting ways of solving the local problem;

• is able to justify their own viewpoint in choosing ways of solving the problem;

• is able to change their own viewpoint based on facts and arguments ( Koval & Dukova, 2019a; PISA 2018 Assessment and Analytical Framewor, 2019; Vostorgova et al., 2019).

The implementation of the criteria requirements was determined on the basis of the analysis of actions and strategies carried out by the team members in solving the project tasks.

The collection of up-to-date information on the level of global competencies formation of the Olympiad participants was carried out using an expert evaluation card. The form included a universal three-level scale: 0 - an absence of indicator display; 1 - indicator appears limited, in the form of separate elements; 2 - indicator appears at all stages of work on the task.

35 people (2019) and 50 people (2020) took part in the Meta-Subject Olympiad. 20 of participants of the year 2020 participated in the Olympiad the previous year (2019) as well. All participants of the Olympiad are students of Krasnoyarsk upper secondary schools.

The effectiveness of the selected diagnostic condition of global competencies was evaluated using F -test: $F=\frac{{MS}_{М/Г}}{{MS}_{В/Г}}$, where ${MS}_{М/Г}$ is the average sum of between-group variance square, and ${MS}_{В/Г}$ is the average sum of intra-group variance square. To obtain statistical characteristics of the development level of global competencies according to the selected criteria indicators, a special sample of students was formed. The first sample included data on 20 students who took part in the Meta-Subject Olympiad in 2019 and 2020. We compared variances of average values of expert assessments for these two years. The second sample consisted of two groups of participants of the 2020 Meta-Subject Olympiad: 1) 20 students who had already had experience in participating in the Meta-Subject Olympiad (2019 participants), 2) 30 students who had not previously participated in such events. We assessed the significance of the differences between the variances of the average expert assessments of the two groups.

Academic added value and delayed pedagogical effect were also assessed. Academic added value was considered as an increment (positive, negative, or zero) in the global competencies of individual students. To assess the increment, we studied the change in the total number of criteria indicators of global competencies formation shown by students in dynamics (control periods: December 2018, February 2019 (I Meta-Subject Olympiad), May 2019, November 2019, February 2020 (II Meta-Subject Olympiad), March 2020). The intensity of criteria indicator display during the control periods was assessed by expert evaluation.

The delayed pedagogical effect was monitored through a change in the attitude of the Olympiad participants to global competencies. In addition, we monitored participants’ understanding of the need to possess these competencies in modern society immediately after the Olympiad and three months later. A survey was used to assess the indicator, which included questions of the initial stage of the study (Table 01 ) along with open questions:

What did the experience of participation in Meta-Subject Olympiad motivate you to do?

How did you benefit/not benefit from the experience of participation in Meta-Subject Olympiad?

Do you see your participation in solving regional (global) problems in the future?

Are you happy with your teamwork results? What skills did you not have to improve the result?

Do you plan to further develop your teamwork skills?

Findings

Initial survey results indicate that students are familiar with global issues and are interested in them (Figure 01 ). Students realize that in today's world it is possible to be successful if you respect other people, are able to work effectively in the team, approach various issues creatively (figure 02 ).

During the project task proposed at the Meta-Subject Olympiad 2020, students showed love for their native city and region. Their projects used the city's main attractions and cultural sites. Despite the young age of the students, they noted the main problems of our time: ecology, transport and logistics within a large city, accessible urban environment, the threat of terrorism, etc. Each group took into account these problems to a certain extent in the development of the project.

See Full Size >

See Full Size >

The effectiveness of the Meta-Subject Olympiad is indicated by the data of the current level of competency development in student groups, who participated in the Olympiads of 2019 and 2020, and the results of their statistical processing. The F -test values obtained during the experiment (for the first sample: with degrees of freedom $df=1,38$; for the second sample: with degrees of freedom $df=1,48$) made it possible to conclude positive dynamics in the development of students’ global competencies of students during the period under review. There were also statistically significant differences in the level of global competencies formation of students participating for the first time in Olympiads and more experienced participants.

According to the results of the conducted study, participants of both Olympiads showed positive dynamics on academic added value. For example, we present a graphical image of the dynamics of the results of the three students who took part in the 2019 and 2020 Olympiads. The image is organised by the total number of criteria indicators students show in the formation of global competencies for certain periods of time. The indicator was considered to be evident if it was recorded at the level of at least "1" (the indicator appears limited, in the form of separate elements). Among the selected students: "Student 1" had the lowest results at the initial stage; "Student 2" and "Student 3" had the same higher level of possession of global competencies. However, later increment in global competencies showed their individual differences (Figure 03 ).

See Full Size >

Observation of the Olympiad participants in the process of their group work in lessons at school, analysis of arguments of their answers in lessons has allowed tracking positive delayed pedagogical effect of their participation in Meta-Subject Olympiad six months and a year after its holding. Students who participated in the Meta-Subject Olympiad became more respectful and attentive to their classmates during group discussions, understood the reason for other views on the problem. Discussion of problems in the class turned into an interactive act. Students have come to understand that a strong argument is a proven fact and that there is not always one correct answer when solving a problem.

Regarding the delayed pedagogical effect, it is necessary to take into account self-assessment of the Olympiad participants on its influence on the change in their views, opinions, abilities and plans. Most students (81.2%) note that participation in the Olympiad has led them to reflect on the global problems of our time, their impact on regional and local problems. Students began to think about how they can contribute to solving those problems. In addition, students (84.7%) acknowledge that the Olympiad allowed them to detect their lack of teamwork skills, such as communication with new people. That encouraged students to determine the goals of further self-development.

Conclusion

The potential of Meta-Subject Olympiad as conditions for development and diagnostics of students’ global competencies has been defined. The goals have been formulated diagnostically in the form of a set of criteria indicators of global competencies formation. The content is of Meta-Subject character. The tasks focus on project activities. Integrated use of evaluation and self-assessment methodologies has been implemented. The criteria of formation of students’ global competencies reflecting their cognitive (knows), active (can), and personal (understands, realizes) components have been formed. The scenario of carrying out Meta-Subject Olympiad as conditions of development and diagnostics of students’ global competencies has been developed. It consists of project task "Design of metro lines"; a set of tasks to be solved on the way to the project; methodological recommendations on organization and work of project groups; criteria expert cards; mathematical tools for assessing the results of students’ project activities.

In the process of implementation of the developed scenario of Meta-Subject Olympiad, we have experimentally confirmed its efficiency as conditions for development and diagnostics of global competencies of students of 10th and 11th forms of secondary school. The experience presented can be extended to other age categories of students.

References

• Andrews, J., & Higson, H. (2010). Graduate employability, ‘soft skills’ versus ‘hard’ business knowledge: A European study. Higher Education in Europe, 33(4), 411-422.
• Avdeenko, N., Denishheva, L., Krasnyanskaya, K., Mikhailova, A., & Pinskaya, М. (2018). Kreativnost' dlya kazhdogo: vnedrenie razvitiya navykov XXI veka v praktiku rossiyskikh shkol [Creativity for Everyone: Integrating the 21st Century Skills in Russian Schools]. Educational Studies Moscow, 4, 282-304. https://doi.org/10.17323/1814-9545-2018-4-282-304
• Beetham, H., McGill, L., & Ittlejohn, A. (2009). Thriving in the 21st century: Learning literacies for the digital age (LliDA project). the Caledonian Academy, Glasgow Caledonian University.
• Byker, E. J., & Putman, S. M. (2019). Catalyzing cultural and global competencies: Engaging preservice teachers in study abroad to expand the agency of citizenship. Journal of Studies in International Education, 23(1), 84-105. https://doi.org/10.1177/1028315318814559
• Byker, E. J., & Thomas, V. (2018). Culturally responsive webquests: Connecting technology with inquiry-based learning. Teaching Social Studies Journal, 18(2), 19-25.
• Cook, L. A., Smith, W. S., Lan, W. Y., & Carpenter, D. (2016). The development of global competencies and global mindedness through global education experiences. International Journal of Global Education, 5(2), 16.
• Deardorff, D. K. (2006). Identification and assessment of intercultural competence as a student outcome of internationalization. Journal of Studies in International Education, 3, 241-266.
• Dorozyński, T., Świerkocki, J., & Urbaniak, W. (2016). Employers expectations vis-à-vis graduates of faculties of economics. Results of a direct study. Comparative Economic Research, 19(2), 93-109. https://doi.org/10.1515/cer-2016–0015
• Glazunova, O. I., & Gromyko, Yu. V. (2019). Two Approaches to Assessing Interactions in Cooperative Work: From PISA Monitoring Studies to Activity-Based Analysis of Collaboration in Project Teams. Psychological Science and Education, 24(5), 58-70. https://doi.org/10.17759/ pse.2019240506
• Händel, M., Artelt, C., & Weinert S. (2013). Assessing metacognitive knowledge: Development and evaluation of a test instrument. Journal for Educational Research Online, 5(2), 162-188.
• Harshman, J. (2016). Critical global competence and the C3 in social studies education. Social Studies, 107(5), 1-5. https://doi.org/10.1080/00377996.2016.119091
• Kang, J. H., Kim, S. Y., Jang, S., & Koh, A.-R. (2018). Can college students’ global competence be enhanced in the classroom? The impact of cross- and inter-cultural online projects. Innovations in Education and Teaching International, 55(6), 683-693. https://doi.org/10.1080/14703297.2017.1294987
• Koval, T. V., & Dukova, S. E. (2019a). Global'nye kompetencii – novyj komponent funkcional'noj gramotnosti [Global competence is a new component of functional literacy]. National and Foreign Pedagogy, 4(61), 112–123. [in Rus.]
• Koval, T. V., & Dukova, S. E. (2019b). Kak ocenivat' umeniya uchashchihsya v sfere global'nyh kompetencij [How to evaluate students’ skills in the area of global competence]. National and Foreign Pedagogy, 4(61), 208–217. [in Rus.].
• PISA 2018 Assessment and Analytical Framework. (2019). https://doi.org/10.1787/b25efab8-en
• Schleicher, A., & Ramos G. (2018). Global competency for an inclusive world. https://www.oecd.org/pisa/aboutpisa/Global-competency-for-an-inclusive-world.pdf
• Schwarz, N. (2007). Cognitive Aspects of Survey Methodology. Applied Cognitive Psychology, 21(2), 277-287. https://doi.org/10.1002/acp.1340
• Shkerina, L. V., Berseneneva, O. V., Zhuravleva, N. A., & Cave M. A. (2019). The meta-disciplinary Olympiad for school students: the new approach to assessing the meta-disciplinary universal educational actions of students. Perspectives of Science and Education, 38(2), 194-211. https://doi.org/10.32744/pse.2019.2.15
• Vogler, J. S., Thompson, P., Davis, D. W., Mayfield, B. E., Finley, P. M., & Yasseri, D. (2018). The hard work of soft skills: augmenting the project-based learning experience with interdisciplinary teamwork. Instructional Science, 46(3), 457-488. https://doi.org/10.1007/s11251–017–9438–9
• Vostorgova, E. V., Mikhailov, V. V., & Syshchenko, A. K. (2019). Model' diagnostiki i razvitiya soft skills shkol'nikov v ramkah podgotovki k sorevnovaniyam WorldSkills Junior [Model of Diagnosis and Developing Schoolchildren‘s Soft skills in Terms of WorldSkills Junior Competition Training]. Education. Science. Scientific personnel, 3, 131-134. https://doi.org/10.24411/2073-3305-2019-10152
• Yashin, A., Klyuev, A., & Bagirova, A. (2018). Designing entrepreneurial education in Russia: hard and soft skills. Ekonomski Vjesnik, 31(2), 261-274.
• Yildiz, M., & Palak, D. (2016). Cultivating Global Competencies for the 21st Century Classroom. International Journal of Information Communication Technologies and Human Development, 8, 69-77. https://doi.org/10.4018/IJICTHD.2016010104