Possibilities Of Dynamic Assessment Of Student Engagement In Online Learning

Abstract

The COVID-19 epidemic has intensified the use of online technologies in human communication around the world. In higher education, online learning has become one of the main types of learning. Although technology (Microsoft Teams, Zoom, etc.) for implementing this method of learning is created and has high functionality, nevertheless, a full-fledged transition to online education requires the formation of the relevant competencies for both the teacher and the student. An important aspect of online education that ensures its effectiveness is student’s engagement in educational activities. The concept of engagement is well developed in pedagogical and psychological sciences. Engagement can be studied at different scales of educational activity, has a variety of types and characteristics that affect the features of the educational process. The article substantiates the possibility of studying short-term (situational) manifestations of engagement based on the analysis of dynamic aspects of students' non-verbal behavior. As noted in numerous studies in the field of social psychology, active group interaction leads to synchronicity of verbal and nonverbal behavioral manifestations of group members. In learning activities, engagement also manifests itself as increased synchronicity and cooperation in students’ and teachers’ behavior. Computer vision and artificial intelligence allow for a constant dynamic assessment of student engagement and reflect its indicators on the teacher's computer screen. Using this assessment as a feedback form will improve the quality of learning in digital environment.

Keywords:

Introduction

In higher education in Russia the COVID-19 epidemic has made online learning one of the main types of education. The massive transition to the online learning format has become a great stress for all participants in the learning process. Although the technology (Microsoft Teams, Zoom, etc.) for introducing such a method of learning is highly functional, the transition to online learning requires the formation of appropriate competencies for both the teacher and the student. As the study reveals (Aleshkovsky et al., 2020) only 22.3% of students in Russian universities calmly perceived the need to change the usual form of learning. Assessing various aspects of learning in the distance format, 57.0% of the surveyed students noted that their motivation decreases, and 40.3% of the respondents also mentioned a decrease in the effectiveness of their work.

Engagement is one of the important characteristics that ensure the effectiveness of the educational process in the digital environment. Although the concept of engagement was initially developed in relation to other types of activities, for example, to laboractivity (engagement of personnel), later this concept was widely applied to the field of education as well (Kiseleva, 2017; Maloshonok, 2016; Vekkaila & Pyhältö, 2016, etc.).

Problem Statement

In the previous section, we identified four theoretical aspects that need to be taken into account when studying the phenomenon of engagement: the variety of definitions of the concept of engagement; the scale of engagement, the relationship with the educational institution; the types and characteristics of engagement; the ways to study engagement. Now we would like to define a framework in each of the highlighted aspects that will enable us to outline the range of requirements to develop a research approach to studying engagement in teaching students online. We have reinterpreted the technological features of modern online platforms as conditions that largely determine the nature of learning activities in the digital environment. In our study, we proceed from the definition of engagement given by Kiseleva (2018) as a multidimensional construct that includes behavioral activity aimed at performing activities, emotional reactions that occur during the performance of activities, cognitive and motivational processes that occur during its performance, as well as understanding of this activity.

A key feature associated with modern online education is essential difficulty in organizing feedback between the student and the teacher that significantly complicates communication. When conducting online classes, it is of great importance for the teacher to receive prompt feedback on the degree of student engagement in learning activities. Such feedback will allow the teacher to quickly respond to changes occurring with the student group, to adjust their activity, the forms of presenting learning material, to change the structure of the lesson, to plan the use of various educational technologies. Currently, a student during online learning can turn off sound and video, which does not allow the teacher to quickly assess the engagement of students and contact them, seeing difficulties. Even on-screen images of students may not be very informative due to their small size and coverage of only a small part of the student's behavior (usually only facial reactions).

These features determine the choice of the scale of educational activity that is necessary for the analysis. The need for prompt feedback from the teacher in relation to the student means an assessment of his behavior in the "micro-scale" (from a few seconds to a few minutes) and "mini-scale" (part of the lesson or the whole lesson). Most accurately, this phenomenon can be described by the concept of “inclusion”. Kiseleva (2018) believes that inclusion is an independent phenomenon that characterizes human behavior, and involves situational concentration of attention on the activity performed and the manifestation of activity aimed at its implementation. Unlike engagement, inclusion, in her opinion, does not involve consideration of the emotional, motivational and value component. In our study, we will consider inclusion as a manifestation of engagement in the "micro-scale" and "mini-scale", in which behavioral manifestations are most accessible to research. Thus, we do not agree with Kiseleva that inclusion is a different phenomenon from engagement. Inclusion, in our opinion, also includes emotional, motivational and value components, but it is difficult to assess and describe their manifestations when analyzing short fragments of behavior. Short-term emotional reactions are supposed to be the most accessible for identification, but the influence of motives, and especially value orientations, can be traced only on a broader scale of analysis of educational activities. Of particular importance to us are the dynamic characteristics of behavior (changes in gestures, micro-movements, verbal reactions), emotional response (primarily in the form of changes in facial expression), as well as manifestations of concentration (eye concentration on the screen).

The most significant characteristic of engagement for the study is the engagement degree. This is the most simple and elementary, as a rule, quantitative form of assessment, while it allows the teacher to obtain generalized (integral) information about the engagement of both a group of students and each student individually. At the same time, it should be taken into account that the engagement degree can vary significantly depending on the organization forms of the educational process used. In particular, at a lecture, engagement can be manifested in concentration on the material presented by the teacher and illustrations to it, while at seminars and practical classes, answers to questions discussed in the classroom, students' communicative activity, their emotional "responsiveness" to the educational situations offered by the teacher can be considered as manifestations of engagement. Thus, it is necessary to take into account the environment in which the classes are held, assignments are done, participating in various educational activities is organized and their characteristic behaviors when assessing the degree of students' engagement in educational activities in the digital environment.

The problem with choosing the method of studying engagement is that the most commonly used ones, such as surveys and interviews, are not suitable for recording its manifestations in such small-time intervals. Methods are needed that can quickly analyze and provide the teacher with information about the students’ engagement degree, which can only be achieved by using automated assessment methods based on quantitative calculations. As noted above, online learning in its current form makes it difficult for a student to communicate with a teacher verbally. Based on the ideas listed above, our study emphasizes the methods that allow recording and analyzing nonverbal behavior and its dynamics in short periods of time, and presenting the obtained data to the teacher in a visual form. Self-assessment methods and other methods can be used, but only as auxiliary ones, for example, to identify the personal significance of educational activities and their content for students, to identify the motives of educational activities, etc.

Research Questions

The discussion proposed in this article is a kind of theoretical research, which is aimed at forming an adequate approach to the study of student engagement in online learning. We believe that this theoretical study precedes future empirical research, the results of which will contribute to improving the quality of teacher-student interaction in the digital environment. Based on the problem statement, we formulated research questions.

• What is student engagement in online learning as a theoretical construct?
• What behaviour can be used to assess the engagement degree? What are the assessment methods?

Purpose of the Study

To propose a methodology for dynamic assessment of student engagement in educational activities in online learning based on non-verbal behavioral manifestations

Research Methods

Taking into account the theoretical nature of the work, we used general scientific theoretical methods: analysis, synthesis, abstraction, deduction, induction, definition of concepts, classification.

Findings

The framework specified by us when formulating the research problem is not a direct indication of a specific research approach, and apparently allows us to develop different approaches with different degrees of effectiveness in achieving the goal. In this article, we propose the use of a research approach that can provide a new heuristic view of the phenomenon of engagement. We believe that it fits well into the framework set by modern online education technologies. Therefore, although our research is theoretical, we expect significant practical results from its application in the future.

The presence of a certain temporal and spatial structure is a distinctive feature of animal communities. For example, fish in schools synchronize the direction and speed of movement with their neighbors, and birds flying in a group synchronize take-off and landing. Similar synchronicity is observed in group behavior of people. Louwerse et al. (2012) define synchronicity as a combined spatial and temporal coordination of behavior. Personal interaction involves several channels - language, gestures, and other behaviors. Within this framework, people can synchronize behavior with each other, which can be one way to reduce the complexity of interpersonal interactions by gradually adapting to each other and reducing the range of possible behaviors. Thus, by becoming more and more similar, the interlocutors greatly simplify the cognitive load required to interact with each other (Fusaroli, et al., 2014).

The study of the phenomenon of synchronicity is most widely used in the study of dialogue. For example, in the model of interactive matching synchronicity is described as consistency due to step-by-step repetition of behavior at all levels of linguistic processing (phonetic, lexical, syntactic, semantic, situational), and is supported by the coupling of gestures accompanying speech, body swaying, gaze, etc. Research has shown that dialog participants who already use a certain syntactic structure are more likely to create new sentences that use the same syntactic structure. Matching also manifests itself in more subtle and rather nonverbal aspects, such as speech speed, pauses, tonality, while matching on one communication channel in many cases seems to facilitate matching on others (Fusaroli et al., 2014).

The most common ways to explain synchronicity that occurs in human interaction use the concepts of imitation and coordination. Imitation can be described as reproducing selected characteristics of another person's behavior with a certain time delay. Examples of unconscious and unintentional imitation can be seen in the case of a yawning person, when the yawn spreads to everyone in the room. There is a correlation of frequencies between the interlocutors in random movements, such as shaking the leg or rubbing the nose, even if neither of them is aware that they are imitating the other. It is noted that newborns already copy the facial expressions of adults. Imitation has social advantages, apparently by inducing an unconscious feeling of sympathy (Louwerse et al., 2012).

The concept of coordination usually refers to more complex and conscious activities. At the same time, imitation is often automatic, and its sequence does not need to be consciously timed with the actions of another person (Louwerse et al., 2012). In the case of coordination, there is a correlation between the actions of several people to achieve a common goal, which does not always consist in similarity. When two people pick up a box, talk on the phone, or make financial calculations, they must perform different actions according to certain time intervals, otherwise the box will fall, the conversation will fail, and the financial transaction will not be completed. In the same way, people coordinate the turns of the dialogue in the right sequence, for example, when one asks a question and the other answers its (Louwerse et al., 2012).

We believe that the phenomenon of synchronicity can be considered as an explanatory mechanism of engagement in the learning process. Student-teacher learning interaction is also obviously a type of social interaction in which synchronicity can manifest itself, based both on imitation (presumably by the student to the teacher) and coordination (for example, when performing cooperative learning assignments). Numerous studies in the field of synchronicity study, as a rule, are aimed at studying communication in social psychology (dialogue, group communication, joint activities), as well as carried out in the field of clinical psychology (communication of the client and the psychotherapist, violations of synchronicity in various mental disorders). The exception is the study (Lafrance & Broadbent, 1976), which investigated the congruence of poses that participants take in the learning process. Therefore, this idea has yet to receive its empirical confirmation when studying the phenomena of synchronicity in various learning contexts. When creating a theoretical construct, thus, we will proceed from the fact that synchronicity can be a marker (indicator) of the engagement degree of participants in the learning process, that is, high synchronicity will indicate a high degree of engagement, and vice versa, low synchronicity –non-engagement or low engagement. The advantage of this approach is the ability to study engagement based on the analysis of not only speech signals, but also analyzing non-speech signals, such as posture, gestures, rhythm of movement, etc. As mentioned in the problem statement section, under online learning, the possibilities of speech communication are very limited. Therefore, the importance of analyzing and interpreting out-of-speech signals increases.

Initially, research in synchronicity studies relied primarily on expert methods, where external experts determined the similarity of poses, gestures, etc., based on observation, usually relying on certain forms of formalized description (Louwerse, et al., 2012). As a rule, these methods were very time-consuming, and the analysis of even small fragments of behavior required considerable time. The use of the expert method, therefore, does not allow for the rapid collection and analysis of information. However, further research followed the course of increasing automation in data collection and analysis. For example, this is the use of methods of so-called "computer vision". In the work by Alexandra Paxton and Rick Dale (2013) tracked changes in pixels from one frame to the next as an indicator that the interlocutors are synchronized in the general movement of the body. This method is also called "motion energy analysis" and has become widely used in a wide variety of modifications (Ramseyer, 2020). Another mathematical method that is becoming increasingly popular is based on a mathematical model used to study nonlinear dynamical systems (Ramseyer, 2020). It is called the method of quantitative analysis of cross-recurrence (CRQA - Cross Recurrence Quantification Analysis) and currently has several modifications, including those adapted for the study of group processes (Dindar et al., 2019). The method quantifies how often two systems exhibit similar patterns of change or movement (Fusaroli, et al., 2014). Thus, developments in the field of studying synchronicity enable to approach the problem of automated analysis of human behavioral activity in group interaction. Despite the fact that this approach has been used primarily in the field of social and clinical psychology, the study by Lafrance and Broadbent (1976) in the field of education has shown encouraging results.

Conclusion

We believe that the field of student engagement research based on the phenomenon of synchronicity of behavior has a great future in view of introducing new technological solutions. Technology of neural networks can become one of the most effective technologies. Existing technological solutions allow monitoring behavior based on the analysis of facial biometric indicators that reflect a variety of emotions (Nezami et al., 2020, etc.). Applied means of digital monitoring (3D sensors PrimeSense, APK "MindReader", ROFES-E01C, Barrier-14, etc.) are widely used currently. We believe that the use of automated assessment using computer vision and artificial intelligence technologies will allow us to quickly assess the impact of pedagogical influences on student engagement and develop teaching technologies that are adequate to the modern online environment.

In the empirical part of our study, we propose to use "Examus" - digital monitoring system which is developed and trained on the basis of neural network technology, as a means of assessing engagement. This system allows to quickly analyze the characteristics of facial expression and behavioral reactions of a person sitting at the monitor, according to two main parameters: the type of emotions displayed by a person and the level of concentration of attention.

The approach proposed in the study should be considered as a promising one to study student engagement in the digital environment. Of course, it is not able to solve all the problems associated with the study of engagement, but we assume that it will be successful within the theoretical framework established by us. An important theoretical issue is also the justification of constructive validity, both due to the variety of theoretical models of the phenomenon under study (as shown above), and in view of its novelty – the direct identification of the synchronicity and engagement in the established educational contexts. However, this approach has advantages as it allows to fill model representation of the behavioral manifestations of engagement with static (through the identification of the compliance of behavior of learning process participants at a specific point in time based on predefined schemes of behavioral reactions) as well as with dynamic assessment (coincidence of tempo-rhythmic characteristics of behavior).

Acknowledgments

Funding: The reported study was funded by RFBR, project number 20-04-60201.

References

• Aleshkovsky, I. A., Gasparishvili, A. T., Krukhmaleva, O. V., Narbut, N. P., & Savina, N. E. (2020). Studenty Rossijskogo universiteta po distancionnomu obucheniyu: ocenka i vozmozhnosti [Russian university students about distance learning: assessments and opportunities]. Vysshee obrazovanie v Rossii, 10, 86-100. DOI: 10.31992/0869-3617-2020-29-10-86-100 [in Rus].

• Alexandrov, D. A., Tenisheva, K. A., & Savelyeva, S. S. (2017). Vzaimosvyaz' mezhdu vneshkol'noj deyatel'nost'yu i uspevaemost'yu podrostkov i samoidentifikaciej [The Relationship Between Extracurricular Activities and Adolescents’ Academic Performance and Self-Concept]. Voprosy obrazovanija, 4, 217-241. DOI: 10.17323/1814-9545-2017-4-217-241[in Rus].

• Astin, A. (1984). Student involvement: a developmental theory for higher education. Journal of College Student Development, 25, 297-308. https://psycnet.apa.org/record/1999-01418-006

• Dindar, M., Alikhani, I., Malmberg, J., Järvelä, S., & Seppänen, T. (2019). Examining shared monitoring in collaborative learning: A case of a recurrence quantification analysis approach. Computers in Human Behavior, 100, 335–344. DOI:

  Google Scholar

  Crossref

• Duka, N. A., & Stovba, A. A. (2016). Ob izuchenii vovlecheniya studentov peduniversiteta v uchebnyj process [On the study of the involvement of students of the pedagogical university in the educational process]. Vestnik Omskogo gosudarstvennogo pedagogicheskogo universiteta. Gumanitarnye issledovanija, 3(12), 85-87. https://www.elibrary.ru/item.asp?id=27370650 [in Rus]

• Fusaroli, R., Konvalinka, I., & Wallot, S. (2014). Analyzing Social Interactions: The Promises and Challenges of Using Cross Recurrence Quantification Analysis. Springer Proceedings in Mathematics & Statistics, 103, 137-155. DOI:

  Google Scholar

  Crossref

• Goffman, I. (2003). Frame Analysis: An Essay on Organisation of Experience. Moscow: Institute of Sociology RAS [in Rus].

• Handelsman, M., Briggs, W., Sullivan, N., Towler, A. (2005). A Measure of College Student Course Engagement. The Journal of Educational Research, 98(3), 184–191.

• Kiseleva, N. V. (2017). Osobennosti vzaimootnoshenij s prepodavatelyami u uchashchihsya s razlichnymi vidami vovlechennosti v uchebnyj process [Features of the relationship with teachers among students with different types of involvement in the educational process]. Tradicii i innovacii v pedagogicheskom obrazovanii. Sbornik nauchnyh trudov III Mezhdunarodnogo kruglogo stola, 194-199. https://www.elibrary.ru/item.asp?id=32723033 [in Rus].

• Kiseleva, N. V. (2018). Social'no-psihologicheskie harakteristiki uchastiya uchashchihsya v obuchenii naprotyazhenii vsej zhizni na razlichnyh etapah processa obucheniya [Socio-psychological characteristics of the involvement of students in continuing education at different stages of the learning process]. Psihologo-pedagogicheskie issledovanija, 10(3), 37-45. DOI: 10.17759/psyedu.2018100304 [in Rus].

• Kiuru, K. V., & Popova, E. E. (2018). Problema vovlecheniya uchashchihsya v process obucheniya v kontekste onlajn-obrazovaniya [The problem of student involvement in the learning process in the context of online education]. Problemy sovremennogo pedagogicheskogo obrazovanija, 59(3), 380-384. https://www.elibrary.ru/item.asp?id=35196994 [in Rus].

• Kiuru, K. V., & Popova, E. E. (2019). Tvorchestvo, sotrudnichestvo, kriticheskoe myshlenie i kommunikaciya kak pokazateli uchastiya uchashchihsya v processe obucheniya [Creativity, collaboration, critical thinking and communication as indicators of student involvement in the learning process]. Problemy sovremennogo pedagogicheskogo obrazovanija, 63(1), 151-155. https://www.elibrary.ru/item.asp?id=37603835[in Rus].

• Lafrance, M., & Broadbent, M. (1976). Group Rapport: Posture Sharing as a Nonverbal Indicator. Group & Organization Studies, 1(3), 328-333. DOI:

  Google Scholar

  Crossref

• Louwerse, M. M., Dale, R., Bard, E. G., & Jeuniaux, P. (2012). Behavior Matching in Multimodal Communication Is Synchronized. Cognitive Science. A Multidisciplinary Journal, 36(8), 1404-1426. DOI:

  Google Scholar

  Crossref

• Maloshonok, N. G. (2016). Vzaimosvyaz' mezhdu ispol'zovaniem Interneta i mul'timedijnyh tekhnologij v uchebnom processe i vovlechennost'yu uchashchihsya [How using the internet and multimedia technology in the learning process correlates with student engagement]. Voprosy obrazovanija, 4, 83. DOI: 10.17323/1814-9545-2016-4-59-83 [in Rus].

• Newmann, F. (1992) Student Engagement and Achievement in American Secondary Schools. Teachers College Press.

• Nezami, O. M., Dras, M., Wan, S., & Paris, C. (2020). Image Captioning using Facial Expression and Attention. Journal of Artificial Intelligence Research, 68, 661-689. DOI:

  Google Scholar

  Crossref

• Pace, R. (1984). Measuring the Quality of College Student Experiences. An Account of the Development and Use of the College Student Experience Questionnaire. Los Angeles: Higher Education Research Institute Graduate School of Education University of California.

• Pascarella, E., & Terenzini, P. T. (2005). How College Affect Students. Vol. 2. A Third Decade of Research. Jossey-Bass.

• Paxton, A., & Dale, R. (2013). Frame-differencing methods for measuring bodily synchrony in conversation. Behavior Research Methods, 45, 329–343. https://link.springer.com/article/

  Google Scholar

  Crossref

• Ramseyer, F. T. (2020). Motion energy analysis (MEA). A primer on the assessment of motion from video. Journal of Counseling Psychology, 67(4), 536–549. DOI:

  Google Scholar

  Crossref

• Schaufeli, W. B., & Bakker, A. B. (2004). Job demands, job resources, and their relationship with burnout and engagement: a multi-sample study. Journal of Organizational Behavior, 25, 293–315. DOI:

  Google Scholar

  Crossref

• Shcheglova, I. A., Koreshnikova, Y. N., & Parshina, O. A. (2019). Rol' uchastiya studentov v razvitii kriticheskogo myshleniya [The role of engagement in the development of critical thinking in undergraduates]. Voprosy obrazovanija, 1, 264–289. DOI: 10.17323/1814-9545-2019-1-264-289. [in Rus].

• Shushlina, M. Y. (2019). Vliyanie novyh form organizacii uchebnogo processa na uroven' vovlechennosti uchashchihsya [The influence of the new forms of the student education organization on student’s involvement]. Forum. Serija: gumanitarnye i ekonomicheskie nauki, 2, 126-133. https://www.elibrary.ru/item.asp?id=38233264 [in Rus].

• Vekkaila, J., & Pyhältö, K. (2016). Doctoral Student Learning Patterns: Learning about Active Knowledge Creation or Passive Production. International Journal of Higher Education, 5(2). DOI:

  Google Scholar

  Crossref

• Voronina, A. A., Shabalina, O. A., & Kataev, A. V. (2019). Metody ocenki vovlechennosti pol'zovatelej interaktivnyh prilozhenij [Methods for assessing engagement of users of interactive applications]. Informacionnye tehnologii v nauke, obrazovanii i upravlenii, 14(4), 70-74. https://www.elibrary.ru/item.asp?id=41585282[in Rus].