Dynamic Adaptive Testing Of Students When Learning English

Abstract

The article analyses the aspects of testing students. The problem of creating a dynamic computerized adaptive test-based approaches: learning through interaction with the teacher and learning through interaction with the environment. The methodologies of the approaches are described in the works of L.S. Vygotsky and J. Piaget. The research questions are due to the specifics of the ecological approach to the dynamic assessment of learning and independent work, focused on the analysis of students as a process of interaction with the environment. Provides a solution for the development of a dynamic computerized test simulator (DCTT) that diagnoses potential training or teaches verb classification problems in English at times. Calculated feedback DCTT forms a numerical assessment (reinforcement) of each action and reward after completing the task. The characteristics of self-study include: the level of independence, labour intensity, pace, etc. As a result of the experiment in a sample of 162 students of three groups of subjects. The first group of students, has implemented autonomous error-free work on the classification of verbs times the English language without any additional training. For the second and third groups, the transition to autonomous learning activities required additional training, with self-regulated frequency reinforcement learning actions. If the I-st and the II-th groups independence level increases, the III - group it jumps level independence. Independence level abruptly decreased while increasing the time of the assignment and increases the number of errors. Criteria of cognitive instability of educational activity of students of all three groups were revealed.

Keywords: Learning activities, dynamic adaptive tests, English times, puzzles, procedural characteristics

Introduction

For the first time, dynamic adaptive testing of human learning activity as a test combining learning with testing was conducted by Luria (1976). Traditional so-called static testing diagnoses learning or learning outcomes in the past (Avanesov, 2012). Static (or standardized) tests refer to tests in which the tester presents: tasks of the test to the subject and writes down the answers without trying to intervene to change, direct or improve the test results. The introduction of dynamic adaptive tests or dynamic assessment of learning activity is due to the need to obtain information about a person's ability to learn, about cognitive functions, processes of change and mediation strategies that are responsible for the cognitive modifiability of the subjects (Feuerstein et al., 2002). Standardized static tests cannot provide this information because the learning process is excluded from the testing procedure. One of the main arguments against standardized tests is that it ignores motivational, emotional, and personality factors. At the same time, the results of a number of works, for example (Chou, 2018; Shrestha, 2017), show that motivational, emotional and personal factors for the success of the test takers are no less important than "pure" cognitive factors. However, in static tests, these factors are not given due consideration. That is, static tests do not provide information about learning and metacognitive processes. The main attention, in static testing, is directed to the question of what is the level of training of the subject in comparison with other subjects.

Dynamic assessment or dynamic adaptive testing provides information about the learning process that allows the teacher to identify specific strategies for effective learning and teacher-student interaction. In dynamic adaptive tests, tasks are formulated so that you can get answers to questions about how the subject processes information, what cognitive functions are formed in him in the process of completing tasks, and how thinking and learning can be improved. The information obtained about changes in the cognitive, motivational and emotional spheres during dynamic testing can be used to develop cognitive learning strategies in the future.

Problem Statement

The main attention of researchers of the potential of learning or learning (Farkov, 2015; Poehner & Lantolf, 2013) is aimed at solving the problem of creating computerized dynamic adaptive learning tests, based on the concept of the zone of proximal development of the socio-cultural development theory of Vygotsky (1978). At the same time, the interactivity of the dynamic assessment reflects the interaction of the mediator (teacher, mediator) and the student. Feedback in such computerized dynamic adaptive learning tests is instructive in nature and simulates the activity of the mediator. The main purpose of dynamic adaptive testing is to diagnose learning potential and changes in cognitive functions during task performance. These changes in cognitive function are the foundations of cognitive intervention to actualize learning potential. In dynamic adaptive tests, tasks are constructed on the basis of their "learning potential", taking into account the possibility of learning important cognitive strategies, enhancing cognitive functions, and measuring cognitive changes. The focus of dynamic adaptive tests is on teaching cognitive strategies and operations. Completing the current assignment prepares the student for a more complex assignment.

In contrast to the closed nature of static testing, the dynamic adaptive testing procedure is open. The role of the testing mediator changes from passive registration (recording) of responses to active mediation of cognitive strategies, rules, operations, etc. In other words, in standardized testing, the tester ensures that the testing procedure is closed, and then calculates the points and interprets the results. In the dynamic assessment process, a mediator (teacher) intervenes in the testing process in order to change the functioning of the test subject and interpret future possible changes, taking into account the current changes during the diagnosis. The interactive process of dynamic assessment is characterized by the regulation of the subject's behaviour: by suppressing impulsivity, ordering and organizing the size of the task. The interaction of the mediator (mediator) with the subject removes the deficits in cognitive functions, enriching the cognitive abilities of the subject.

In standardized static testing, the focus is on the end product of the learning process: learning. Dynamic adaptive testing focuses on cognitive processes that cause changes in specific cognitive functions that require development, as well as motivational and emotional factors that affect the functioning of the cognitive sphere. In other words, the emphasis is on the cognitive components of the subjects, such as the nature of cognitive learning and strategies, and the mediator's intervention that facilitates these changes. Standardized testing diagnoses learning, and dynamic adaptive testing diagnoses learning potential or learnability, including unique and qualitative aspects of the subject's behaviour. Standardized testing provides answers to the “what” and “how much” questions, dynamic adaptive testing of the learning process “how” and “why”.

This paper shows that dynamic adaptive testing of the self-learning process can answer not only the “how” and “why” questions, but also solve the problem of quantitatively describing and diagnosing the components of learning, i.e. answer the questions "what" and "how much".

Research Questions

The research questions considered in the article are due to the need to diagnose and quantitatively describe the potential of learning or learnability, as well as to identify correlations between the components of learning ability of subjects using various cognitive strategies for finding solutions to problems of sentence classification, according to the main tenses of the English language. The existing dynamic adaptive tests of the learning process interpret the qualitative aspects of learning potential, and analyse the lack of cognitive functions and the intermediary efforts required to change them. Learning potential, or learnability, is a dynamic metric that paves the way for a deeper study of the factors that prevent a learner from completing an assignment. Learnability is an integral dynamic parameter of learning activity (Farkov, 2015). Learning components include:

• temporal pace of learning;
• economy of thinking or laboriousness of learning;
• flexibility of thinking or the ability to act in the face of uncertainty or changing conditions;
• independence of thought or sensitivity to external help and self-awareness, exclusion of mistakes;
• features of generalization and abstraction of signs or the ability to compress educational information; mindfulness of actions or self-esteem of learning actions.

Purpose of the Study

The aim of the study is to diagnose the components of learning, based on the use of computerized dynamic adaptive self-learning tests. The interactivity of such tests is provided by the interaction of the student and the electronic problematic environment. Interaction represents reinforcements for the learner's actions, which are rewards for doing right and penalties for doing wrong. In this case, the feedback is of a numerical, evaluative nature. As the frequency of incorrect actions decreases, the level of independence increases, and the relative frequency of reinforcement of the student's actions decreases. Upon achieving error-free activity, the student passes into a state of autonomous, completely independent activity.

Research Methods

Methods for achieving the research goal are based on the theory of reinforcement learning (Sutton & Barto, 2014) and the theory of human development by Piaget (2003). The instrumental basis for the study is computerized dynamic adaptive self-learning tests (Bortnovsky et al., 2010; Noskov et al., 2019). Their difference from dynamic adaptive learning tests developed on the basis of the sociocultural theory of Vygotsky (1978), is that the process of human self-learning occurs as a result of interaction with the electronic problematic environment. Human development is based on the principle of balance between the processes of accommodation and assimilation of information. The context of dynamic adapted testing in the form of mediator actions through recommendations, help and feedback is replaced by the context of dynamic adapted testing in the form of self-regulation of the frequency of reinforcement. Accordingly, a sense of competence is replaced by a sense of independence. Interpretation of test results in dynamic adaptive self-learning tests is mostly objective, since the subject's reactions are responses to reinforcements.

The nature of the assignments is due to the fact that they are intended for self-study, and not for learning. Problem solving activities are completed by the student, not the teacher, as in the case of dynamic adaptive learning tests developed on the basis of Vygotsky (1978).

Figure 1 shows the interface of the dynamic adaptive self-learning test simulator "Times of the English language". On the left side of the interface there are:

• "DONE" button, after the classification task has been completed;
• a sensor with an activity actiogram, provides reinforcement to the actions of the subject (sets the motivation for the process); 3.the statusogram sensor, determines the level of independence of the subject after completing the task (sets the motivation for achievements);
• 3.Puzzle viewing window with sentences in English. On the right side of the interface there is a working field of the English tenses classification table.

See Full Size >

The test subject selects a puzzle in the puzzle view window and clicks on the corresponding cell of the table with a question mark. The puzzle from the viewport takes the corresponding position in the table of the times of the English language. If the choice is made correctly, then the subject receives a positive reinforcement +1, if wrong, then reinforcement - 1. Actions related to viewing puzzles are not reinforced. They have a zero numerical rating, but are taken into account in the task execution data protocols to determine the labour intensity and time rate.

Findings

The experiment involved 162 students. The execution time and the number of tasks were not limited. Dynamic testing ended when level 10 of independence was reached, which corresponded to autonomous error-free activity in the classification of sentences by the times of the English language. All subjects began their activity from the 1st level of independence, at which every action of the subject was reinforced. As a result of the experiment, the entire set of subjects was divided into three subsets: I. the subset of students, who can be characterized, as well as Budoff (1987), as “scorers”. II. a subset of subjects with a monotonically increasing level of independence; III. a subset of subjects with a non-monotonically increasing level of independence.

Figure 2 shows a histogram of the distribution of subjects for the indicated subsets.

See Full Size >

The number of the level of independence of educational activity of the subject is determined from the expression:

$U_v=1+INT\left(10\times I\right)\left(1\right)$

$U_v=\mathrm{1,2},3,\cdots ,\mathrm{9,10}\left(2\right)$

Here:

$I=1-H\left(p\right), ifp\ge 0.5 \left(3\right)$

$I=0, ifp<0.5, \left(4\right)$

where: $H=-p{\text{log}}_2\hspace{0.17em}p-\left(1-p\right){\text{log}}_2\left(1-p\right),$ informational entropy of the subject's actions, $p$– the relative frequency of correct actions of the previous task. The value is calculated after solving the next problem. The value $I$ varies from 0 to 1. On the display screen, the value $I$ is displayed by the levels of independence of the subject's activity.

The leaps in the levels of independence testify to the cognitive instability of the subjects caused by the lack of external information, which they stopped receiving during the transition to higher levels of independence. Cognitive instability for these subjects leads to an increase in the number of errors, labor intensity, and time to complete the task (Dyachuk et al., 2016). Upon completion of the task, the electronic problematic environment increases the relative frequency of reinforcements and, accordingly, the level of independence of the subject decreases abruptly. In Figure 3, graph number II, the subject had two breakdowns in the level of independence, before he reached level 10 of independence and completed the dynamic test. The monotonous increase in the level of independence includes two subsets of subjects I - "scorers" and III.

In both cases I and III, subjects may experience states of cognitive instability. But the cognitive unstable state arising from a lack of information (the number of reinforcements decreases) is compensated by the subject by mobilizing internal mental resources. In this case, there is no regression of activity and the monotony of the increase in the level of independence remains. However, if there is a shortage of external information, the task execution time may increase. This indirectly indicates that the subject is in a state of cognitive instability, which manifests itself in an increase in the time it takes to complete the task.

See Full Size >

For 50% of the "scorers" the time for completing the task at the 10th level of independence has increased. This can be considered as indirect evidence of the state of cognitive instability of the subjects. Overcoming the cognitive stress caused by disabling the reinforcement sensor results in wasted time. However, these subjects do not make mistakes, in contrast to the subjects of group III (see Figure 4). Figure 4 shows the histograms of the distribution of subjects II and III of the subset by the number of performed posterior, during the transition to autonomous activity (level 10).

See Full Size >

From the analysis of the histograms in Figure 4, it can be seen that subjects with jumps in the levels of independence reach the 10th level of independence on average, for a larger number of tasks, i.e. the labor intensity and the time to achieve autonomous activity are greater for them than for subjects with a monotonously increasing sequence of levels of independence.

Conclusion

Modern conditions of a multilingual society place special demands on the level of English proficiency. This is especially true for university students. "Doubling" of consciousness in the process of learning a foreign language requires new approaches to assessing the student's learning ability, on the basis of which it is possible to build an individualized trajectory of his development. The article demonstrates the capabilities of dynamic computer tests-simulators of the English language, which make it possible to intensify the solution of problems of the classification of verbs by the tenses of the English language and to diagnose learning.

References

• Avanesov, V. S. (2012). History of the pedagogical theory of measurements. Pedagogical measurements, 3, 3-26.

• Budoff, M. (1987). The validity of learning potential assessment. In C.S. Lidz (Ed.). Dynamic assessment: An interac-tional approach to evaluating learning potential. (pp. 53-81). New York.

• Chou, M. H. (2018). Speaking anxiety and strategy use for learning English as a foreign language in fulland partial English-medium instruction contexts. TESOL Q, 52(3), 611-633.

• Dyachuk, P. P. (Jr.), Dyachuk, P. P., Karabalykov S. A., & Shadrin, I. V. (2016). Diagnostics of unstable cognitive states of active agents. In NEUROINFORMATICS: Collection of scientific papers: in 3 parts, (pp. 259-270).

• Bortnovsky, S. V., Drozdova, L. N., Dyachuk, P. P., Dyachuk, P. P. (Jr.) & Shadrin, I. V. (2010). Management of adaptation of students in problem environments and diagnostics of self-regulation processes of educational actions. Publishing house of KSPU named after V.P. Astafiev.

• Farkov, A. V. (2015). Teachableness of students in mathematics: diagnostic problems. 5 - 11 grades. VAKO.

• Feuerstein, R., Falik, L. H., Rand, Y., & Feuerstein, R. S. (2002). The dynamic assessment of cognitive modifiability. In The learning propensity assessment device: theory, instrument and techniques. ICELP Press.

• Luria, A. R. (1976). Cognitive Development. Its Cultural and Social Foundations. Harvard.

• Noskov, M. V., Peregudova, I. P., Dyachuk, P. P., & Denisenko, O. I. (2019). Dynamic adaptive audio tests-simulators as a means of monitoring bilingual education. Informatics and Education, 10, 46-54.

• Piaget, J. (2003). Psychology of intelligence. Peter.

• Poehner, M. E., & Lantolf, J. P. (2013). Bringing the ZPD into the equation: capturing L2 development during computerized dynamic assessment (C-DA). Lang Teach Res, 17(3), 323-342.

• Shrestha, P. N. (2017). Investigating the learning transfer of genre features and conceptual knowledge from an academic literacy course to business studies: exploring the potential of dynamic assessment. JEngl Acad Purp, 25, 1-17.

• Sutton, R. S., & Barto, E. G. (2014). Reinforcement learning. BINOM. Knowledge laboratory.

• Vygotsky, L. S. (1978). Mind in society. Harvard University Press.