The article highlights the problem of assessing the achievements of mentally retarded schoolchildren in an interactive environment. The purpose of the study is to determine and test the requirements to the diagnostic procedure, developed on the basis of the general outcome measurement method for monitoring the achievements of mentally retarded schoolchildren in an interactive environment. The paper determines such requirements as the allocation of the progressive indicator system for a certain ability or skill testing; structuring tasks based on letter sets and evenly distributing them into difficulty levels; presentation of tasks within the indicator by groups; short assessment time; etc. These requirements were tested on the example of initial reading skills formation indicator assessment; the results were analyzed applying the methods of statistical analysis. To determine the reliability differences or coincidences were found between the stable results of two inclusions in a close time period. To determine the validity of relationship (correlation) of the assessment results in an interactive environment with the results of reading aloud or a verbal response was revealed. The results of approbation do not contradict with the assumption that compliance with the indicated requirements in the assessment of reading indicators in students with mental retardation in an interactive environment allows obtaining reliable and valid data. Thus, the requirements to the assessment procedure of the achievements of mentally retarded schoolchildren in an interactive environment have been determined, which allow obtaining accurate results in a short time.
Currently monitoring problems in education is gaining special relevance; a number of not only Russian and foreign, but also recent international studies have been focused on them (Farley et al., 2016; Hill & Lemons, 2015; Hopfenbeck et al., 2018; Jones et al., 2018; Kearns et al., 2015). Regarding the children with learning difficulties, the problems of both "starting" assessment and tracking the effectiveness of learning are actualized; new technologies and practical tools are in demand for planning the expected possible results in the near future and obtaining information about the efficiency of correctional and pedagogical schemes, the so-called "feedback". For example, in many countries’ education systems due to availability of different levels of support for students with learning difficulties, procedures for evaluating the response to intervention (RTI) are widely discussed (Solheima et al., 2018). One can note a tendency towards convergence of specialists opinions from different countries - representatives of various scientific schools and methodological foundations of the RTI-model of diagnostics, in particular, having something in common with the concept of “zone of proximal development” developed by L.S. Vygotsky.
In the context of our study the method of general outcome measurements (GOMs) (Wallace et al., 2010; Wayman et al., 2009) developed on the basis of a curriculum-based measurement (CBM) (Deno, 2003), is of interest. GOMs are tailored to accommodate learners with cognitive impairments. In addition, the student's answer through recognizing the correct option of the three suggested, which was originally used within the framework of this method to assess the achievements of students with a lack of speech, is extremely convenient for translation into a "programming language".
The possibilities of using the modified Russian-language version of the GOMs method for assessing achievements of students with mental retardation (on the example of reading skills assessment) have been confirmed in a series of studies conducted on the basis of the KSPU named after V.P. Astafiev in 2016 - 2020 (Mamaeva et al., 2020).
The research conducted allows us to conclude that the requirements of GOMs are universal for different languages and different cultural contexts. Also, on the basis of the KSPU named after V.P. Astafiev the research was done, the results of which do not contradict with the possibility of applying a modified version of GOMs for monitoring the reading skills of mentally restarted students using computer technologies on the local media (tablet computer with a touch screen). The relevance of tracking the effectiveness of teaching reading skills to children with mental retardation is due, on the one hand, to the importance of this skill for socializing in community, and on the other hand, to persistent difficulties that children of this category experience in mastering the initial reading skills (Afacan et al., 2017; Ainsworth et al. al., 2016). Apart from that, in recent years interest in the use of computer technologies in education has been growing; new domestic and foreign research accompanied with practical studies regularly appear (Soboleva et al., 2017).
Assessment using computer technology has a number of advantages:
- objectivity, independence from expert opinion;
- possibility of including children with the lack of speech in the assessment system;
- optimization of time spent on processing the results (computer processing is done automatically)” (Mamaeva et al., 2020, p 228).
Carrying out assessment in an interactive environment, in contrast to a local medium, also ensures the "transparency" of its results, while not contradicting with the principles of deontological ethics. Since the results of a particular child are available only to a limited number of people, it allows organizing interaction with the family and carry out assessment in a remote mode (which is especially important in the context of the COVID-19 pandemic). However, the transfer of assessment from a local media to an interactive environment requires clarification of the procedure requirements and their approbation in new conditions.
The problems of the research are to determine and test the procedure requirements to assess the learning achievements of mentally retarded schoolchildren in an interactive environment.
In accordance with the problem indicated, the following research questions arise:
- What are the requirements for the procedure to the assessment of mentally retarded schoolchildren learning achievements in an interactive environment?
- Is the data obtained in an interactive environment in compliance with the requirements we have designated (using the example of assessment indicators of the initial reading skills formation) reliable and valid?
Purpose of the Study
The aim of the study is to determine and test (using the example of assessment indicators of the initial reading skills formation) the requirements for the assessment procedure developed on the basis of the GOMs method for monitoring the learning achievements of mentally retarded schoolchildren in an interactive environment.
Requirements for assessment procedure
To answer the first question of the research, while determining the requirements to the procedure of mentally retarded schoolchildren achievements testing in an interactive environment, we relied on the main provisions of the GOMs method. However, the original version of the GOMs method, proposed by the University of Minnesota, was modified taking into account:
- specifics of the spoken and written Russian language;
- specifics of teaching children with mental retardation to read in Russian;
- features of interactive environment.
The following requirements are defined as basic:
Selecting the system of progressive indicators for a certain ability or skill assessment. This requirement is one of the key ones in the method and correlates with the principle of “gradual learning” in Russian correctional pedagogy with the focus on “zone of proximal development”. To assess the formation of initial reading skills, in cases where training is based on a sound analytical-synthetic method, we have identified the following indicators:
- knowing letters;
- language analysis skills (recognizing the first letter in a word, the last letter in a word, the presence of a letter in a word, syllabic analysis);
- reading syllables;
- reading words;
- reading sentences (matching a picture to a sentence; finishing a sentence with a word that matches the meaning).
For children who are not able to master reading on the basis of generally accepted approaches, alternative reading methods are proposed: visual perception, pictograms, “reading” of pictographic sentences.
Some of the above indicators have been tested on paper, local media using computer technologies (tablet computer with a touch screen) and in an interactive environment; others are at the stage of development and testing. This report will present the results of testing the requirements for the procedure of the initial reading skills formation assessment in mentally retarded schoolchildren in an interactive environment using the example of the following indicators: recognizing the first and last letters in a word, reading syllables and reading words.
Structuring of diagnostic tasks not only according to indicators, but also letters. Since the process of studying letters in mentally retarded schoolchildren is time-consuming (sometimes for several years) and students initially start reading syllables, words and sentences using a limited set of letters, it is appropriate to start tracking the formation of these indicators also with a limited letter sets. The tasks are divided into three groups.
Thus, assessment according to our proposed algorithm presupposes an initial testing the students’ alphabet skills, and then the rest of the indicators based on the letter set located in the "zone of proximal development" of a particular student. At the same time, within each set both recommended and additional indicators are shown. For example, in the set 3 (all letters) the recommended indicators are: reading syllables, words, sentences; the rest of the indicators are considered as extra.
It should be noted that one more difference from the foreign version, where all tasks within the indicator were considered as equivalent, is the allocation of difficulty levels within the indicator and an even distribution of tasks into difficulty levels. For example, according to the indicator "recognizing the first letter in a word" the first level of difficulty is identifying the first vowel in a word; the second level of difficulty is the first consonant from the consonant cluster; the third level of difficulty is the first consonant from the fusion syllable. Tasks for testing each of the indicators are presented sequentially by groups; each includes one task of all difficulty levels of the indicator in a random sequence; the choice of a task from a specific difficulty level is carried out randomly. Thus, the “phasing” we offer in the modified version of the GOMs method is more specific, which allows us to obtain more accurate data on the skills formation within each specific indicator, since the program automatically calculates the number of correct recognitions both for each indicator and for each difficulty level within indicator.
Due to the fact that the screening time for each indicator is limited, we have defined a requirement - no more than three difficulty levels within the indicator.
Short assessment time (three minutes per indicator). This requirement is indicated in GOMs method and is due to such psychophysical characteristics of students with cognitive disorders as: low performance, increased fatigue, distraction, difficulty concentrating, reduced motivation (Zashchirinskaia et al., 2017). Subsequent to allocation of difficulty levels within the indicator this provision is modified: for each difficulty level within the indicator the time is distributed evenly: 60 seconds per level in the case of three difficulty levels within the indicator, 90 seconds per level in the case of two difficulty levels within the indicator.
The student's answer recognizing the correct option from the three proposed. When completing the tasks a child must choose the correct answer from the three offered. Correct answers are automatically calculated per unit of time (3 minutes per indicator), both for the indicator as a whole and for each difficulty level within the indicator. During the research we used both showing the answer with a computer mouse, and with the help of a pointing gesture (in this case, the teacher controls the mouse), as well as with the help of a student's touching of the correct option on the touch screen. The choice of electronic medium and a method of display were made by the teacher, based on the motor characteristics of a particular student and material and technical support in a particular organization. At the same time, while working with each particular student during the experiment, only a uniform technical solution was used, since hypothetically it can be assumed that a change in conditions of presenting material will affect the quantitative diagnostic indicators. Comparison of the results when using different electronic media and different recognition methods was not carried out within the framework of this study, however, we are considering it as a further perspective.
The rule of "three errors" in the original foreign version presupposes the end of the test when a student makes three consecutive errors. Compliance with this requirement, on the one hand, allows to reduce the test time, while maintaining the accuracy of the results, and also prevents fatigue and students’ negative reaction to repetitive mistakes when presenting tasks that are beyond their proficiency level or not understood, which is especially difficult for students with mental retardation and the brain control of the affect is difficult. Based on foreign studies on the possibility of applying the rules of both two and three errors (Wayman et al., 2009), we settled on the “three errors” rule. However, its implementation in our program has its own specifics associated with the introduction of difficulty levels within the indicators: in the case of three consecutive incorrect choices in any difficulty level, the testing of only this level is stopped; testing by the indicator as a whole for the rest of the difficulty levels is continued according to the time assigned.
Using the first three tasks for each indicator as training. This provision of the GOMs method is used in our proposed version without changes and involves the use of a system of prompts with a gradual increase in the assistance provided with each subsequent repetition of the task (four presentations of training tasks are acceptable), which allows a student with mental retardation to navigate when applying knowledge in new conditions. The simplest tasks are used as trainers, which are not used as estimates afterwards. The results of training tasks are not taken into account when scoring.
Providing organizing assistance by repeating instructions at the first incorrect choice when performing assessment tasks (assessment tasks are tasks starting from the fourth within each indicator). If the choice is made with an error or does not fit within a certain time limit (from 5 to 30 seconds, depending on the indicator), then the question is repeated, but already expanded. The correct choice from the first presentation of the task in each assessment task is awarded with 1 point, while for the correct choice after repeating the question no points are awarded. This requirement is due to the need for additional assistance associated with the insufficient formation of motivational, energy and regulatory components of cognitive activity in children with mental retardation (Dmitriev & Verkhoturova, 2019; Verkhoturova et al., 2020). Repeating instruction in this case reduces the risk of simplifying instructions by the learners ("degrading" to random choice).
The possibility of up to four inclusions in a close time period (within two weeks) to obtain reliable results, which is an important condition for ensuring data reliability. Carrying out several inclusions is especially important when learning how to work with the program, as well as in case of a situational decrease in indicators results; for this, the program provides a function for selecting the basic value of the indicator for a period. The need to comply with this requirement was confirmed earlier through a series of studies in 2016 - 2020 (Mamaeva et al., 2020).
Interpretation of assessment results based on their quantitative and qualitative analysis. Scoring by indicators and difficulty levels is done automatically, but it is on the basis of quantitative and qualitative analysis that a teacher chooses a letter set, assessment indicators, the base indicator values for the period, and also supplies and interprets the numerical values, taking into account the qualitative features of the tasks, observation of behavior and children condition during the test period. For example, a student who chooses a "random guess" strategy while reading sentences may score more points than a student who completes assignments consciously, but slowly, and whose reading by syllables skill is just forming.
It should also be noted that within the framework of this message there is no need to present requirements for the content and design of tasks for each specific indicator (for example, material selection, voice instructions, images, font design, etc.), which is of a private nature and belong to exactly one specific indicator.
These requirements to the content and procedure for assessing the initial reading skills formation in mentally retarded schoolchildren in an interactive environment were tested in terms of the following indicators: recognizing the first and last letters in a word, reading syllables and reading words.
Determining the reliability and validity of data
To answer the second question of the research (to determine the reliability and validity of data obtained in an interactive environment in compliance with the requirements indicated), a series of studies was organized in the period from 2018 to 2020 on the above indicators. Testing of different indicators was carried out on the basis of different organizations, with different students and at different times. The main criteria for inclusion in the experiment were: the presence of mental retardation and the formation of the indicator (the examined indicator of the initial reading skills formation at the time of assessment should be in the "zone of proximal development"). Data on the organization of the experimental study by indicators are shown in Table 1.
The experimental study was organized as follows:
- to determine the reliability between two and four inclusions were carried out within two weeks, and then the results of two stable inclusions were compared;
- to determine the validity the interdependence between the data on recognitions in an interactive environment and the results of reading syllables and words aloud or a verbal answer to a question about the first and last letters in a word was analyzed.
The results were reviewed using the methods of statistical analysis, the software package "Statistica 10" and nonparametric statistical tests for related samples were used: Wilcoxon's test, sign test, and Spearman's correlation coefficient was calculated.
Initial analysis data is presented in Table 2.
As can be seen from Table 2, for all indicators the medial, maximum and minimum values of recognitions during two stable inclusions carried out over two weeks are very close, which is consistent with the assumption of the tested sets of tasks reliability. The medial, maximum and minimum values of recognitions and the corresponding values of the verbal responses are also close, but the results of the recognitions are higher than the results of the verbal responses. This is explained by the fact that recognizing and reading aloud (or naming the first and last sounds in a word) are close, but not identical; reading aloud has a more complex psychological structure. At the same time, the results of the recognitions in an interactive environment and the results of verbal answers can be closely interrelated, which corresponds to the results of our earlier study on a local medium, and does not contradict with the assumption of the task sets validity tested in an interactive environment (Mamaeva et al., 2020). To confirm the outcomes on reliability and validity of tasks set placed in an interactive environment in compliance with the above requirements, the results of approbation were analyzed using methods of statistical analysis.
Determining the reliability of task sets
To determine the reliability of the tasks sets, the results of two stable inclusions were compared for each indicator in a close time period (up to two weeks).
As can be seen from Table 3 for all indicators, the obtained p-value values (Wilcoxon, Sing Test) allow us to conclude that there is no statistical significance of changes in the indicator when repeating inclusions for two weeks. The results coincide at a significance level of 0,05. Therefore, task sets placed in an interactive environment in compliance with the requirements indicated are reliable.
Determining the validity of the task sets
To determine the validity of the task sets for each indicator the interdependence (correlation) was determined between the results obtained in an interactive environment by the recognition method and the results obtained applying reading aloud.
As can be seen from Table 4 for all indicators, a close relationship was revealed between the data obtained in an interactive environment by the recognition method and the data obtained by traditional methods of reading aloud (or verbal response). The correlation coefficient reaches statistical reliability for all indicators, which allows us to make a conclusion about the validity of task sets placed in an interactive environment in compliance with the requirements we have indicated. Since according to indicators I and II (recognizing the first and last letter in a word) not the reading skill itself is examined, but one of the prerequisites for its formation, namely, elementary forms of sound-letter analysis, the correlation coefficient with the results from naming is somewhat lower than in indicators III and IV, within which testing of the syllables and words reading is carried out.
It is effective to assess the learning achievements of students with mental retardation in an interactive environment based on a modified Russian-language version of the GOMs method (general outcome measurement).
To obtain more accurate data in a short time, it is recommended to carry out assessment in an interactive environment taking into account the following requirements: allocation of progressive indicators system for diagnosing a certain ability or skill; even distribution into difficulty levels (no more than three levels within the indicator); presentation of tasks within the indicator by groups, each including one task of all difficulty levels in a random sequence; short assessment time (three minutes per indicator); a student's answer by recognizing the correct option out of the three suggested; completion of test for each difficulty level within the indicator when making three consecutive errors; using the first three tasks as training; applying organizing assistance; the possibility of up to four inclusions in a close time period to obtain reliable results. To assess the initial reading skills formation it is advisable to structure the tasks according to the letter sets.
These requirements were tested in an interactive environment using the example of assessing the initial reading skills formation; the reliability and validity of task sets was confirmed by the following indicators: recognizing the first and last letters in a word, reading syllables and reading words.
At the same time, the results presented in this article do not reveal all the questions on the problem of monitoring the learning achievements of students with disabilities in an interactive environment. As a further perspective, it is assumed to expand and test the task sets for assessing the achievements of students in other educational areas; determine sensitivity to changes in the developed task sets; clarify the requirements to the monitoring procedure for students of various nosological groups, as well as those with mental retardation complicated with motor and behavioral disorders.
We would like to thank: Administration and teachers of educational organizations of Krasnoyarsk and Krasnoyarsk Territory, who took part in approbation: "Krasnoyarsk School No. 3", "Krasnoyarsk School No. 5", "Krasnoyarsk School No. 8", "Zheleznogorsk Boarding School". Masters graduates of KSPU named after V.P. Astafiev who took part in data collection (Voloshina Polina Aleksandrovna, Kilina Svetlana Yurievna).
Afacan, K., Wilkerson, K. L., & Ruppar, A. L. (2017). Multicomponent reading interventions for students with intellectual disabilities. Remedial and Special Education, 39(4), 229- 242. DOI:
Ainsworth, M. K., Evmenova, A. S., Behrmann, M., & Jerome, M. (2016). Teaching phonics to groups of middle school students with autism, intellectual disabilities and complex communication needs. Research in Developmental Disabilities, 56, 165-176. https://www.deepdyve.com/lp/elsevier/ teaching-phonics-to-groups-of-middle-school-students-with-autism-7ICzoO60o9
Deno, S. L. (2003). Developments in curriculum-based measurement. The Journal of Special Education, 37(3), 184-192. https://eric.ed.gov/?id=EJ785942
Dmitriev, A. A., & Verkhoturova, N. Yu. (2019). Social and psychological aspects of emotional response in childhood. Journal of Siberian Federal University. Humanities & Social Sciences, 15(4), 537-545. http://journal.sfu-kras.ru/article/125202
Farley, D., Anderson, D., & Irvin, Sh. (2016). Modeling reading growth in grades 3 to 5 with an alternate assessment.RemedialandSpecialEducation,38(4),195-206. DOI:
Hill, D. R., & Lemons, C. J. (2015). Early grade curriculum-based reading measures for students with intellectualdisability.JournalofIntellectualDisabilities,19(4),311-325. DOI:
Hopfenbeck, T. N., Lenkeit, J., Masri, Y. El., Cantrell, K., Ryan, J., & Baird, J. (2018). Lessons Learned from PISA: A Systematic Review of Peer-Reviewed Articles on the Programme for International Student Assessment. Scandinavian Journal of Educational Research, 62(3), 333-353. DOI:
Jones, F. G., Gifford, D., Yovanoff, P., Al Otaiba, S., Levy, D., & Allor, J. (2018). Alternate assessment formats for progress monitoring students with intellectual disabilities and below average IQ: an exploratory study. Focus on Autism and Other Developmental Disabilities, 34(1), 41-51. DOI:
Kearns, J. F., Kleinert, H. L., & Thurlow, M. L. (2015). Alternate assessment as one measure of teacher effectiveness: implication for our field. Research and Practice for Persons with Severe Disabilities, 40(1), 20-35. DOI:
Mamaeva, A. V., Antonova, T. S., Denisova, X. Y., & Kilina, S. Y. (2020). Achievement Monitoring Technology Of Mentally Retarded Schoolchildren On The Touch Screen. European Proceedings of Social and Behavioural Sciences EpSBS, 90(27), 226-235. https://www.europeanproceedings.com/proceedings/EpSBS/volumes/icest2020
Soboleva, E. V., Sokolova, A. N., Isupova, N. I., & Suvorova, T. N. (2017). Use of training programs based on gaming platforms for improving the effectiveness of education. Novosibirsk State Pedagogical University Bulletin,7(4), 7-25.
Solheima, O. J., Frijtersa, J. C., Lundetræ, K., & Uppstada, P. H. (2018). Effectiveness of an early reading intervention in a semi-transparent orthography: A group randomised controlled trial. Learning and Instruction, 58, 65-79. DOI:
Verkhoturova, N. Y., Dmitriev, A. A., & Dmitrieva, S. A. (2020). On the problem of relevance to develop emotionally competent behavior in children with intellectual disabilites. E3S Web of Conferences, 210, 18109. DOI:
Wallace, Т., Tichá, R., & Gustafson, K. (2010). Technical characteristics of general outcome measures (GOMs) in reading for students with significant cognitive disabilities. Reading& Writing Quarterly, 26(4), 333-360. DOI:
Wayman, M. M., Tichá, R., Wallace, T., Espin, C. A., Wiley, H. I., Du, X., & Long, J. (2009). Comparison of different scoring procedures for the CBM maze selection measure. Technical Report. University of Minnesota. https://www.progressmonitoring.org/pdf/tr10asls.pdf
Zashchirinskaia, O. V., Nikolaeva, E. I., Rybnikov, V. Y., & Bysova, V. M. (2017). Vliyaniye vnutrisemeynoy kommunikatsii na spetsifiku vospriyatiya zhestov detmi mladshego shkolnogo vozrasta s legkoy umstvennoy otstalostyu [The influence of family communication on the specificity of the gestures perception by children of primary school age with mild mental retardation]. Vestnik Novosibirskogo gosudarstvennogo pedagogicheskogo universiteta [Novosibirsk State Pedagogical University Bulletin], 7(5), 81-93.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
About this article
25 September 2021
Print ISBN (optional)
Economics, social trends, sustainability, modern society, behavioural sciences, education
Cite this article as:
Mamaeva, A. V., Antonova, T. S., Denisova, X. Y., Bryukhovskih, L. A., & Kuidina, E. E. (2021). The Requirements To Assessing Of Mentally Retarded Students Achievements In Interactive Environment. In I. V. Kovalev, A. A. Voroshilova, & A. S. Budagov (Eds.), Economic and Social Trends for Sustainability of Modern Society (ICEST-II 2021), vol 116. European Proceedings of Social and Behavioural Sciences (pp. 1474-1484). European Publisher. https://doi.org/10.15405/epsbs.2021.09.02.165