Perspectives Of Future Teachers On Formative E-Assessment Using The Classroom Response System

Abstract

The new European Higher Education Area involves a change in the understanding of learning and the teacher's activity. In this respect, the new culture of higher education highlights the importance of integrating assessment into the training process. E-assessment, a term used to describe technology-facilitated assessment, can be a useful and efficient in terms of cost and time to implement formative and continuous evaluation. Providing immediate feedback during the formative e-assessment process is very important in learning and teaching. Pedagogical research on mobile learning as an independent method or learning enrichment tool is relatively new. This study investigates the perspectives of future teachers on formative e-assessment and the effectiveness of using the classroom response systems (CRSs), which are technology-based formative assessment tools, in improving students’ learning. The study employed quantitative and qualitative research methods by a questionnaire to collect data from 134 first- and second-year students in initial teacher training program from Technical University of Cluj-Napoca, Romania. The findings revealed that students believe in the importance of formative e-assessment and receiving immediate feedback which is supported using CRS. Practically, all students engaged in this process felt the e-assessment added value to their learning, saved learning time, created a more active and fun learning environment and they would like to see it implemented in other courses. By encouraging students to correct errors and to receive award marks, it has improved students’ learning experience. The results are discussed considering relevant research to suggest recommendations for improving e-assessment implementations in initial teacher training.

Keywords: Formative e-assessmentclassroom response systemmobile learninginitial teacher training

Introduction

The new culture of higher education highlights the importance of integrating assessment into the training process, decisively influencing the process of helping students to learn and understand their progresses in learning. In recent decades, the focus has been on integrating assessment into learning and its role in contributing to the development of this process in a formative way. For example, Bransford, Brown, and Cocking ( 2000) mention that evaluation is a crucial element for effective learning. As Black and Wiliam ( 1998) assert, evaluation becomes continuous and is integrated into learning and curriculum to stimulate student development.

The English vocabulary nuances the meaning of the evaluation activity by using the word “assessment”, a term that emphasizes the process of collecting, reviewing and using data, for the purpose of improvement in the current performance. Evaluation is described as an act of passing judgement based on a set of standards. The etymological meaning (given by the Latin term “assidere” = “to stand by”) suggests the link between assessor and assessee throughout the didactic process, on the one hand, and the link of the evaluation activity with the teaching and learning activities, on the other hand. The basic difference between assessment and evaluation lies in the orientation (i.e. while the assessment is process oriented, evaluation is product oriented). The distinction between evaluation and assessment is from whole to part, because evaluation “involves looking at all the factors that influence the learning process, such as syllabus objectives, course design, materials, methodology, teacher performance and assessment. Assessment and evaluation are often linked, because assessment is one of the most valuable sources of information about what is happening in a learning environment” ( Harris & McCann, 1994, p. 2).

Formative assessment is “assessment of the regulation of training approaches, carried out continuously, systematically, analytically, directly put into the service of the individual and his training, which tends to improve it, to make it more efficient and, why not, more enjoyable” ( Bocoș, Raduţ-Taciu, & Stan, 2016, p. 134). Hattie and Timperley ( 2007) and Nicol and Macfarlane-Dick ( 2006) stated that the feedback is most effective when it is directly related to clearly defined learning objectives and that the effective formative feedback is not based solely on monitoring progress towards these specific objectives, but it must encourage students to develop effective learning strategies. Formative assessment is the major area of interest in learning activities that “offers teachers and students continuous, real-time information that informs and supports training” ( Ramsey & Duffy, 2016, p. 6). In modern didactic docimology, formative assessment and self-assessment aim to achieve four fundamental objectives: students’ awareness of school weaknesses and intuition of the strategies for overcoming obstacles, providing the opportunity to report on the requirements of the teacher and the curriculum, ensuring the effective exchange of information between student and teacher, transforming the student from the evaluated subject in the evaluator of his/her own school performance ( Stan, 2008). Assessment is often viewed as an activity or an event. However, assessment, particularly formative assessment, is more properly viewed as a process. Thus, assessment must now be considered as a learning situation in its proper meaning (assessment for learning or formative assessment), in addition to fulfilling its main function of verifying what students know and what they can do at a particular moment of learning (assessment of learning or summative assessment).

With the advent of technology and its role in education, a large body of research has developed into investigating the role of technology in the educational process and its effects in improving the interactive educational environment. E-assessment, a term used to describe the assessment facilitated using Information and Communication Technologies (ICT), can be a useful and efficient way in terms of cost and time to implement formative and ongoing evaluation. A large number of online and mixed courses have been developed by higher education programs to respond to “the various needs and desires of students and the need for longer time to meet growing curricular requirements” ( Garrison & Kanuka, 2004). In higher education, the focus remains on the summative evaluation, while formative evaluation receives little attention despite its essential role in promoting learning. For this reason, Pachler, Daly, Mor, and Mellar ( 2010) and Wang, Wang, and Huang ( 2008) recommended a reorientation of the focus on online formative assessment to create student-centred learning and assessment environments. Thus, Pachler et al. ( 2010) used the term formative e-assessment defined as “the use of ICT to support the iterative process of gathering and analysing information about student learning by teachers as well as learners and of evaluating it in relation to prior achievement and attainment of intended, as well as unintended learning outcomes” (p. 716). In the same sense, Gikandi, Morrow, and Davis ( 2011) define online formative assessment as a presentation of formative assessment in online learning and blended learning where the teacher and students are detached from time and/or space and where a considerable amount of teaching/learning events are driven by web-based technologies. Pachler et al. ( 2010) declare the domain of formative e-assessment as highly complex, because it is integrated into the teaching and learning process and because technology “reshuffles the context of teacher-student interaction” (p. 720).

Extending wireless distance learning has led to the emergence of mobile learning as an “extension of e-learning” ( Brown, as cited in Park, 2011, pp. 80), a new way to obtain, process and transmit information for educational purposes using mobile technology equipment. Pedagogical research on mobile learning as an independent method or learning enrichment tool is relatively new. Thus, mobile learning has emerged as a potential educational method, but also as a supporting tool in the educational process, especially through mobile phones and tablets/iPads ( Ceobanu, 2016). There are a number of new technologies and software introduced free or at affordable prices that help teachers with formative assessment during the training process and enhancing learning and evaluation. One of these technologies is the Classroom Response System (CRS). Irving ( 2015) asserted that these tools “assist in the formative assessment process by supporting classroom environments that allow students and teachers to assess learning and providing mechanisms to present information about student learning during instructional sequences” (p, 380). These technologies include, but are not limited to: Clickers, Kahoot, Socrative, Quiz Socket, RecaP, Ombea, Top Hat, VotApedia, Poll Everywhere etc. In a synthesis by Dunn, Richardson, Oprescu, & McDonald ( 2013) the benefits of using a CRS include improving attentiveness, improving attendance, improving engagement in course when used well, enhancing teacher-student interaction by providing immediate feedback, especially in large classes, and allowing students to remain anonymous. Research results have shown that CRSs have raised questions and feedback when technology is integrated with pedagogy and had a positive effect on students' attitudes and academic performance. The help given by technology through these systems is seen in activating student thinking, providing immediate feedback, motivating participation and promoting knowledge-centred discussions. Essential features of CRSs help teachers to effectively transform the class from teacher-centred to student-centred, help to assess students' learning by questioning the topic, collecting student responses instantly and quickly, and finally projecting the answers of the entire class. All that is required to bring CRSs into action is the use of two devices – the computer of the class teacher and student's mobile phone/tablet.

Problem Statement

A number of studies provided evidence of the significant contribution that technology brings to improve teaching, learning and evaluation that positively influence students' knowledge and skills. Most students today grew up with digital technologies such as the internet, smart phones and tablets. Teachers are under pressure to review programs to provide more efficient and effective training. It is essential to use the training time in the most appropriate possible ways. To successfully cope with the current technological challenges, training efforts are needed both from teachers and from their trainers during initial and continuing training ( Glava, 2008). It has been widely recognized that e-assessment can contribute to improving the quality of student learning experience, and many researches have been done on attitudes towards e-assessment from academic staff. Relatively little research has recorded students’ perspectives on formative assessment experiences using mobile-phone-based classroom response systems, especially the students' perspectives as future teachers in their initial teacher training program.

Research Questions

The study will try to answer the following two questions in terms of students' perceptions:

Purpose of the Study

The main purpose is to investigate students' perceptions about the effectiveness of using formative e-assessment and of tools’ implementation in the traditional teacher training activities through CRSs, such as Socrative, Kahoot, Poll Everywhere in improving students’ learning. The study sample consisted of 134 first and second year level students attending the initial teacher training program at Technical University of Cluj-Napoca, Romania. Out of the 134 participants, 55 are men and 79 are women, 71 subjects are enrolled on the electrical profile, 36 subjects on the building profile and 27 subjects on the mechanical profile, 59 subjects in first year and 75 subjects in second university year.

Research Methods

The study employed mixed research methods: quantitative by using the means and standard deviations and qualitative by analysing students’ responses to four open-ended questions. A questionnaire was used to collect data from 134 students. The first part of the questionnaire focused on the students’ general views on formative e-assessment within university activities (14 items). The second part consisted of 10 items on the efficiency of using the CRSs with multiple-choice, true/false and fill in the blanks questions implemented in teacher training activities. The statements are rated on a five-point Likert scale ranging from one (Strongly Agree) to five (Strongly Disagree). For open-ended questions, four questions were used to ask students about their perception on how effective is using CRSs in assessment. The open-ended questions gave the participants the opportunity to elaborate and explain in-depth their perception regarding the use of CRSs as tools for formative e-assessment to improve learning. Using mixed methods allows for data collection, so that deeper understanding can be achieved. The tests were conducted in an online CRS environment through student mobile phones. The feedback was immediately given to both the researcher and the students and these feedbacks were taken into consideration during the training.

Findings

In order to investigate the perceptions of future teachers on the importance and effectiveness of using formative e-assessment, the mean for each item of the questionnaire was calculated. The analysis based on observed scores for each item showed a high level of opinions for formative e-assessment (Table 1 ). The descriptive statistics results showed that the total sample mean score was at a high level (M = 2.31, SD = 0.38). The highest mean was found in the item: Feedback given is fast (M = 1.50, SD = 0.61) and the lowest mean was found in the item: Formative e-assessment favours some students more than others (M = 3.47, SD = 1.08).

The positive student perception towards formative e-assessments is hard to be contested. As it was anticipated, students were positively predisposed towards the use of technology in assessment. Students are, by a large majority, satisfied with online assessment, throughout the training period, confirming the existing literature ( e.g. Dermo, 2009). This type of assessment provides the teacher with the information required to adjust the teaching methods. On the other hand, students must be informed of the assessment results as soon as possible. The method is very well received if the assessment results are analysed by the teacher and the style or contents of the course are changed where necessary. This is called washback effect. It is most beneficial if formative assessment is a continuous process that, thanks to the efforts of teachers and students, contributes to curriculum development and preparation for the other type of assessment – summative assessment. The feedback obtained from the online tests or quizzes may allow to identify and correct any deficiencies or difficult concepts. It is also important to identify the nature of problems in filling the online quizzes and to eliminate them. Furthermore, in the eyes of the students, online assessment is considered to be more correct than paper-based one. The participating students appreciate the benefits of the online quizzes, the most valued being the possibility of obtaining the results immediately after the quiz is completed by means of feedback, and the possibility of verifying the current level of knowledge. However, in the present study some participants had been familiar with the use of Moodle platform, for example, and this has certainly contributed to their positive stance.

The author of this study used CRS technology in Pedagogy courses and seminars in the initial teacher training program and was interested in finding out its effectiveness for formative e-assessment to enhance student’ learning. Therefore, to further explore the usefulness of implementing CRS as a technology tool in aiding formative e-assessment in the classroom, students generally indicated that they agree on the usefulness of implementing this technology (M=1.78, SD=0.43). As students, participants showed that they generally liked to use CRSs (M=1.35, SD=0.59) and they helped them to check their progress in learning and to master the subjects they learned (M=2.23, SD=0.81). Furthermore, they stated that they plan to use CRSs with their students when they will become teachers (M=1.63, SD=0.78). Students who participated in this study know the features that a CRS provides its users (e.g. is easy to use in the course, it helps students to get immediate feedback about their responses, the feedback is useful, the scoring was correct, the questions were well written, different questions were offered) with a range of means between 1.37–1.69 and standard deviations between 0.57–0.73. For item “I prefer to work marked by CRS than by a teacher as human tutor” (M=3.2, SD=0.95), 36.6 % of the future teachers responded with disagree and strongly disagree, while 47% of them were undecided. Not so much the audio-visual message delivered by ICT is likely to produce educational effects, as its efficient integration into an active didactic strategy, designed by the teacher, whose presence remains necessary.

Although, the participants’ qualitative responses covered a wide range of aspects, the researcher highlights the most important themes emerged from these responses. In responses to the first open-ended question “How do you compare the assessment by CRSs with the assessment from other courses?” participants overwhelmingly agree that CRS technology is an effective tool in evaluation and the learning process. The assessment provided by CRSs is objective, interesting, accessible, fast, easy to use and fun. One student stated, “I think it is a fast and efficient method of evaluation.” On the same line, another respondent wrote, “I think it is a more interactive method that draws your attention and preserves it over the evaluation period.” A third student stated, “CRS is totally objective, and this is not always available for a teacher.” Among students' comments, we note that assessment by CRSs is more flexible, friendly, applications are easy to use, feedback is received faster, it helps to fix the information at the end of the activity, without favouritisms and differences between students. Most students found that using CRS is an effective way to interact in large classes. It is obvious that new generations (millennials) want to use technology in their daily life and using it in the class has a positive effect on students learning, as many researchers said ( Preszler, Dawe, Shuster, & Shuster, 2007; Schell, Lukoff, & Mazur, 2013). At the question “What were the things you liked the most at CRSs?” many of the respondents report the speed of feedback, the correct scoring, the ease with which CRSs can be used by several students at the same time, interaction with technology (mobile phones), precise questions, the possibility of reviewing the questions and answers to identify where everyone was wrong, displaying a ranking. Another aspect was saving the learning time. One student wrote, “I liked the short time I had to answer the questions” and another responded “I liked the most that everything was at an alert pace, that it put me the concentration capacity ‘in moving’ and that it awakened the competitive spirit in me. And I had fun.” Among the participants' answers to the question, “What were the things you liked the least at CRSs?” was the time too short for the answer. One student wrote “I didn’t like the fact that if you answered later, you received a lower score.” Another negative aspect identified was “the lack of detail in the assessment of a wrong answer”. In this regard, during or at the end of each assessment, the answers to all the questions, both the correct ones and the wrong ones, were discussed. The other participants’ answers to the above question are about technical difficulties and problems. For the open-ended question “What improvements would you recommend to CRSs?”, they indicated a longer response time, a greater emphasis on communication and interaction, without fear of embarrassment if the student did not answer correctly, introduction of varied items and a better internet connection. The positive impact of using technology to add value to the learning process, as presented in the results of this study, is in line with what several researchers have argued ( Irving, 2015; Ramsey & Duffy, 2016). One of the main drivers for the introduction of e-assessment is often the claim that it has the advantages of saving time and better use of resources. The participants pointed out that the use of CRS in the didactic activity provides objectivity, commitment, accessibility, speed, ease of use and even fun, which, eventually, aid the learning process. The combination of reward points and anonymous responses with instant feedback means that students are free to make mistakes without fear of social embarrassment (in front of peers or the teacher) or fear of adversely impacting their notes. The systems’ ability to provide anonymous participation with private accountability is a critical feature of CRSs. Students are free to contribute without the fear of possible public humiliation and without worrying about more vocal students who dominate the discussion. Regarding the problems, challenges and difficulties encountered in using Socrative, Kahoot, Poll Everywhere, most of the participants pointed out that technology resources and support are the main issues. As with any technology, users may experience technological problems sometimes, some due to university-specific issues (the internet connection fails) or computer-specific issues (i.e., slow browser response). In the literature, there is a strong agreement that CRSs promote learning when coupled with appropriate pedagogical methodologies. A part of the students’ learning takes place in the class and it is therefore essential that the two modes complement each other to ensure that the disadvantages of one mode is outweighed by the other ( Gibbs, 2006).

Conclusion

The most obvious finding of this study is that using technology-based tools, such as Socrative, Kahoot, Poll Everywhere, enhances formative assessment and, consequently, improves students’ learning. Furthermore, these tools help in providing individualized learning and engaging students with the feedback which, in turn, leads to creating an effective teaching and learning environment and makes the didactic activity interesting, informative and fun. Further empirical research is needed to investigate the effectiveness of using technology-based tools for formative assessment on students’ achievements and performance.

The role of teachers is to create opportunities for their students to engage in practical experiences, thus encouraging them to (re)interpret their academic performance. Students' understanding of their assessment processes reveals the need to specifically and permanently include assessment in the didactic activities, in a manner that is not fixed on instrumentation and on a single methodology. So, the initial training program is an important space in which the knowledge construction of future teachers occurs. Thus, the current study suggests that further research is needed to highlight the assessment practices undertaken during initial training. As a result of such research, perhaps, teachers and students can learn to use assessment as an active and ongoing process, to include it into the curriculum as a set of activities, rather than thinking about it as a singular event. The results of this study can also be supported by teachers' need as instructional designers to design more interactive and engaging courses. However, the results of this study are limited to the extent of their generalization in situations with more advanced educational technology tools and to more innovative and integrated models, from traditional instructional methods to twenty-first century training strategies that guarantee student autonomy. Based on the results and findings, this study recommends teachers: to engage the students in formative assessment process to gauge understanding and correct misconceptions; to integrate technology in courses and seminars because it improves students’ learning; to use new digital apps and software to help them implement formative assessments in their activities; to understand that formative assessment is a major component within didactic activities, that provides teachers and students continuous and real-time information to support teaching and learning.

References

• Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: principles, policy & practice, 5(1), 7-74. https://doi.org/10.1080/0969595980050102
• Bocoș, M.-D., Raduţ-Taciu, R., & Stan, C. (2016). Dicționar praxiologic de pédagogie (vol. II). Pitești: Editura Paralela 45.
• Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How People Learn. Washington, DC: National Academy Press.
• Ceobanu, C. (2016). Învățarea în mediul virtual. Ghid de utilizare a calculatorului în educație. Iași: Editura Polirom.
• Dermo, J. (2009). E-assessment and the student learning experience: A survey of student perceptions of e-assessment. British Journal of Educational Technology, 40(2), 203.
• Dunn, P. K., Richardson, A., Oprescu, F., & McDonald, C. (2013). Mobile-phone-based classroom response systems: Students’ perceptions of engagement and learning in a large undergraduate course. International Journal of Mathematical Education in Science and Technology. https://doi.org/10.1080/0020739X.2012.756548
• Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95-105. https://doi.org/10.1016/j.iheduc. 2004.02.001
• Gibbs, G. (2006). Why Assessment is Changing. In C. Bryan & K. V. Clegg (Eds.) Innovative Assessment in Higher Education. London: Routledge.
• Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351. https://doi.org/10.1016/j.compedu.2011.06.004
• Glava, C. (2008). Implementarea noilor tehnologii în formarea cadrelor didactice. In M. Bocoș & I. Albulescu (Eds.), Studii de pedagogie universitară (pp. 167-186). Cluj-Napoca: Editura Presa Universitară Clujeană.
• Harris, M., & McCann, P. (1994). Handbooks for the English Classroom. Assessment. Oxford: Macmillan Heinemann.
• Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
• Irving, K. (2015). Technology-assisted formative assessment. In M. J. Urban & D. A. Falvo (Eds.), Improving K-12 STEM education outcomes through technological integration (pp. 380–398). https://doi.org/10.4018/978-1-4666-9616-7.ch017
• Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090
• Pachler, N., Daly, C., Mor, Y., & Mellar, H. (2010). Formative e-assessment: practitioner cases. Computers & Education, 54(3), 715-721. https://doi.org/10.1016/j.compedu.2009.09.032
• Park, Y. (2011). A pedagogical framework for mobile learning: Categorizing educational applications of mobile technologies into four types. International Review of Research in Open and Distance Learning, 12(2), 78-102. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/791/1788
• Preszler, R. W., Dawe, A., Shuster, C. B., & Shuster, M. (2007). Assessment of the effects of student response systems on student learning and attitudes over a broad range of biology courses. CBE—Life Sciences Education, 6(1), 29–41. https://doi.org/10.1187/cbe.06-09-0190
• Ramsey, B., & Duffy, A. (2016). Formative assessment in the classroom: Findings from three districts. Michael and Susan Dell Foundation and Education, 1. Retrieved from https://education-first.com/wpcontent/uploads/2016/05/MSDF-Formative-Assessment-Study-Final-Report.pdf
• Schell, J., Lukoff, B., & Mazur, E. (2013). Catalyzing learner engagement using cutting-edge response systems in higher education. Cutting-Edge Technologies in Higher Education, 6, 233-261. https://doi.org/10.1108/S2044-9968(2013)000006E011
• Stan, C. (2008). Clasic și modern în docimologia didactică universitară. In M. Bocoș & I. Albulescu (Eds.), Studii de pedagogie universitară (pp. 41-62). Cluj-Napoca: Editura Presa Universitară Clujeană.
• Wang, T.-H., Wang, K.-H., & Huang, S.-C. (2008). Designing a web-based assessment environment for improving pre-service teacher assessment literacy. Computers & Education, 51(1), 448–462.