Students’ Experiences On The Assessment Of Learning Outcomes In Online Environment
Abstract
During the last decades e-learning has awaken the interest of numerous high education institutions, which wanted to innovate and to flexibilize in the educational environment. Online learning environment is considered to be an ecosystem which integrates technology with teaching and learning practices, being an significant indicator of innovation. Many universities in the world integrated e-learning in their educational systems but the studies regarding its effectiveness did not produce cogent data. Most studies highlight that e-learning effectiveness is influenced by the training of both teachers and students. The crisis caused by COVID 19 pandemics determined high education institutions to implement distance learning abruptly, without any previous training. We propose to analyse students’ perception upon the assessment experiences of learning outcomes in the online environment, the formative feedback, the difficulty of assessment items, the difficulty of the evaluation type, interaction with the teacher in online evaluation, satisfaction with the results of summative evaluation ) in relation with students’ personal characteristics (personal efficacy, the level of digital competences, satisfaction with the assessment activity in the online environment, the attitude towards the assessment in online environment). The investigation had been carried out based on a questionnaire elaborated by us applied on a sample of 146 students (traditional, non-traditional). The analysis of students’ assessment experiences in online environment can offer useful data with a view to improve the organization of the didactic process in online environment as well as to develop training programs for teachers for online assessment.
Keywords: Feedback formative, online assessment, online learning environment, sumative assessment, student perception
Introduction
The crisis caused by COVID affected the whole educational system and high education institutions were under the necessity to introduce unprecedented changes in order to ensure continuity in education, by the sudden transition from traditional face-to-face learning to online learning, which offered new learning experiences. Rapidly, online learning became an emergent method implemented by all high education institutions. The change of the learning environment implied rapid adaptations not only from high education institutions which were obliged to develop digital tools and instruments instantly and from teachers who were obliged to integrate information and communication technologies and new teaching technologies in their instruction programs but also from students, who were under the necessity to mobilize their personal resources which helped their learning.
Within this context, we consider that the experience for those involved represents an opportunity to explore the participatory factors so that efficient strategies can be created in developing training programs in online environment.
Problem Statement
Online learning and innovation in education
During the last decades, most innovation initiatives in education underlined the necessity to design and implement virtual learning environments through the use and integration of new information and communication technologies, laying the base of e-learning. E-learning is the result of amalgamation between information and communication technology and education and allows the creation of innovative mechanisms not only in the learning environment but also in didactic processuality. E-learning as an educational tool became important through enrolling information and communication technology progress, being considered a new model or providing information in the educational domain (Malik, 2010), a new learning approach which exploits the potential of information and communication technologies in order to provide and receive educational content (Negash & Wilcox, 2008).
Trying to define online learning, the specialty literature offers us different approaches according to the field related to ICT, education, informatics, etc. From a pedagogical perspective, Khan (2005) defines online learning as an innovative approach for delivering a well-designed, learner-centred, interactive and facilitated learning environment to anyone, anywhere, anytime, by utilising the attributes and resources of various digital technologies along with other forms of learning materials suited for an open and distributed learning environment.
Online learning meets the current demands of involved actors through the accessibility and rapid availability of informational contents, standardized contents, personalized training, interactivity, trust and comfort (Taha, 2014).
Current tendencies accentuate the necessity of accessible educational opportunities, determining flexibilization and attractively of learning environments, in which students have the possibility to learn according to their available time, space and location (Burgess & Russell, 2003). The solution proposed by educational policies is the encouragement of online learning and its hybridization to offer students active learning environments with a view to stimulate their engagement in learning (López-Pérez et al., 2011).
Online learning environment is considered an ecosystem which integrates technology with teaching and learning practices, by including different based-technology platforms, being a significant indicator of innovation (Eze et al., 2018).
Thus, the need arises that high education institutions should adapt their study programs to this type of learning. Simply moving pedagogy from one medium into another was not enough to ensure quality learning (Henriksen et al., 2020). The organization of the didactic process for online environment implies not only adaption and administration of the learning content, ensuring the flexibility of the teaching methodology and the necessary means for the instruction process but mainly the development of digital competences of the involved actors, teachers and students equally.
Although online training is a worthwhile approach for instruction, training packages should not be delivered indirectly (Ogbonna et al., 2019; Rosenberg, 2001). Training in virtual environment depend on several factors such as availability and opening for knowledge, students’ competence to use digital technique (Kim et al., 2019), curriculum centered on students’ training needs, a robust infrastructure and students’ capacity to adapt (Ogbonna et al., 2019; Ssekakubo et al., 2011).
The question, which arises, is the following: to what extent can virtual environment answer the needs of the student-centred learning paradigm, where collaboration is stimulated and where students build their own knowledge and develop strategies of critical thinking and improve their problem-solving skills?
The impact of online environment learning
Online environment leaning is defined as an interactive learning offered by the internet through the agency of electronic devices and by the use of different learning (ex. Moodle) and communication (ex. Zoom, Microsoft Teams, Google Meet) platforms.
Online environment learning can be done synchronous and asynchronous, according to the type of communication and the collaboration form of the participants. Asynchronous learning takes place when participants are not simultaneously in the same virtual space, but several means of communication are used – discussion forums, e-mail, web resources provided by the teacher. This instructional time is achieved independently from the teacher’s presence. It implies flexibility, students access the learning environment in terms of their own availability.
Synchronous learning presumes direct interaction, in real time and uses digital communication channels which offer the possibility of common activity and/or sharing information. If asynchronous learning implies self-organization and self-rhythm, synchronous learning implies collaboration and interaction. Asynchronous learning leads to a higher cognitive task, low activation and greater ambiguity in communication (Blau et al., 2017), while synchronous learning implies immediate feedback, interpersonal communication and associates with levels of engagement and motivation in learning (Ogbonna et al., 2019).
The impact of online environment learning had been measured through the results obtained by students (Safavi, 2008). The effects upon learning outcomes are closely connected to the manner in which technology is used as an instructional means (Ogbonna et al., 2019). Empirical studies on the ICT impact do not detect a consistent relation between technical availability and students’ learning (Kozma, 2005).
Cosgrove and Olitsky (2015) analyse in a large study the impact of the three modalities of organizing learning (traditional, online and hybrid) upon learning outcomes and underlines that there is no proof that a type of study is better than others in acquiring content information. It had been observed that students who learned traditionally manifest a high level of content retention than those who participated in online or hybrid environment.
As concerns the impact of mode of delivering courses on students, high levels for hybrid and online learning had been detected as compared to traditional learning (Collopy & Arnold, 2009)
Comparing the results on learning using the synchronous versus asynchronous form, studies of Hrastinski (2008) revealed the fact that asynchronous learning allows the improvement and a rise in performance while synchronous learning facilitates communication, interest and engagement.
As regards the students’ expectations and expectancy about online learning environment, the studies show that students have high expectations from this form of organizing the learning environment, even if they doubt the stringency of online courses and learning (Keramidas, 2012).
Performance assessment in online environment
A very important aspect related to students’ expectations towards online environment refers to the approach of feedback and assessment in the learning process (Wyss et al., 2014). Feedback and assessment are essential for learning (Paechter et al., 2010). Pertinent and relevant feedback is considered in online learning as a critical component of efficient learning (Deggs et al., 2010).
The real challenge of an efficient answer with online learning environment refers to authentic assessment. This is directly related to measuring and appreciation of learning performances in online environment. Assessment strategies must be adapted to students’ needs and to the characteristics of online environment. In online environment, students expect from their teachers, beside clear instructions and explicit and stringent tasks, a pertinent and relevant feedback so that performance assessment should be trustworthy. Concurrently, they need support to assess the information they have, they need frequent opportunities to perform in tasks and to receive feedback about their performance, and they need chances to reflect to what they learned, to what they should do to acquire contents but also about the assessment process.
Strategies of formative assessment (formative feedback) and summative assessment can be used in online environment.
Online environment increases the chances for immediate and continuous feedback, facilitating engagement and self-regulation in learning for students (Wolsey, 2008), motivation and academic performance. The effectiveness of formative feedback during asynchronous activity depend on the time students have to compose and reflect upon understanding contents or upon expressing an opinion about a subject under debate (Gikandi et al., 2011; Vonderwell et al., 2007).
Since assessment certifies the development of competences at the end of a study program and is achieved through objective tests, having well-defined objectives and contents, the effectiveness of summative assessment in online environment is supported by specialty literature (Oosterhof et al., 2008).
Research Questions
What is students’ perception on the assessment activity of academic outcomes in online environment?
Is there a relation between students’ perception on the assessment activity of academic outcomes in online environment and their personal characteristics?
Purpose of the Study
The constative-type pilot study proposes to analyse students’ perception on assessment experiences of learning outcomes in online learning environment (formative feedback, the difficulty of assessment items, the difficulty of assessment type, interaction with the teacher and peers in online assessment, satisfaction with the results of summative assessment ) in relation with students’ personal characteristics (personal efficiency, level of digital competences, satisfaction with the assessment activity in online environment, the attitude towards the assessment in online environment.
Research methods
Sample
The sample is made up of 146 Romanian students currently attending initial training stage for didactic profession –82, first year of study, 64, 3-rd year of study: 6 males and 140 females, average age 27.83 years (SD=6.14). As concerns professional experience, 92 are traditional students and 54 are non-traditional students. Participants in the study had no online learning experience before COVID 19 pandemic crisis.
Instruments
The questionnaire with 32 items to identify the experiences concerning the assessment of academic outcomes used in online environment had been elaborated by us.
The questionnaire has the following scales: - 4 items (α = .92);- 1 item - (true/false, matching, multiple choice), - 1 item - (essay, portfolio, debate, reflection), – 5 items (α = .62), – 3 itemi (α = .81), - 4 itemi (α = .86), – 5 items (α = .72),– 4 itemi (α = .72), -5 itemi (α = .79),
The items of the questionnaire had been elaborated based on the concepts in the specialty literature, being grouped on the above-mentioned scales. Three expert educators contributed to the content validation of the assessment tool, by evaluating the relevance of the items at the targeted scale. The assessment had been finalized on a binary scale 1- adequate, 0-inadequate. Based on frequency analysis, 32 items having the highest frequencies had been chosen for the final form of the questionnaire.
The evaluation scale used was a five-point Likert scale type (1 - strongly disagree; 5 - strongly agree).
The questionnaires had been administered online, the participation being voluntary and unpaid.
Findings
Based on SPSS statistics analysis, the results of our study are presented in Table 1.
Students were asked to evaluate the perceived difficulty of assessment items in online environment; thus, first of all, students perceived matching type items (52.8%) and multiple -choice (48.21%), while as regards the difficulty of the assessment type, the first place is taken by essay (84.25%) and portfolio (62.45%).
The qualitative analysis of students’ perceptions on assessment in online environment highlights some important aspects, namely the fact that the average scores at the dimensions of the questionnaire are low (Table 1). Inferential analysis of the differences between average values reveals the fact that there are no significant differences between 1-st year students’ perceptions and 3-rd year students ‘ perceptions at the following dimensions: formative feedback (t (144)=-0.650, p=.201), interaction with teacher and peers in online assessment (t (144)=1.798, p=.006), satisfaction with the results of summative assessment (t (144)=0.381, p=.384), level of digital competences (t (144)=.684, p=.172), attitude towards assessment in online environment (t (144)=1.009, p=.029), effectiveness in online environment (t (144)=1.921, p=.006). Significant differences are found between the two analysed groups only at the dimension satisfaction with online assessment activity (t (144) = 2.716, p=.000). It has been found that the average score at this dimension is higher at 1-st year students (m=11.23, SD=2.623), as compared to 3-rd year students’ average score (m=8.51, SD=1.616). The enthusiasm of 1-st year students is reflected in the satisfaction they have towards the learning-assessment environment, albeit an atypical one, which confirms their new professional status as a student.
The analysis at the level of questionnaire dimensions underlines the fact that students associated moderately formative feedback with interaction with teacher and peers (r=0.535, p≤0.000), with the attitude towards online environment (r=0.423, p≤0.000), but also with the effectiveness in online environment (r=0.433, p≤0.000). Feedback within formative assessment represents a central point which is not only an indicator of satisfaction towards the assessment activity but also an element which confers assessment the fulfilment of its goals.
Another component of the assessment activity is interaction with teacher and peers, this dimension having strong associations with the level of satisfaction towards the assessment activity (r=0.529, p≤0.000) and towards the effectiveness of assessment in online environment (r=0.624, p≤0.000).
The estimation of the level of digital competences is related to moderate associations on the investigated dimensions, less on the satisfaction with summative assessment, where there are no significant associations identified. It was found that in online environment summative assessment, the student’s digital competence had not been taken into consideration but mainly the competences targeted by the contents delivered within the disciplines of the curriculum.
We also observe that students’ attitude towards assessment in online environment is directly but poorly influenced by both the level of digital competences (r=0.175, p≤0.000), and mainly by the social interaction online environment can support (r=0.588, p≤0.000) and by formative feedback (r=0.423, p≤0.000).
To summarize, we can observe that the two sides of learning outcomes assessment in online environment, formative feedback and interaction with teacher and peers are interrelated to personal aspects which belong to digital competences, to satisfaction with the assessment of summative results in online environment but also with aspects related to the perception on the effectiveness of online assessment, to satisfaction with online assessment and to students’ attitude towards online assessment. The results reveal aspects which can be improved so that the alternative offered by online environment should meet the demands of a quality assessment in the didactic process.
Conclusions
Assessment in online learning contexts includes distinct characteristics compared to traditional assessment, mainly due to the lack of interactivity generated by the nature of asynchronous character. Understanding students’ experiences as regards the perception on the assessment of the competences acquired in online environment enables the improvement in organizing online environment didactic process. Teachers must plan the didactic assessment strategy in online environment so that they ensure effectiveness and authenticity in learning.
The results of the study concerning the perception on the experience upon the assessment of learning outcomes in online environment reveals the fact that the teacher must modify the structure of the course, the support materials and assessment strategies so that they allow a higher retention of information and ensure learning performances.
It is very important to integrate formative assessment in online and hybrid environment by implementing well-structured assessment strategies so that they facilitate the development of online learning communities, as predecessors of interactive collaborative learning, ensuring the prerequisites of academic learning (Akyol et al., 2009). The design of tasks involving collaboration among students and the introduction of authentic learning experiences which serve students’ interests will improve students’ social presence in online environment, learning environment and students’ satisfaction with online courses.
The assessment of learning outcomes in online environment must be based on permanent monitoring, immediate formative and constructive feedback in order to help students activate their self-regulation strategies in learning, an essential condition in obtaining high academic performances.
References
Akyol, Z., Garrison, D. R., & Ozden, Y. (2009). Online and blended communities of inquiry: exploring the developmental and perceptional differences. International Review of Research in Open and Distance Learning, 10(6), 65–83.
Blau, I., Weiser, O., & Eshet-Alkalai, Y. (2017). How do medium naturalness and personality traits shape academic achievement and perceived learning? An experimental study of face-to-face and synchronous e-learning. Research in Learning Technology, 25, 1-23.
Burgess, J. R. D., & Russell, J. E. A. (2003). The effectiveness of distance learning initiatives in organizations. Journal of Vocational Behaviour, 63(2), 289–303.
Collopy, R., & Arnold, J. M. (2009). To blend or not to blend: Online and blended learning environments in undergraduate teacher education. Issues in Teacher Education, 18, 85-101. https://ecommons.udayton.edu/edt_fac_pub/15
Cosgrove, S. B., & Olitsky, N. H. (2015). Knowledge retention, student learning, and blended course work: Evidence from principles of economics courses. Southern Economic Journal, 82, 556-579.
Deggs, D., Grover, K., & Kacirek, K. (2010). Expectations of adult graduate students in an online degree program. College Student Journal, 44(3), 690-699.
Eze, S. C., Chinedu-Eze, V. C., & Bello, A. O. (2018). The utilisation of e-learning facilities in the educational delivery system of Nigeria: A study of M-University. International Journal of Educational Technology in Higher Education, 15(34), 1-20.
Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351.
Henriksen, D., Creely, E., & Henderson, M. (2020). Folk pedagogies for teacher educator transitions: approaches to synchronous online learning in the wake of COVID-19. Journal of Technology and Teacher. Education, 28(2), 201–209.
Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educase Quarterly, 31(4), 51–55.
Keramidas, C. G. (2012). Are undergraduate students ready for online learning? A comparison of online and face-to-face sections of a course. Rural Special Education Quarterly, 31(4), 25-32.
Khan, B. H. (2005). Learning features in an open, flexible, and distributed environment. AACE Journal, 13(2), 137-153.
Kim, H. J., Hong, A. J., & Song, H.-D. (2019). The roles of academic engagement and digital readiness in students’ achievements in university e-learning environments. International Journal of Educational Technology in Higher Education, 16(1). 1–18.
Kozma, R. B. (2005). Monitoring and evaluating of ICT for education impact: A review. In D. A. Wagner, B. Day, T. James, R. B. Kozma, J. Miller, & T. Unwin (Eds.), Monitoring and evaluating of ICT in education projects: A handbook for developing countries. infoDev/World Bank. https://www.infodev.org/infodevfiles/resource/InfodevDocuments_9.pdf
López-Pérez, M. V., Pérez-López, M. C., & Rodríguez-Ariza, L. (2011). Blended learning in higher education: Students’ perceptions and their relation to outcomes. Computers & Education, 56(3), 818–826.
Malik, M. W. (2010). Factors Effecting Learner’s Satisfaction Towards E-Learning: A Conceptual Framework. OIDA. International Journal of Sustainable Development, 2(3), 77-82.
Negash, S., & Wilcox, M. V. (2008). E-Learning Classifications: Differences and Similarities. In S. Negash, M. Whitman, A. Woszczynski, K. Hoganson, & H. Mattord (Ed.), Handbook of Distance Learning for Real-Time and Asynchronous Information Technology Education, 1-23.
Ogbonna, C. G., Ibezim, N. E., & Obi, C. A. (2019). Synchronous versus asynchronous e-learning in teaching word processing: An experimental approach. South African Journal of Education, 39(2), 1–15.
Oosterhof, A., Conrad, R. M., & Ely, D. P. (2008). Assessing learners online. Pearson.
Paechter, M., Maier, B., & Macher, D. (2010). Students' expectations of, and experiences in e-learning: Their relation to learning achievements and course satisfaction. Computers & Education, 54(1), 222-229.
Rosenberg, M. J. (2001). E-Learning: Strategies for Delivering Knowledge in the Digital Age. McGraw-Hill, New York.
Safavi, A. A. (2008). Developing countries and E-Learning program development. Journal of Global Information Technology Management, 11(3), 47– 64.
Ssekakubo, G., Suleman, H., & Marsden, G. (2011). Issues of Adoption: Have e-Learning Management Systems Fulfilled Their Potential in Developing Countries? The Proceedings of the South African Institute of Computer Scientists and Information Technologists Conference on Knowledge, Innovation and Leadership in a Diverse, Multidisciplinary Environment. Cape Town, October 2011, 231-238.
Taha, M. (2014). Investigating the Success of E-Learning in Secondary Schools: The Case of the Kingdom of Bahrain. (Publication No. 27542827). [Doctoral dissertation]. Brunel University London.
Vonderwell, S., Liang, X., & Alderman, K. (2007). Asynchronous discussions and assessment in online learning. Journal of Research on Technology in Education, 39(3), 309–328.
Wolsey, T. (2008). Efficacy of instructor feedback on written work in an online program. International Journal on ELearning, 7(2), 311–329.
Wyss, V. L., Freedman, D., & Siebert, C. J. (2014). The development of a discussion rubric for online courses: Standardizing expectations of graduate students in online scholarly discussions. TechTrends, 58(2), 99-107.
Copyright information
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
About this article
Publication Date
10 April 2023
Article Doi
eBook ISBN
978-1-80296-961-0
Publisher
European Publisher
Volume
5
Print ISBN (optional)
-
Edition Number
1st Edition
Pages
1-1463
Subjects
Education sciences, teacher education, curriculum development, educational policies and management
Cite this article as:
Stan, M. M. (2023). Students’ Experiences On The Assessment Of Learning Outcomes In Online Environment. In E. Soare, & C. Langa (Eds.), Education Facing Contemporary World Issues - EDU WORLD 2022, vol 5. European Proceedings of Educational Sciences (pp. 242-250). European Publisher. https://doi.org/10.15405/epes.23045.25