Metacognitive Awareness (MA) Impact on Writing Performance: A Digital Language Assessment
Abstract
There has been a decline in Malaysia’s English performance since the pandemic started as evidenced in the 2020’s SPM results which had a decline in performances as compared to the previous year. As English is one of the pre-requisite pass subjects for the university entrance, it is omnipotent for the students to have good proficiency in the language. The aim of the study is to improve the pre-university students’ performance in the writing component of Malaysian University English Test (MUET) through digital Metacognitive Awareness (MA) training (MA). A single group-intervention study with a pretest-posttest design was conducted in a Malaysian public university involving 30 pre-university who consented to participate in the study. Findings show that the average mean score of the pre-test is 57.35 (s.d. 4.84). After 10 weeks of intervention, the mean score of the post-test is 67.54 (s.d. 8.89), indicating 10.19 increase. In addition, the post-test recorded the highest score of 78.72, compared to 64.8 in the pre-test. Overall, results seem to indicate that MA had a positive effect on the pre-university students’ writing performances. The findings from the study are beneficial towards lecturers who teach MUET, students who will be sitting for MUET and for MUET and ESL instructors who will continue conducting their assessment digitally.
Keywords: Language assessment, metacognitive awareness, writing
Introduction
There has been many initiative done by Malaysia to further establish the education system. One of the efforts includes making English as one of the three compulsory pass subjects for the Malaysian Certificate of Education (MCE) which is equivalent to the General Certificate of Secondary Education (GCSE) in the United Kingdom. This action was carried out in the first wave during the implementation of the Malaysia Education Blueprint 2013-2025. Wave One which was carried out in 2013-2015, focuses on the strengthening and improvement of the current system followed by the structural adjustments in Wave Two and in Wave Three which takes place from 2021-2025 where the focus is given towards the chance of excellence with the increase of operation flexibility (Iber, 2014; Ministry of Education Malaysia, 2013). Thus, we cannot deny that English plays a significant role in Malaysia’s education system.
The syllabus which has been developed in terms of the learning standards in Malaysia has been designed so that the language learnt in school can be used in the learners’ daily lives, academic and professional setting. The curriculum focuses on the development of literacy and critical literacy of the learner, and this aligns with the National Philosophy of Education. The National Philosophy of Education seeks to have a holistic development for the students in terms of intellectual, emotional, and spiritual potential. Although English has secured an important role in the Malaysian education system, the students’ performance level and language proficiency for English has been in a state of decline (Razali, 2013). This is further worsened with the Covid-19 pandemic, where the Minister of Education Datuk Mohd Radzi Md Jidin said in a press conference that students SPM English performance declined in 2020 (Harun, 2021). The students’ performances in English examination are not in a satisfactory level and this is worse in students’ writing examination (Azman, 2016). Among the four skills that are taught in English which are writing, speaking, listening, and reading, writing is perceived to be one of the hardest skills for Malaysian ESL teachers to teach (Vengadasamy, 2006). It is in fact a skill that is found to be daunting for Malaysian ESL teachers to teach (Kee et al., 2018; Maarof et al., 2011).
Several factors contribute as the reasons that could affect a learner’s writing performance. One of the factors that align with this research is Metacognitive Awareness (MA). Metacognitive, a term that was initially coined by Flavell in 1979 relates metacognitive to the ability of a learner to be aware of their thoughts and cognitive thoughts through a variety of strategies which include monitoring their thoughts, organising their thoughts, and adapting their thoughts. Teng (2018) mentions that metacognitive can be understood as the ability of the learners to self-regulate and monitor their learning and whereby the learners are able to use higher order thinking skills (HOTS) in their learning process. In writing, learners are required to rationalise, argue, and synchronise their idea for their writing to be coherent. A good piece of writing requires good planning as learners who are metacognitively aware of their writing process can understand the nature of the writing process and the purpose of their writing (Ramadhanti et al., 2019). This allows the essay produced by the learners to be more well developed. Learners who are metacognitively aware of their learning process can monitor and manage the effectiveness of their language learning, thus this study will study the effects of metacognitive awareness towards learners’ writing performances.
However, before going further, it is important to highlight that during the pandemic, assessments executed to measure the students’ writing performance were conducted digitally, thus it is vital to understand the relevancy of digital assessment. According to Appiah and Van Tonder (2018), digital assessment (DA) can be defined as the use of digital tools and technologies in a lesson to design, to administer, to store, and to assess the students' assessment performance in the educational settings. In recent years, DA has established itself in the education system, especially in the higher education institutes (Al-Fraihat et al., 2020). It cannot be denied that integrating technology in the classroom offers a plethora of benefits for teachers and learners.
DA has many benefits and one of the benefits of DA is that it has increased the flexibility and accessibility of assessment for both the instructor and the students. Although the LMS is not a new, but we cannot deny the practicality of it. With the LMS, generating a comprehensive data report has been made easier. With the generation of this report, it enables the instructors to analyse the learners; performances and also identify the pattern of the students for a fraction of time as compared to traditional assessment (Tomasik et al., 2018). This data-driven approach enables educators to customize their teaching strategies, utilise different teaching strategies and approaches in their lesson and provide the learners with targeted support so that the learners are bot to receive quality education. Additionally, digital assessments provide immediate and personalized feedback. The LMS is built in with automated scoring and feedback mechanism. This enables the learners to receive immediate feedback, hence allowing learners to identify their strengths and areas for improvement in real-time without having to wait as much time as compared to traditional lesson. Furthermore, DA's immediate feedback instantly clears the students’ misunderstandings or unclarity in the topic subject, thus increasing the opportunities for teachers and students to act on the feedback (Awang, 2022). This timely feedback encourages a culture of continuous improvement for both the teachers and students in the teaching and learning process, thus the learners to make timely adjustments to their learning strategies and teachers in their pedagogy approaches. Therefore, there is no denying the importance of digital assessment to improve learning outcomes in the digital age.
As a pre-requirement to do their bachelor’s degree, students in Malaysia are required to take an English test that measures their level of proficiency. This test is known as the Malaysian University English Test (MUET). The main purpose of this paper is to ensure that students are prepared and are equipped with the ability to use English in different contexts and situation that is befitting for higher educations. MUET is a test that consist of examinations that cover all the four skills in ESL which are listening, speaking, reading and writing. The range of score starts from one mark to 360 marks. Students who sit for this test will be divided into five bands, Band 1 which is the lowest band students can achieve and Band 5+ which is the highest band a student can achieve. Each paper has a weightage of 25% and maximum score of 90 marks (Malaysian Examination Council, 2019).
The MUET format has been aligned to the proficiency level and the marking scheme of the Common European Framework of Reference (CEFR). In the writing test, students will be tested on two tasks; task A which focuses on the students’ ability to use textual stimulus, containing a few notes and being able to contextual the task and task B which uses a textual stimulus, containing a few sentences or statements. Task A has been aligned with the CEFR standard to test the proficiency level for A2-C1 and B1-C1f or task B. In task A, students are required to write a letter, or an email based on the stimulus given. Students are required to write an essay with the length of 100 to 130 words. Meanwhile in task B is an extended writing essay. Students will be given a scenario that gives students enough information so that they will be able to extend the essay. A form of reflective essay is the type of essay required by students to write and this could be written in the form of a discursive essay, an argumentative essay, or a problem-solution essay. Students are required to think critically and gather all the information given to produce an essay. The length of the essay for task B must be at least 250 words.
Although several studies have been conducted on this factor, there has been a limited number of studies that study on metacognitive in depth, especially when the assessments are conducted digitally. This study would like to investigate on how in a digital assessment setting can metacognitive awareness affects the learners’ writing performance among pre-university students in Malaysia. Thus, this study attempts to discover the role of metacognitive awareness and how it relates to writing performance.
Problem Statement
The syllabus which has been developed in terms of the learning standards in Malaysia has been designed so that the language learnt in school can be used in the learners’ daily lives, in the academic and professional setting. In language learning, mastering writing skills allows the learners to develop and improve both productive (speaking and writing) and receptive skills (reading and listening) as all these skills are interconnected (Namaziandost et al., 2018; Saed & AL-Omari, 2014).
Although English has secured an important role in the Malaysian’s education system, the students’ performance level and language proficiency for English have been in a state of decline (Razali, 2013). The students’ performances in English examination are not in a satisfactory level and this is worse in students’ writing examination (Azman, 2016), thus it is important to note that learners who are metacognitively aware of their learning process can monitor and manage the effectiveness of their language learning, thus improving their performances.
Research Questions
This following question was formulated for this study:
What effect does the implementation of digital Metacognitive Awareness (MA) training have on pre-university students' writing performance?
Purpose of the Study
The purpose of this paper is to identify the impact of digital Metacognitive Awareness training on the students’ writing performance among pre-university students in Kota Kinabalu, Malaysia. This study yields preliminary findings for the main project aiming at identifying the real causes affecting writing performances where assessment is done digitally.
Research Methods
This study employed a quasi-experimental design, one-group pretest-posttest design to collect data of the administration of MUET pre-test and post-test digitally. Cranmer (2017, p. 1124) explains that a common example of a one-group pretest-posttest design is in the form of curriculum or assessment whereby the design allows the researchers to assess the effectiveness of the curriculum. This is done by testing the students on the subject matter earlier in the research period and later introducing the topic in the intervention period. After the intervention period has completed, the students are reassessed on the topic, thus the differences in the students’ knowledge at the end of the period as compared to the beginning of the testing period is a result from the knowledge that they have gained from the instructor’s intervention.
The test papers used by the instructors in this study were past year MUET questions which has been endorsed by the Malaysian Examination Council. The intervention and the papers were assessed by the MUET trainers. Statistical analyses were conducted using IBM SPSS, version 29. Mean values, standard deviation, lowest scores, and highest scores were obtained from the test scores. Statistical Package for Social Science. The research is anonymous, and the participants only points out their age and grade
Participants
The recruitment of participants upon this study consisted of 30 students from a public university in Kota Kinabalu, Malaysia. These students were pre-university students from University Malaysia Sabah (UMS), specifically from ASASI. The age group of the sample were specified and restricted to a specific age, which was the age range of 18 years old. The student samples were taken from the second semester students who were in their first year. These attributes to identify the study population to be reliable as the study was conducted during the pandemic, thus there were difficulties in the recruitment of test subjects.
Metacognitive Awareness (MA) Intervention
During the first session of the experiment, the researcher conducted a pre-test which involved the selected group. In the pre-test the students were yet to be exposed to metacognitive awareness strategies that could be applied in the writing. The students were given the same test paper and were needed to answer the tests within 80 minutes. After the completion of the pre-test, the researcher began the intervention. Throughout the intervention duration, the students attended classes that focused on training the students in methods of applying metacognitive awareness strategies in their writing.
The lecturer trained the students for a period of 10 weeks. The students were exposed to the format of the paper and how to answer the questions according to the rubrics that had been set up by the Malaysian Examination Council (MEC). The students were exposed to metacognitive skills that involved knowing your limits, self-monitoring, modify, skimming, rehearsing and self-test. The students were required to train themselves and apply the metacognitive skills in their writing tasks. The lecturer facilitated the students throughout the process. The writing tasks were given to the students inside of class hours where the feedback on their writing was given to the students by their lecturer.
After the completion of the intervention session, the students were required to take a post-test which was the MUET trial paper. Similar to the pre-test, the students were given the MUET writing examination paper which consisted of two sections. For this paper, the students were required to answer the two tasks from the paper within 80 minutes. The paper was collected by their trainer and was graded according to the rubrics set by the MEC.
Findings
RQ 1: What effect does the implementation of digital Metacognitive Awareness (MA) training have on pre-university students' writing performance?
Findings from the results show that the students writing performances before being exposed to Metacognitive Awareness Training were lower in the pre-test. This can be seen by the results of the students whereby majority of the students (27 students) achieved a Band 4 as the highest band in the pre-test and three students attained a Band 3.5 in their pre-test. The results obtained show that majority of the students were independent users of the language as the baseline for independent users was in the range of Band 3 to Band 4.5 (Malaysian Examination Council, 2019). This is in line with the study that was conducted by Kansızoğlu and Cömert (2020) that involved 55 7th grade students in Turkey. The results from this study show that both the control group and experimented group had shown lower marks in their pre-test. This is due to both groups not being exposed to Metacognitive Awareness Training in their lesson.
Results from the study show that there was a significant positive relationship between the post-test and the learners' writing performances. From the study conducted, there was a significant increase in marks in the students' results. The overall improvement can be seen in the increase of students who received a Band 5 in the post-test, which is as many as 15 students. As compared to the results in the pre-test, the students were unable to achieve a Band 5. As compared to the results in the pre-test, there is a total of 15 students who were able to achieve a Band 5 in the post-test. The results obtained show that one-third of the students who were independent users of the language had improved to become proficient users of the language as the baseline for proficient users was in the range of Band 5 to Band 5+ (Malaysian Examination Council, 2019). Another notable key finding can be seen where students S19, S30, and S15, who all received band 4 and S10 who received a Band 3.5 in the pre-test were in the top 5 scorers post-test where all the students received a Band 5. A significant increase in marks can be seen for student S10 who received a Band of 3.5 and achieved a Band 5 in the post-test.
Comparison of Pre-test and Post-test Result (Band) After Metacognitive Awareness Training
This section presents the findings that have been discovered based on the comparison has been made on the students’ writing performances before and after the training on Metacognitive Awareness. The findings that have been gathered based on the data gathered from the examination marks before and after being exposed to Metacognitive Awareness Training were analysed using descriptive analysis.
From Table 1, the average score (mean) in the pre-test was 57.25 out of 90 scores. This shows that the students scored just above the passing point before being introduced to Metacognitive Awareness training in their writing whereby the passing point is at 45 marks. However, after being exposed to Metacognitive Awareness Training, their writing performance had an average score (mean) improvement of 10 marks to 67.5 marks. In the pre-test, the median score of the students is at 57.6 and increased to 76.92 in the post-test. The students demonstrated an improvement from a Band 4 which is at the range mark of 52.6 - 64.24 to a Band 5 which is at the range mark of 73.25 - 82.5. This indicates that the students had improved from being an independent (Band 4) writing proficiency to a proficient (Band 5) writing proficiency. In the post-test, the lowest score obtained is 49.3 (Band 3) and the highest score is at 78.72 (Band 5). For the standard deviation, in the pre-test the students’ SD were narrower as compared to the post-test, whereby the students achieved a standard deviation of 4.84 in the pre-test and an 8.89 in the post-test. This indicates that the students have become more heterogenous in the post-test.
The results of the analysis obtained in Table 2, showcase that there was an overall improvement in the students’ writing performance between the pre-test and the post-test. In the pre-test, there were no students who achieved a Band 5 in their writing. However, after Metacognitive Awareness Training was integrated into their lesson, a total of 15 students were able to reach Band 5 in their post-test. The number of students who were in Band 3.5 in the pre-test decreased to 0 in the post-test. The number of students who achieved a band 4 in the post-test had decreased to 10 students from 25 students in the pre-test. The number of students who achieved 4.5 in the post-test also saw an improvement from two students in the pre-test to five students in the post-test.
Conclusions
The results from the previous studies showed consistency with the results in the current study. This strengthens the argument that students who were not exposed to Metacognitive awareness in the teaching displayed an overall lower mark in their writing performances. However, after the intervention of Metacognitive Awareness in the lecture that was conducted digitally, the students had an increase in their marks. This study also discovered that the number of students who obtained a Band 5 had also increased from 0 to 15 students, thus supporting the results that were obtained from this study. The reason that contributes to the students’ lower marks in the pre-test could be due to the students not knowing how to utilise metacognitive awareness in their writing. Another factor could be contributed due to the prevalence of the misconception that improvement in the mastery of the grammar aspect of a language will cause an increase in the learners’ writing performance, thus the students only focus on the grammar instead of the overall writing process. This belief has become detrimental as it denies low-performing students the opportunity to explore their own process of writing.
The result from the post-treatment data is also in line with studies conducted by Kansızoğlu and Cömert (2020) whereby the experimented group who were taught using Flipped Classroom Model that was based on Metacognitive Writing Awareness. In this study, the control group which was exposed to MA achieved significantly higher marks in their post-test as compared to the control group who were not exposed to Metacognitive Awareness Training. The findings from this study also correspond with the study conducted by Rahmat et al. (2021), whereby the group who was exposed to Metacognitive Awareness Training displayed better writing techniques which improves their writing performance and writing became much easier for the students, especially with the presence of team members. Another study that was conducted by Cer (2019), that investigated the use of Metacognitive Awareness in writing and how it affects the high school students’ marks discovered that their marks had a significant improvement. The results from the study showcased that the average marks of the students had improved for the experiment from 9.54 in the pre-test to 14.77 in the post-test with the highest mark increasing from the overall mark of 20 with 15 marks in the pre-test to 17 marks in the post-test.
Based on the results obtained from the study, it can be concluded that Metacognitive Awareness Training improves the students’ writing performances. There are some limitations to the study that could affect the accuracy of the results acquired in this study. One of the limitations of the study is the sample size. Due to the small sample size, which is only 30 students, the results acquired may be an isolated case as it does not represent the majority of ESL learners. However, the sample of this size is still relevant as the results of the study are relevant to this specific study group. The second limitation that was faced while conducting the present study was in the sampling of data. Purposive sampling was employed in this study; thus, it may not fully represent the predefined population which is the ESL learners and only represent a small and purposive sample of the population. There is room for improvements that could be done in the future research. As this study was a purely quantitative study, there is room for further improvement by integrating qualitative elements in future studies. This can be done by inserting students’ essay that contains their writing before and after the intervention period.
References
Al-Fraihat, D., Joy, M., Masa'deh, R. e., & Sinclair, J. (2020). Evaluating E-learning systems success: An empirical study. Computers in Human Behavior, 102, 67-86. DOI:
Appiah, M., & Van Tonder, F. (2018). E-assessment in higher education: A review. International Journal of Business Management & Economic Research, 9(6), 1454-1460.
Awang, M. I. (2022). The Digitalization of Learning Assessment. Proceedings of International Conference on Multidiciplinary Research, 4(1), 1-7. DOI:
Azman, H. (2016). Implementation and Challenges of English Language Education Reform in Malaysian Primary Schools. 3L The Southeast Asian Journal of English Language Studies, 22(3), 65-78. DOI:
Cer, E. (2019). The Instruction of Writing Strategies: The Effect of the Metacognitive Strategy on the Writing Skills of Pupils in Secondary Education. SAGE Open, 9(2), 215824401984268. DOI:
Cranmer, G. A. (2017). One-group pretest-posttest design. M. Allen (Ed.), The Sage Encyclopedia of Communication Research Methods (pp. 1124). Sage Publication, Inc.
Harun, H. N. (2021). Improved SPM results despite pandemic [NSTTV]. News Straits Time. https://www.nst.com.my/news/nation/2021/06/697466/improved-spm-results-despitepandemic-nsttv
Iber, G. (2014). English language teaching in Malaysia: The case study for dual track English curriculum [Short Communication]. Advances in Language and Literacy Studies, 5(4), pp 68-71.
Kansızoğlu, H. B., & Cömert, Ö. B. (2020). The effect of teaching writing based on flipped classroom model on metacognitive writing awareness and writing achievements of middle-school students. TED EĞİTİM VE BİLİM. DOI: 10.15390/eb.2020.8823
Kee, L. L., Razali, A. B., Noordin, N., & Samad, A. A. (2018). The role of digital technologies in facilitating the learning of esl writing among tesl pre-service teachers in Malaysia: A review of the literature. The Journal of Asia TEFL, 15(4), 1139-1145. DOI:
Maarof, N., Yamat, H., & Kee, L. L. (2011). Role of teacher, peer and teacher peer feedback in enhancing ESL students writing. World Applied Sciences Journal 15 (Innovation and Pedagogy for Lifelong Learning), 29-35.
Malaysian Examination Council. (2019). 800 Malaysian University English Test (MUET): Regulation and Test Specifications. Ministry Examination Council.
Ministry of Education. (2013). Malaysia Education Blueprint 2013-2015 (Preschool to Post-Secondary Education). Ministry of Education.
Namaziandost, E., Saray, A. A., & Esfahani, F. R. (2018). The effect of writing practice on improving speaking skill among pre-intermediate EFL learners. Theory and Practice in Language Studies, 8(12), 1690. DOI: 10.17507/tpls.0812.16
Rahmat, N. H., Aripin, N., Razlan, Z., & Khairuddin, Z. (2021). The Influence of Metacognitive Scaffolding on Learning Academic Writing Online. International Journal of Education, 13(3), 48. DOI:
Ramadhanti, D., Ghazali, A. S., Hasanah, M., & Harsiati, T. (2019). Students' Metacognitive Weaknesses in Academic Writing: A Preliminary Research. International Journal of Emerging Technologies in Learning (iJET), 14(11), 41. DOI:
Razali, A. B. (2013). Malaysian teachers’ conceptions and uses of digital technology in English writing training: A multiple case study. [Doctoral dissertation]. ProQuest Dissertations & Theses Global. https://search.proquest.com/docview/1426850 316?accountid=27932
Saed, H. A., & AL-Omari, H. A. (2014). The effectiveness of a proposed program based on a mind mapping strategy in developing the writing achievement of eleventh grade EFL students in Jordan and their attitudes towards writing. Journal of Education and Practice, 5, 88–109.
Teng, M. F. (2018). A learner-based approach of applying online reading to improve learner autonomy and lexical knowledge. Revista Española De Lingüística Aplicada/Spanish Journal of Applied Linguistics, 31(1), 94–123. DOI:
Tomasik, M. J., Berger, S., & Moser, U. (2018). On the development of a computer-based tool for formative student assessment: Epistemological, methodological, and practical issues. Frontiers in Psychology, 9. DOI:
Vengadasamy, R. (2006). Red-eyed over red ink: Alternatives to error hunt. GEMA Online Journal of Language Studies, 6(2), 89-102.
Copyright information
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
About this article
Publication Date
25 September 2023
Article Doi
eBook ISBN
978-1-80296-964-1
Publisher
European Publisher
Volume
7
Print ISBN (optional)
-
Edition Number
1st Edition
Pages
1-929
Subjects
Language, education, literature, linguistics
Cite this article as:
Suyansah, A. I., Ismail, N., Said, N., & Jawing, E. (2023). Metacognitive Awareness (MA) Impact on Writing Performance: A Digital Language Assessment. In M. Rahim, A. A. Ab Aziz, I. Saja @ Mearaj, N. A. Kamarudin, O. L. Chong, N. Zaini, A. Bidin, N. Mohamad Ayob, Z. Mohd Sulaiman, Y. S. Chan, & N. H. M. Saad (Eds.), Embracing Change: Emancipating the Landscape of Research in Linguistic, Language and Literature, vol 7. European Proceedings of Educational Sciences (pp. 837-845). European Publisher. https://doi.org/10.15405/epes.23097.75