Assessing Digital Literacy Competence for Online Reading Among Postgraduate Students
Abstract
This study examines the digital literacy competence of postgraduate students in a Malaysian private university, focusing on their knowledge of digital technology and their ability to perform online reading activities in the English language. Data from 84 participants were collected through a questionnaire survey and test. Results indicate that 70% of the students possess Intermediate-level competence overall. However, a detailed analysis reveals below-average performance in online reading and comprehension tasks such as locating, synthesizing, and evaluating online information. This study emphasizes the importance of digital literacy instruction at the postgraduate level to sustain quality education and to ensure that they can graduate on time. Aligning with UNESCO's Sustainable Development Goals (SDGs), which aim for digital literacy attainment among youths and adults by 2030, this research contributes to fostering critical users in the digital environment and addressing the specific competence gaps that were identified. Targeted interventions can enhance students' proficiency and prepare them for the demands of the digital age.
Keywords: Comprehension skills, digital literacy, online reading, postgraduate study
Introduction
In the 21st century, the exponential growth of digital technology, especially in the post-pandemic era has resulted in the massification of education, especially in the form of online learning. As the geographical barrier is diminished, access to quality education is increased, especially at the postgraduate level. As a result, students are expected to acquire a set of skills in a new form of literacy, dubbed 'digital literacy', to take advantage of the various technologies and digital information available. In line with providing quality education to all, UNESCO's Sustainable Development Goals (SDG) Target 4.4 has acknowledged the importance of digital literacy and positions it as one of the skills to be attained by youths and adults by 2030 (UNESCO Institute of Statistics, 2019). To be specific, Indicator 4.4.2 calls for countries to track the percentage of youth and adults who have achieved at least a minimum level of proficiency in digital literacy.
Nonetheless, in the higher education (HE) sector, there has been a mismatch and a misconception regarding the minimum standards of digital literacy skills required for admission. A report by Coldwell-Neilson (2020) for the Department of Education, Skills & Employment in Australia highlights that the standard is not well articulated, if it does at all exist. This would affect the quality of learning and the future workforce with the increasing digitization. At the same time, due to the demand of today's learning environment, the current students are expected to be digitally literate to ensure they can complete their studies on time. The number of PhD holders produced in Malaysia is estimated to reach 60,000 by 2023 (Baydarova et al., 2021). However, the output remains low (Da Wan et al., 2023) due to various challenges that resulted in low and late completion of studies (Shariff et al., 2015). So, this begs the question, what is the minimum standard of digital literacy for postgraduate students in Malaysia, and how do we achieve this standard?
Digital Literacy for Postgraduate Students
Extant studies have recorded many variations of the terms to define digital literacy, including 'computer literacy' and 'information literacy' (Bawden, 2001). However, this study uses the umbrella term 'digital literacy', encompassing many different digital technologies and skills (Ala-Mutka, 2011) required in today's landscape. The study further contextualizes the definition according to the Digital Literacy Global Framework (Law et al., 2018), particularly in reference to Area 1: Information and Data Literacy, which is the ability to articulate information needed and to judge the relevance of the source and its content.
At the postgraduate (PG) study level, digital literacy, particularly on information and data, is considered a basic competence that enables students to conduct academic reading and research for their master's and Doctoral dissertations and thesis in English. The PG students enrolled in HE institutions in Malaysia use English as a second (ESL) or foreign language (EFL). They need to demonstrate competency in performing online research and comprehension activities effectively in the English language, using a combination of search engines and online databases for information gathering and using digital tools for information and data analysis. They must then be able to filter critically (Drew, 2012) and evaluate online information (Kingsley & Tancock, 2014) to ensure that their dissertation and thesis are credible. This is in line with the New Literacies Perspectives (Leu et al., 2004), which view data and information literacy as a multimodal process that includes the following: the ability to locate information based on questions, the ability to critically evaluate the usefulness of the information and the ability to synthesize the information to answer those questions. In doing so, students need to be able to navigate the online environment effectively and familiarize themselves with features of non-linear reading, such as hypertexts and links (Sain et al., 2019). At the same time, they must be proficient in English to effectively utilize online information databases such as Google Scholar.
Nonetheless, there is scant empirical research documenting students' competence and training in digital literacy, especially at the PG level in Malaysia. This may have stemmed from the myth that the current generation of students who enrolled in HE institutions are 'digital natives' (Kirschner & De Bruyckere, 2017; Margaryan et al., 2011) or that they may have already gained the skills and language proficiency, as well as built the competence during their previous undergraduate studies (Coldwell-Neilson, 2020). To no surprise, there is a mismatch between the faculty's expectations and the student's actual skills in digital literacy (Coldwell‐Neilson, 2018), as studies have found that students do not have the proficiency for learning digitally nor are they provided with training (Kirschner & De Bruyckere, 2017). To add to that, PG students in Malaysia, have also been struggling to perform well in critical reading and academic writing in English thus affecting their ability to graduate on time (Kaur & Sidhu, 2014; Lim et al., 2016). Thus, the coalescence of these issues and challenges could potentially affect the quality of PG education.
To address this issue, this research project implemented a six-month treatment project to examine and enhance PG students' basic competencies to enable them to complete their studies successfully and graduate on time. The first step in the curriculum design of the project was to conduct a needs analysis of students' current competencies in which one of the areas explored was digital literacy for ESL and EFL postgraduate students. Therefore, this study aims to gauge their competence, especially in performing online research activities based on the following questions:
What is the current level of digital literacy competence possessed by first-semester PG students?
How do the PG students perform their online research and comprehension activities in the English language?
Methodology
This study employed a descriptive research design involving 84 first-semester PG students who are ESL and EFL users at the Faculty of Education in a Malaysian private university located in Petaling Jaya, Selangor. The data were collected quantitatively using two research instruments, a questionnaire and a digital literacy competence test. The questionnaire comprised two main sections. Section A explored the students' demographic competence and basic computer literacy knowledge. Section B examined the respondents' information about their digital literacy background and the challenges they faced in online reading and research activities.
The test, which was adapted from Sain et al. (2019), measures their competence in digital literacy in two parts; Part A – Knowledge of Digital Tools for Research (7 marks) and Part B – Online Research and Comprehension Skills (9 marks). For Part B, the respondents had to complete two tasks:
Task 1 – Locating and synthesizing online information (4 marks)
Task 2 – Evaluating online information (5 marks)
To gauge the level of digital literacy competence, the following descriptors of Basic, Intermediate and Advanced users (Asega, 2014) were used:
Basic – Users can perform some online searching using search engines, put together information from multiple sources and know that not all online information is reliable.
Intermediate – Users can browse the Internet for information, can search for information online, select appropriate information, compare different information sources, and combine the information from multiple sources effectively.
Advanced – Users can use a wide range of strategies for information searching, browsing, and synthesizing, are critical about the information found, and can cross-check and assess the validity and credibility of the information.
For each level, the participants were evaluated based on their ability to use the Internet to perform online searching, to synthesize the information and finally to evaluate the credibility of online information.
Results
The findings are reported based on the analysis of the questionnaire followed by the digital literacy competence test.
Profile of the Respondents
The respondents of this study comprised 19 (23%) male and 65 (77%) female students, a majority of whom were between the ages of 30 to 39 years old. Of the 84 respondents, 58 were pursuing their Ph.D. (69%), while the remaining 26 were pursuing master's degrees (31%). All these respondents were taking a PG degree in education by research mode.
Section A also examined the respondents' experience with digital technology used for academic purposes and their level of digital literacy competence. As reported in Table 1, these respondents mostly had less than ten years of experience in using digital technology and the Internet for academic purposes and perceived their competence as Intermediate users.
Postgraduate Students' Digital Literacy Competence
The first objective of this study was to identify the first-semester students' current level of digital literacy competence upon enrolling in the PG study programs. The mean scores for the digital literacy competence test are presented in Table 2. Based on the average mean score of the two tasks, the respondents performed at the low-intermediate level (M=7.41), performing better in Task A than in Task B. The respondents' performance in Task B was at the Basic level (M=2.32), which did not reflect the overall level of competence.
The respondents' scores were categorized at the Basic, Intermediate and Advanced competence levels and presented in Table 3. Similar to the respondents' perceived competence, a majority of 70.2% or 59 respondents fell into the Intermediate User category.
Postgraduate Students' Performance in Online Research and Comprehension Activities
The second objective of this study was to examine the first-semester PG students' performance in completing online research and comprehension tasks. Since digital literacy in this study is defined as the ability to search and analyze online information in English, their skills were tested.
Results from Part B of the Digital Literacy Competence test were analyzed and tabulated in Table 4. The respondents had to complete two tasks of nine questions. As seen in the table, the respondents' mean scores did not even achieve half of the maximum score for each task.
The responses were further analyzed to identify any common patterns. The following problems were identified based on the three skills – the ability to locate, synthesize, and evaluate online information (see Table 5). For the first skill, respondents who did not manage to successfully complete the task may have been confused by online jargon such as Uniformed Resource Locator (URL) and ended up providing the names of search engines. For the task that required them to locate the author's name, some respondents ended up writing the name of the website instead.
As for the ability to synthesize information from multiple sources, some respondents lacked the language ability to paraphrase and combine the information obtained. As a result, they ended up plagiarizing and copying the content. Their poor online research skills were also evident in their inability to evaluate online information. The respondents could not identify that the information was outdated or inaccurate or verify that the online materials' author was not credible.
The respondents were asked to share their opinions regarding the challenges in completing the questionnaire's online research and comprehension activities. The findings in Table 6 below provide some insights into their poor performance in the online research tasks. In terms of locating online information, among the challenges they faced were difficulty in deciding the keywords in English for the search task as well as finding suitable materials due to the overwhelming abundance of search results.
When it comes to synthesizing the information using multiple sources, the challenges came in deciding which information to use and analyzing the information effectively, which may have resulted in the responses being copied directly from the source materials. The challenges faced in evaluating online information, meanwhile, were reflective of the types of responses provided as they were unable to decide whether the information was authentic or whether the author was credible.
There were two other issues highlighted by the respondents, which may have contributed to the poor score. Among others, the respondents had poor Internet connectivity or were unable to access the content, thereby affecting their completion of the task. Others stated language barriers, i.e., the difficulty in comprehending the source materials or writing accurate responses in English.
Discussion
So, the question that begs us is, what can be learned from these findings? To address the first research question, we identified that the PG students in this study possessed low-intermediate digital literacy skills based on the digital competence test. This may be a cause for concern since the majority of the respondents are pursuing their doctoral degrees and rely heavily on digital technology to complete their theses and dissertations. The test result indicated a digital divide whereby students may have the knowledge of the digital tools required for academic research but fail to successfully apply their knowledge in performing basic online research and comprehension activities in the English language. It is worth noting that most of the respondents in this study had less than ten years of experience using digital technologies for academic purposes. This may have impacted their performance, indicating a lack of preparedness and proficiency for digital learning (Kirschner & De Bruyckere, 2017). Similar studies such as Vishnu et al. (2022) also found that their respondents had moderate-level competence in the information and data literacy component. However, their PG respondents were reported to have scored higher means but only with a significant difference compared to first-year undergraduates.
Another part of the findings, which addressed the second research question, suggested that PG students struggled to cope with the multidimensional skills required for online reading using the English language. Based on their responses in the online research activity, many could not competently show the ability to decide which information to be used, as well as to interpret and use the information effectively, which involves multifaceted processes of sophisticated application in online environments (Afflerbach & Cho, 2009; Leu et al., 2013). This resulted in them giving basic or even inaccurate responses when locating online information. As performing online reading activities, particularly in evaluating and synthesizing information critically, is acknowledged as a 'difficult feat' (Kingsley et al., 2015), the students may lack the knowledge and strategy to read digitally effectively and in critical ways. Therefore, these findings imply that strategy training is pertinent to ensure that students are able to perform these processes effectively (Sain et al., 2019). Previously, it was found that giving direct instructions on skills like synthesizing was found to improve student's performance (Castek, 2008). Similarly, using tested frameworks to support instructions has helped to support better information processing (Zhang & Duke, 2011).
The inclusion of digital literacy instruction at the tertiary level is timely due to the shift to online learning post-pandemic and crucial to help the students complete their studies successfully and graduate on time. Researchers have noted that the current graduate-on-time percentage is very low, with an average completion rate of 4.84 years as opposed to the four years standard (Ismail et al., 2011). Moreover, it was found that poor completion rate at the PG level had stemmed from limited technical skills and knowledge related to academic research (Muthukrishnan et al., 2022), reading and critical thinking skills (Kaur & Sidhu, 2014), and poor writing skills (Lim et al., 2016); all of which are now mostly conducted using digital platforms. Thus, embedding digital literacy in the PG curriculum is instrumental to equipping the students with the necessary skills before they are expected to embark on intensive research activities for their thesis and dissertation writing.
Conclusion
The findings in this study indicated that the first-semester students, who enrolled in the PG programs, possessed rather limited competence in digital literacy, particularly with regard to online research and comprehension skills in the English language. Although the study was primarily focused on Information and Data Literacy, its findings underscore the need for educational initiatives to support students in becoming critical users of digital tools and enhancing their online research skills. As there had been no clearly defined criteria to set the minimum standard of digital literacy, let alone the specified skills required for PG study, there could be a decline in the quality of PG education. With Malaysia's current attrition rate worrying (Ministry of Education, 2016), the situation could worsen if no countermeasures are taken.
Although this study was limited to a small sample of PG students from one private university, its findings are worthy of a call to action for HE institutions to develop targeted interventions and programs that address the specific digital literacy gaps identified among PG students. By ensuring that students receive comprehensive digital literacy instruction, universities can contribute to the sustainability of quality education and prepare individuals for the demands of the 21st-century digital landscape. PG curriculum should also consider embedding or improving digital literacy instructions for ESL and EFL learners as a way to support the students' PG journey. A diagnostic tool should also be considered to identify the students' level of basic competence in digital literacy and other skills required for PG study. Lastly, there should also be a refined entry requirement to inform potential students of the skills and competencies expected from them. With a collective concern and effort by HE institutions and students alike, quality PG education can be sustained and the minimum standard of digital literacy competence in SDG 4.4 can be achieved.
Acknowledgements
This paper is part of a research project funded by the Fundamental Research Grant Scheme (FRGS)(FRGS/1/2018/SSI09/SEGI/01/1), Ministry of Higher Education & SEGi University, Malaysia.
References
Afflerbach, P., & Cho, B. Y. (2009). Identifying and describing constructively responsive comprehension strategies in new and traditional forms of reading. Handbook of Reading Comprehension Research, 69-90.
Ala-Mutka, K. (2011). Mapping digital competence: Towards a conceptual understanding. Sevilla: Institute for Prospective Technological Studies, 7-60.
Asega, K. F. (2014). Assessing digital literacy levels: the design of an instrument. International Applied Linguistics Association Conference 2014 (AILA 2014).
Bawden, D. (2001). Information and digital literacies: a review of concepts. Journal of Documentation, 57(2), 218-259. DOI: 10.1108/eum0000000007083
Baydarova, I., E Collins, H., & Ait Saadi, I. (2021). Alignment of Doctoral Student and Supervisor Expectations in Malaysia. International Journal of Doctoral Studies, 16, 001-029. DOI:
Castek, J. M. (2008). How do 4th and 5th grade students acquire the new literacies of online reading comprehension? Exploring the contexts that facilitate learning [Unpublished doctoral thesis]. University of Connecticut.
Coldwell‐Neilson, J. (2018). Digital literacy expectations in higher education, Proceedings of the Australasian Society for Computers in Learning in Tertiary Education, ASCILITE 2018, 25‐28 November. Geelong, Australia. 103‐112. https://ascilite.org/wp‐ content/uploads/2018/12/ASCILITE‐2018‐Proceedings.pdf
Coldwell‐Neilson, J. (2020). Unlocking the code to digital literacy. https://ltr.edu.au/resources/FS16-0269_Coldwell-Neilson_Report_2020.pdf
Da Wan, C., Azman, N., & Sirat, M. (2023). Productivity of Malaysian universities: Boon or bane?. In The Performance of Asian Higher Education (pp. 114-137). Routledge.
Drew, S. V. (2012). Open Up the Ceiling on the Common Core State Standards: Preparing Students for 21st-Century Literacy—Now. Journal of Adolescent & Adult Literacy, 56(4), 321-330. DOI: 10.1002/jaal.00145
Ismail, A., Abiddin, N. Z., & Hassan, A. (2011). Improving the Development of Postgraduates' Research and Supervision. International Education Studies, 4(1). DOI:
Kaur, S., & Sidhu, G. K. (2014). Evaluating the Critical Literacy Practices of Tertiary Students. Procedia - Social and Behavioral Sciences, 123, 44-52. DOI:
Kingsley, T. L., Cassady, J. C., & Tancock, S. M. (2015). Successfully promoting 21st century online research skills: interventions in 5th-grade classrooms. Reading Horizons, 54(2), 5.
Kingsley, T., & Tancock, S. (2014). Internet inquiry: Fundamental competencies for online comprehension. The Reading Teacher, 67(5), 389-399.
Kirschner, P. A., & De Bruyckere, P. (2017). The myths of the digital native and the multitasker. Teaching and Teacher Education, 67, 135-142. DOI:
Law, N., Woo, D., de la Torre, J., & Wong, G. (2018). A global framework of reference on digital literacy skills for indicator 4.4.2. UNESCO Institute for Statistics Information Paper, (51), http://uis.unesco.org/sites/default/files/documents/ip51-global-framework-reference-digital-literacy-skills-2018-en.pdf
Leu, D. J., Kinzer, C. K., Coiro, J. L., & Cammack, D. W. (2004). Toward a theory of new literacies emerging from the Internet and other information and communication technologies. Theoretical Models and Processes of Reading, 5, 1570–1613. DOI:
Leu, D., Kinzer, C., Coiro, J., Castek, J., & Henry, L. (2013). New Literacies: A Dual-Level Theory of the Changing Nature of Literacy, Instruction, and Assessment. Theoretical Models and Processes of Reading, 1150-1181. DOI:
Lim, P. C., Sidhu, G. K., Chan, Y. F., Lee, L. F., & Jamian, L. S. (2016). Assessing Writing Skills of Postgraduate Students: Perspectives of Supervisors and Supervisees. Assessment for Learning Within and Beyond the Classroom, 31-41. DOI:
Margaryan, A., Littlejohn, A., & Vojt, G. (2011). Are digital natives a myth or reality? University students' use of digital technologies. Computers & Education, 56(2), 429-440. DOI:
Ministry of Education. (2016). Enhancing Academic Productivity and Cost Efficiency (University Transformation Programme Silver Book). Malaysia
Muthukrishnan, P., Sidhu, G. K., Hoon, T. S., Narayanan, G., & Fook, C. Y. (2022). Key factors influencing graduation on time among postgraduate students: A PLS-SEM approach. Asian Journal of University Education (AJUE), 18(1), 51-64.
Sain, N., Nawi, S. M., Yusof, G. K., Sidhu, G. K., Bown, A., & Fluck, A. (2019). Investigating Malaysian undergraduates' online research performance in embracing education 4.0. In 1st International Conference on Education & Languages for Students & Adults Learners (ELSA 2019) (pp. 44-62).
Shariff, N. M., Ramli, K. I., Ahmad, R., & Abidin, A. Z. (2015). Factors contributing to the timely completion of PhD at the Malaysian public higher educational institutions. International Journal of Humanities Social Sciences and Education (IJHSSE), 2(1), 256-263.
UNESCO Institute of Statistics. (2019). Tools to Help Countries Measure Digital Literacy. http://uis.unesco.org/en/news/tools-help-countries-measure-digital-literacy
Vishnu, S., Raghavan Sathyan, A., Susan Sam, A., Radhakrishnan, A., Olaparambil Ragavan, S., Vattam Kandathil, J., & Funk, C. (2022). Digital competence of higher education learners in the context of COVID-19 triggered online learning. Social Sciences & Humanities Open, 6(1), 100320. DOI:
Zhang, S., & Duke, N. K. (2011). The Impact of Instruction in the WWWDOT Framework on Students' Disposition and Ability to Evaluate Web Sites as Sources of Information. The Elementary School Journal, 112(1), 132-154. DOI:
Copyright information
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
About this article
Publication Date
25 September 2023
Article Doi
eBook ISBN
978-1-80296-964-1
Publisher
European Publisher
Volume
7
Print ISBN (optional)
-
Edition Number
1st Edition
Pages
1-929
Subjects
Language, education, literature, linguistics
Cite this article as:
Nawi, S. M., Sidhu, G. K., & Kamil, A. S. S. (2023). Assessing Digital Literacy Competence for Online Reading Among Postgraduate Students. In M. Rahim, A. A. Ab Aziz, I. Saja @ Mearaj, N. A. Kamarudin, O. L. Chong, N. Zaini, A. Bidin, N. Mohamad Ayob, Z. Mohd Sulaiman, Y. S. Chan, & N. H. M. Saad (Eds.), Embracing Change: Emancipating the Landscape of Research in Linguistic, Language and Literature, vol 7. European Proceedings of Educational Sciences (pp. 865-874). European Publisher. https://doi.org/10.15405/epes.23097.78