Developing ESL Teachers’ TPACK Through Evaluation of Digital Resources

Abstract

This study explores the assessment and integration of ESL digital resources as part of the technological pedagogical content knowledge (TPACK) development of ESL pre-service teachers. The research aimed to address the overwhelming abundance of online ESL resources by helping PSTs identify and utilize appropriate resources for their instructional needs. A total of 56 second-year ESL pre-service teachers participated in digital competency training, focusing on the appraisal and use of digital resources. Through collaborative efforts, the participants developed 30-minute language activities and resources by integrating identified technological resources within a teaching simulation. The evaluation of 11 ESL resources from various categories was based on criteria such as content quality, pedagogical coherence, feedback and self-correction, motivation, usability, customization, and sharing. Quantitative data were collected using a rating form with five-point items corresponding to the evaluation criteria. The findings highlighted the strengths of different platforms across multiple constructs. Tiktok, Youtube, Quiziz, and Wordwall consistently demonstrated high mean values, indicating their effectiveness and usefulness for instructional purposes. However, platforms like Jamboard and Mentimeter identified areas for improvement to better align with pedagogical criteria. These research outcomes provide valuable insights for educators in selecting suitable platforms that align with their instructional goals and cater to the needs of their students.

Keywords: Digital competency, language learning, private college, Super Star

Introduction

Creating an optimal learning environment is a common goal and an ongoing effort for educators as they strive to provide the best learning experience for students. Recent studies have shown that language educators are experimenting with various strategies and technology in order to equip pre-service teachers with the technological knowledge and skills they will need in the future (Said et al., 2021; Singh & Kasim, 2019). It is also crucial for pre-service English as a Second Language (ESL) teachers to be able to design creative lessons and keep their technological knowledge up-to-date so that they can effectively adopt and use educational technology in their teaching (Huang, 2021). In the TESL program at a public university in Malaysia, the Technological Pedagogical Content Knowledge (TPACK) framework (Figure 1) is utilized to integrate technology into academic courses within the program. One specific course entitled ‘ICT in English Language Teaching (ELT)’ strives to provide explicit instruction directed at developing the ESL pre-service teachers’ TPACK.

See Full Size >

TPACK framework outlines the constructs representing different aspects of knowledge that teachers need to effectively integrate technology in their instruction and how they are combined and interrelated. Table 1 presents the 7 constructs of TPACK and the definition based on Koehler and Mishra (2006), and the topics in the ICT for ELT. To support Malaysia’s Ministry of Education (MoE) recently announced Digital Learning, this course strives to equip ESL pre-service teachers with knowledge and skills in using a plethora of web and digital resources for ESL teaching and learning. The final assessment required the participants were required to develop interactive instructional materials using appropriate web or digital resources based on the knowledge they gained through theoretical and practical lecture sessions. When examining the TPACK framework, the course can be recognized as striving to combine Content Knowledge (CK) and Technology Knowledge (TK) to form Technological Content Knowledge (TCK). Moreover, it has the potential to enhance the development of Technological Pedagogical Knowledge (TPK) and Pedagogical Content Knowledge (PCK) among ESL pre-service teachers.

In supporting TPACK development, scholars have put forward the idea of digital competency training. With the increasing ubiquity of digital technologies across various domains, the acquisition of new skills and competencies is necessitated. In teacher education, digital competence has emerged as a substantial body of research (Basilotta-Gómez-Pablos et al., 2022; Çebi & Reisoglu, 2020; Milutinović, 2020). Digital competency refers to the compulsory skills, knowledge and attitudes for effective use of technology in teaching and learning required for all teachers. Discrepancy and gaps were identified in offering of digital competency training in teacher education programs which were often not tailored to the diverse needs of students and teachers (Basilotta-Gómez-Pablos et al., 2022). Therefore, there is stronger call for more specialized training which should be more practical and experiential.

The digital competency training for this study focused on evaluation of web resources and applications for ESL teaching and learning due to the proliferation of ESL online resources since the advent of the Internet in the 1990s. Presently, these resources are readily accessible to meet the diverse needs of users worldwide. A search conducted on Google on May 21st, 2023 yielded over 52 million hits associated with online ESL resources, with more than 154,000 hits on Google Scholar, and hundreds of apps on Google Playstore. This indicates a burgeoning interest and demand for such resources, as they continue to be made available to interested users. Moreover, these resources are integral components of scholarly research, highlighting their essential role in education, particularly within the context of ESL. The utilization of online resources as tools for language teaching and learning tools have long been associated with positive outcomes for language learning such as Facebook (Wongsa & Son, 2022), Tiktok (Hu & Du, 2022), Jamboard (Okmawati & Tiarina, 2022), Youtube (Rodríguez-Peñarroja, 2022), Powtoon (Susanti, 2022), Wordwall (Mazelin et al., 2022), Edpuzzle (Mai et al., 2022), Storyboard That (Wahyuningsih et al., 2020), Quizizz (Pramudita, 2023) and Wakelet (Compagnoni, 2022). While potentially enticing, the use necessitates caution as there are inherent challenges in using online resources materials without possessing the skills and knowledge to align them to pedagogy and content. Consequently, the usefulness of these resources may be compromised. Evaluating suitable resources involves encountering various challenges and issues, including assessing the credibility and objectivity of web authors, verifying the currency and accessibility of web content, and determining the relevance and comprehensiveness of the information provided (Aguayo & Ramírez, 2020; Alhabdan, 2021).

There are a number of evaluation framework guiding language teachers to appraise language resources. For instance, over the past 20 years, researchers and instructional designers have frequently used Reeves' (1994) assessment of various forms of digital resources, which includes analysing the epistemology, pedagogical philosophy, underlying psychology, goal orientation, experiential value, teacher role, programme flexibility, value of errors, motivation, accommodation of individual differences, learner control, user activity, cooperative learning, and cultural sensitivity (Chen, 2016). Another evaluation framework proposed by Smith and Ragan (2004) emphasized the importance of scrutinizing instructional materials or programs by examining their content, task, and context. Evaluating the content quality offered by language resources is crucial, particularly in relation to their intended educational purpose. Since these resources are designed for learning purposes, it becomes essential to assess the alignment of pedagogical methods with language skill development within the learning activities.

In the case of mobile apps which refer to softwares installed on portable devices, evaluating their usability, customization options, and sharing capabilities holds great value. Chen (2016) proposes seven key elements for evaluating language-learning resources, including content quality, pedagogical coherence in language skill development, feedback and self-correction mechanisms, motivation, usability, customization, and sharing. In his study, Chen conducted a study exploring the affordances of seven mobile apps for language learners: Duolingo, Youdao Dictionary, Speak English Fluently, Brain POP, Speak English- Listen, Speak, Compare and Voice Tube. Findings of the study highlight the possibility of integrating mobile apps into language learning, especially for adult learners. Chen further implies the need for ESL or language instructors to be equipped with necessary skills by investing in effort to incorporate mobile apps into ESL classroom and design.

Research Questions

This following question was formulated for this study:

What is students’ evaluation of the 11 ESL digital resources based on Chen (2016) evaluation rubric ?

Purpose of the Study

The purpose of this paper is to explore ESL pre-service teachers’ evaluation of 11 ESL digital resources based on Chen (2016) seven criteria in order to develop their TPACK.

Research Methods

In this study, 56 second-year ESL pre-service teachers underwent 2-week training on evaluating and developing digital materials based on 11 apps for language learning. Table 2 lists the eleven identified apps based on its type and function. They were required to work in a group of 5. Each group was assigned one app. In the first week, they were instructed to explore the app, and then design a learning activity/resource. Outputs expected were 10-minute video on how to use/navigate the app, an outline for a 15-minute activity and one learning resource. A 30 minute discussion was also conducted on how to evaluate app based on Chen (2016) rubric. In the second week, each group had to conduct 20-minute simulation of the language activity integrating the assigned app. Brief discussions were also held at the end of each presentation to record any important remarks from the group and peers. At the end of the training, all students had to rate the 11 apps based on Chen (2016) rubric.

For this study, Chen’s (2016) rubric for evaluating digital resources including mobile apps were adopted. A form consisting 8 items was distributed to the 56 participants. They were asked to rate on a scale of 1 to 5 based on 7 criteria of content quality, pedagogical coherence regarding language skills, feedback and self-correction mechanisms, motivation, usability, customization, and sharing. Table 3 provides the explanation for each criterion.

Data was quantitatively analysed using descriptive statistics of mean values and standard deviation. Based on the evaluation standards outlined by Genc (2023) for Likert scale (5-point) items, ratings ranging from 1.00 to 1.79 are categorized as a very low level, ratings between 1.80 and 2.59 are classified as a low level, ratings between 2.60 and 3.39 are considered a moderate level, ratings between 3.40 and 4.19 are regarded as a high level, and ratings between 4.20 and 5.00 are considered a very high level.Qualitative data from the open-ended item was thematically analysed.

Findings

Table 5 summarizes ESL pre-service teachers evaluation of 11 identified digital applications for language learning. Overall, the mean values for all contructs and apps exceeded the lower limit cut-off value for high level (Genc, 2023).

Based on the evaluation criteria in Table 5, the data reveals varying levels of performance across different apps. Among the evaluated constructs, content quality which tests the applications based on their ability to enable learners to improve their English skills is an essential criteria. Tiktok emerges as the app with the highest mean value (4.39) in this category, indicating a strong performance in delivering high-quality content. Other platforms like Youtube (4.52) and Wordwall (4.32) also demonstrate commendable scores in terms of content quality. However, apps such as Jamboard (3.61) and Mentimeter (4.00) have slightly lower mean values, suggesting room for improvement in this construct. Pedagogical coherence, another important criterion, evaluates the platforms' ability to provide logical and coherent instructional approaches. Youtube (4.63) stands out as the application with the highest mean value in this aspect, indicating a well-structured and coherent pedagogical approach. Tiktok (4.38) and Facebook (4.18) also exhibit strong performances in terms of pedagogical coherence. On the other hand, Jamboard (3.72) and Powtoon (3.54) show slightly lower mean values, indicating a need for enhancing the coherence of their pedagogical strategies.

Feedback and self-correction play a crucial role in supporting students' learning and growth. Digital resources like Quiziz (4.63) and Tiktok (4.38) receive high mean values in this aspect, suggesting effective feedback mechanisms. Similarly, Storyboard That (4.18) and Edpuzzle (4.18) also exhibit strong performances in providing adequate feedback and encouraging self-correction. However, Jamboard (3.88) and Mentimeter (4.13) have comparatively lower mean values, indicating scope for improvement in their feedback and self-correction mechanisms. Motivation is another crucial construct influencing students' engagement and interest. Tiktok (4.55) and Youtube (4.55) demonstrate the highest mean values in this category, suggesting their effectiveness in fostering student motivation. Applications such as Quiziz (4.64) and Powtoon (4.45) also perform well in motivating students. However, Jamboard (3.46) receives a relatively lower mean value, indicating a need to enhance motivational features within the platform.

Users can adapt applications through customization to their unique requirements and tastes. When it comes to this, Tiktok (4.57) and Quiziz (4.61) excel thanks to their wide customisation choices. Applications like Storyboard That (4.11) and Wordwall (4.23) also offer noteworthy levels of customization. The slightly lower mean values of Mentimeter (3.98) and Jamboard (3.86), however, point to possible areas for improving customising features. The architecture of exchange assesses the platforms' capacity to promote collaboration and content exchange. The app with the highest mean value in this area is Tiktok (4.43), which shows that users strongly promote sharing and teamwork. In terms of sharing, websites like Wordwall (4.27) and Storyboard That (3.77) also display noteworthy performances. However, the significantly lower mean values of Mentimeter (3.75) and Jamboard (3.51) indicate potential

Overall, the analysis of the data reveals the strengths and potential of various applications across multiple constructs. Notably, Tiktok, Youtube, Quiziz, and Wordwall demonstrate consistently high mean values across a range of metrics, showcasing their efficacy and usefulness for instructional purposes. Conversely, Jamboard and Mentimeter highlight areas for improvement to meet pedagogical criteria. These findings provide valuable insights for ESL pre-service teachers and educators, enabling them to make informed decisions when selecting platforms that align with their instructional goals and student needs. The positive outlook based on these findings aligns with previous studies that have examined the use of these applications in language learning. Wongsa and Son (2022) endorsed the use of Facebook, emphasizing its ability to enhance students' motivation and attitude towards learning English through interactive and flexible learning environments that foster communication and collaboration. Hu and Du (2022) found that TikTok can facilitate language proficiency and offers affective benefits, although its impact on language learning compared to traditional classroom-based methods is not substantial. Okmawati and Tiarina (2022) also recommended the use of Jamboard for motivation in language learning. Similarly, Rodríguez-Peñarroja (2022) and Mazelin et al. (2022) supported the use of Youtube and Wordwall, respectively, for their positive impact on motivation, participation, understanding, and engagement in ESL classrooms. Edpuzzle (Mai et al., 2022), Powtoon (Susanti, 2022), Storyboard That (Wahyuningsih et al., 2020), and Quizizz (Pramudita, 2023) were praised for fostering creativity, idea development, and enjoyable yet competitive lessons. Compagnoni (2022) provided strong support for Wakelet in promoting multiliteracies among language learners, emphasizing its capacity for content personalization, interpersonal and intrapersonal dialogue, self-reflection, and intercultural considerations.

Conclusion

In conclusion, the evaluation of various educational platforms based on different criteria provides valuable insights for educators seeking to integrate technology into their instructional practices. Tiktok, Youtube, Quiziz, and Wordwall demonstrate strong performances across multiple constructs, including content quality, pedagogical coherence, feedback and self-correction, motivation, usability, customization, and sharing. These platforms exhibit features that support effective teaching and learning experiences. On the other hand, platforms like Jamboard and Mentimeter show potential for improvement in specific areas, indicating the need for further enhancements to better meet the pedagogical needs of educators. By considering these findings, educators can make informed decisions about selecting appropriate platforms that align with their instructional goals and promote engaging and effective learning experiences for their students.

Acknowledgments

The reported study was funded by University Malaysia Sabah under the New Appointment Scheme (SLB: Skim Lantikan Baharu), project number SLB2230 “Supporting ESL Teachers' Technological Pedagogical Content Knowledge (TPACK) for Teaching and Learning in Suburban Areas of Sabah”.

References

• Aguayo, N., & Ramírez, C. M. (2020). Does technical assessment matter? Functionality and usability testing of websites for ESL/EFL autonomous learners. Research in Learning Technology, 28(1063519), 1–26. DOI:

  Google Scholar

  Crossref

• Alhabdan, A. (2021). Evaluating English Learning Websites and Digital Resources From the Perspective of Saudi English Language Learners: Technical Assessment. Theory and Practice in Language Studies, 11(6), 639-651. DOI:

  Google Scholar

  Crossref

• Basilotta-Gómez-Pablos, V., Matarranz, M., Casado-Aranda, L. A., & Otto, A. (2022). Teachers’ digital competencies in higher education: A systematic literature review. International Journal of Educational Technology in Higher Education, 19(1). DOI:

  Google Scholar

  Crossref

• Çebi, A., & Reisoglu, I. (2020). Digital competence: A study from the perspective of pre-service teachers in Turkey. Journal of New Approaches in Educational Research, 9(2), 294–308. DOI:

  Google Scholar

  Crossref

• Chen, L.-L. (2016). A model for effective online instructional design. Literacy Information and Computer Education Journal, 7(2). DOI:

  Google Scholar

  Crossref

• Compagnoni, I. (2022). Developing multiliteracies through digital storytelling a case study of learners of Italian as a foreign language. El.Le, 11(2), 225–244. DOI:

  Google Scholar

  Crossref

• Genc, O. (2023). Identifying principal risk factors in Turkish construction sector according to their probability of occurrences: a relative importance index (RII) and exploratory factor analysis (EFA) approach. International Journal of Construction Management, 23(6), 979-987. DOI:

  Google Scholar

  Crossref

• Huang, Y. (2021). TPACK Framework for Online EFL Teacher Training in Higher Education. Adult and Higher Education, 3(1), 95–102. DOI:

  Google Scholar

  Crossref

• Hu, H., & Du, K. (2022). TikTok in Mobile-Assisted English language learning: An exploratory study. International Journal of Information and Education Technology, 12(12), 1311–1320. DOI:

  Google Scholar

  Crossref

• Koehler, M. J., & Mishra, P. (2006). Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge. Teachers College Record, 108(6), 1017–1054. http://one2oneheights.pbworks.com/f/MISHRA_PUNYA.pdf

• Kontkanen, S., Dillon, P., Valtonen, T., Renkola, S., Vesisenaho, M., & Väisänen, P. (2016). Pre-service teachers' experiences of ICT in daily life and in educational contexts and their proto-technological pedagogical knowledge. Education and Information Technologies, 21(4), 919-943. DOI:

  Google Scholar

  Crossref

• Mai, T. M., Tran, L. T. D., Nguyen, M. T. X., Le, L. V. P., & Vo, N. H. K. (2022). Evaluating the design and delivery of an online community-based course to develop school teachers’ tpack for emergency remote teaching. Call-EJ, 23(4), 162–186.

• Mazelin, N., Maniam, M., Jeyaraja, S. S. B., Ng, M. M., Xiaoqi, Z., & Jingjing, Z. (2022). Using wordwall to improve students’ engagement in ESL classroom. International Journal of Asian Social Science, 12(8), 273–280. DOI:

  Google Scholar

  Crossref

• Milutinović, V. (2020). Examining the digital competencies of pre-service teachers. Professional Competences For Teaching In The 21st Century, August, 373–391. DOI:

  Google Scholar

  Crossref

• Okmawati, M., & Tiarina, Y. (2022). Proceedings of the 5th international conference on language, literature, and education (ICLLE-5 2022). In Proceedings of the 5th International Conference on Language, Literature, and Education (ICLLE-5 2022). Atlantis Press SARL. DOI:

  Google Scholar

  Crossref

• Pramudita, R. F. (2023). Quizizz as technology to enhance EFL students’ active participation in the classroom. Journal of English in Academic and Professional Communication, 9(1), 11–18. DOI:

  Google Scholar

  Crossref

• Reeves, T. C. (1994). Evaluating what really matters in computer-based education. Computer education: New perspectives, 219-246.

• Rodríguez-Peñarroja, M. (2022). Integrating project-based learning, taskbased language teaching approach and youtube in the esp class: A study on students’ motivation. Teaching English with Technology, 22(1), 62–81.

• Said, N., Soekarno, M., Nik Mohamed, N. Z., & Pudin, C. S. J. (2021). the Feasibility of a Personal Learning Environment (Ple) To Support Academic Writing: Esl Pre-Service Teachers’ Perspectives. International Journal of Education, Psychology and Counseling, 6(43), 110–121. DOI:

  Google Scholar

  Crossref

• Singh, C. K. S., & Kasim, Z. M. (2019). Pre-service teachers’ mastery of technological pedagogical content knowledge for teaching English language. Universal Journal of Educational Research, 7(10D), 24–29. DOI:

  Google Scholar

  Crossref

• Smith, P. L., & Ragan, T. J. (2004). Instructional design. John Wiley & Sons.

• Susanti, H. (2022). Students’ responses toward the implementation of powtoon in EFL classroom at a remote area. International Journal of Modern Languages and Applied Linguistics, 6(3), 17–33. DOI:

  Google Scholar

  Crossref

• Wahyuningsih, E., Santihastuti, A., Kuriniawati, I., & Arifin, U. M. (2020). Storyboard That Platform to boost students ’ creativity : Can it become real ? IOP Conference Series: Earth and Environmental Science PAPER, 1–6. DOI:

  Google Scholar

  Crossref

• Wongsa, M., & Son, J. B. (2022). Enhancing Thai secondary school students’ English speaking skills, attitudes and motivation with drama-based activities and Facebook, Innovation in Language Learning and Teaching, 16(1), 41-52. DOI:

  Google Scholar

  Crossref