A Thematic Analysis of Language Learners’ Online Learning Experiences: Contents and Assessments
Abstract
The Fully Online Learning Community (FOLC) framework which was used in conceptualizing, designing, and researching collaborative online learning posits a dimensional structure comprises four constructs specifically social presence, cognitive presence, collaborative learning and digital space which interrelate with one another. As movement restrictions were executed universally due to Covid-19 pandemic, both learners and instructors around the globe were compelled to fully embrace and be part of this online community. Hence, this paper aims to look into the experiences of a group of language programme learners with regards to their cognitive presence – course contents and assessments while engaging in online learning. This qualitative study employed thematic analysis (TA) in coding the data derived from 256 responses from a total of 320 respondents of an open-ended questionnaire. From the TA, five dimensions were identified to correspond with the themes developed for course contents and four for assessments respectively. The findings are imperative in assisting instructors, curriculum developers and designers in devising as well as reviewing an online learning curriculum to improve the existing curriculum and practices of the language programme.
Keywords: Assessments, cognitive presence, course contents, online learning experiences, thematic analysis
Introduction
The landscape of education globally shifted tremendously when the world was hard hit by the Covid-19 pandemic in 2020. What was initially perceived as impossible, undoable, and unimaginable in a blink of an eye became feasible and achievable and that is proven true when the traditional, conventional teaching and learning which had taken place in brick-and-mortar schools for years was abruptly replaced by virtual classrooms. Instead of being compelled to be in the four-wall classrooms, both learners and instructors were forced to meet online. This undeniably was challenging at the very beginning where the students need to adapt themselves to the new environment computer-led training in virtual classrooms from traditional classroom, which is a challenging task (Sanchez-Gordon & Luj´an-Mora, 2014) yet became a norm and a well-accepted practice in due course.
Online learning or virtual learning absolutely has become a familiar practice these days. This was the approach primarily employed by instructors during the pandemic to ensure the teaching and learning still went on amidst the movement restrictions implemented throughout the globe and its usage persists till now. Through this learning approach and environment, students have freedom in learning and get connected with their teachers anywhere they want (Singh & Thurman, 2019). In Malaysia, online learning has been in existence for quite a while. According to Hussin et al. (2009), the Malaysian Higher Learning Institutions had implemented online learning started in the late 1990s. With the pandemic, it has gained its popularity even more where it has started to dominate the education system and is being widely incorporated into curriculum particularly in higher learning institutions. Schools and universities across the country are beginning to redesign learning spaces to enable this new norm of education, fostering more interaction with the use of technology as an enabler (Md Yusuf et al., 2021).
As many schools and tertiary level institutions are redesigning their curriculum to incorporate online learning, instructors, curriculum developers and designers should ensure there are uniform and standardised learning goals and objectives in place in ensuring the needs and aims of both the learners and the institutions are not put at stake and that the learning process and progress is not interrupted and compromised by such implementation. To probe learners’ insights regarding how they perceive their online learning experience as put forth by FOLC framework (Van Oostveen et al., 2016) on the cognitive presence, a thematic analysis technique is employed. This analysis allows some investigation on whether online/ virtual course contents and assessments are as effective and impactful as they were when carried out physically in normal, conventional classroom settings.
Thematic analysis is a technique used for analysing qualitative data that involves reading through a set of data and looking for patterns in the meaning of the data to find themes. Lochmiller (2021) asserts that thematic analysis is one of the most used but often poorly defined approaches in the qualitative research community. This is in tandem with Michelle and Lara (2020) who posit that thematic analysis is a widely used, yet often misunderstood, method of qualitative data analysis. It is a useful and accessible tool for qualitative researchers, but confusion regarding the method’s philosophical underpinnings and imprecision in how it has been described have complicated its use and acceptance among researchers. The method is principally concerned with the identification of patterns which are then reported as researcher-generated themes. It is a method for identifying, analyzing and reporting patterns (themes) within data (Braun & Clarke, 2006). This definition, according to Lochmiller (2021) presumes that an analyst produces a generalized understanding of coded data based on the recurring application of codes and the patterns associated with those codes. The more frequently codes appear in the data set, the more likely it is that the analyst will state that code as the basis of a theme.
Research Questions
This paper aims to look into the experiences of a group of language programme learners with regards to their cognitive presence in terms of course contents and assessments while engaging in online learning. The following are the research questions:
What were the students’ perceptions pertaining to the course contents in their online learning experience?
What were the students’ perceptions pertaining to the course assessments in their online learning experience?
Research Methods
To investigate the language learners’ online learning experiences in terms of the course contents and assessments in LG120 programme, this study employed a qualitative method. A survey form was used in collecting the data. 256 responses were received from a total number of 320 respondents who were students of LG120 programme across five semesters. The respondents answered the end-of-semester open-ended questionnaire consisting of four constructs – content, assessment, lecturers’ delivery and peer interaction. The responses were later coded and subsequently the themes were determined using a six-stage thematic analysis established by Braun and Clarke (2006).
The first stage involved in the study was examining and getting familiar with the data from the responses obtained to be categorised into the cognitive constructs- course contents and course assessments. Any repetitive submission from any of the respondents was eliminated.
The second stage was generating the initial codes. The responses were categorised into positive (strength), negative (weakness), neutral (+-) and not applicable (NA) codes. To do this, any recurring keywords in the responses were highlighted and based on these repeated keywords, the coding themes were developed.
The third stage involved was coding the responses into the first set of developed themes. At this stage, 14 themes were identified for course content construct. The themes were 1) lifelong and independent learning, 2) well organised, 3) informative material, 4) lesson recordings, 5) enhancing knowledge, 6) mentoring, 7) references, 8) expertise, 9) instructor traits, 10) mode fit for the subject matter, 11) platforms used, 12) mode suitability, 13) internet connectivity and 14) time period. Meanwhile, for the course assessment, 14 themes were also discovered which were 1) platform, 2) guidance, 3) instructions, 4) collaboration, 5) area of interest, 6) inspiration, 7) reflection, 8) independence, 9) feedback, 10) usage of platforms, 11) mode suitability, 12) internet connectivity, 13) time period and 14) references.
The fourth stage was reviewing and finalising the themes developed earlier. At this stage, the earlier themes were re-grouped and re-labelled. As for the course content, the refined and finalised themes were 1) suitability of content, 2) learning outcome, 3) learning materials, 4) instructions, 5) feedback, 6) usefulness of content, 7) learning platform, 8) learning activity, 9) guidance and 10) learning time in comparison to 14 themes identified previously. On the other hand, for course content, the refined themes were 1) assessment platform, 2) ease of assessment tasks, 3) assessment time, 3) meaningfulness of assessment tasks, 5) usefulness of assessment tasks, 6) feedback, 7) evaluation as well as 8) internet connection.
The fifth stage was matching the finalised themes with online learning design dimensions by Means et al. (2014). At this stage, the themes are grouped according to the suitable dimensions based on the two constructs – course contents and course assessments. Whichever theme recognised to not belong to any suitable dimension is labelled as distinctive and missing and was reported as stand-alone themes.
The sixth and the final stage was writing the report. A descriptive data analysis was employed in reporting the findings of this study.
Findings
Table 1 displays the nine dimensions of online design postulated by Means et al. (2014) which are matched against the finalized themes derived.
Course contents
The findings of the study are reported based on the constructs – course contents and course assessments matched against the online learning design dimensions. The findings for course contents as displayed in Table 2 are further elaborated as follows:
Modality
The first dimension of the online learning design that matches the findings of the study is modality. Modality in the context of course content refers to the learning platform used in delivering the content of the course whether it is carried out fully online, a blend of both online and Face-to-face (F2F) sessions or web enabled F2F.
One respondent stated that:
“All classes are very packed with infos, and suitable for odl”.
This response indicates the respondent believes that the content of the course can be delivered well despite using an online platform. Another respondent was quoted saying:
“Lessons could be more enjoyable if done physical”.
This response shows that the student thinks lessons will be more fun and meaningful if they were carried out physically/F2F.
Based on the responses received, they show that there is a mixture of opinions from the respondents where some find lessons can still be effectively delivered using online platforms while the others believe the lessons will be conveyed more effectively if it is conducted physically.
Pacing
The second dimension that corresponds with the findings is pacing. In reference to course content, pacing refers to how the students navigate through their learning process ranging from self-paced (students accomplish their learning goals or objectives at their own speed) or class-paced (students accomplish their learning goals or objectives at the speed set by the instructor or following the majority) or class-paced with some self-paced elements (a combination of both). Based on the excerpt in Table 2, since the statement does not explicitly display the type of pacing experienced by the respondent, it can be deduced that the pacing type is class-paced since typically the instructor is the one setting the duration for lectures/ teaching and assessment duration and students often are left with less space to decide for themselves. Hypothetically, this respondent reports the class-paced practiced did not effectively hold his/her attention in grasping the content of the subject.
Pedagogy
The third dimension which corresponds to the study findings is pedagogy. Pedagogy refers to the approach or method employed by the educator in delivering the content of any intended and designated course to the learners. Means et al. (2014) classify pedagogy into four categories specifically expository, practice environment, exploratory and collaborative. The two responses as quoted in Table 2 demonstrate the learners’ satisfaction with the approach employed by the lecturer(s) whereby they were provided with learning materials deemed necessary to enhance and consolidate their understanding of the courses learnt. However, other responses indicate the respondents’ discontentment with the practice used as they believed they should have been given notes and recorded lectures to better strengthen their understanding of the courses. The use of learning materials by the instructors is one of the themes constructed mapped against the model. All these responses highlight the use of learning materials by instructors as part of their online teaching pedagogy.
In addition to learning materials, another theme found to match the online learning design dimension: pedagogy is the instructors’ instructions. The following responses are amongst those recorded.
“More exposure/explanation would be good”.
“Good in teaching us how to do citation and references correctly”.
“The lecturer gave detailed explanations for every lessons …”.
Based on the responses above, the respondents were having mixed reactions regarding the instructions provided by the instructors during the online learning sessions.
Instructor role online
Another theme found to match against the dimension is instructor role. This refers to how the instructors conduct and manage the class in the online learning environment. Based on the responses quoted as displayed in Table 2, they demonstrate the respondents’ opinions on the role played by the instructors. Despite online platforms were used instead of the common and conventional f2f method, some respondents still believed that they can still manage their learning due to the effective role played by the instructors while the other commented that the instructor should have provided more guidance.
Source of feedback
Means et al. (2014) postulated that immediate feedback supports the learners’ online learning experiences. The feedback can be in the form of automated responses set and made available on whichever platforms used by the instructor once the students have accomplished the learning activities or have accomplished a certain milestone as laid out in the course plan. The feedback could also be the instructor’s personal, genuine feedback. Based on the finding of the study, one respondent was recorded stating that:
“Easy to understand as the lecturer provided us with notes, slides, consultation time, and exercises”.
The respondent’s statement corresponds to what Means et al. (2014) claim that online learning environments are designed with the expectation that the instructor will be responsible in providing feedback on students’ online work. The availability of feedback provided by the instructor makes it easy for him/her to comprehend and navigate the content of the subject learnt.
Independent/ stand-alone themes under course contents
In addition to the above discussed themes and dimensions, there are a few other themes created under the course content construct which were deemed unfit to be categorised into any of the above dimensions. The first stand-alone theme deemed unfit is suitability of content. This theme refers to how the respondents perceive the appropriateness of the contents of the courses they undergo. Below are among the respondents’ statements regarding this theme.
“Learning content is tailored to the needs of LG students”.
“The individual project was a great idea for students to reach out on their own”!
“The course itself is very informative as it helps increases my knowledge on what and how to overcome issues at my workplace professionally”.
Based on the above excerpts, the respondents perceive the course contents offered in the LG120 programme are suitable and fulfil their needs as language learners as well as are beneficial in equipping them with necessary knowledge for future workplace use. Yet, there are also some negative responses obtained demonstrating the respondents’ dissatisfaction with regard to the contents of the courses that they took. Among the responses are as follows:
“The informations and content were too lengthy”.
“The content was hard to understand in one go. There was no textbook due to PKP”
“... there were no teaching on digital proofreading which was the highly expected session from us”.
Another theme deemed unfit to be included in any of the dimensions in the chosen model is learning outcome. Learning outcome refers to specific knowledge, skills and/or expertise that students should be able to grasp once they have completed the courses taken. The following are among the responses recorded:
“The content was very structured and insightful, thus aiding in further understanding of the subject”.
“I’ve become more confident to talk in front of my classmates”.
“I managed to find out my creativity level, which was average, but learning this course truly helped”.
“The content helped me a lot in doing the assessments”.
The final theme which is considered to be unfit and is appropriate to be treated as a stand-alone is usefulness of content. This theme refers to how the respondents perceive the practicality and effectiveness of the courses taken in assisting them academically or personally.
“This course contains new interesting knowledge that motivates me to discover more about the content that are given”
Table 3 above depicts the excerpts obtained from the respondents. These findings are further explained below.
Assessments
Modality
The first dimension that corresponds to the findings under the course assessment construct is modality which refers to the online platforms utilized in carrying out assessment for the courses taken. Based on the responses obtained, it can be deduced that the respondents have mixed perceptions regarding the chosen online platforms used. Few believe the chosen platforms were suitable in carrying out the assessments while few others deemed the platforms used were inconvenient to them due to the features of the platforms themselves alongside having Internet connectivity issue.
Pacing
The second dimension identified is pacing. Pacing in relation to course assessment refers to how the students complete any assessments assigned online. From the excerpts in Table 3, generally, it can be concluded that the students had a challenging time in completing their assessments virtually. Considering that ODL allows them to set their own learning pace, it seems that most of them could not cope up with the demands of the assessments due to few factors such as the amount of time given to complete the assessments as well as too many assessments share similar deadlines, leaving the students to feel overwhelmed and eventually affect their self-motivation.
Role of online assessment
The third dimension which corresponds to the findings is role of online assessment. Online assessment is meant to provide a measure of student performance and to determine how much has been accomplished by both students and instructors. The online assessment too serves as an input to grade along as a measuring method to identify students at risk of failure and to gauge how much to scaffold the students if the assessment scores are to be considered as the reflection of the students’ comprehension and mastery of the course learnt. One of the responses obtained is:
“It'll be nice if madam could discuss with us on what did we do wrong with our first assessment, so that we could prevent from repeating the same mistakes when we do our next assignment”.
The above excerpt displays the assessment assigned serves as a tool to determine the student’s milestone in comprehending their course content that is reflected through their test/ assessment scores. Simultaneously, this also serves as feedback for the instructor to enhance or improvise his/her approach in teaching. Based on the above quote, hypothetically, it can be inferred that the respondent did not have any unpleasant experience in completing the online assessment. The problem was because he/ she was not provided with necessary feedback that can assist him/ her to prepare for the other assignment. Another response quoted is:
“I don’t fully agree with assessment that requires a certain amount of likes and views as part of their marks. In my opinion, it is unfair for people who does not have a lot of friends. But it is called a community engagement for a reason, right”.
The above response clearly indicates the respondent’s dissatisfaction on how he/she was evaluated and graded. Another respondent stated:
“The toughest for me because the level of confusion I had during the assessment was insane”.
This response is in tandem with the first two responses quoted earlier in which the respondent was struggling to understand the assessment assigned online perhaps due to several possible factors namely unclear instructions, unsuitable platform used and others.
The following excerpts, nonetheless, are among those positive responses recorded.
“It helped me see my future career better”.
“It was indeed a great experience to get to feel in real workplace environment where the industry people gave their suggestions on our future career paths”.
“so far it has been interesting and bringing me out of my comfort zone because i am a shy person and doesnt really enjoy being the center of attention”.
“Assessments were very straightforward, leading to lesser struggles in completion of said assessments”.
The above excerpts display the respondents’ contentment as to how the assessments designated assist them in giving information about their learning state, their potential grade as well as the possibility of experiencing failure.
Source of feedback
The final dimension identified is source of feedback. All the responses received serve as the source of feedback which can be used for the betterment and improvement specifically the course assessment and the language programme module at large. The responses/ statements quoted under role of online assessment dimension above managed to capture the recommendations made by the students in relation to how they perceived their online assessment. A myriad of suggestions was put forward and this truly proves the recommendations presented reflect their online learning experiences. The excerpts on source of feedback are those which have been shown and explained earlier in their respective themes. They, nevertheless, also highlight and emphasize the feedback given by the respondents based on their experience and perceptions of the assessments that they were required to complete. These feedback are indeed invaluable to the language programme developers and designers in reviewing their existing curriculum to better suit the needs of the language learners and to optimise the effectiveness of the assessments carried out.
Independent/ Stand-alone themes under assessments
One of the themes which is identified to be unsuitable to be categorised into any of the created theme is meaningfulness of assessment tasks. Among the excerpts extracted from the respondents’ responses are:
“I really enjoyed completing the assessment where I am able to reflect what I have learned in the lessons”.
“I personally love how students are given liberty to choose their desired topics for the assessment”.
“I really enjoy doing the assessments with the guidance from my beloved lecturer and my fellow classmates”.
Based on the responses quoted above, the respondents felt the assessments designated were fun and meaningful in assisting them to apprehend the course contents better apart from being able to flaunt their creativity in completing the tasks.
Another independent theme identified is usefulness of assessment tasks. The following are the responses received:
“This subject gives me more exposure and experience on how to deal with real community.”
“An amazing project, I was able to come out with my own project and make it happen”.
“All assessments that I have done is very helpful in order for me to learn and understand more about literature.”
“The assessments are really helpful to increase my understanding of the subject. However, the platform used which is Microsoft Team is inconvenience in terms of long loading time and the format in assessment keeps on shifting when submitted through the platform”.
The respondents perceived the assigned assessments were beneficial and practical in terms of helping them to be exposed to new things and surroundings, giving them the platform to materialize their own ideas and thoughts into project as well as enhancing their comprehension on the subjects learnt.
Conclusions
Based on the findings, five dimensions of online learning were identified to correspond with the themes developed for course contents which are modality, pacing, pedagogy, instructor role and source of feedback and four themes for the assessment construct namely modality, pacing, source of feedback as well as role of online assessment which was further narrowed down to provide student or teacher with information about learning state, input to grade and identify students at risk of failure. Since the findings of the study were derived via TA, it can be concluded that TA is an effective technique in analysing large and rich quantitative data since TA focuses on examining themes or patterns of meanings within data (Braun & Clarke, 2006).
Since TA is a qualitative analysis technique where the researcher decides on any appropriate themes based on recurring codes and patterns, it is deemed a simplistic method of data analysis as compared to the other more well-recognised techniques such as interpretative phenomenological analysis and grounded theory. However, great consideration should be placed as any attempt in employing TA should be done deliberately so as to ensure no single step/ phase is skipped that can hamper the validity of the research findings that employs TA method.
References
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. DOI:
Hussin, H., Bunyarit, F., & Hussein, R. (2009). Instructional design and e-learning: examining learners’ perspective in Malaysian institutions of higher learning. Campus-Wide Information System, 26(1), 4–19. DOI:
Lochmiller, C. R. (2021). Conducting thematic analysis with Qualitative Data. The Qualitative Report, 26(6), 2029-2044. DOI:
Md Yusuf, A. H. S., Maulan, S., Fang Jing, H., Jailani, A. I., & Ayob, N. M. (2021). A thematic analysis of online learning experiences among students of a language programme: Instructors’ delivery and peer interaction. International Journal of Academic Research in Business and Social Sciences, 11(6), 168–175. DOI:
Means, B., Bakia, M., & Murphy, R. (2014). Learning online what research tells us about whether, when and how. Routledge Taylor & Frances. DOI:
Michelle, E. K., & Lara, V. (2020). Thematic analysis of qualitative data: AMEE Guide No. 131. Medical Teacher, 42(8), 846-854. DOI:
Sanchez-Gordon, S., & Luj´an-Mora, S. (2014). MOOCs gone wild. 8th International Technology, Education and Development Conference Proceedings (pp. 1449-1458). https://www.researchgate.net/publication/264047273_MOOCs_Gone_Wild
Singh, V., & Thurman, A. (2019). How many ways can we define online learning? A systematic literature review of definitions of online learning (1988-2018). American Journal of Distance Education 33(4), 289–306. DOI:
Van Oostveen, R., DiGiuseppe, M., Barber, W., Blayone, T., & Childs, E. (2016). New conceptions for digital technology sandboxes: Developing a Fully Online Learning Communities (FOLC) model. In EdMedia+ Innovate Learning (pp. 665-673). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/173015/
Copyright information
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
About this article
Publication Date
25 September 2023
Article Doi
eBook ISBN
978-1-80296-964-1
Publisher
European Publisher
Volume
7
Print ISBN (optional)
-
Edition Number
1st Edition
Pages
1-929
Subjects
Language, education, literature, linguistics
Cite this article as:
Jing, H. F., Ayob, N. M., Maulan, S., Jailani, A. I., & Md Yusuf, A. H. S. (2023). A Thematic Analysis of Language Learners’ Online Learning Experiences: Contents and Assessments. In M. Rahim, A. A. Ab Aziz, I. Saja @ Mearaj, N. A. Kamarudin, O. L. Chong, N. Zaini, A. Bidin, N. Mohamad Ayob, Z. Mohd Sulaiman, Y. S. Chan, & N. H. M. Saad (Eds.), Embracing Change: Emancipating the Landscape of Research in Linguistic, Language and Literature, vol 7. European Proceedings of Educational Sciences (pp. 816-828). European Publisher. https://doi.org/10.15405/epes.23097.73