Content Validity of the School-Based Star-CBT Module: Thoughts From School Counsellors
Abstract
Lacking structured modules creates gaps in managing mental health problems among adolescents in Malaysia. Hence, a new Cognitive Behavior Therapy (CBT) module called the school-based Shine Through Any Roadblocks (STAR-CBT) intervention has been introduced. This study extends previous work which focused on engagement with school counsellors. The study aims to bridge the gap between feasibility and practicality of the school-based STAR-CBT module through validation and contextualization by school counselors in Malaysia. The study participants were 20 secondary school counselors (N=20) from Klang Valley, Malaysia who were recruited using a simple random sampling method. The program was delivered virtually by the founder of the module. The validity of the module was assessed both quantitatively and qualitatively via a semi-structured interview, written review, and content validity ratio (CVR) based on Ecological Validity Model (EVM) framework. Overall, the findings revealed that school-based STAR CBT module showed good content validity, fulfilling the consensus requirements across the experts. Quantitatively, the CVR scores supported the validity of the eight sessions in the module while the qualitative review demonstrated the feasibility and practicality of the module. The participants collectively agreed the module should be implemented by school counselors for adolescents with depression whilst agreeing that this module is helpful, practical, and useful. Implementation of a locally validated module would further provide access to evidence-based psychosocial interventions in all Malaysian schools.
Keywords: Adolescent, CBT, depression, school counselor, validation
Introduction
The National Health and Morbidity Survey (NHMS, 2019) indicated that almost 424,000 children were found to have mental health problems in Malaysia. Key findings of this national survey indicated that mental health problems occurred predominantly among girls, children aged between 10-15 years old, living in rural areas, and those from below 40% economic status (or the B40 group). The survey also indicated that 42.9% of school children have peer problems, 15.9% have conduct problems, 2.3% have hyperactive problems and 8.3% have emotional problems. A year earlier, a study among adolescents indicated about 6.2% of adolescents have suicidal ideation. The crucial factors include gender, ethnicity, anxiety or worry, lack of close friends, and poor peer support (Chan et al., 2018). The study also found those who ever had sexual intercourse and had been in a physical fight have a higher tendency to have suicidal ideation (Chan et al., 2018).
Certainly, from 2020 until 2022, the COVID-19 pandemic created remarkable challenges for adolescents and young people. The unplanned and dramatic changes in teaching-learning methods from conventional to online learning, personal conflict (such as negative thoughts of being at home, disrupted daily activities, changing sleep patterns, and excessive internet use), and conflict with other family members may precipitate mental health problems (Amran, 2020; Amran et al., 2021; Amran & Jamaludin, 2021). Various psychological disturbances, from distress (such as grief, adjustment disorder, and acute stress disorder) to psychiatric disorders (such as mood disorder, psychosis, and suicidal tendency) occur following quarantine, social and physical distancing, and school closure (Ramadhan et al., 2020).
Despite such worrying mental health problems among Malaysian adolescents, specific modules to guide counselling and intervention to help this vulnerable group are sparse. Only a few modules are available locally with limited use. For example, the Rational Emotive Education Module on irrational beliefs and stress is suitable only for boarding school students (Mahfar et al., 2019), and the Life Skills Education program which aims to improve the emotional health and self-esteem of adolescents is prepared only for orphanages (Mohammadzadeh et al., 2019). Another Cognitive Behavioural Therapy module developed by local researchers is, however, applicable only to depressed adults (Mukhtar, 2011). The mean age of participants in the eight CBT sessions by Mukhtar (2011) was 40 to 46 years. Recently, an attempt to adapt spiritual aspects to CBT among depressed adults has also been made, but the effectiveness of its module is still being researched (Subhas et al., 2021).
Limited CBT modules targeting adolescents further increase the difficulties faced by mental health professionals in handling adolescents’ psychological disorders and this is exacerbated by the fact that in Malaysia, such professionals are already short in supply (Dahlan et al., 2018; Martadza et al., 2019). Lacking structured modules on which to base interventions creates worrying gaps in managing mental health problems among adolescents in this country. Hence, a new CBT module called the school-based Shine Through Any Roadblocks (STAR- CBT) intervention has been introduced (Saw et al., 2019; Saw et al., 2020). The STAR-CBT module comprises eight group-based sessions that are implemented among secondary school students for two months. It has demonstrated excellent efficacy in lowering the participants' levels of depressive symptoms and automatic negative thoughts (Saw et al., 2020). However, so far, only clinical psychologists have used and tested the module. Giving adolescents access to psychotherapy is extremely difficult given Malaysia's dearth of professional psychologists (Ng, 2013). Giving counsellors the authority to use CBT may be a workable solution to the therapist shortage. According to an estimation by the Malaysian Ministry of Women, Family, and Community Development, the number of available counsellors capable of practising cognitive-behavioral therapy (CBT) in 2021 will be around 9800. In 2023, there was a significant increase in the number of registered counsellors in Malaysia, surpassing initial estimates (Dasar, Pelan Perkhidmatan Kaunseling Dirangka Untuk Tingkat Aksesibiliti Perkhidmatan, 2023). Hence, the study aims to bridge the gap by validating and contextualizing the school-based STAR-CBT program among school counsellors.
Purpose of the Study
The purpose of this study was to validate and contextualize the school-based STAR-CBT program among school counsellors.
Methodology
Participants
The study participants were 20 secondary school counselors (19 females and 1 male) from the Klang Valley, Malaysia. The participants were recruited using a simple random sampling method using a computer-generated sampling procedure. Inclusion criteria were secondary school counselors who were well-versed in the Malay language. No experience using CBT approaches was needed before the intervention program.
Measures
Content validity ratio
The Content Validity Ratio (CVR) was assessed based on the following content; 1) How your activities affect your mood 2) How your thoughts affect your mood 3) How your relationship affects your mood. The participants were asked to rate all sessions using the Likert Scale i.e., either not necessary, useful not essential, or essential.
Focus group interview
The Focus group interview was done as per the Ecological Validity Model (EVM) framework developed by Bernal et al. (1995) which outlines what to adapt in the delivery and content of the module. EVM specified eight dimensions comprising language, persons, metaphors, content, concepts, goals, methods, and context. A semi-structured interview was employed to capture opinions on the feasibility and practicality of the intervention module.
Procedure
Before the implementation of the program, all participants were briefed about the study. They were required to sign an informed consent before the implementation of the program.
The program was scheduled for eight sessions in total (Saw et al., 2019 & 2020). The participants were strongly encouraged to complete all sessions. However, they are allowed to withdraw from the program at any time without any penalties.
The program was delivered via Google Meet and conducted in the Malay language. The program was conducted by the founder of the module and assisted by one clinical psychologist and two psychiatrists who all have more than 10 years of experience using CBT in mental health services.
At the end of the program, the participants were required to rate the STAR-CBT module based on the Content Validity Ratio (CVR) 3 core contents. After that, the participants (also considered Subject Matter Expert-SME) were invited for a focus group interview. The focus group interview was conducted by an independent psychiatrist to avoid bias. All interviews were recorded and transcribed by the research team members for analysis.
Results
Table 1 provides additional information about the participants’ demographic characteristics and educational background. All participants managed to complete the module; hence, zero attrition rate was achieved. Most of the participants were female, married, and aged between 20 to 59 years old. The majority of the participants have a postgraduate level of education.
The validation of the STAR CBT module was based on outputs derived from the SMEs via Content Validity Ratio (CVR) assessment and semi-structured interview i.e. focus group interview. Quantitative analysis was performed manually using the CVR formula, while qualitative outputs were based on the analysis and interpretation of the semi-structured interview from the eight EVM dimensions that served as themes. The following presents the outcomes of this study.
Quantitative results using content validity ratio (CVR)
Table 2 shows the CVR values for the core contents of the module per session, as rated by the participants. As illustrated, a panel of six expert reviewers required a minimum CVR of .42 based on a one-tailed test (.05 significance level) to yield good validity (Lawshe, 1975). All eight core contents were above the minimum CVR values.
Lawshe’s CVR formula was employed to determine the content validity of the gathered data. The formula is as follows: CVR = (ne – N/2)/(N/2), where, CVR = content validity ratio, ne = number of SME panelists indicating ‘essential’, and N = total number of SME panelists.
Qualitative review of the STAR-CBT module
The EVM is a framework for behavioral intervention in treating culturally diverse populations. It guides the development and the documentation processes toward attaining ecological validity. The eight dimensions of EVM include language, persons, metaphors, content, concepts, goals, methods, and context (Domenech Rodríguez et al., 2011).
The summary of the practicality and feasibility of the STAR-CBT module, as recommended by the panel of school counsellors, revolves around the eight EVM dimensions. Rosselló and Bernal (1996) asserted that the use of local language reflects cultural integration. The dimension of in terms of choice of words, concepts, phrases, and syntax used in the module was evaluated as appropriate and comprehensible. This was highlighted by the reviewers,
“…the terms and words can be understood”; “overall okay”; “terms easily understood”; “I feel that the sentences in the module are comprehensible, and however, if it is unclear, counsellors can guide the students easily”
The dimension of postulates that racial/ethnic similarities between therapist and participants in an intervention better shape the therapeutic relationship (Rosselló & Bernal, 1996). The panel of SMEs in the study were representative of the different races in Malaysia reflecting a diverse contribution towards validating the module. Secondly, the familiarity of the school counselors as well as being the most integral individual in the targeted area, i.e., the school serves as a matching. The images, layout, and design of the module were rated as culturally suitable and sensible.
The dimension of refers to the use of symbols and concepts that are collective in a group (Rosselló & Bernal, 1996). Reviewers added that certain part of the module can be in point-form, mind-map, while inserting take home message at the end of each module.
As for the dimension of, the subject’s personal, customs, and cultural values were integral (Rosselló & Bernal, 1996). Two reviewers commented on the application homework as a part of misalignment or rather a barrier for Malaysian adolescents, as stated below;
Students might not do the homework”; “Students might not be able to complete the homework”
“More activities needed to be added”
Another modification made to a dimension that reflects the belief system of the targeted population’s culture is Its ultimate role is to attain consonance with both culture and context using appropriate constructs in the intervention model (Rosselló & Bernal, 1996). Although well-being awareness has been widely spread across the nation, stigma remains an issue. To avoid stigma and peer pressure while acknowledging the efforts in following the intervention, the following reflects a related suggestion;
“…student be given certificate of accomplishment”
refer to the dimension that values the agreement of goals of treatment between the therapist and participants, which must be established in the intervention model (Rosselló & Bernal, 1996). The intervention goals are presented in the module with guided step-by-step sessions. The panel reviewers appreciated all the presented sessions and noted that they could better understand the issues faced by their students through the module. The journey from one to eight sessions was meaningful, with the perception of tapping into reducing the depressive symptoms among the students in a holistic manner. The goal of the intervention module is congruent with both the therapist and the participants.
“…each other the session is helpful and meaningful. Each client is different and the techniques can help holistically; “the strong factor of the module is that it can help the students to change their perception by disputing…and to be more meticulous in pinpointing the student’s belief system.”
A suggestion was made to the dimension related to. This dimension highlights the procedures required for treatment intervention (Rosselló & Bernal, 1996). The panel of expert reviewers felt that the number of sessions was fair for the Malaysian cultural context. However, the suggestion of including a follow-up session after the intervention program was considered beneficial. This is in line with the notion that the length of sessions did not report any significant improvement as mentioned by Mukhtar (2011). Concerning extending the module to a younger age group, the module might need to be shortened to cater to them as mentioned by the reviewers.
Lastly, the dimension of looked into the socio-cultural elements, including treatment barriers. This is reflected in the.
“…this module is suitable to our culture…quite relevant but must relate it to their style, their daily routine”
Although this intervention is feasible for the Malaysian context, better outcomes can be sought by including elements of psychospirituality while keeping the commitment of the dyads in the intervention. The ultimate goal of this research is to ensure the module suits all secondary school students in Malaysia. According to Orellano-Colón et al. (2014), one who is known for or aware of the targeted populations can effectively facilitate the program. Thus, school counsellors are the most suitable candidates to run the intervention. Counsellors with adequate training in the module and supervision may take up the role of conducting the intervention.
The qualitative assessments to determine the content validity of the STAR CBT module were also composed of written reviews from the panel of experts. The review summary encompassed the suitability of the module title, the directions of the module (concise, clear, and complete), the content areas covered (appropriate, and complete), and some comments/suggestions (McKenzie & Clark, 1999).
All participants who underwent the workshop were invited for a focus group review. A semi-structured review was employed for the focus group to capture opinions on the feasibility and the practicality of the STAR CBT intervention module.
The participants provided valuable and helpful opinions for program evaluation purposes. They claimed that they could effectively understand moods, thoughts, and behavior concerning the presentation of depressive symptomatology. Another participant claimed,
“This program has taught effective ways to help the students in a guided manner”
The feasibility of the program in terms of session length, information saturation, as well as execution and practicality of the program as a school-based intervention, was explored from the stance of the participants. One participant expressed, As for execution, the participant suggested the module be presented to the ministry to be implemented as an official practice in all secondary schools. Exposure and module sharing should be done in all secondary schools in every state in Malaysia. All the participants agreed and supported the notion that this program should be run as a school-based program (i.e.,
Conclusion
This study aims to validate and contextualize the STAR-CBT module for use among secondary school adolescents in Malaysia. The study participants i.e. the school counsellors collectively agreed the module is suitable for implementation by the school counsellors for adolescents with depression. The majority of the participants found this module very helpful, practical, and useful for them. Furthermore, they also highlighted that the module needs to be featured as a guide in the school intervention program.
Additionally, the participants stated that the module content is simple, easy, and appropriate. They also commented that the homework is doable, clear, and can be easily understood by the students and counsellors. This factor could be attributed to their educational background. As stated in the demographic profile, most of the participants have a postgraduate level of education and they are familiar with CBT. Hence, understanding this module might not be a problem for them.
In terms of language usage, participants also agreed on the clarity and conciseness of language used in the module. Participants agreed that they did not have any issues with the language used in the module. This could be explained by the fact that the module had undergone a comprehensive validation and translation process by previous researchers (Saw et al., 2019) which contributes to the above-mentioned factors.
Regarding the attrition rate, all the participants were able to complete the module as scheduled. No drop-out was reported. It could be due to the participants being committed and interested in this module. The majority reported that they were excited to participate in this study as it could help and add to their knowledge in dealing with adolescents with depression.
This study also demonstrated some limitations that must be considered which firstly is the small number of participants. Future studies should consider having more participants for richer results. Secondly, it is suggested that future studies include more participants from East Malaysia, i.e.; Sabah and Sarawak to see how the locals from East Malaysia responded to this module as their culture and language are more diverse than in West Malaysia.
To the best of our knowledge, this is among the first studies to validate and contextualize the STAR-CBT module among school counsellors in Malaysia. We believe that the involvement of school counsellors would help in early intervention, especially for adolescents with depression. Their engagement would allow more adolescents to receive and have access to CBT skills and hence reduce the number of depression cases in the future.
Data Availability Statement
Data is available on request from the authors.
Declaration of Conflicts Interests
The authors would like to declare that the research was conducted per research ethical guidelines and report no conflict of interest.
Funding/Ethics
This study has been supported by the University Teknologi MARA MyRA Grant (600-RMC/MYRA 5/3/LESTARI/112/2020). Ethical approval was obtained from the Research Ethical Committee of Universiti Teknologi MARA.
Acknowledgments
The authors would like to express their gratitude and appreciation to Geran Penyelidikan MyRA 2020-2022 and Universiti Teknologi MARA(UiTM) for the support of this research project. Sincere gratitude is also extended to all participants and those who were involved in this study.
References
Amran, M. S. (2020). Psychosocial risk factors associated with mental health of adolescents amidst the COVID-19 pandemic outbreak. In (pp. 0020764020971008): SAGE Publications Sage UK: London, England.
Amran, M. S., Bakar, A. Y. A., & Ifdil, I. (2021). Adolescents’ Anxiety during COVID-19 Pandemic in Malaysia. Acta Counseling and Humanities, 1(2), 99-100.
Amran, M. S., & Jamaludin, K. A. (2021). The Impact of Unplanned School Closures on Adolescent Behavioral Health During the Covid-19 Pandemic in Malaysia. Frontiers in public health, 9.
Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity for outcome research: Issues for the cultural adaptation and development of psychosocial treatments with Hispanics. Journal of Abnormal Child Psychology, 23(1), 67–82.
Chan, Y. Y., Lim, K. H., Teh, C. H., Kee, C. C., Ghazali, S. M., Lim, K. K., Khoo, Y. Y., Tee, E. O., Ahmad, N. A., & Ibrahim, N. (2018). Prevalence and risk factors associated with suicidal ideation among adolescents in Malaysia. International Journal of Adolescent Medicine and Health, 30(3). https://doi.org/10.1515/ijamh-2016-0053
Dahlan, R., Abd Ghani, M. N., Yahaya, R., & Tuan Hadi, T. S. (2018). Child and Adolescent Mental Health Service (CAMHS), Terengganu, Malaysia: milestones so far and the paths to the future. London journal of primary care, 10(4), 113-117.
Dasar, pelan perkhidmatan kaunseling dirangka untuk tingkat aksesibiliti perkhidmatan. (2023, October 9). Retrieved from https://www.bharian.com.my/berita/nasional/2023/10/1162815/dasar-pelan-perkhidmatan-kaunseling-dirangka-untuk-tingkat
Domenech Rodríguez, M. M., Baumann, A. A., & Schwartz, A. L. (2011). Cultural adaptation of an evidence based intervention: From theory to practice in a Latino/a community context. American Journal of Community Psychology, 47(1-2), 170–186.
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575.
Mahfar, M., Noah, S. M., & Senin, A. A. (2019). Development of rational emotive education module for stress intervention of Malaysian boarding school students. Sage Open, 9(2), 2158244019850246.
Martadza, M., Saedon, U. I., Darus, N., Badli, T. S.-Z. T., Ghazalan, S. A., & Yunus, W. M. A. W. M. (2019). Patterns of referral to clinical psychology services in the ministry of health Malaysia. The Malaysian journal of medical sciences: MJMS, 26(6), 111.
McKenzie, J. F., & Clark, J. K. (1999). Establishing content validity: Using qualitative and quantitative steps. American Journal of Health Behavior, 23(4), 311-318.
Ministry of Women, F. a. C. D. (2022). Registered Counselor 2021, Retrieved from https://www.data.gov.my/data/ms_MY/dataset/kaunselor-berdaftar-mengikut-negeri-dan-jantina/resource/ea803a61-2a39-441d-a135-bfee16cc1754
Mohammadzadeh, M., Awang, H., Ismail, S., & Kadir Shahar, H. (2019). Improving emotional health and self-esteem of Malaysian adolescents living in orphanages through Life Skills Education program: A multi-centre randomized control trial. PloS one, 14(12), e0226333.
Mukhtar, F. (2011). Predictors of Group Cognitive Behaviour Therapy outcomes for the treatment of depression in Malaysia. Asian Journal of Psychiatry, 4(2), 125-128.
Ng, W. S. (2013). Counseling and Psychotherapy in Malaysia. In R., Moodley, U. P., Gielen, & R. Wu (Eds.), Handbook of counseling and psychotherapy in an international context. Routledge.
Orellano-Colón, E. M., Varas-Díaz, N., Bernal, G., & Mountain, G. A. (2014). Achieving ecological validity of occupation-based interventions for healthy aging. Physical & Occupational Therapy in Geriatrics, 32(4), 368–380.
Ramadhan, M. H. A., Putri, A. K., Melinda, D., Habibah, U., Fajriyah, U. N., Aini, S., Prananjaya, B. A., & Ikhsan, D. S. (2020). Children's Mental Health in the Time of COVID-19: How Things Stand and the Aftermath. Malaysian Journal of Medical Sciences, 27(5), 196-201. https://doi.org/10.21315/mjms2020.27.5.15
Rosselló, J., & Bernal, G. (1996). Adapting cognitive behavioral and interpersonal treatments for depressed Puerto Rican adolescents. In Hibbs, E. D., & Jensen, P. S. (Ed), Psychosocialtreatments for child and adolescent disorders: Empirically based strategies for clinical practice (pp. 157-185). Washington, DC, US: American Psychological Association.
Saw, J. A., Tam, C. L., & Bonn, G. (2019). Development and validation of a school-based cognitive-behavioural therapy (CBT) intervention for Malaysian high school students with depressive symptoms. Asia Pacific Journal of Counselling and Psychotherapy, 10(2), 171-187.
Saw, J. A., Tam, C. L., Thanzami, V., & Bonn, G. (2020). Contextualized School-Based Cognitive Behavioral Therapy (CBT) Intervention for Malaysian Secondary School Students. Frontiers in psychiatry, 11, 565896.
Subhas, N., Mukhtar, F., & Munawar, K. (2021). Adapting cognitive-behavioral therapy for a Malaysian muslim. Medical Journal of the Islamic Republic of Iran, 35, 28.
Copyright information
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
About this article
Publication Date
26 December 2023
Article Doi
eBook ISBN
978-1-80296-965-8
Publisher
European Publisher
Volume
4
Print ISBN (optional)
-
Edition Number
1st Edition
Pages
1-109
Subjects
Education, educational psychology, pedagogy, positive pedagogy, special education, second language teaching
Cite this article as:
Saw, J. A., Razali, S., Hashim, N. A., & Omar, S. A. (2023). Content Validity of the School-Based Star-CBT Module: Thoughts From School Counsellors. In A. Güneyli, & F. Silman (Eds.), ICEEPSY 2023: Education and Educational Psychology, vol 4. European Proceedings of International Conference on Education and Educational Psychology (pp. 45-54). European Publisher. https://doi.org/10.15405/epiceepsy.23124.4