The Development Of Critical Thinking Skills In High School – A Case Study

Abstract

The Teaching and Learning International Survey (TALIS), conducted by the OECD in 2013, shows that teachers in Romania believe that they are the best-trained teachers in the 34 countries participating in this study. Thus, 99.8% of Romanian teachers feel well or very well prepared in terms of the content of the subject taught, and 98.5% of teachers feel good or very well prepared in terms of pedagogy and instruction of the subject taught. For comparison, only 72% of Finnish teachers (where students have excellent results) declare themselves well or very well trained in the content of the subject taught, and 64% declare themselves well or very well trained in the pedagogy of the subject given. In contrast, unlike in Romania, students from Japan and Finland ranked second and fifth in the last international PISA ranking. This paper describes a study to investigate the impact of an integrated curriculum development on a third-year course in a vocational high school. The course focuses on developing critical thinking skills, in part by using contextualised complex problems. Upon conclusion of the study and analysis of the data, we observed significant gain in those skills measured with a standard instrument: Watson Glaser Critical Thinking Appraisal (WGCTA).

Keywords: Modular instructioncritical thinking skillsWGCTA

Introduction

For everyday activities, we take a certain amount of trust, and this saves us from having to recheck every detail. We have to decide on how much information is really required and what level of doubt is acceptable for each new circumstance. Similarly, critical thinking involves: identifying correctly when we need to gain more information; selecting effectively the right type and level of information for the purpose.

The term, considered to be a very general one, has been described in several variants: either “critical reasoning” (“critique”), referring to the development of reason for certain beliefs, the evaluation of actions by common logical means, or “critical argumentation”, which implies identifying and evaluating arguments in everyday life.

Lipset (1995) says this competence falls into the hands of the modern citizen: decision-making capacity in democratic societies is a process at the end of which a group reaches consensus through discussions, debates and analysis. The decision-making act should be more than just the accumulation of opinions expressed. Opinions must be confronted in the public sphere, and all participants in the public discourse must carefully listen to each other’s arguments. To make authentic democratic decisions, no group should be excluded.

In the educational field, Dewey (1910) proposes the term “reflexive thinking”, in his book How We Think, as “a number of features that distinguish the superior use of the rational faculty of men from its minimal functioning and routine”. The author uses the term “reflexive thinking” to describe this concept as an active, careful and persuasive consideration of an opinion or any form of knowledge in the light of the evidence they support and the conclusions they wish to base.

Lewis (2005), in his book The Power of Critical Thinking, believes that “critical thinking is systematic because it involves distinct procedures and methods. It requires clear evaluations and formulations as it is used both in evaluating your existing opinions (yours or others) and in designing new ones. It operates according to rational standards in that they are judged from the perspective of how they are reasonably well-founded. Critical thinking implies, of course, logic. Logic is the study of good reasoning or good inferences and of the rules that govern them. Critical thinking is, however, more comprehensive than logic, as it implies not only logic but also the truth or falsity of statements, the evaluation of arguments and evidence, the use of analysis and investigation, the application of more competencies that help us decide what is worth believing or doing”.

The Teaching and Learning International Survey (TALIS), conducted by the OECD in 2013, shows that teachers in Romania believe that they are the best-trained teachers in the 34 countries participating in this study. Thus, 99.8% of Romanian teachers feel well or very well prepared in terms of the content of the subject taught, and 98.5% of teachers feel good or very well prepared in terms of pedagogy and instruction of the subject taught. For comparison, only 72% of Finnish teachers (where students have excellent results) declare themselves well or very well trained in the content of the subject taught, and 64% declare themselves well or very well trained in the pedagogy of the subject given. In contrast, unlike in Romania, students from Japan and Finland ranked second and fifth in the last international PISA ranking (OECD, 2014). So, what is the problem in our system of education?

Problem Statement

At the end of 2017, Kitchen, Fordham, Henderson, Looney and Maghnouj (2017) presented the PISA 2015 test results, which reconfirmed the serious situation in Romania. Thus, the results of the Romanian students show a slight improvement compared to PISA in 2012, but they are still worrying – almost a quarter of the Romanian students had very poor performance in all three tests (reading, mathematics and sciences), that is, they are illiterate in those fields.

The quality of education in a country cannot exceed the quality of teaching, writes Andreas Schleicher, head of the OECD, director for Education and Skills. But in Romania, and this is a paradox, the best teachers develop the weakest results for students. The question that the educational system must ask is why this has happened. If we want students to leave school prepared for adulthood, we need to make sure they have experienced and mastered the skills they will need in a context that accurately reflects the world outside the school walls.

‘Higher-order’ skills, like analysing and synthesising information, are extremely valuable – as are ‘wider’ skills, such as working well in teams, using initiative, problem-solving, critical thinking and creativity. These are the skills that employers are looking for, and these are the skills we need for the 21st century. And if we want our curriculum to teach these skills, our assessments need to focus on them. In Romania, the curricula are not developed around such skills.

In the last two decades, critical thinking has become imperative to integrate students into the workplace. In describing the critical thinking competence, we will operate with the following evaluation criteria:

• Identifying basic ideas, messages of communication

• Recognition of the logical path in a reasoning task

• Controlling and evaluating the quality of arguments

• Recognizing information connections

• Differentiation of relevant and irrelevant information

• Examining the credibility of messages

• Considering and evaluating action variants

• The issue of value judgments

• Formulating opinions based on the argument

The benefits of practicing and developing logical abilities (critical thinking skills) are localised in at least three areas: persuasion, knowledge and co-operation. The logical abilities to accept a belief based on solid arguments or constraining evidence can protect the subjects from the collateral effects of persuasion (commercials, media manipulation, political promises etc.). The practice of logical abilities can also lead to the expansion of knowledge through reasoning, in the sense of inferring new information (conclusions) from their previous knowledge (premises), and not in the sense of dependence on vague generalisations or slogans, habits and stereotypes of thought. Benefits due to logical abilities are also manifested in public life when they teach people to enter into cooperative or subordination relationships based on shared goals.

The module “Learning for tomorrow”

The module “Learning for Tomorrow” has as a great argument, a feature that develops logical and critical thinking; cultivates respect and self-respect, in parallel with the acceptance of pluralism under its many aspects; creates information management skills at high standards and communication skills; creates opportunities for approach and collaboration (teacher-student, student-student, student-teacher), thus developing social skills; customises the act of learning by giving the student the possibility to perceive, organize and represent the informative material according to his/her dominant intelligence (spatial/visual, logical); capitalises on the acquired competencies for enhancing performance in learning, thus ensuring the harmonious formation of the student’s personality; analyses the processes and phenomena in a global and sequential way, thus helping to discover mutual ties; creates the learning situations necessary to acquire transferable key competencies tailored to the needs of society; familiarises with the diversity of knowledge areas and the purely disciplinary approaches; helps students to discover their affinities, talents, aspirations; practically prepares students to capitalise on the diversity of learning experiences in personal, educational and career choices.

This module aims to develop a rationale for the reorientation of education for sustainable development and encourages reflection on the challenges involved. It is an introductory preparatory module that is necessary for the cross-curricular modules, generally following the key concepts of sustainable development. It is imperative to go through the initial module and assess the stage of development of critical thinking skills acquired by students. The “Learning for Tomorrow” module was designed using the experiential learning methods developed by Kolb (1984): experience, processing the experience, generalising, applying (table 01 ).

See Full Size >

Research Questions

Transversal skills that students need in order to pursue a college degree or build a career (e.g. solving problems, communication skills, time management, teamwork etc.) is a pressing issue of education worldwide, but especially in the Romanian system. One of the most common criticisms aimed at young people new to the adult workplace is that they lack initiative. They cannot solve problems for themselves and constantly need to be told what to do. Therefore, is the school institution charged with this responsibility? How do teachers develop autonomy in thinking or develop critical student thinking if learning tasks only include elements that keep memorizing events or steps to solve problems, and if, when evaluating, students are only asked to reproduce them? Is it important for teachers to take these aspects into account when designing students’ learning experiences? The research focuses on three questions:

What is the approach to critical thinking competence?

Is interdisciplinary teaching a solution for the development of critical thinking skills in students?

Is modular delivery an effective strategy for developing critical thinking skills in students?

Purpose of the Study

The purpose of this research is to restructure the perspectives and practice of pre-university school space by considering the need to develop critical thinking skills.

In order to achieve our purpose, we have used the following research hypotheses:

Hypothesis 1.1 Subjects in the economic experimental group 1 who will follow the module “Learning for tomorrow” activities will have statistically significant higher results in the Watson-Glaser Critical Thinking posttest than in the pretest.

Hypothesis 1.2. Subjects in the economic experimental group 1, who will follow the module “Learning for tomorrow” activities, will have significantly more statistically significant results in the Watson-Glaser Critical Thinking posttest than in the pretest.

Research Methods

The Watson-Glaser Critical Thinking Appraisal has been used to compare students on entry and exit program, make comparisons between different levels and investigate correlations between critical thinking and variables. All test items include problems and arguments based on situations experienced in the daily workplace, classrooms, social media and others. The tool has five subsets designed to evaluate different elements of critical thinking, comprise deduction, inference, the relevance of assumptions, interpretation and evaluation of arguments.

In our study, we included 56 high-school teenagers from the same institution, but in different classrooms, at the same level, namely the 9th grade.

In many field contexts, especially when evaluating intervention programs, there is no possibility to distribute participants into groups. This is also the case for our research group, as they are already assigned to classrooms, making it difficult to re-distribute them as other factors may distort the results of the research. So, the entire research group will be organized into 2 groups, an experimental group and a control group. The control group is used to detect the maturation effect. In this case, the experimenter wishes to administer an experimental modification to the already established group, so he cannot manipulate any of the elements of the group, and this is not desirable as the experiment aims to analyse its features. The condition is that either of the two groups can be an experimental group. As an advantage, we also have an increase in the level of knowledge in the control group, which is the maturing effect. Practically, through the control group, we determine the maturing effect. As a disadvantage, we can remind that external factors influencing the outcome may arise, the possible differences between the two classrooms, between the two teachers, the parents of the children etc. This design is very useful in real situations when one can reasonably assume that the two groups are similar.

So, after performing these activities, the experimental group consisting of 28 students (same as the control group, gender balanced) participated in 10 learning experiences of 50 minutes each, all described in the module “Learning for tomorrow”. It is worth mentioning that, in order to eliminate any risk of disturbing the research data, the collection of these data was dropped according to the type of subjects. To analyse the research data, in both the pretest and test phases, the coefficients of internal consistency for the validity of the research were reviewed. The research tool used predominantly for the Critical Thinking Competence Questionnaire was directly offered, and the time spent solving the items took 45 minutes.

Data from student research were processed using the SPSS (Statistical Package for Social Sciences), version 2.0. Therefore, for the two groups of subjects, correlation tests for two independent groups (t-test for independent samples), a correlation test and other statistical tests were used to verify the fulfilment of objectives and statistical assumptions. It is important to note that these actions were carried out by independent researchers.

The Mann-Whitney test (or Wilcoxon-Mann-Whitney) is sometimes used in clinical experiments to compare the effectiveness of two treatments. It is an alternative to the t-test if the data does not have a normal distribution. While the t test is relevant for the population average, Mann-Whitney is commonly used for population medians. The Mann-Whitney test is used as an alternative to the t-test when the data does not have a normal distribution. The test can detect differences in form and spread, as well as the median. The Wilcoxon test is a non-parametric bivariate test used to identify the statistical significance of identified differences for variables derived from dependent samples (repeated measurements or measured variables of the same respondents) measured by ordinate scales, regardless of the type of distribution. ANOVA test repeated measurements. Despite the name, it is also a statistical test used to determine the significance of the difference in the difference between three or more samples (dependent or independent) measured on a proportional scale.

Findings

We analyse the results obtained in the post-experimental group, after verifying distribution normality, using the graphical (subjective) and statistical (Kolmogorov-Smirnov and Shapiro-Wilk) method. In the Table of Critical Thinking and posttest group of Tests of Normality (table 01 ), we have the result obtained in the Kolmogorov-Smirnov test (28) = 0.217, p = 0.001 and the result in the Shapiro-Wilk test (28) = 0.846, p = 0.001. As the results in both tests are statistically insignificant, it follows that the variable is not normally distributed, which will be followed by the Mann-Whitney U test. The same results are obtained with the posttest test group, Shapiro-Wilk test score = 0 .981, p = 0.005.

Table 02 Results Critical Thinking Experimental Group Control Group - Mann-Whitney U Test Statistics present the results of Mann-Whitney U test, Wilcoxon W test values, U-value transformation in the Z-score and associated significance threshold. From this table, interest is the Z value and the significance threshold (sig). As can be seen, Z = -6.010, p = 0.001, therefore there are significant differences between the two groups in terms of the Watson-Glaser Critical Thinking Appraisal test.

The results show that the subjects in the experimental group have a higher ranking than those in the control group (41.59 vs. 15.41), it shows that the teenagers in the experimental group who went through the program mode “Learning for tomorrow” get better results on the Watson-Glaser Critical Thinking test compared to subjects in the control group who have not gone through the experimental module.

Calculating the size of r using the formula, we obtained √ (`) (6.010x 6.010) / 56 = 0.645. Given the size effect r = 0.645, according to Cohen’s criteria, it has resulted that the effect of the experimental module on the subjects in the experimental group was strong, in terms of their Watson-Glaser Critical Thinking performance. In presenting the results, we will calculate the median, this being more appropriate for nonparametric tests.

For the second hypothesis, we tested the normality of data distribution and the pretest results, and critical thinking competence data were evaluated using the Shapiro-Wilk statistical and graphical test. We note that the result in the K-S test = 0.097, p = 0.020 and the result in the S-W test is 0.960, p = 0.034. Data were normally distributed. Since the data analysed in the posttest (described for the previous hypothesis testing) were not normally distributed and the sample had low volume, we applied the Wilcoxon test (nonparametric test).

The Critical Thinking Table 03 presents the results of the comparison test. As S = - 4.625, p = 0.0001, there are significant differences between the critical thinking measured before the intervention and the one measured after the intervention. To see the meaning of the difference, we analyse the values in the Sum of Ranks column, from the Critical Thinking Pretest and Posttest Experimental Ranks group, and we will report to the highest rank. In our case, the high value is 406.00 and corresponds to positive ranks, i.e. situations where the ranks of the posttest are higher than the pretest (after and before the ranks). We also calculate the effect size (applying the formula used in previous situations) and the median values for each of the two pair variables √ (`) (-4.625x-4.625) / 28 = 0.76.

In order to check whether Watson-Glaser Critical Thinking test results were influenced by the experimental module, we applied the Mann-Whitney U test. The results show a statistically significant difference between groups (U = 25.50, Z = -6.010, p = 0, 001), the subjects in the experimental group achieving significantly higher results (Mdn 144.00) compared to the teenagers in the control group (95,000). The effect shown by the data above indicates that r = 0.645, which means, according to Cohen's criteria, that the experimental module is a powerful one and has influenced the subjects’ performance in the Watson-Glaser Critical Thinking test, and therefore they have developed critical thinking skills by participating at the module “Learning for tomorrow” activities.

Regarding the second hypothesis, about the competence of the subjects to think critically, we have found that it is more developed after the intervention compared to the previous situation. The described results indicate that the experimental intervention activities developed in the module “Learning for tomorrow” had a significant effect, resulting in significant differences between the posttest and pretest variables, z = -4.625b, p = .001, the critical thinking of the subjects being more developed after the intervention. The effect size is r = 0.76, which highlights a strong effect of the experimental intervention on the development of critical thinking competence measured using the Watson-Glaser Critical Thinking test, which means we reject the null hypothesis and accept the one we discussed.

Major curriculum revisions should not be made based solely on the results of the critical thinking tests used in this study. It is necessary a large group. Group means should be evaluated and compared to national standards for several years to get trended aggregate data before major decisions are made.

A longitudinal study should be conducted to evaluate which critical thinking teaching strategies show significant improvements in teenagers over a specified period of time.

Conclusion

As can be seen, both statistical assumptions have been confirmed. If we refer to the hypothesis regarding the comparison of the control group with the experimental one, from the perspective of the development of critical thinking skills, as a result of pupils’ participation in the activities of the “Learning for tomorrow” module, the effect obtained was 0.645, a sufficiently large coefficient, to Cohen. The results for the second hypothesis comparing the results obtained by the experimental group before and after the experimental activities also confirm that the development of such learning experiences in school can significantly influence the increase of the 21st-century competencies in students (critical thinking, in this case).

It is important to note that the teacher who teaches activities for the module “Learning for Tomorrow” has been trained for 2 years to develop these activities with the target group. Looking at the results, we can deduce some considerations and recommendations:

- teacher training for the development of critical thinking skills should be part of their initial training (at present in Romania the development of these competencies among teachers is a need for their development, but not covered in official government programs);

- the interest of teachers in the development of these competencies to students is very weak, because the national tests are not focused on these competencies, they are more focus on memorisation, which is very easy to measure;

- teenagers’ motivation for learning could be stimulated with the help of the proposed activities (in Romania, the student motivation for learning is the weakest in the world, according to PISA 2013 results).

As a conclusion, we can say that it is absolutely necessary, in a world in which it is not known how the future students will look, to develop competent students transferable to any field. We must learn to learn in contexts that allow students to celebrate new skills to ensure that learning is seen as relevant. The most successful teachers realize that teaching is not a joke and the role of them is to remember students the magic of learning, the process of this and make it matter. A perfect learning cycle stimulates them to further their whole development, as a part of the cognitive but also emotional part, in order to use their learning in a real adult world. School activities are like a boat and students, and together with the teacher (together, not only) are the ones who roam to reach the shore on an unknown island. None (the teacher or pupils) of them knows what is waiting for them, but they have to be prepared for anything, so during this journey, they have to train continuously to deal with the unspoken challenges.

References

1. Dewey, J. (1910). How we think. Chicago: D. C Heath & Co Publishers.
2. Kitchen, H., Fordham, E., Henderson, K., Looney, A., & Maghnouj, S. (2017). OECD reviews of evaluation and assessment in education: Romania 2017. Paris: OECD. Retrieved from https://www.unicef.org/romania/OECD_Reviews_en_pt_web.pdf
3. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development (Vol. 1). Englewood Cliffs, NJ: Prentice-Hall.
4. Lewis, V. (2005). The power of critical thinking, New York: Oxford University Press.
5. Lipset, S. M. (Ed.). (1995). The encyclopedia of democracy. Washington, DC: Congressional Quarterly.
6. OECD. (2014). Raport național – Analiza mediului educațional din România – TALIS 2013. [Centrul Naţional de Evaluare şi Examinare]. Retrieved from http://rocnee.eu/sites/default/files/2017-11/Raport%20national%20TALIS%202013%20fa.pdf