Semantic Differential Method As A Pedagogical Diagnostics Tool In University Learning Process

Abstract

The study of motivational semantic attitude in students of Saint-Petersburg University of State fire service of EMERCOM of Russia towards the subject of Chemistry was conducted aiming at obtaining the pedagogical diagnostics method that considers students’ emotional perception of the educational process and that makes it possible to promptly identify negative tendencies arising within the process of basic subjects learning. The comparative analysis of the first- and third-year students’ and cadets’ attitude toward the subject of Chemistry was conducted using the developed semantic differential survey. The data obtained in the course of the study was processed by the factor analysis method. As a result, four main factors reflecting the students’ attitude toward the subject of chemistry were determined: Factor 1 - «Interest and Utility» (the attitude to chemistry as a particular field of knowledge), factor 2 - «Personal Attitude» (the attitude to the subject), factor 3 - «Emotional Satisfaction» (emotional satisfaction with chemistry studying), factor 4 - «Intellectual Accessibility» (cognitive component of chemistry studying). The study reveals learners’ different attitudes toward the subject depending on the type of course they take (free tuition cadets and fee-paying students). It is noted, that the lack of motivation for studying in many respects is caused by the learners’ misunderstanding of chemistry relevance to the professional disciplines set. The semantic differential method is proposed to be universal for pedagogical diagnostics and students’ emotional state monitoring in the course of their studies at the university.

Keywords: MotivationPedagogical diagnosticsProcess of educationSemantic differentialThe subject of Chemistry

Introduction

Due to the fair criticism on the qualification and quality of young specialists in recent years, particular pedagogical issues have become of major importance, namely, the question of improving the quality of education, provoking interest towards the future profession and scientific work, self-education. Despite the reforms carried out recently, significant financial support for the system of education, material and technical base upgrade of higher education institutions – yet graduates are noted to lack proper qualification, they show the inability to be competitive at the modern labor market as well as to manage necessary tasks. One of the young specialists’ poor training reasons is said to be insufficient learning motivation and immature professional self-consciousness at the junior years of studying at the university (Demchenko, 2013).

The issues of providing top quality education and its compliance with world standards are the major ones in Russian pedagogical science nowadays (Razinkina et al, 2018). Whereas, both the quality and the quantity of knowledge acquired by students as well as the teachers’ work effectiveness are still determined by the academic progress record. However, the studies conducted proved that academic progress neither always determine the attitude to the subject nor guarantee the quality of the obtained knowledge. Nevertheless, academic progress depends on many factors, namely: financial opportunity, scholarship and education fee, gender, ability to adjust to the university conditions, the desire to affirm in a group, ability to manage free time, etc. The attitude toward the subject and the chosen major should be considered as a combination of cognitive, affective-evaluative and behavioral factors. The content of every component in the course of studies changes regularly, which forms the scope of acquired knowledge and motivation for its obtaining and application (Baskin & Svidzinskaya, 2018; Brown et al., 2015; Chan, 2014; Heng & Karpudewan, 2015; Shipunova, Mureyko, Serkova, Romanenko, & Romanenko, 2016).

It is impossible to determine the forms and methods of learning activity stimulation and set a positive attitude toward the university discipline in young people basing just on monitoring and analysis of their academic progress. It is important to consider the students’ emotional perception of the learning process and its dynamics when finishing one year and entering the next one (Shipunova, 2014). Identification and consideration of individual students’ individual peculiarities is impossible without proper organization of pedagogical diagnostics - such teacher's activity which has to be based on both intuitive and rational aspects and aimed at revealing the abilities and professional preferences of future specialists, whereas, the rational aspect of pedagogical diagnostics implies both qualitative and quantitative indicators application (Baskin & Gutnik, 2013). It is crucial to work out fast and clear diagnostics methods that will allow tracking the factors both causing learning motivation decrease and encouraging the creation of comfortable educational environments.

Of particular importance is the pedagogical diagnostics the basic subjects study with junior students of technical universities. Once entering the university, first-year students are usually determined to get new knowledge, however, when they realize that they have to study a number of basic disciplines: Mathematics, Physics, Chemistry, Foreign Languages – their interest decreases, due to misunderstanding of the subjects’ relevance to their future profession that leads to the decrease in the level of subject material mastering. The means of provoking the learners’ desire to study the subject, increasing their intrinsic motivation for these basic subjects is always under discussion. The methods of motivation encouragement vary according to the individual characteristics of a learner and are determined by particular pedagogical diagnostics results. The diagnostics is conducted via surveys, observational and comparative methods depending on the group of students taken as subjects and specifics of a subject. (Mezentseva, 2017; Kogan, Khalyapina, & Popova, 2017).

Moreover, students’ emotional state track record, as well as their attitude toward the subject and educational process monitoring can be also implemented by means of a survey using psychosomatic methods. These methods make it possible to provide a quantitative and qualitative description of the subjective, emotional attitude of the respondent toward any of the objects or circumstances of his environment and inner world (Kozhevnikova & V’yzhanina, 2016).

It appears that in Russia the psychosemantic methods for studying attitudes toward academic disciplines and academic curriculum creation and correction are not widely applied. On the contrary, the world psycho-pedagogical science these methods are universally used for educational process analysis of such subjects as Chemistry, Mathematics, Biology etc., mainly natural science disciplines (Baskin & Svidzinskaya, 2018).

Two of the most common types of elements used in psychometric attitude inventories or scales include semantic-differential and Likert-type items. These formats consist of a question stem followed by several response options. The Likert-type scale method, a psychometric scale commonly involved in research that employs questionnaires , is most popular in terms of studying attitude to science in general and its particular fields. Science attitude is measured through questionnaires consisting of Likert-scale items, that include a declarative statement followed by several levels of agreement such as “strongly agree/agree/neither agree nor disagree/disagree/strongly disagree”. In particular, five or seven-point Likert scales ranging from 12 to 76 items were developed in order to determine the attitude to the subject of Chemistry depending on the aims and objectives of an experiment (Lovelace & Brickman, 2013; Sumarni, Susilaningsih, & Sutopo, 2018).

The semantic differential method was developed in 1957 under the guidance of the American scientist Osgood for an individual perception of certain phenomena study. The basis of the method is the theory that the evaluated object is important for an individual not only because of its objective content but also due to the personal attitude of a person to an object. Semantic-differential items use a bipolar adjective list that examinees use to select a response option out of a range of values that best matches their view. As a result, unconscious associative links are derived between the subjects in the consciousness.

The semantic differential method is widely used in studies of cognitive consciousness aspects and individual self-awareness. Practically speaking, it is a combination of controlled associations method and scaling procedures. This psychsemantic method encompassing a wide scope of indicators affecting the attitude of a person to an object of research is designed in a way that a respondent cannot predict its final result within the course of the survey. This makes it possible to conduct pedagogical diagnostics and attain an objective picture of the individual’s emotional-semantic attitude to the object. This method makes it possible to characterize the emotional attitude to the subject both qualitatively and quantitatively (Kozhevnikova & V’yzhanina, 2016).

Problem Statement

The pedagogical diagnostics method development that will consider students’ emotional perception of the educational process and make it possible to quickly identify negative tendencies, arising in the course of education, which would contribute to increasing the students’ learning, as well as adjust the educational process forms and select the teaching techniques that would boost the potential of the whole group as well as each individual.

Research Questions

• Use of semantic differential method for assessing the students’ attitude toward the university subjects, in particular, to the subject of Chemistry and chemistry-related disciplines.

• Analysis of different attitude toward the subject of Chemistry and chemistry-related disciplines depending on the form of studying (free tuition – cadets, fee-paying – students).

• Mathematical processing of the data obtained by the semantic differential method using factor analysis in order to identify the factor influence, comparison of their structures for different groups of learners and evaluating the effectiveness of the variables determining differences (or, on contrary, identity) of the structures in these groups.

Purpose of the Study

The research of motivational and students’ semantic attitude toward the subject of Chemistry and chemistry-related disciplines using the semantic differential method as in the course of training first-year students and the possible transformation of this attitude in senior years, when the chemistry course is completed.

Research Methods

The first and third-year students of Saint-Petersburg University of State fire service of EMERCOM of Russia majoring in Fire Safety (cadets and students), Forensic Examination (cadets and students), Mining (students) and System Analysis and Management (cadets) were selected by the authors to become the objects of the present study. 265 people participated in the study, 128 of them were first-year students (55% of respondents were the cadets, 45% were students), 137 people were third-year students (45% cadets, 55% students). The selection of this university was particularly interesting, as the educational process is conducted according to two different systems. On the one hand, there are cadets – future firefighters and rescue officers who are under control of the course officers and have to follow strict discipline rules as well as carry out both educational and service duties. On the other hand, there are students, who do not have these restrictions. Cadets study for free and get a scholarship, whereas the majority of students (77% among the first-year students and 73% among the third-year students) pay for their tuition.

The choice of Chemistry as the subject of the present research was determined by the fact that this discipline serves as the base for the further special engineering training of experts, firefighters and rescuers of top qualification. Moreover, it serves as the base for combustion and explosion processes studying, fire safety of technological processes, fire tactics, forensic methods of research and etc. Students start learning it from their first year, where they master General, Inorganic and Organic Chemistry and end up learning it in the third year with combustion processes mastering. During studying, Chemistry requires a wide range of pedagogical practices: lectures, practical exercises and experimental research in laboratories. The topics covered in the chemistry course are related both to the surrounding environment: ecology, medicine, - and to the future professional activities: fires and their elimination, mine accidents, natural disasters consequences.

The semantic differential method was suggested to assess the motivational and semantic attitude of students to the subject of Chemistry and related disciplines, study the dynamics of this attitude. The selection of this method is determined by its wide application, due to which a number of techniques of its involvement in terms of the subject of Chemistry studying were developed and described. Besides, the method is quite demonstrative, universal, easily-modified when analyzing various situations arising in the educational process. In 2003, Dalgety used a semantic differential format to develop items to measure attitudes toward Chemistry and chemistry self-efficacy (Dalgety, Coll, & Jones, 2003). In 2008, Bauer developed the attitude measurement method toward the subject of Chemistry using 20 pairs of antonyms. The author revealed the attitude transformation dynamics toward the subject of Chemistry and demonstrated that the attitude to the subject depends on learning progress, length of learning and students’ preliminary knowledge. Bauer reported a four-factor solution described as: Interest/Utility; Emotional Satisfaction; Anxiety; Intellectual Accessibility. One single item was independent and described with the term “Fear” (Bauer, 2008). This method became widespread and was used for attitude to Chemistry estimation of college-level students in the Pacific Islands (Brown, et al, 2014). In 2011 Bauer method was refined by Xu and Lewis (2011) by reducing the number of items from its original 20-item form and five latent constructs to eight adjective pairs in two factors: “Emotional satisfaction” аnd “Intellectual accessibility” (Brandriet, Xu, Bretz, & Lewis, 2011; Xu & Lewis, 2011). In 2015 shortened semantic differential method was used in Turkey for analysing high school students’ attitudes toward Chemistry (Kahveci, 2015; Sen, Yilmaz, & Temel, 2016), the method was evaluated to be eligible in such culturally different countries as Australia and Saudi Arabia (Xu, Alhooshani, Southam, & Lewis, 2015).

Since in Saint-Petersburg University of State fire service of EMERCOM of Russia the subject of Chemistry is one of the basic subjects and it serves as a basis for further learning of professional subjects, the authors found it important to evaluate the students’ idea of how significant and useful chemistry knowledge is for its further application in professional field. Survey forms developed by foreign scientists did not reflect the applied nature of chemistry; therefore, a new survey form was developed taking into account the specifics of chemistry studies by future firefighters and rescuers. Moreover, the survey was suggested to be independent of pressure on respondents to obtain the expected results and differentiate the attitude to the subject from the attitude toward the teacher holding the class. Students themselves were involved in the process of selecting antonyms. They were asked to come up with the most suitable definitions, which, from their point of view could characterize the subject of Chemistry. As a result, a form containing 20 antonyms was compiled. Basing on the pair of opposing emotional evaluation characteristics, respondents had to measure their attitude to the subject on the 7-point scale. The value on the scale changed from +3 (absolute match) to -3 (minimum match) through zero. Whereas antonyms characterized the object from positive to negative and vice versa, this excluded the positional style of answers when filling in the form.

The forms were printed on separate sheets, which were filled in by respondents manually without asking any personal identity information. The time allocated for filling in the form was 5-10 minutes.

The study was conducted with the first-year students after they completed the General Chemistry course that is 50% of the full chemistry course; and with the third-year students after they completed the Chemistry of combustion processes course, which is an individual discipline for fire safety students as well as a part of the Theory of combustion and explosion course for students from other majors.

The results of the survey were processed by factor analysis. This statistical method combines qualitative and structural-quantitative analysis and allows us to identify internal object characteristics that are not directly observed. Factor analysis is a multidimensional statistical method intended to explain interconnections between variables. Factor analysis helps identify survey items that show similar response patterns. A group of similar items is called a factor. Each factor is interpreted as a reason for the joint variability of the group of variables that have maximum module factor loads in this factor. The factors identified as a result of the analysis can be used for complex phenomena explanation when managing various scientific-pedagogical tasks, particularly, in a comparative pedagogical experiment. (Lebedeva, 2003, 2015).

The internal structure of data was assessed using the principal components method. The structure was explored using Varimax rotation. Data were analyzed in terms of suitability for factor analysis. Both the Kaiser–Meyer–Olkin (KMO) and Bartlett’s test suggested that factor analysis was feasible (Table 01 ) (Sen, Yilmaz, & Temel, 2016).

Findings

The factor analysis results are shown in Table 02 . An initial four-factor solution, based on eigenvalues >1, was suggested. These 4 factors yielded 57.0-65.0% of the total variance which was an acceptable value for the explained variance. The correlation between factors lies within -0.28 – 0.38 which proofs the independence between factors.

The threshold for significant factor loadings of the items for the first-year cadets and students was accepted as 0.50 a commonly used threshold value that indicates strong loading. Factor loads on some of the variables for the third-year students appeared to be less than 0.50 as they were used to assess the attitude toward chemical disciplines changing dynamics.

The positive factor loading value means that the respondent’s attitude to the subject tends to match the first adjective, while the negative corresponds to the second one.

Considering the interconnections structure in each factor the authors described them as follows: factor 1 - «Interest and Utility», factor 2 - «Personal Attitude», factor 3 - «Emotional Satisfaction», factor 4 - «Intellectual Accessibility». The labels of the factors attempt to capture the theme represented by the antonyms loading on each factor. Reflecting the attitude toward Сhemistry, factors 1 and 2 combine, first of all, behavioral and partially emotional and cognitive components, factor 3 is an affective-evaluative component, and factor 4 is the cognitive component of the attitude.

The data given in Table 02 proves that students differentiate the attitude to Chemistry as the field of knowledge (factor 1) from the attitude to Chemistry as the subject they study (factor 2). Junior learners, particularly students, show very high factor loadings in “Developing”, “In demand”, “Interesting” items. Although students, compared to cadets, who begin their studies with introductory practice, hardly understand what is the sense of learning Chemistry (low indicators “Widespread”, “Dominant”), both groups of respondents are determined to study. Despite problems and difficulties arising when learning the subject (high “Complicated” indicator), cadets note that the course lacks logic (high “Chaotic” indicator), while students characterize Chemistry as very tense, nevertheless, the emotional satisfaction from the educational process indicator for both groups is very high (factor 3). The low value of “Pleasant” item for students correlates well with the high factor loading of “Tense” item for the same group of respondents, whereas the low rates of “Friendly” item for cadets might be explained by the fact that academic debts for this group lead to disciplinary sanctions and restrictions. However, some negative tendencies in the attitude toward the subject can be identified already in the first year. The subject Chemistry seems to be too classical, dull, unvaried, theoretical for learners and the acquired knowledge does not line up into the logical system. At the same time, cadets note that intellectual involvement in the course of education is quite low.

The attitude to the subject of Chemistry and related disciplines changes accordingly in the process of education. Studying special subjects cadets and students realize the importance of Chemistry as the science for their future profession (high factor loadings of “In demand” and “Beneficial”, students transferred the item “Practical” to the characteristics, as a professional field), the subject is becoming less difficult and more intellectually involving, experimental studies in a laboratory reduce factor loadings in “Classical” and “Dull” items, however, on the whole, the data of the analysis shows the increase of negative tendencies in the process of studying. Young people, particularly students, note the loss of interest in Chemistry (low values in “Developing”, “Interesting”, “Colorful” items, allocation of the latter one into the emotional perception field). The presentation of the study material at the lessons still lacks logic for learners (high factor loading “Chaotic”) and seems to be quite monotonous.

Of particular concern is the factor of emotional satisfaction with the learning process, which took the second place for all respondent groups according to its importance. While cadets express just disappointment with the chemistry studies, “Emotional Satisfaction” factor for students shows deep emotional disappointment and dissatisfaction. The decrease in the factor loading in the “Tense” item for students and its increase for cadets can be explained by the fact that students are more free, they do not undergo pressure from the course officers and are not afraid to get bad grades; whereas for cadets, on the contrary, this might lead to additional duties, penalties, bans on access to the city, which is extremely unpleasant for the third-year students. Despite such a drastic decline in the indictors of emotional dissatisfaction from Chemistry and chemistry-related subjects, the academic progress rate for the third-year students is higher than for the first-year ones. Thus, the average score in Chemistry, depending on the chosen major in the first year for cadets is 3.3 – 4.3; for students is 3.6 – 4.0. As for the third-year learners, cadets increased their scores in Chemistry of combustion processes up to 3.8 – 4.5 and students up to 3.3 – 4.0 which again indicated the ambiguity of the motivation for studying and academic performance indicators correlation.

Negative attitude to a subject cannot develop the motivation to learn. Considering the motivation as a complex multidimensional construct consisting of three components: value (purpose for engaging in a learning task), expectancy (learning beliefs and self-efficiency) and emotional beliefs, it should be recognized that emotional dissatisfaction produces an undermining effect on all three components (Chan, 2014). In this regard, the task of a teacher is to change the attitude to the subject at the right time, make it more intellectually-accessible, convince future firefighters, rescuers, experts that it is impossible to become a specialist capable of evaluating the danger of an emergency, the dynamics of its development and give quick and precise recommendations for its prevention or elimination without a good command of basic mathematic laws, physics and chemistry.

Conclusion

• The quality of education improvement and, consequently, the formation of the top-class specialist should be based on their clear understanding of the acquired knowledge potential practical application in the future. Teaching activity in this regard should aim at learning motivation development considering particular peculiarities of study groups as well as individuals.

• Effective education is not possible without students’ emotional state monitoring. The semantic differential method makes it possible to provide a qualitative and quantitative evaluation of the respondent’s attitude to the process of studying, to a particular subject and the dynamics of his attitude changing during the development of the subject.

• The semantic differential method in combination with mathematical statistics method, particularly factor analysis can be recommended for pedagogical diagnostics in order to reveal negative trends that arise in the course of education, correct the learning process forms and methods, identify and consider individual peculiarities of the learners and eliminate negative tendencies leading to the learning motivation decrease.

References

1. Baskin, Y. G., & Gutnik, I. Y. (2013). The possibilities of pedagogical diagnostics in ensuring education quality. Problems of technosphere risk management, 1, 158-162.
2. Baskin,Y.G., & Svidzinskaya, G.B. (2018). Application of semantic differential method in psycho - pedagogical studies at the universities of EMERCOM of Russia. Bulletin of St. Petersburg State University Fire Service of EMERCOM of Russia, 1, 93-100.
3. Bauer, C. F. (2008). Attitude toward chemistry: A semantic differential instrument for assessing curriculum impacts. Journal of Chemical Education, 85(10), 1440- 1445. doi.org/10.1021/ed085p1440
4. Brandriet, A. R., Xu, X., Bretz, S. L., & Lewis, J. E. (2011). Diagnosing changes in attitude in first-year college chemistry students with a shortened version of Bauer's semantic differential. Chemistry Education Research and Practice, 12(2), 271-278. doi.org/10.1039/C1RP90032C
5. Brown, S. J., Sharma, B. N., Wakeling, L., Naiker, M., Chandra, S., Gopalan, R. D., & Bilimoria, V. B. (2014). Quantifying attitude to chemistry in students at the University of the South Pacific. Chemistry Education Research and Practice, 15(2), 184-191. doi: 10.1039/C3RP00155E
6. Brown, S. J., White, S., Sharma, B., Wakeling, L., Naiker, M., Chandra, S., Gopalan, R., & Bilimoria, V. (2015). Attitude to the Study of Chemistry and Its Relationship with Achievement in an Introductory Undergraduate Course. Journal of the Scholarship of Teaching and Learning, 15(2), 33-41. doi: 10.14434/josotl.v15i2.13283
7. Chan, J. Y. K. (2014). Effect of students’ affective characteristics on learning and achievement in first-year general chemistry (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 3581195)
8. Dalgety, J., Coll, R. K., & Jones, A. (2003). Development of chemistry attitudes and experiences questionnaire (CAEQ). Journal of research in science teaching, 40(7), 649-668. doi.org/10.1002/tea.10103
9. Demchenko, O.Y. (2013). Cognitive peculiarities of the State Firefighting Service of the Russian Extraordinary Situations Ministry students’ professional self-consciousness. Discussion, 2, 97-102.
10. Heng, C. K., & Karpudewan, M. (2015). The interaction effects of gender and grade level on secondary school students’ attitude towards learning chemistry. Eurasia Journal of Mathematics, Science & Technology Education, 11(4), 889-898.
11. Kahveci, A. (2015). Assessing high school students' attitudes toward chemistry with a shortened semantic differential. Chemistry Education Research and Practice, 16(2), 283-292. doi: 10.1039/c4rp00186a
12. Kozhevnikova, O.V., & V’yzhanina, S.A. (2016). Psychosemantics. The semantic differential method: textbook for University students. Izhevsk: Publishing center "Udmurt University".
13. Lebedeva I. P. (2003). The specifics of the application of factor analysis in educational research. Education and science, 2(20), 10-25.
14. Lebedeva I. P. (2015). Factor analysis of quality date? Topical issues of modern science, (2), 128-131.
15. Lovelace, M., & Brickman, P. (2013). Best practices for measuring students’ attitudes toward learning science. CBE—Life Sciences Education, 12(4), 606-617. doi.org/10.1187/cbe.12-11-0197
16. Mezentseva, M. E. (2017) The role of a university teacher in the modern world. Professional Culture of the Specialist of the Future. Procedings of the international conference, June 14-15, 68-77.
17. Kogan, M.S., Khalyapina, L.P. & Popova, N.V. (2017). Professionally-oriented content and language integrated learning (CLIL) course in higher education perspective. In L. G. Chova, A. L. Martínez, & I. C. Torres (Eds.). ICERI 2017 Proceedings: 10th International Conference of Education, Research and Innovation (pp. 1103-1112). Seville, Spain: ICERI
18. Razinkina, E., Pankova, L., Trostinskaya, I., Pozdeeva, E., Evseeva, L., & Tanova, A. (2018). Student satisfaction as an element of education quality monitoring in innovative higher education institution. E3S Web of Conferences, Vol. 33, p. 03-43. EDP Sciences. doi.org/10.1051/e3sconf/20183303043
19. Sen, S., Yilmaz, A. & Temel, S. (2016). Adaptation of the Attitude toward the Subject of Chemistry Inventory (ASCI) into Turkish. Journal of Education and Training Studies, 4(8), 27-33. doi: 10.11114/jets.v4i8.1485
20. Shipunova, O. D. (2014). The role of relation to values principle in the social management practices. The existential-communicatory aspect. Middle East Journal of Scientific Research, 19(4), 565-569. doi.10.5829/idosi.mejsr.2014.19.4.123684
21. Shipunova, O. D., Mureyko, L. V., Serkova, V. A., Romanenko, I. B., & Romanenko, Y. M. (2016). The Time Factor in Consciousness Construction. Indian Journal of Science and Technology, 9(42), 1-10. doi.10.17485/ijst/2016/v9i42/104226
22. Sumarni, W., Susilaningsih, E. & Sutopo, Y. (2018). Construct Validity and Reliability of Attitudes towards Chemistry of Science Teacher Candidates. International Journal of Evaluation and Research in Education (IJERE), 7(1), 39-47. doi.org/10.11591/ijere.v1i1.11138
23. Xu, X., Alhooshani, K., Southam, D., & Lewis, J. E. (2015). Gathering psychometric evidence for ASCIv2 to support cross-cultural attitudinal studies for college chemistry programs. In Affective dimensions in chemistry education (pp. 177-194). Berlin, Heidelberg: Springer
24. Xu, X. & Lewis, J. E. (2011). Refinement of a chemistry attitude measure for college students. Journal of Chemical Education, 88(5), 561-568. doi.org/10.1021/ed900071q