Students’ Curricular Preferences for Computer-Based Instruction and Evaluation

Abstract

This case-study highlights the results of an empirical research, whose purpose was to identify and analyze the curricular preferences towards computer-based instruction and evaluation of students studying in the field of Primary and Preschool Pedagogy specialization, from the University of Craiova, Faculty of Letters, Department of Communication, Journalism and Education Sciences. We elaborated a questionnaire in order to identify the curricular preferences of the above mentioned students (n=92), aiming in the same time to analyze the relationship between two major variables: students’ interest regarding the attractiveness of disciplines as part of the educational curriculum and students’ perceptions concerning academic subjects’ utility with regard to the professional training curriculum. The results have confirmed the conclusions of several previous surveys regarding the curricular preferences and have opened, in the same time, new perspectives, for further research and development in the field of professional training curriculum, from the perspective of the new information and communication technologies.

Keywords: Computer-assisted evaluationcomputer-based instructionnew information and communication technologiesdistance educationvirtual learning communities

Introduction

The idea for this paper started with the study of the primary and preschool pedagogy students’

curriculum preferences, on one hand, and on the other hand in the analysis of the importance that virtual

learning communities have in developing a wide range of professional competences and identity. It is

very easy to observe, from our experience as educators, that computers are increasingly used by students in educational activities to the point that they are almost ubiquitous in our life, generating strong specific

interactions (Vlăduțescu, 2012; Strungă, 2015). However, although there is considerable evidence to

support the idea that the use of the new information and communication technologies (NICT) has positive

effects for students’ attainment, the integration of computer based-instruction and evaluation in university

curriculum seems challenging and sometimes even difficult. It is very important to also highlight that the

potential to harness the NICTs in educational activities exists, because there are almost 10 million users

of internet in Romania and the internet penetration was 56.3% in 2015 (Eurostat, 2015). There are, of

course, a variety of reasons for the insufficient integration of computer based-instruction and evaluation

in university curriculum, starting, in our opinion, with: the underfunding of education and related NICT

infrastructure, a perceived ambiguity of terms such as computer-based instruction and evaluation (CBIE)

in pedagogical literature, few studies in the field of curriculum needs analysis (including students’

curriculum preferences), the absence of a robust digital national agenda for university education,

underdeveloped digital competences of both students and educators and so on. Interactive CBE implies

the existence of a partnership between educational agents, which is based on collaboration and

negotiation processes and which targeting the empowering trainee and mobilization of its initiatives in

evaluation and learning processes.

In order to develop the integration of CBIE in the university curriculum, it is essential to take into

account, in the needs analysis process, students preferences and attitudes towards curriculum (Bunăiaşu,

2011). Starting from this approach, aiming to increase students’ attainment and develop the curriculum,

we have constructed the following research questions: a) How many students are interested in taking part

of CBIE activities, integrated in the preschool and primary education specialization curriculum? b) What

is the relationship between students’ interest regarding the attractiveness of disciplines as part of the

educational curriculum and students’ perceptions concerning academic subjects’ utility with regard to the

educational curriculum?

Paper Theoretical Foundation and Related Literature

The main theoretical foundations of this paper are threefold. First, in the context of activity theory,

in order to identify the ‘zone of proximal development’ (Vygotsky, 1980), teachers and professors need to

first understand students’ learning preferences and needs. Through this process, we can plan on how to

maximize students’ educational potential by using computer-based instruction and evaluation. Second,

the e-learning theory developed by Mayer and Moreno was extremely useful in designing previous

computer-based instruction programs for preschool and primary education specialization from our

university. Many principles such as contiguity, segmenting, signaling, personalization, learner, pre-

training, redundancy (Moreno & Mayer, 2007) were actively used in developing the learning

management systems we used (Strungă, 2015). In this context, we also successfully tested similar

research instruments with high school students (Strungă, 2008) and university students (Strungă &

Bunăiașu, 2013). Third, we used Harasim’s online collaborative learning theory in order to construct

virtual learning communities and determine if there are any improvements in students’ performance

(Harasim, 2012). A major theoretical foundation related to students’ curricular preferences is based in

studies focused on the ‘pupil voice’ (Keys & Fernandes, 1993; Blatchford, 1996; Rudduck & Flutter, 2000). Rudduck and Flutter state that ‘we need to tune in to what pupils can tell us about their

experiences and what they think will make a difference to their commitment to learning and, in turn, to

their progress’ (p. 75). Other authors also mention that recent research on pupils’ perspectives in the UK

has been correlated either to the development of school-based strategies based on consultation with pupils

on effective classroom practice, or to aspects of curricular evaluation (Deaney et al., 2003).

Many specialists observed that the absence of a clear, consistent definition of CBIE makes the task

of analyzing the effects of these technology-based interventions very difficult (Archer, 2014). For

instance, the type of solution or “technology” used, the purpose of implementation, and the method of

intervention can be very different across the studies previously carried out. It was proposed a model

analyzing three roles for technology in education: technology as a tutor, technology as a teaching aid, and

technology as a learning tool, that can be used in CBIE (Archer, 2014). In the context of our paper, we

used the second understanding for CBIE, technology that includes (but it is not limited to) the use of rich

media formats, games, virtual learning communities, mobile applications, etc. Computer-based

instruction (CBI) was understood in our research as an instructional paradigm, which uses computer

technology to deliver training and educational materials to students. The main purposes of the interactive

assessment are: to stimulate self-evaluation capacity of the subject involved in this process, of its

awareness required and to increase the self-confidence of the trainer.

The study of computer-based instruction and evaluation is an interdisciplinary research field with

an extraordinary development in recent years. One of the best ways to highlight the impact NICT have in

education is, in our opinion, to study the tertiary meta-analyses, that include several meta-analyses and

many more individual studies.

First, the tertiary meta-analysis done by Archer et al. revealed initially that “Results from previous

meta-analyses show very little evidence in support of NICT for literacy interventions. There is so much

variation in the types of ICT interventions such as the technology used and the software programs used”

(Archer et al, 2014, p. 141). However, trying to explain the inconsistent outcomes among previous meta-

analyses, Archer et al. proposes a new approach that takes into account the training and support of those

conducting interventions and attention to the fidelity of the intervention program as moderator variables.

Overall, these two variables are positively correlated with the success of the program implementation.

According to the authors, “This review demonstrates that NICT interventions can prove to be more

effective when implementation factors such as support and training are employed, measured, and clearly

reported” (Archer et al, 2014, p. 146). The study took into account four previous meta-analyses: 1) the

first meta-analysis studied the use of computer-assisted instruction (CAI) on reading in middle and high

school students. Eight studies met the inclusion criteria. The researchers reported a mean effect size of

+0.10; 2) the second meta-analysis found similar results concluding that the supplementary CAI programs

studied are not producing significant effects in upper elementary reading; 3) the third meta-analysis was

conducted on the use of ICT in the field of literacy learning. Of the 20 included studies that were included

in the study, only four studies to be statistically relevant, with one of these having a negative effect size;

4) A fourth meta-analysis examined if information and communication technologies were effective in

teaching English. They concluded that the studies were too heterogeneous, in both the written

composition and the ICTs used, to conduct a meta-analysis.

Second, another study conducted by the Scottish Government in 2015 entitled “Literature Review

on the Impact of Digital Technology on Learning and Teaching” reviewed the 5 meta-analyses. The

researchers concluded there is: a) conclusive evidence that digital technologies can support educational

attainment in general (in math and science particularly); b) Indicative evidence that it can support

educational attainment in literacy and help close the gap in attainment between groups of learners and c)

Promising evidence that digital technologies can provide assistance for many issues including

overcoming challenges by learners, development of employability skills and career pathways, improved

communication with parents and increasing time efficiencies for teachers (APS Group Scotland, 2015).

Third, a research report by the United States Department of Education from entitled “Evaluation of

Evidence-Based Practices in Online Learning A Meta-Analysis and Review of Online Learning Studies”

from 2009, presented a meta-analysis of 50 study effects, 43 of which were drawn from research with

older learners and found that students in online conditions performed modestly better, on average, than

the students learning the same material by traditional face-to-face instruction, with an average effect size

of +0.20 favoring online conditions (Means et al, 2009). The research team added that the online learning

is effective for both undergraduates (mean effect of +0.30, p < .001) and graduate students or

professionals (+0.10, p < .05).

Fourth, a recent tertiary meta-analysis by Al Zahrani and Laxman concerning the impact of mobile

learning in higher education reviews 8 meta-analyses. The authors concluded that “This study shows

there is a significant absence of attention being paid to pedagogical details in conceptualizing m-learning

research”, adding that “Without reference to theoretical and pedagogical issues, studies of m-learning will

not necessarily further our understanding of how m-learning can contribute to successful learning

outcomes globally” (Al Zahrani & Laxman, 2015, p. 86).

All these studies highlight the idea that CBIE can have a very important impact in the university

curriculum but only if adequate training and support are offered (both at the beginning and during the

implementation), if there is a constant attention to the fidelity of the intervention and careful integration

in the larger knowledge management model of the university.

Methodology

3.1.Sample

Since our case-study is an approach to optimize the digital curriculum for Primary and Preschool

Pedagogy specialization, we applied the research instrument to a sample of undergraduate students

enrolled in the study program organized by University of Craiova (at both Craiova and Drobeta Turnu-

Severin) - 240 students, of which participated in the investigation a total of 92 subjects, 46 from the first

year and 46 from the second year. Consequently, the survey’s results are representative to students of

Primary and Preschool Pedagogy specialization from the University of Craiova. The average age of

subjects participating in research was 26 years, median age was 25 years, 98% of the subjects were

female and 2% male, 59 resided in urban areas and 41% in rural areas.

3.2.Procedure

The survey was carried out between February and May 2016 with the permission of University of

Craiova’s rectorate. We included in this study students from the first and second year of their

undergraduate training (primary and preschool specialization) using systematic random sampling. The

research instrument was applied during the seminar activities for the subject “Methodology of

Educational Research” and “Theory and Methodology of Curriculum”.

3.3.Instrument

For the purpose of this study, we constructed a questionnaire with 10 items entitled “Students’

Digital Curriculum Preferences Inventory” (SDCPI). The first item (I1) included a list with all the

subjects from the preschool and primary education specialization’s curriculum and the students were

asked to specify how useful and interesting would be to include CBIE for each discipline. All the answers

from the first two items were codified on a Likert scale from 1-5 (1 - not useful at all and 5 – very useful,

respectively 1 - not interesting at all and 5 – very interesting). The next two items (I2 and I3) asked

students what subjects they think should be added or removed from the curriculum. Items I4, I5 and I6

included questions regarding the access to various electronic devices (smartphones, laptops, tablet

computers, smart TV, desktop), how useful these devices are for their professional development and how

much they use them. The last items (I7, I8, I9 and I10) gathered factual data concerning the year of study,

age, gender, family and residence. Overall, the questionnaire included 8 closed questions (I1, I2, I5-I10)

and 2 open questions (I3, I4) and was based on previous studies in the field of curriculum preferences

(Strungă & Bunăiașu, 2013). The research instrument was pretested in the framework of our seminar

“Methodology of Educational Research” and students’ observations were included in the revised version.

The research instrument was also reviewed by three other colleagues from the Department of Communication, Journalism and Education Sciences and their feedback was added in the last version of

the questionnaire. Before using the questionnaire, we conducted a reliability test in IBM SPSS 24 for

SDCPI, and the Cronbach’s alpha coefficient was equal to .993.

3.4.Objectives and Hypotheses

The main objectives of this study were: a) to identify the current digital curricular preferences

which could be the basis of a needs analysis profile for our specialization; b) to make use of current

digital infrastructure in order to improve the digital curriculum; c) to recommend new ways for using

CBIE in our classes. We elaborated the following hypothetic statements: first, if we apply the SDCPI, the

CBIE preferences are different for mandatory subjects than the preferences for specialty subjects and

second, if we apply the SDCPI, the CBIE preferences of first year students are different from the second

year students. In the context of this paper, we will use the concept of “hypotheses” when referring to the

hypothetic statements mentioned above.

3.5.Data Analysis Procedure

We calculated the means for the 47 subjects included in the curriculum for preschool and primary

education specialization (3 years) from the University of Craiova and also a general mean (MG) for all the subjects. For each subject we calculated the means regarding the grade of utility and interest associated

with CBI and CBE. The means were then ranked in both ascending and descending order, with the aim of

understanding the curriculum preferences for the general sample. Additionally, in order to analyze the

difference between the two groups (first year students and second year students), we conducted the Mann-

Whitney U test with IBM SPSS 24, adequate for measuring Likert-type ordinal scales such as those we

used in the study. The Z coefficients obtained from the Mann-Whitney U test were also ranked in both

ascending and descending order.

Results

The students from the preschool and primary education specialization considered that the

following subjects are adequate for computer based instruction and evaluation activities: physical

education (2.2), philosophy of education (2.6), intercultural education (2.8), educational alternatives (2.8),

sociology of education (2.8), comparative pedagogy (2.9). The students from the preschool and primary

education specialization considered that the following subjects are less adequate for computer based

instruction and evaluation activities: psychopedagogy of game (3.9), theory and methodology of

instruction (3.9), planning and implementation of educational projects (3.7), theory and methodology of

curriculum (3.7), didactics of extracurricular activities (3.6), inclusive education (3.5). The general mean

(MG) for all subjects and both years was 3.2. A surprising result was that the preferences for CBE was higher than expected.

The students from the first and second year gave very different answers for the following subjects:

Didactics of mathematics in primary school ( Z =-7.16, p=0,00 respectively -7.08, p=0.00), theory and

methodology of instruction ( Z =-6.70, p=0.00 respectively Z =-6.64, p=0.00), mathematics for preschool

and primary education ( Z =-7.00, p=0.00 respectively Z=-7.11, p=0.00), pedagogical practicum ( Z =-6.52, p=0.00 respectively Z =-6.46, p=0.00), Romanian literature ( Z =-6.54, , p=0.00 respectively Z =-6.16,

p=0.00), especially evaluation. We observed significant differences between the answers from first year

students and second year students for most of the variables studied in our paper.

Discussions

First, the results of the study highlighted the fact that the students from preschool and primary

education specialization from the University of Craiova are moderately interested to take part in CBIE

activities (MG=3.2). It is worth mentioning that many of the subjects that ranked very high in students’ preferences - intercultural education (2.8), educational alternatives (2.8), sociology of education (2.8),

comparative pedagogy (2.9) - are disciplines with great potential of openness toward the community and

social environment. This is a great advantage from the standpoint of CBIE because NICTs have the

potential to build bridges at local, regional, national and European levels. However, there was no

difference between mandatory and specialty subjects, in terms of CBIE preferences. Thus, the first

hypothesis was not confirmed. The analysis also suggested that students are more interested in face-to-

face instruction for disciplines such as psychopedagogy of game (3.9), theory and methodology of

instruction (3.9), planning and implementation of educational projects (3.7), theory and methodology of

curriculum (3.7), didactics of extracurricular activities (3.6), inclusive education (3.5), that are highly

related to direct experience and pedagogical practicum. The awareness concerning this issue was

observed especially in the answers of the second year students.

Second, the Mann-Whitney U test confirmed the differences between the two groups for the

majority of subjects included in the curriculum of preschool and primary education specialization. Very

high differences were found for disciplines such theory and methodology of instruction ( Z =-6.70, p=0.00

respectively Z =-6.64, p=0.00), mathematics for preschool and primary education ( Z =-7.00, p=0.00

respectively Z=-7.11, p=0.00), Romanian literature ( Z =-6.54, p=0.00 respectively Z =-6.16, p=0.00),

which suggests that sometimes the preferences can be polarized and a monolithic approach to curriculum

could not be the best solution when addressing the students’ needs, preferences and representations. The

dynamics of students’ preferences can be used in the curriculum development and needs analysis process.

In conclusion, the second hypothesis was confirmed.

Conclusions

One of the most important conclusions of this study was the need for a better integration of CBIE

in the curriculum for preschool and primary education specialization from the University of Craiova; the

analyses carried out so far highlighted which subjects should include more CBIE activities from the

perspective of the students. The openness of the students towards the NTIC can create new opportunities

to creatively use virtual learning communities and other instruments especially in transnational study

programs (Aristovnik, 2012; Strungă, & Florea, 2014; Sava & Danciu, 2015; Stoian, A.C., 2016), as other

studies have shown. We agree with Al Zahrani and Laxman that “M-learning can only bring about an

improvement in learner outcomes when it is matched by the application of pedagogical practices that take

into account the characteristics and opportunities presented by m-learning and recognize the demands of

the differentiated educational and cultural contexts it will be used in” (Al Zahrani & Laxman, 2015, p.86).

And this observation should not necessarily be limited to m-learning, insofar is relevant even for the

larger field of CBIE especially in relationship to adult and distance education programs (Strungă &

Martin, 2012; Sava & Danciu, 2015). In delivering educational programs that integrate CBIE, studies

have clearly demonstrated that training, support and fidelity of implementation are highly correlated with

greater attainment (Archer et al, 2014). However, more studies are necessary in order to understand better

the dynamics of students’ preferences, that take into account not only the dimensions analyzed in this

study (interest and utility) but also other factors that can have a greater impact in the overall attitude. At

the level of recommendations, in addition to the previous observations, we propose the following: the

development of virtual research and study of the learning behavior of students, the development of a

virtual curriculum at preschool and primary education specialization, elaborating a new virtual evaluation

and assessment model, creating new constructivist virtual learning strategies, building international

virtual learning communities, linking virtual learning communities with virtual professional development

communities, innovating virtual pedagogical practicum, integrating e-mentorship and e-internship as new

methods complementary to CBIE, innovating the virtual relationships, interactions and norms established

between the members of the virtual learning community (Strungă, 2015).

References

1. Al Zahrani, H., & Laxman, K. (2015). A critical meta-analysis of mobile learning research in higher
2. education. The Journal of Technology Studies, 41(2), 74–89.
3. APS Group Scotland. (2015). Literature review on the impact of digital technology on learning and
4. teaching. Edinburgh. Retrieved from http://www.gov.scot/Resource/0048/00489224.pdf.
5. Archer, K., Savage, R., Sanghera-sidhu, S., Wood, E., Gottardo, A., & Chen, V. (2014). Examining the effectiveness of technology use in classrooms : A tertiary meta-analysis. Computers & Education, 78, 140–149.
6. Aristovnik, A. (2012). The impact of ICT on educational performance and its efficiency in selected EU and OECD countries : a non-parametric analysis. The Turkish Online Journal of Educational Technology, 11(3), 144–152.
7. Blatchford (1996). ‘Pupils’ views on school work and school from 7-16 years’, Research Papers in Education, 11, 263-288 Bunăiaşu, C. M. (2011). Proiectarea şi managementul curriculumului la nivelul organizaţiei şcolare. Bucureşti: Editura Universitară.
8. Deaney et al (2003). Pupil perspectives on the contribution of information and communication technology to teaching and learning in the secondary school. Research Papers in Education, 18 (2), 141-165.
9. Eurostat (2015). Digital economy and society statistics - households and individuals. Retrieved from http://ec.europa.eu/eurostat/statisticsexplained/index.php/Digital_economy_and_society_statistics_-_households_and_individuals.
10. Harasim, L. (2012). Learning theory and online technologies. Routledge.
11. Keys & Fernandes (1993). What do students think about school?, Slough: NFER.
12. Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies (Center for Technology in Learning). (M. Barbara, T. Yukie, M. Robert, B. Marianne, & J. Karla, Eds.) (Vol.
13. 15). Centre for Learning Technology. Retrieved from http://repository.alt.ac.uk/629/1/US_DepEdu_Final_report_2009.pdf.
14. Moreno, R. & Mayer, R. (2007). Interactive multimodal learning environments.Educational Psychology Review, 19(3), 309-326.
15. Rudduck & Flutter (2000). Pupil participation and pupil perspective: Carving a new order of experience, Cambridge Journal of Education, 30, 1, 75-88.
16. Sava, S. L., & Danciu, L. (2015). Students ’ perceptions while enrolling in transnational study programs.
17. Procedia - Social and Behavioral Sciences, 180 (November 2014), 448–453.
18. Slavin, R. E., Cheung, A., Groff, C., & Lake, C. (2008). Effective reading programs for middle and high
19. schools: a best-evidence synthesis. Reading Research Quarterly, 43,290-322.
20. Slavin, R. E., Lake, C., Chambers, B., Cheung, A., & Davis, S. (2009). Effective reading programs for the
21. elementary grades: a best-evidence synthesis. Review of Educational Research, 79, 1391.
22. Stoian, A. C. (2016). The efficiency of differentiated learning - independent learning situations versus
23. collaborative learning. Network Intelligence Studies, IV(1), 51–59.
24. Strungă, A.C. (2008). Investigaţia feedback-ului curricular ca şi premisă a optimizării procesului de învăţământ. Analele Universităţii din Craiova, Seria Psihologie – Pedagogie, VII (17-18), 193-215.
25. Strungă, A. C., & Martin, C. (2012). Mentoring and eMentoring in entrepreneurial education: CReBUS perspective. Romanian Journal of Education Sciences, XX(1), 101–109.
26. Strungă, A. C., & Bunăiașu, C. M. (2013). The investigation of the curricular preferences of students from primary and preschool pedagogy specialization. Premises for a model of action and socio - pedagogical intervention. Review of Research and Social Intervention, 40(1), 61–77.
27. Strungă, A. C. (2014). Imaginile mentale europene și identitate profesionalã în formarea cadrelor didactice: aplicații în domeniul învãțãmântului primar. București: Editura Universitară.
28. Strungă, A. C., & Florea, C. A. (2014). The integration of creativity management models into universities’ virtual learning communities. Network Intelligence Studies, II(2), 287–294.
29. Strungă, A. C., (2015). The integration of virtual learning communities into universities’ knowledge management models. Procedia - Social and Behavioral Sciences), 197/2015, 40(1), 2430-2434 Vlăduțescu, Ș. (2012). Relationship and communication networks. Journal of Community Positive Practices, 4, 790–797.
30. Vygotsky, L. (1980). Mind in society: The development of higher psychological processes. New York: Harvard University Press.