Supervisors’ experiences with e-portfolio in pre-service teacher education

Abstract

This study is part of a larger study that explores how the use of an e-portfolio can improve student teachers’ development of profession related competences in a workplace setting. More specifically, we focus on the potential of an e-portfolio for enhancing and managing feedback provided by supervisors. For this study we collected data from nine school-based and four university-based teacher education internship supervisors about the perceived usability of one assessment e-portfolio which was used during a four-month internship period in which they supervised thirteen student teachers of the final (fifth) year of the primary school teacher curriculum.

Keywords: E-portfolioassessment e-portfoliousabilitysupervisionteacher education

Introduction

The most powerful sources for professional learning in the workplace and critical for the development of professional expertise are considered to be feedback on and assessment of activities (Ericsson, Charness, Feltovich, & Hoffman, 2006; Hattie, 2009). However, an insufficient use and low quality of personalized feedback limits the impact on learning (Miller & Archer, 2010). Recent studies in different sectors demonstrate the use of portfolios to help assess workplace-based learning and to support lifelong learning and employability (e.g. Van der Schaaf, Stokking & Verloop, 2008). Additionally, electronic portfolios have been introduced to collect evidence about the development on various competencies.

This study is part of a larger study that explores how these potential benefits, especially related to enhancing feedback on learning, can improve student teachers’ development of profession related competences. Prior to moving to the requirements of feedback, we needed to establish the most crucial activities student teachers need to develop in their initial teacher training (see also Krull & Leijen, 2015). For this purpose, several innovative concepts such as core practice (Grossman, Hammerness, & McDonald, 2009; Windschitl, Thompson, Braaten, & Stroupe, 2012; Zeichner, 2012), entrusted professional activity (EPA) (Ten Cate, 2013; Ten Cate & Scheele, 2007), and rubrics (Brookhart & Chen, 2015; Dekker-Groen, Van der Schaaf, & Stokking, 2011) were utilized to develop a framework to assess and foster the development of student teachers’ performance-based competency requirements. This resulted in an assessment rubric involving five professional roles, 12 or 11 professional activities (12 Estonian context / 11 Dutch context), and five or four performance levels (five for Estonian context/ four for Dutch context) for each activity (see Leijen et al, 2017). Following, the assessment framework was implemented in an e-portfolio environment and tested by a small group of student teachers and supervisors. In the current study, we focus on the teacher education internship supervisors’ perceptions and experiences with one assessment e-portfolio. More specifically we aim to understand what the perceived usability of the assessment e-portfolio was.

Methods

Data was collected about the experience nine school-based supervisors and four university-based supervisors had with an e-portfolio called EPASS during a four-month internship period in which they supervised 13 student teachers of the final (fifth) year of the primary school teacher curriculum. All the supervisors observed the lessons given by the student teacher and provided a score on the student teachers’ professional activities (see Leijen et al, 2017) in the e-portfolio. Moreover, the supervisors were all asked to provide written feedback on these activities. However, if the university-based supervisors had access to the e-portfolio and had a possibility to observe the development of the student teachers in the system, the school-based supervisors received only the feedback protocol via email and could not observe the development of the student teachers in the e-portfolio. Nine school-based supervisors filled in the questionnaire about the usability of the e-portfolio for giving feedback to students. The questionnaire is based on different constructs describing usability. The constructs were Job-fit (Thompson et al. 1991); Complexity (Thompson et al. 1991); Ease of use (Moore and Benbasat, 1991), and Facilitating conditions (Thompson et al. 1991). The questionnaire was submitted online and supervisors were asked to answer via email. Data from the university-based supervisors was collected during a group interview session. Descriptive statistics of the questionnaire data were calculated. Qualitative analysis of the open-ended responses and of the interviews was conducted following the thematic analysis procedure (Braun & Clarke, 2006) in which the researcher looks for themes and sub-themes underlying the qualitative data.

Results

The school supervisors mostly agreed that evaluating students via e-portfolio decreased the time for assessment, increased the effectiveness and quality of assessment, and supported the development of assessment skills. Using the e-portfolio for evaluation also seemed to be easy for them (see also Figure 1 ).

See Full Size >

The results from the open-ended responses of the school supervisors and focus group interview with the university supervisors indicated that the e-portfolio helped the supervisors to have a broader view on the professional activities of the students. The assessment rubric provided them with a better structure for evaluating students. Although the supervisors did indicate that it is not possible to evaluate all the professional activities in each lesson but requires a longer period of evaluation time. One critique regarding the assessment rubric was that some of the descriptions for the indicators were too long and included several different skills and knowledge, which made the assessment difficult.

In the case of our study, the school supervisors were external assessors. This means that they did not have access to the students’ portfolios but were only able to fill in a form in order to give feedback to students. Therefore the school-based supervisors did not have an overview of how the e-portfolio influenced students’ learning and professional activities, which would also have been valuable to them. The supervisors approved the idea about decreasing the use of paper in assessing the students. They were also positive that they could fill in the protocol in the evening and had time to think about the feedback in more detail. In our study the supervisors also wanted to have an opportunity to write additional feedback that did not concern any of the activities in the rubric. Supervisors considered it important to learn about using the e-portfolio before they started using it. Both university and school supervisors found that it would have been good to have an instructional seminar on how to use the system before the internship.

One general issue concerning the use of e-portfolios that was confusing for the supervisors was the question of the relationship between written feedback in the e-portfolio and oral feedback to students. The supervisors pointed out that oral feedback is very important or even more important than written feedback in the professional development of the student teachers. Therefore they were concerned whether the students still receive oral feedback in addition to the feedback inserted in the e-portfolio. Moreover, the supervisors expressed the belief that the amount of feedback highly depends on the supervisor. Moreover, some of the university supervisors had their own assessment instrument and the e-portfolio was extra work for them.

Conclusion

The results showed that the school supervisors mostly agreed that evaluating students via e-portfolio decreased the time for assessment, increased the effectiveness and quality of assessment and supported the development of their assessment skills. However, it was also pointed out that student teachers also need oral feedback during their internship and portfolio-based written feedback should not decrease opportunities for receiving oral feedback at the workplace. This shows that for more successful implementation the role of e-portfolio-based feedback in relation to other forms of feedback needs careful consideration and consolidation among teacher educators.

Acknowledgements

This study was conducted within the framework of “Workplace-Based e-Assessment Technology for competency-Based Higher Multi-Professional Education” (WATCHME) project supported by the European Commission 7th Framework Programme (grant agreement No. 619349).

References

1. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101.
2. Brookhart, S. M., & Chen F. (2015). The quality and effectiveness of descriptive rubrics. Educational Review (3), 343-368.
3. Dekker-Groen, A. M., Van der Schaaf, M. F. & Stokking, K. M. (2012). Performance standards for teachers supporting nursing students’ reflection skills development. Journal of Nursing Education and Practice, 2(1), 9–19.
4. Ericsson, K. A., Charness, N., Feltovich, P. J., & Hoffman, R. R. (Eds.). (2006). The Cambridge handbook of expertise and expert performance. New York: Cambridge University Press.
5. Grossman, P., Hammerness, K., & McDonald, M. (2009). Redefining teaching, re‐imagining teacher education. Teachers and Teaching: Theory and Practice, 15(2), 273–289.
6. Hattie, J. (2009). Visible learning. A synthesis of over 800 meta-analysis relating to achievement. Routledge: Oxon.
7. Krull, E. & Leijen, Ä. (2015). Perspectives for Defining Student Teacher Performance-Based Teaching Skills Indicators to Provide Formative Feedback through Learning Analytics. Creative Education, 6(10), 914–926.
8. Leijen, Ä., Slof, B., Malva, L., Hunt, P., van Tartwijk, J., & van der Schaaf, M. (2017). Performance-Based Competency Requirements for Student Teachers and How to Assess Them. International Journal of Information and Education Technology, 7(3), 190–194.
9. Miller, A. & Archer, J. (2010). Impact of workplace-based assessment on doctors' education and performance: a systematic review. British Medical Journal, 341.
10. Moore, G. C., & Benbasat, I. (1996). Integrating diffusion of innovations and theory of reasoned action models to predict utilization of information technology by end-users. In Diffusion and adoption of information technology (pp. 132-146). Springer US.
11. Ten Cate, O. (2013). Nuts and Bolt of Entrustable professional activities. Journal of Graduate Medical Education, 5(1), 157–158.
12. Ten Cate, O, & Scheele, F. (2007). Competency-Based Postgraduate Training: Can We Bridge the Gap between Theory and Clinical Practice ? Academic Medicine, 82(6), 542–547.
13. Thompson, R. L., Higgins, C. A., & Howell, J. M. (1991). Personal computing: toward a conceptual model of utilization. MIS quarterly, 125-143.
14. Van der Schaaf, M. F., Stokking, K. M., & Verloop, N. (2008). Teacher beliefs and teacher behaviour in portfolio assessment. Teaching and Teacher Education, 24(7), 1691-1704.
15. Van der Schaaf, M.F., & Stokking, K.M. (2011). Construct Validation of Content Standards for Teaching. Scandinavian Journal of Educational Research, 55(3), 273-289.
16. Windschitl, M., Thompson, J., Braaten, M., & Stroupe, D. (2012). Proposing a core set of instructional practices and tools for teachers of science. Science Education, 96(5), 878–903.
17. Zeichner, K. (2012). The Turn Once Again Toward Practice-Based Teacher Education. Journal of Teacher Education, 63(5), 376–382