A ‘Quality of Use’ Model in Blended Learning

Abstract

Blended learning is considered as a new research area. Previous scholars suggested more studies required to explore and confirm the quality dimensions for this area. Hence, the aim of this study is to develop a predictive model based on ‘quality of use’ concept in blended learning using flipped classroom. The model will be incorporated with contemporary variables, namely student engagement and student interaction. Data were obtained using survey method. A total of 299 undergraduate students participated in the study. Data were analyzing using SmartPLS. The finding revealed that the enhancement of ‘quality of use’ model by integrating student engagement and student interaction attained the validity and reliability of a predictive model for blended learning using flipped classroom. However, this model is only applicable for the context on this study. The replication of similar study need to be conducted in order to confirm the variables used in similar research context.

Keywords: Quality of usepredictive modelblended learningSmartPLS

Introduction

Blended learning is considered as a new research area. One of the main issues in blended learning measurement is the disagreement in terminology use in publications, which lead to the difficulties in developing suitable measurements for this area (Graham, Woodfield, & Harrison, 2013). Previous scholars suggested more studies needed to explore and confirm the quality dimensions for blended learning (Shelton, 2010; Teo, 2010). A substantive discussions about theory, developing new theoretical framework that accompanied by empirical research, are seen as appropriate approaches to increase the understanding in this area(Halverson, Graham, Spring, Drysdale, & Henrie, 2014). Hence, the aim of this study is to develop a predictive model based on ‘quality of use’ concept in blended learning using flipped classroom. The model will incorporate two contemporary variables, which are student engagement and student interaction.

Literature Review

Quality of Quality of use is one of the important issues in the area of information system (Folmer & Bosch, 2004). Bevan (1995) in his article ‘Measuring usability as quality of use’ explained, quality was not solely about the attribute of the product but also in the context of use including interactions of users, task and environment in its application. The results of these experiences were known as the ‘quality of use’. Measuring ‘quality of use’ is important because it leads to the determination of whether the attributes of products attain user perception related to satisfaction, efficiency and effectiveness (Bevan, 1995).

Satisfaction is one of the essential elements in blended learning at tertiary level because it reflects the quality of the program offered by universities. Other than satisfaction, efficiency and effectiveness are also important in education. Effectiveness measures the ability of students to complete assessments based on the subject’s requirement (Aspiazu, 2013). While efficiency refers to the ability of students to complete their assessments in the specified timeframe (Alnanih, Ormandjieva, & Radhakrishnan, 2013). Besides the above mentioned variables, factors related to students learning such as student engagement and student interactions, are reported to have a huge impact in achieving learning goal (Athabasca University, 2004; Findlay-thompson, Saint, & Mombourquette, 2014).

Blended learning offers a new research areas that require fundamental discussions with results from empirical research. Unfortunately, less empirical studies were conducted in the area of blended learning(Halverson et al., 2014), hence, the development a comprehensive model such as ‘quality of use’ in blended learning among the digital natives generation becoming crucial.

Teaching the digital natives generation creates new challengers to educators, because this generation requires different methods of teaching and learning. Flipped classroom for example is one of the approach in blended learning that offers a combination of dual teaching modalities namely face to face and online learning (Bristol, 2014). This approach seems to be a suitable learning strategy for digital native generation (McLaughlin et al., 2013, 2014).

Research Methodology

The respondents of this study were undergraduates’ students from a public university in Malaysia. The inclusion criteria used in the sample selection were students who enrolled the Fundamental of Entrepreneurship (ENT300) subject for the semester June to October 2015. ENT300 was delivered using flipped classroom since semester June 2012 in this university. Data were obtained using survey method. A total of 325 students participated in the study, however only 299 questionnaires were qualified for further analysis due to quality of responses.

The measurement for this study were adapted from various scholars. Satisfaction and student interaction measurement were adapted from Kuo, Walker, Schroder and Belland (2014), effectiveness and efficiency were adapted from (Finstad, 2010), and students engagement from Dixson, (2010). All continuous items are measured using 6 point Likert scale ranging from ‘1’ refers to ‘strongly disagree to ‘6’ refers to ‘strongly agree’. In order to improve the content validity, a professor, one senior lecturer and three students were asked to review the questionnaires. Amendments had been made before the final copy of the questionnaires were distributed.

Findings

This section will discuss about the finding for this study. First the profile of respondents are presented followed by the analysis of predictive model through the analysis of the measurement model. Prior to starting the analysis, the author ensure the estimates in the path-modelling is normal. The result indicates that the algorithms converged after Iteration 4 thus confirming that the estimation is normal (Hair, Hult, Ringle, & Sarstedt, 2014; Wong, 2013). The authors employed Partial Least Squares Structural Equation Modeling (PLS-SEM) method of analysis through SmartPLS based on the suggestion by previous scholars (Hair et al., 2014; Hair, Ringle, & Sarstedt, 2012), as the aim of this study is to develop a predictive model by focusing on explaining the variance in the dependent variable while accessing the model. The predictive measurement model is prepared based on the guideline provided by Hair, Hult, Ringle and Sarstedt (2014).

4.1 Profile of Respondents

From 299 respondents who participated in this study, majority of the respondents were female represented by 208 students (69.6%) and male 91 students (30.4%). Most of the students were 20 years old and the highest respondents were from Diploma in Sport Studies and Diploma in Planting Industry Management with 53 students (17.7%). Lastly, the respondents were from semester 5 (65.2%) and semester 4 (34.8%). Apart from the information obtained from this survey, that all respondents are Malay with Bumiputra status. A detail of the respondents’ profile is presented in the Table 1 .

4.2 Measurement Model

All measurement in the model are reflective constructs. The estimates in the path-modelling indicates that the algorithms for this model converged after iteration 4, thus confirming that the estimation is normal [22, 21]. Results from this study reveal that the value for composite reliability (CR) are 0.885 (Satisfaction), 0.908 (Efficiency), 0.910 (Effectiveness), 0.929 (Student Interactions) and 0.872 (Student Engagement). This results demonstrated that all variables have high level of internal consistency reliability(Hair et al., 2014).

Validity assessment for reflective measurement model are evaluated using convergent and discriminant validity. Convergence validity measured using average variance extracted (AVE). Result for this study exhibits that the value for AVE of the measured variables were 0.606 (Satisfaction), 0.711 (Efficiency), 0.718 (Effectiveness), 0.504 (Student Interactions) and 0.533(Student Engagement) respectively. Therefore the convergent validity for constructs is adequate(Hair et al., 2014).Discriminant validity for this model is measured using Fornell-Larcker Criterion(Hair et al., 2014) as shows in Table 2 . A squared AVE values represented by value on the diagonal and the interconstruct correlations value represented by off the diagonal. It can be seen that all AVE values higher than interconstruct presented by correlation values, thus indicating discriminant validity.

Discussions

The aim of this study is to develop a predictive model based on ‘quality of use’ concept in blended learning using flipped classroom approach by incorporating contemporary issues, which are student engagement and student interaction. From the measurement model analysis using SmartPLS, the result indicates that, the enhancement of ‘quality of use’ model is reported as valid and reliable. The enhancement of contemporary variables namely student engagement and student interaction to the existing ‘quality of use’ concept are applicable in explaining blended learning using flipped classroom. This conclusion can be drawn by the result of composite reliability (CR), average variance extracted (AVE) and discriminant validity using Fornell-LackerCiterions.

The result clearly indicates that student interaction has relationship with satisfaction, efficiency and effectiveness. This empirical finding supports previous literatures from similar context that has positive impacts between student interactions and ‘quality of use’ variables (Marina, 2009; Piccoli, Ahmad, & Ives, 2001; Zhang, Zhou, Briggs, & Nunamaker, 2006). This study also reveals that student engagement has influence on satisfaction, effectiveness and student interaction. This empirical finding supports previous scholars who reported similar result in the same research context.

Conclusion and Recommendations

This study provide an empirical finding to support the development of a comprehensive ‘quality of use’ predictive model. This model was developed based on the original concept introduced by Baven (1995), where the main variables identified were satisfaction, effectiveness and efficiency. The enhancement of this predictive model incorporated two contemporary variables namely student engagement and student interaction.

This predictive ‘quality of use’ model is able to explain blended learning using flipped classroom. Therefore, this model is applicable to be used by blended learning administrator to gauge the perception of users based on the tested constructs, especially in the flipped classroom setting.

However, this finding only represent the context of this study. Hence, more studies need to be replicated to confirm the model, specifically the relationship between student engagement and student interaction in various context in blended learning.

References

1. Alnanih, R., Ormandjieva, O., & Radhakrishnan, T. (2013). A New Quality-in-Use Model for Mobile User Interfaces. 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement, 165–170.
2. Aspiazu, L. Z. (2013). Development of a Model for Security and Usability.
3. Athabasca University. (2004). Theory and practice of online learning. (T. Anderson & F. Elloumi, Eds.) (Vol. 36). Canada: Athabasca University.
4. Bevan, N. (1995). Measuring usability as quality of use. Software Quality Journal, 4, 115–150.
5. Dixson, M. D. (2010). Creating effective student engagement in online courses : What do students find engaging ? Journal of the Scholarship of Teaching and Learning, 10(2), 1–13.
6. Findlay-thompson, S., Saint, M., & Mombourquette, P. (2014). Evaluation of a flipped classroom in an undergraduate business course. Business Education & Accreditation, 6(1), 63–71.
7. Finstad, K. (2010). Interacting with Computers The Usability Metric for User Experience. Interacting with Computers, 22(5), 323–327.
8. Folmer, E., & Bosch, J. (2004). Architecting for usability: a survey. Journal of Systems and Software, 70(1-2), 61–78.
9. Graham, C. R., Woodfield, W., & Harrison, J. B. (2013). A framework for institutional adoption and implementation of blended learning in higher education. The Internet and Higher Education, 18, 4–14.
10. Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2014). A primer on partial least squares structural equation modeling (PLS-SEM). SAGE Publications, Inc.
11. Hair, J. F., Ringle, C. M., & Sarstedt, M. (2012). Partial Least Squares: The Better Approach to Structural Equation Modeling? Long Range Planning, 45(5-6), 312–319.
12. Halverson, L. R., Graham, C. R., Spring, K. J., Drysdale, J. S., & Henrie, C. R. (2014). A thematic analysis of the most highly cited scholarship in the first decade of blended learning research. The Internet and Higher Education, 20, 20–34.
13. Marina, P. (2009). Digital Game-Based Learning in high school Computer Science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1–12.
14. McLaughlin, J. E., Griffin, L. M., Esserman, D. a, Davidson, C. a, Glatt, D. M., Roth, M. T., … Mumper, R. J. (2013). Pharmacy student engagement, performance, and perception in a flipped satellite classroom. American Journal of Pharmaceutical Education, 77, 196.
15. McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, L. M., … Mumper, R. J. (2014). The Flipped Classroom: A Course Redesign to Foster Learning and Engagement in a Health Professions School. Academic Medicine, 89(2), 236–243.
16. Piccoli, G., Ahmad, R., & Ives, B. (2001). Web-Based Virtual Learning and a Research Framework Environments : a Preliminary Assessment of Effectiveness in Basic It Skills Training. MIS Quarterly, 25(4), 401–426.
17. Shelton, K. (2010). A Quality Scorecard for the Administration of Online Education Programs: A Delphi Study. University of Nebraska.
18. Teo, T. (2010). Development and validation of the E-learning Acceptance Measure (ElAM). The Internet and Higher Education, 13(3), 148–152.
19. Wong, K. K. (2013). Partial Least Squares Structural Equation Modeling (PLS-SEM) Techniques Using SmartPLS. Marketing Bulletin, 24(October), 1–32.
20. Zhang, D., Zhou, L., Briggs, R. O., & Nunamaker, J. F. (2006). Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness. Information & Management, 43(1), 15–27.