A Process In Developing A Quality Model For Educational Web Application

Abstract

Educational web application has become a popular platform among academicians in delivering daily routine task over the last decade. To ensure the application meets user satisfaction is not a straightforward process. A questionnaire has been used in determining the characteristics needed in web-based integrated student assessment or in short, WBISA application. Rasch Measurement Model (RMM) has also been applied in constructing the quality model of WBISA application. In the beginning, there is a need to identify and remove item misfit, person misfit which affect the model development in achieving the initial model. Furthermore, a category response from the respondents needs to be analysed to identify the pattern of responses to determine whether they follow the rules applied in the Principal Component Analysis (PCA). Seven misfit items and ten misfit persons have been removed from the final analysis which results in good and acceptable PCA value as suggested by Rasch Model. The Person reliability is 0.96 (Excellent) and item reliability is 0.88 (Good). The model error shows a value of 0.25 which is considered as very good. The PCA values increased from 34.3% to 41.9%. Using Rasch, a model with eight constructs consisting of Usability, Reliability, Efficiency, Functionality, Supportability, Availability, Security, and Integrity has been established.

Keywords: Educational, higher learning institution, quality model, web application

Introduction

Web application becomes a popular medium for delivery content in Higher Learning Institution (HLI) among the academicians nowadays. This is because the characteristics belong to web application which supports multiple contents and serves various large scale application such as e-commerce, information distribution and other activities (Offutt, 2002). However, the quality of software plays an important role in business success in many organizations.

Various quality models exist in providing specific characteristics that act as guideline in producing a quality application and therefore cause satisfaction to the user. The popular quality model from gurus in software engineering are Boehm quality model, McCall’s quality model, Dromey’s quality model, FURP’s quality model and ISO 9126’s quality model. A comparative studies have been done in analysing characteristics that exists in five established quality model mentioned above (Al-Qutaish, 2010; Nur Razia et al., 2016). Basically, all the models have their own list of characteristics. Examples of the characteristics are usability, functionality, and reliability. Each characteristic has their own specific definition.

This paper is structured as follows: Section 2 describes problem statement while Section 3 provides a description of the research questions. This is followed by Section 4 which discusses the purpose of the study. Section 5 which describes the research method used in the study. Section 6 which proves the main findings of the case study and finally Section 7 that sets out our conclusion.

Problem Statement

Previous studies conducted by researcher focusing more on e-commerce, information system, health, and other domain while several research focusing on specific domain application such as in telecommunication (Nor Fazlina Iryani & Mohamad Khatim, 2013), and Business-to-Business (B2B) application (Behkamal et al., 2009).

According to (Al Nawaiseh et al., 2020), the characteristic for educational application is different compared to other applications. In educational application, the profit value is not an important characteristics since the user of the application is academicians and students only. A study conducted by (AL-Nuaimi et al., 2022) focusing in evaluation a learning management application during covid 19 era by applying PLS-SEM technique. Beside that, research implemented by (Muqtadiroh et al., 2020) claimed that assurance, empathy, responsiveness, reliability, and website content are the factors influenced student intention in using e-learning application. Osman (2022) proposed a quality model for Sudanese educational institutions with six elements which are aesthetics, ease of use, rich contents, multimedia, reputation, and security. Another researcher conducted proposed a quality model on service acceptance of using educational applications (Bsharat et al., 2019).

However, there are limited number of studies investigating the quality model for educational web application. Therefore, to overcome these issues, this paper will discuss the process or steps in producing quality model for educational web application in public institution focusing on WBISA application.

Research Question and Research Objective

The overall research question and research objective is list as below Table 1.

Purpose of the Study

The intention of the current study is to investigate the characteristic for educational web application and discuss of the processes in developing the quality model for educational web application

Research Method

The characteristics are identified by previous studies and through a preliminary study conducted to the targeted respondents. The steps in obtaining the suitable characteristics have been discussed by (Nur Razia et al., 2018). These characteristics are later used in constructing the questionnaire of the survey based on the respondents’ feedback or suggestion. The characteristics are usability, reliability, efficiency, functionality, supportability, availability, security, and integrity.

The questionnaire consists of two parts, part A and part B. Part A was to address the respondent’s demographic such as HLI they are working, contact email, working experience, occupation, and experience in using web-based application. Part B consists of quality characteristics and the item associated with it needed by WBISA application. The survey was administered using two methods via online and manual medium. According to Joshi et al. (2015), the Likert type scale could be used to understand about the options or perceptions of respondents related with the items in the questionnaire. The respondents were asked to specify the frequency of each characteristic and they may choose only one of the options. Their answer was a choice on a four-point scale consist of “Strongly Disagree”, “Disagree”, “Agree”, ”Strongly Agree”. The “Strongly Disagree” is referred as 1, “Disagree” is referred as 2, “Agree” is referred as 3 and “Strongly Agree” is referred as 4. Table 2 presented the questionnaire used in this study.

Findings

Summary statistics for item

The analysis for item summary statistics indicated that item reliability value did not differ much before the data was cleaned. The item reliability after clean data is 0.88. However, the separation index of item increased to 2.54 logit. Table 4 shows model error increased from 0.20 logit to 0.25 logit. The separation index increased from 2.66 logit to 2.54 logit. The value of MNSQ was 0.98 logit and ZSTD was -.1, which is near to expectation of 1 and 0. This discloses that the 49 items are targeting the right type of respondents in measuring the latent traits and the produced data is at a reasonable prediction level of the responses to the items.

See Full Size >

Rasch Analysis applies the Principal Component Analysis (PCA) of the residuals to know how much variance of the instrument measuring what were supposed to be measured. Figure 1 depicts that the raw variance explained by measures is 41.9 percent increased by 7.6 percent. Previously, before the data is clean, the value is 34.3 percent. Fisher (2007) stated that the minimum requirement value of variance is 40%. Therefore, this value can be considered as accepted value.

See Full Size >

Conclusion

In this paper, a process in developing a quality model for educational web application has been discussed. Each characteristic has their own influence and effects towards the educational web application model. First, the characteristics of WBISA have been identified, then followed by a proposed quality model which has been established. To identify the characteristics of WBISA, a survey has been conducted to the target respondents. The analysis has been done using Rasch Measurement Model approach.

This research contributes in:

a) Identifying characteristics of educational application

b) Proposing quality model for educational web application, referred to as WBISA application

For future work, the proposed quality model needs to go through the verification process from panels who are experts in educational environment. The panel experts may agree or disagree with the proposed quality model.

Acknowledgments

The author would like to thank Universiti Selangor for supporting the research and valuable panels for comments and suggestion towards this paper. The author would also like to thank all the academicians who participated in online survey and face to face medium.

References

• Al Nawaiseh, A. J., Helmy, Y., & Khalil, E. (2020). A new software quality model for academic information systems “case study e-learning system.” International Journal of Scientific and Technology Research, 9(1), 3822–3833. http://www.ijstr.org/final-print/jan2020/A-New-Software-Quality-Model-For-Academic-Information-Systems-case-Study-E-learning-System.pdf

• AL-Nuaimi, M. N., Sawafi, O. Al, Malik, S., Al-Emran, M., & Selimb, Y. F. (2022). Evaluating the actual use of learning management systems during the covid-19 pandemic: an integrated theoretical model. Interactive Learning Environment.

  Google Scholar

  Crossref

• Al-Qutaish, R. E. (2010). Quality Models in Software Engineering Literature: An Analytical and Comparative Study. Journal of American Science, 6(3), 166–175. http://www.jofamericanscience.org/journals/am-sci/am0603/22_2208_Qutaish_am0603_166_175.pdf

• Behkamal, B., Kahani, M., & Akbari, M. K. (2009). Customizing ISO 9126 quality model for evaluation of B2B applications. Information and Software Technology, 51(3), 599–609.

  Google Scholar

  Crossref

• Bond, T. G., & Fox, M. C. (2015). Applying The Rasch Model - Fundamental Measurement in the Human Sciences. Journal of Educational Measurement, 40(2). Routledge.

  Google Scholar

  Crossref

• Bsharat, M., Ibrahim, O., & Bsharat, S. (2019). Proposing a new model for quality of service acceptance. International Journal of Recent Technology and Engineering, 8(2), 1995–1999.

  Google Scholar

  Crossref

• Fisher, W. P. J. (2007). Rasch Measurement Transactions of the Rasch Measurement SIG. American Educational Research Association, 21(1), 1087–1096.

• Joshi, A., Kale, S., Chandel, S., & Pal, D. (2015). Likert Scale: Explored and Explained. British Journal of Applied Science & Technology, 7(4), 396–403.

  Google Scholar

  Crossref

• Lawshe, C. (1975). A Quantitative Approach To Content Validity. Content Validity II, 28(1), 563–575. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.9380&rep=rep1&type=pdf

• Muqtadiroh, F. A., Herdiyanti, A., & Puspitasari, N. (2020). The e-Learning Quality Model to Examine Students’ Behavioral Intention to Use Online Learning Platform in a Higher Education Institution. Khazanah Informatika: Jurnal Ilmu Komputer Dan Informatika [Khazanah Informatika: Journal of Computer Science and Informatics] 6(2), 176–183.

• Nor Fazlina Iryani, A. H., & Mohamad Khatim, H. (2013). Software Quality Model for Telecommunication Industry in Malaysia. Jurnal Teknologi UTM, 63(1), 13–19. https://journals.utm.my/index.php/jurnalteknologi/article/view/1354

• Nur Razia, M. S., Saliyah, K., & Nor Azliana Akmal, J. (2016). A Review on Software Quality Attributes for Web-based Application. International Conference on Engineering and Applied Science 2016, (1), 181-191.

• Nur Razia, M. S., Saliyah, K., & Nor Azliana Akmal, J. (2018). Identification of software quality characteristics on academic application in Higher Education Institution (HEI). Journal of Telecommunication, Electronic and Computer Engineering, 10(2–7), 133–136.

• Offutt, J. (2002). Quality attributes of Web software applications. IEEE Software, 19(2), 25–32.

  Google Scholar

  Crossref

• Osman, A. S. A. (2022). Assessing the Quality of Educational Websites in Sudan using Quality Model Criteria through an Electronic Tool. International Journal of Advanced Computer Science and Applications, 13(1), 329–334.

  Google Scholar

  Crossref

• Rasch, G. (1960). Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests. Educational tests and measurements. Danmarks Paedagogiske Institut. https://books.google.com.my/books/about/Probabilistic_Models_for_Some_Intelligen.html?id=aB9qLgEACAAJ&redir_esc=y

• Şengül Avşar, A. (2019). Comparison of Person-Fit Statistics for Polytomous Items in Different Test Conditions. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 10(4), 377–393.

  Google Scholar

  Crossref

• Shi, L. (2022). Applying Rasch Measurement to Validate the Colorado Learning Attitudes about Science Survey. Conference: AERAAt: San Diego, 1–15.

• Taherdoost, H. (2018). Sampling Methods in Research Methodology; How to Choose a Sampling Technique for Research. SSRN Electronic Journal, January 2016.

  Google Scholar

  Crossref