Conducting Pilot Test of Assistive Courseware for Low Vision (AC4LV) Learners: Initial Round

Abstract

This is an ongoing study which related to the development of Assistive Courseware for Low Vision (ACLV) learners. As discussed in the previous study, low vision learners face difficulties in their learning activities particularly in terms of information accessibility, navigationability, and pleasurability. Due to that, AC4LV has been developed based on the Conceptual Design Model of AC4LV which was constructed by utilizing User Centred Design (UCD) approach. Expert review was conducted to validate the proposed model. Having developed the prototype it has to be tested to the intended subjects. Qualitative approach was found the most suitable technique to make test to the low vision learners as they are in children age. Prior to run the actual testing pilot test has to be conducted to ensure the actual testing could be run smoothly. Therefore, this study reports the initial round of conducting the pilot test of AC4LV learners. The test was segmented into four segments; (i) briefing, (ii) observation, (iii) focus group interview, and (iv) closing. The results indicates that there are problems in terms of the feasibility of the prototype and data collection method.

Keywords: Assistive Technology (AT);Pilot testAssistive Courseware (AC)Low vision learners

Introduction

Low vision refers to a person who has a profound visual disability, but still retains some useful eyesight. They have a limited ability to discriminate visual detail. Most of the available learning materials do not support them particularly in terms of content presentation. Nurulnadwan, Nur Hazwani, and Ariffin (2011) Nurulnadwan, Ariffin, Siti Mahfuzah, and Mohd Saifullizam (2013) has explore and reveal the problems faced by the low vision learners in their learning activities. As a solution, a conceptual design model of a learning material designed specifically for the low vision learners has been constructed as explained in Nurulnadwan, Ariffin, and Siti Mahfuzah (2014a) Nurulnadwan, Ariffin, and Siti Mahfuzah (2014b) Nurulnadwan, Ariffn, Aiti Mahfuzah, (2014c). The model has been validated through expert review method (Nurulnadwan, Ariffin, & Siti Mahfuzah, 2015a). Next (Nurulnadwan, Ariffin, & Siti Mahfuzah, 2015b) explains the validation process of the proposed model through prototyping method. Furthermore, AC4LV has to be tested to the intended subjects in order to confirm that it could fulfill the low vision learners learning needs in terms of information accessibility, navigationability, and pleasurability. By referring to Hartson and Pyle (2012) user experience testing is the most suitable type of evaluation for this study. Prior to run the actual testing, pilot test should be conducted. Therefore, the main objective of this study is to conduct the pilot test of AC4LV learners. This study discussed the initial round of the test and referred it as pilot test 1.

Special testing requirement

When working with disabled learners as the research subjects, it is important to adopt creativity, multi-method, and flexible approaches to tailor to their needs (Shaw, Brady, & Davey, 2011). They are special, who are less predictable than the adolescent or adult computer users. This means written instructions are meaningless because they are not able to understand the concepts used for adults (Raisamo et al., 2006). Similarly, Shaw et al. (2011) also do not advocate with questionnaire particularly self-completion type for children under 12 years old because they may not respond genuinely or their response are influenced by biasness. Also, questionnaire or web-based survey is certainly not appropriate if the research subjects are sensitive (Shaw et al., 2011). In this study, the subjects have restriction in their sense of seeing. Even though if the questionnaire is read-aloud, it is very hard for them to make decision in choosing the most appropriate answer. Many aspects would influence them in making decision, such as confused with the scales and finally getting tired because this study has to cover three aspects of user experience as mentioned in the earlier section. Eventually, the biased responses may affect the credibility of this study. Hence, quantitative approach is less applicable for this study. Thus, multidisciplinary aspects have to be covered in order to obtain the truthful results. As of the sensory impairment, VI children including low vision can have unclear concepts about trying out new things which differ from the sighted children. For that reason, concepts connected to the testing situation have to be carefully explained and presented to ensure that the subjects have the correct understanding (Raisamo et al., 2006). Accordingly, the testing procedures for this study cover the aspects of communication, disabilities, social and developmental psychology, and special pedagogy as suggested by (Raisamo et al., 2006). They are applied in both techniques of qualitative approach; observation and interview. The next section discussed the methodology utilized to conduct the pilot test.

Methodology

To seek findings of user experience on AC4LV and to increase the credibility, validity, and reliability interpretation of the findings, this study manage to have the initial round of pilot test. Fig. 1 illustrated the summary of activities involve to carried out this study.

Figure 1: Fig. 1. Summary of activities
Fig. 1. Summary of activities
See Full Size >

Pilot test with low vision learners

The purpose of pilot test with low vision learners in qualitative approach is different than in quantitative approach. In this study, pilot test was carried out with the main objective to investigate the wellness and consistency of user experience testing. Meanwhile, the specific objectives are (i) to determine the feasibility of the proposed model (prototype of AC4LV) and (ii) to identify any problems with the equipment and data collection methods (Herrington, 1997). This investigative work is important in determining if there are flaws, limitations, or other weaknesses in the prototype of AC4LV, equipment, or data collection methods which allow this study to make necessary revisions prior to commencing the main study of user experience testing as suggested by Turner (2010) and (Kim, 2010). To achieve both sub-objectives, this study manages to execute the initial round of pilot test which are discussed in the following subsections.

Pilot test I

The initial round of pilot test was carried out at one of the Integration Primary School (Visually-impaired) in Malaysia. Seven low vision children between nine and 12 years old involved in this test. Four of them were males while the remaining were females. However, one of the girls has to be excluded, because she was standard two, which is beyond the scope of this study. So, the total of the subjects considered in the test is six.

The reasons for scoping that range of age are as discussed in Nurulnadwan et al. (2013). This number is sufficient for a pilot test of qualitative research to represent the population of low vision learners locally or broadly as they are homogeneous subjects.

Procedure

Subjects were expected to feel more comfortable in their natural environment such as their routine class. Accordingly, the test was carried out in their routine class they are used to. The test was segmented into four segments; (i) briefing, (ii) observation, (iii) focus group interview, and (iv) closing. Before the testing commenced, two laptops were setup with AC4LV. In addition, recording sheets, video recorder, audio recorder, and screen recorder are prepared to record the observation and interview session.

Having setup the equipment, the subjects were asked to get into two groups. This study managed to have four subjects in Group 1 and two subjects in Group 2. One laptop was attached to one group. They were setting to sit comfortably next to each other as seen in Fig. 2 and Fig. 3. They were briefed by introducing the team of the researcher and the purpose of this study. Finishing the first session, the second session started by playing the AC4LV to them. Think-aloud protocol was utilized in obtaining the truthful results. 30 minutes was enough for them to explore all the topics and activities in the AC4LV. That allocation time was also used by the researcher and the team member to record the observation in the form of video, notes, and photographs. Then, focus group interview was carried out in the third session of the testing. In this session, semi-structured interview questions wereaddressed to them. Having finished this session, the session was closed by thanking and giving token to the subjects.

Figure 2: Fig. 2. Group 1
Fig. 2. Group 1
See Full Size >
Figure 3: Fig. 3. Group 2
Fig. 3. Group 2
See Full Size >

Findings and Discussion

The pilot test I revealed a number of problems and inadequacies that have to be attended to, prior to conducting the main study. The adjustments that needed to be made are related to the feasibility study of the prototype itself and the practicalities of the data collection methods.

Problems with AC4LV (feasibility study of the prototype)

The AC4LV was completed seven days before the pilot study commenced. It was expected that there was no problem with AC4LV after quality checking and packaging. However, the use of prototype in the pilot test 1revealed a little “bugs” that needed to be attended to. The problems are (i) suddenly “ activity 2” stucks, which could not be accessed. So, the subjects were quite frustrated, (ii) there are noise in some of the topics and activities that needed to be removed, (iii) the screen resolution of AC4LV played in laptop for Group 2 was not fix to the screen size when viewed in full screen presentation and (iv) the timing of transition was quite bored . Thus, none of the “bug” was found serious to obstruct the data collection of the pilot study. Nevertheless, all of the problems were brought to the attention of developer who corrected them for the use in the main testing.

Problems with the Data Collection Method

The initial round of the pilot test also revealed some problems with data collections methods primarily with the equipment setting, which impedes the fluency of the user experience testing. All problems encountered with data collection methods, their causes, and solutions are listed in Table 1 .

Table 1 -
See Full Size >

Conclusion

It is important for this study to attach one computer for one subject in order for them to get the experience of using AC4LV individually. This is due to the results obtained from the observation, that one of the subjects of Group 1 showed uninterested to join the testing as he faced restricted access to the laptop. Furthermore, it was found that audio is not applicable to be played through loud speaker, which could disturb each of the group. Even this procedure has not affected the whole results but the experience of using the AC4LV is important to each of the subject. However, this study found that this type of testing would encourage the subjects to have active social interaction among each other regarding the content in the AC4LV. This influences them for not feeling afraid or introverted during the focus group session, so that they cooperate positively. Due to the detected problems, the second round of pilot test has to be conducted in future works of this study.

References

  1. Hartson, R., & Pyle, P. (2012). The UX book, process and guidelines for ensuring a quality user experience. London: Elsevier.
  2. Herrington, J. (1997). Authentic learning in interactive multimedia environments. (Doctoral Dissertation, Cowan University Faculty of Science, Technology and Engineering, 1997). Retrieved from http://www.academia.edu/176919/Authentic_learning_in_interactive_multimedia_environments
  3. Kim, Y. (2010). The pilot study in qualitative inquiry: Identifying issues and learning lessons for culturally competent research. Qualitative Social Work, 10(2), 190–206.
  4. Nurulnadwan, A., Nur Hazwani, M. R., & Ariffin, A. M. (2011). Visually impaired children’s acceptances on assistive courseware. American Journal of Applied Sciences, 8(10), 1019–1026.
  5. Nurulnadwan, A., Ariffin, A. M., Siti Mahfuzah, S., & Mohd Saifullizam, J. (2013). Preliminary investigation on creative educational content for visually-impaired (VI) learners. In H. Badioze Zaman, P. Robinson, O. Patrick, T. K.Shih, & S. Velastin (Eds.), Advances in Visual Informatics (3rd ed., pp. 408–417). Switzerland: Springer International Publishing.
  6. Nurulnadwan, A., Ariffin, A. M., & Siti Mahfuzah, S. (2014a). A comparative analysis on conceptual design model of Assistive Courseware (AC) for visually-impaired learners (AC4VI). Australian Journal of Basic and Applied Sciences, 8(4), 75–80. Retrieved from http://ajbasweb.com/old/ajbas/2014/Special/75-80.pdf
  7. Nurulnadwan, A., Ariffin, A. M., & Siti Mahfuzah, S. (2014b). Critical analysis in proposing a conceptual design model of assistive courseware for low vision (AC4LV) learners. International Journal of Computer Applications, 92(10), 18–25.
  8. Nurulnadwan, A., Ariffn, A. M., & Siti Mahfuzah, S. (2014c). Conceptual design model of assistive courseware for low vision (AC4LV) learners. The Proceedings of International Conference on Advances in Educational Technology (ICAET ’14), 1–12.
  9. Nurulnadwan, A., Ariffin, A. M., & Siti Mahfuzah, S. (2015a). Expert review conceptual design and development model of assistive courseware for young low vision (AC4LV) learners. International Journal of Conceptions on Management and Social Sciences, 3(2), 35–39.
  10. Nurulnadwan, A., Ariffin, A. M., & Siti Mahfuzah, S. (2015b). The design principles of assistive courseware for low vision (AC4LV) learners. ARPN Journal of Engineering and Applied Sciences, 10(3), 1447–1456.
  11. Raisamo, R., Hippula, A., Patomäki, S., Tuominen, E., Pasto, V., & Hasu, M. (2006). Testing usability of multimodal applications with visually impaired. MultiMedia, IEEE, 13(3), 70–76.
  12. Shaw, C., Brady, L., & Davey, C. (2011). Guidelines for research with children and young people. NCB Research Centre, London.
  13. Turner, D. W. (2010). Qualitative interview design : A practical guide for novice investigators. The Qualitative Report, 15(3), 754–760. Retrieved from http://www.nova.edu/ssss/QR/QR15-3/qid.pdf

Copyright information

This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

About this article

Cite this paper as:

Click here to view the available options for cite this article.

Publisher

Future Academy

First Online

18.12.2019

Doi

10.15405/epsbs.2016.08.32

Online ISSN

2357-1330