Towards Using Eye-Tracking and Consumer-Grade Electroencephalogram Devices To Detect Usability Issues in Mobile Applications

Abstract

Despite the importance of cognitive workload in examining the usability of smartphoneapplications and the popularity of smartphone usage globally, cognitive workload as one attribute of usability tends to be overlooked in Human Computer Interaction (HCI) studies. Moreover, limited studies that have examined the cognitive workload aspect often measured some summative workloads using subjective measures (e.g., questionnaires). A significant limitation of subjective measures is that they can only assess the overall, subject-perceived cognitive workload after the procedures/tasks have been completed. Such measurements do not reflect the real-time workload fluctuation during the procedures. They, therefore, are not useful for pinpointing poor designs in user interfaces that are associated with cognitive workload surges in the user’s brain during a task. This dissertation used mixed methods to empirically study (1) the reliability of an eye-tracking device (i.e., Tobii Pro Nano) and a low-cost electroencephalogram (EEG) device (i.e., MUSE 2) for detecting real-time cognitive workload changes during N-back tasks, and (2) the potential to use the increased cognitive workload detected during tasks to pinpoint user interface areas containing potential usability issues in mobile applications. Results suggest that (1) the EEG measurements collected by MUSE 2 are not very useful as indicators of cognitive workload changes in our setting; (2) eye movement measurements collected by Tobii are useful for monitoring cognitive workload fluctuations and tracking down interface design issues in a smartphone setting; (3) more specifically, the maximum pupil diameter is the preeminent indicator of cognitive workload surges; and (4) cognitive workload surges may be caused by design issues. One usability issue has been detected and fixed this way in a mobile application designed by a National Science Foundation (NSF) sponsored project. In conclusion, the pupil diameter measure combined with other subjective ratings would provide a comprehensive user experience assessment of mobile applications. They can also be used to verify the successfulness of a user interface design solution in improving user experience.