From

Abstract

This item is not the definitive copy. Please use the following citation when referencing this material: Dillon, A. and Morris, M. (1998) From "can they?" to "will they?": extending usability evaluation to address acceptance. AIS Conference Paper, Baltimore, August 1998. Introduction: usability engineering: Within the human-computer interaction (HCI) community, there exists a long and rich research paradigm on "usability engineering (UE)." Within the usability engineering tradition, usability is operationally defined as the effectiveness, efficiency and satisfaction with which specified users can perform particular tasks in a given environment (see e.g., Shackel 1991, Nielsen, 1993). Effectiveness answers: can users perform their tasks? Efficiency means: what resources do users expend to achieve a given outcome (e.g., time, effort)? Finally, satisfaction measures assess how well users like the application. From this perspective, usability is contextually defined in operational terms that designers can see as targets to meet, for example: "Users should be able to perform specified tasks with new tool after W minutes training, with X% effectiveness, at least Y% efficiency, and Z% greater satisfaction than with old interface" where W < infinity, and 0< [X, Y, Z] <100. The strengths of the usability engineering approach include: 1. The use of operationalised measures that are negotiated in context, 2. The direct coupling of usability to tasks the tool must support, 3. The capability of negotiated targets to fit into an iterative design process, and 4. The decoupling of the usability construct from interface features Each of these strengths gives the approach value to the software industry where design practices require targets to be met and where the success of a new tool is determined contextually rather than in any absolute manner. Thus, the usability engineering paradigm has enjoyed a wide range of support from industry. Nonetheless, there are associated weaknesses of this approach. Some of these weaknesses include: 1. Usability criteria are dynamic, not fixed, 2. Usability is thus contextually determined so what works in one context may not work in another and design practices must continually ground themselves in work practices 3. Determining usability criteria requires considerable analytic skill, 4. Generalization beyond context is difficult, 5. Criteria do not determine re-design advice While the approach advocated by usability engineers of deriving appropriate targets for design and testing to meet is useful, it is clear that usability does not fully determine actual system use (see Dillon and Morris 1996). Thus, it is possible that designers may produce a well engineered artifact that meets set criteria, but still fails to gain the acceptance of discretionary users. In other words, usability is a necessary but insufficient determinant of use.