BUSINESS MODELING SYSTEMS: COMPARING LEARNING PERFORMANCE AND IDENTIFYING LEARNING COMPLEXITIES

Abstract

Information system professionals need methods to systematically obtain data about interactive computer systems from the users' perspective. This research was concerned with business modeling systems and measuring the performance of subjects learning fundamental operations of the system through training and actual "hands-on" use. The objective was to develop a methodology to measure and evaluate two specific systems: IFPS and VISICALC; to compare subjects' performance in building simple financial projection models and in using the models to answer questions; and to identify learning difficulties associated with the systems. Subjects were business school students with no previous business modeling system experience. Formal measurement was made of the time taken to complete three planning tasks through use of the system. The model user was also the model builder. Performance in decision making was not investigated. Informal observations were made about difficulties the subjects encountered when learning the system. Subjects received two hours of classroom instruction and approximately one hour of "hands-on" training with a specific system. Then under controlled laboratory conditions, they individually demonstrated their ability to use the system. The type and frequency of errors made was also recorded. Subjects were required to use the system until they had completed all tasks correctly. Subjects using IFPS completed tasks in a shorter time than those using the positional system VISICALC. The research methodology was successful in providing feedback for training and design modifications, and also provided for a successful comparison of differences in learning performance when analyzing the effects of previous experience and type of system. Results showed a wide range of performance by subjects, even though they had similar backgrounds. Results suggest that a keyword system is easier to learn for novice users who are learning the fundamental operations of a business modeling system.