Automating test generation for discrete event oriented real-time embedded systems

Abstract

The purpose of this work is to provide a method for the automatic generation of test scenarios from the behavioral requirements of a system. The goal of the generated suite of test scenarios is to allow the system design to be validated against the requirements. The benefits of automatic test generation include improved efficiency, completeness (coverage), and objectivity (removal of human bias). The Model-based Codesign method is refined by defining a design process flow. This process flow includes the generation of test suites from requirements and the application of these tests across multiple levels of the design path. An approach is proposed that utilizes what is called a requirements model and a set of four algorithms. The requirements model is an executable model of the proposed system defined in a deterministic state-based modeling formalism. Each action in the requirements model that changes the state of the model is identified with a unique requirement identifier. The scenario generation algorithms perform controlled simulations of the requirements model in order to generate a suite of test scenarios applicable for black box testing. A process defining the generation and use of the test scenarios is developed. This process also includes the treatment of temporal requirements which are considered separately from the generation of the test scenarios. An algorithm is defined to combine the test scenarios with the environmental temporal requirements to produce timed test scenarios in the IEEE standard C/ATLAS test language. An algorithm is also defined to describe the behavior of the test environment as it interprets and applies the C/ATLAS test programs. Finally, an algorithm to analyze the test results logged while applying the test scenario is defined. Measurements of several metrics on the scenario generation algorithms have been collected using prototype tools. The results support the position that the algorithms are performing reasonably well, that the generated test scenarios are adequately efficient, and that the processing time needed for test generation grows slowly enough to support much larger systems.