THEORETICAL AND PRACTICAL ASPECTS OF OPTIMIZATION IN FINITE WORD-LENGTH DIGITAL SIGNAL PROCESSING

Abstract

We consider a general class of digital signal processing problems in which some sytem parameters restricted to some finite set of values are selected based on the minimum meansquare error criterion. This class of problems are relevant in the design of modern communication, radar, and antenna array systems under finite word-length con-constraints. Linear problems allowing infinite precision values are given by the classical Wiener solutions. However, under the finite word-length constraints, optimum solutions can be obtained in principle by using techniques from integer quadratic programming problems. Practical solutions obtained in this manner, such as from the branch-and-bound algorithm, usually use a large amount of active computer memory space and thus are not applicable to real-time on-board processing situations. In this paper, we shall consider various theoretical and practical aspects of exhaustive search algorithms over shrinking sequences of constrained regions known to contain the desired optimum solution. Properties and bounds on the number of points to be searched as well as specific system problems shall be presented.