Self-optimizing stochastic systems: Applications to stochastic shortest path problem, stochastic traveling salesman problem, and queueing systems.

Abstract

We investigate stochastic systems which have a set of control parameters and a performance criterion. By operating the system at fixed control parameters, noisy performance values are observed. (The values are noisy due to the inherent stochastic nature of the system.) Certain relevant distributions of the system are assumed unavailable. The task is to develop algorithms that guide the system to optimal parameter settings based on its operating history. Consider the stochastic shortest path problem, where the time to traverse an edge is given by a random variable whose distribution is unavailable explicitly. The optimality criterion (to be maximized) is the probability of going from a given source node to a given terminal node within a specified critical time period. By choosing a particular path and traversing it, realizations of the distributions, i.e. time to go from the source node to the terminal node on that path are observed. Or consider the M/M/1 queue. Here, the control parameter is the average service time. The performance criterion is the sum of cost of the server and cost of system time of a customer in steady-state. By choosing a particular average service time and serving accordingly, a noisy observation of the total cost is obtained. We combine an asymptotically optimal random search method for finding the global optimum of a function with problem-specific local search techniques. Such a combination results in efficient solution procedures for the above problems. This conclusion is reached by applying the procedure to problems for which the optimum solutions are known. The main contribution of the study is in demonstrating that "reasonable" performance can be achieved for the proposed optimization problems in "reasonable" time by exploiting problem-specific structures to advantage. The generality of the method should allow others to use it in different optimization settings than ours. Also, the self-optimizing aspect of these methods and the stochastic versions of the local search techniques are new.