This monograph applies the relative optimization approach to time
nonhomogeneous continuous-time and continuous-state dynamic systems. The
approach is intuitively clear and does not require deep knowledge of the
mathematics of partial differential equations. The topics covered have
the following distinguishing features: long-run average with no
under-selectivity, non-smooth value functions with no viscosity
solutions, diffusion processes with degenerate points, multi-class
optimization with state classification, and optimization with no dynamic
programming.
The book begins with an introduction to relative optimization, including
a comparison with the traditional approach of dynamic programming. The
text then studies the Markov process, focusing on infinite-horizon
optimization problems, and moves on to discuss optimal control of
diffusion processes with semi-smooth value functions and degenerate
points, and optimization of multi-dimensional diffusion processes. The
book concludes with a brief overview of performance derivative-based
optimization.
Among the more important novel considerations presented are:
- the extension of the Hamilton-Jacobi-Bellman optimality condition from
smooth to semi-smooth value functions by derivation of explicit
optimality conditions at semi-smooth points and application of this
result to degenerate and reflected processes;
- proof of semi-smoothness of the value function at degenerate points;
- attention to the under-selectivity issue for the long-run average and
bias optimality;
- discussion of state classification for time nonhomogeneous continuous
processes and multi-class optimization; and
- development of the multi-dimensional Tanaka formula for semi-smooth
functions and application of this formula to stochastic control of
multi-dimensional systems with degenerate points.
The book will be of interest to researchers and students in the field of
stochastic control and performance optimization alike.
