This book gives an exposition of recently developed approximate dynamic
programming (ADP) techniques for decision and control in human
engineered systems. ADP is a reinforcement machine learning technique
that is motivated by learning mechanisms in biological and animal
systems. It is connected from a theoretical point of view with both
adaptive control and optimal control methods. The book shows how ADP can
be used to design a family of adaptive optimal control algorithms that
converge in real-time to optimal control solutions by measuring data
along the system trajectories. Generally, in the current literature
adaptive controllers and optimal controllers are two distinct methods
for the design of automatic control systems. Traditional adaptive
controllers learn online in real time how to control systems, but do not
yield optimal performance. On the other hand, traditional optimal
controllers must be designed offline using full knowledge of the systems
dynamics. It is also shown how to use ADP methods to solve multi-player
differential games online. Differential games have been shown to be
important in H-infinity robust control for disturbance rejection, and in
coordinating activities among multiple agents in networked teams. The
focus of this book is on continuous-time systems, whose dynamical models
can be derived directly from physical principles based on Hamiltonian or
Lagrangian dynamics.