Dyanimic programming is a very useful idea used in the study of control problem. We
first consider a deterministic control problem to show the idea. We derive the dynamic
programming principle and then the dynamic programming equation (or
Hamilton-Jacobi-Bellman equation, HJB equation). We discuss the verification
theorem. The theorem state that a solution of HJB equation is the value function of
the control problem. It also shows that a control derived from the solution is optimal.
The HJB equation usually is a PDE. We also briefly discuss the concept of viscosity
solution. This is very useful for HJB equation. The similar ideacan be also used to
study the stochastic control problem. We can formally derivedynamic programming
principle and HJB equation. There is a major difference between deterministic and
stochastic control problem. A rigorous proof of the dynamic programming principle
usually may require some continuity property of the value function.
Reference：1. Hanspeter Schmidli, Stochastic Control in Insurance
2. W. H. Fleming and2. W. H. Fleming and H.M. Soner, Control Markov Processes and Viscosity Solutions