*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************
Title: Feynman-Kac Numerical Techniques for Stochastic Optimal Control
Date: Monday, August 9th, 2021
Time: 11:00 AM - 1:00 PM (EDT)
Location: BlueJeans meeting (https://bluejeans.com/303268392/2685)
Kelsey Hawkins
Robotics Ph.D. Candidate
Institute for Robotics and Intelligent Machines
Georgia Institute of Technology
Committee:
Dr. Panos Tsiotras (Advisor) – School of Aerospace Engineering, Georgia Tech
Dr. Dmitry Berenson – Electrical Engineering and Computer Science Dept., University of Michigan
Dr. Sam Coogan – School of Electrical and Computer Engineering, Georgia Tech
Dr. Evangelos Theodorou – School of Aerospace Engineering, College of Computing, Georgia Tech
Dr. Kyriakos Vamvoudakis – School of Aerospace Engineering, Georgia Tech
Abstract:
Three significant advancements are proposed for improving numerical methods in the solution of forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control (SOC) problems. First, we propose a novel characterization of FBSDE estimators as either on-policy or off-policy, highlighting the intuition for these techniques that the distribution over which value functions are approximated should, to some extent, match the distribution the policies generate.
Second, two novel numerical estimators are proposed for improving the accuracy of single-timestep updates. In the case of LQR problems, we demonstrate both in theory and in numerical simulation that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems.
Third, we propose a new method for accelerating the global convergence of FBSDE methods. By the repeated use of the Girsanov change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree.
