*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************
Title:
Bridging MCMC and Optimization
Abstract:
In this talk, I will discuss three ingredients of optimization theory in the context of MCMC: Non-convexity, Acceleration, and stochasticity. I will focus on a class of non-convex objective functions arising from mixture models. For that class of objective functions, I will demonstrate that the computational complexity of a simple MCMC algorithm scales linearly with the model dimension, while optimization problems are NP-hard.
I will then study MCMC algorithms as optimization over the KL-divergence in the space of measures. By incorporating a momentum variable, I will discuss an algorithm which performs accelerated gradient descent over the KL-divergence. Using optimization-like ideas, a suitable Lyapunov function is constructed to prove that an accelerated convergence rate is obtained.
Finally, I will present a complete recipe for constructing stochastic gradient MCMC algorithms that translates the task of finding a valid sampler into one of choosing two matrices. I will then describe how stochastic gradient MCMC algorithms can be applied to applications involving temporally correlated data, where the challenge arises from the need to break the dependencies when considering minibatches of observations.
Bio:
Yian Ma is currently a post-doctoral fellow at University of California, Berkeley, hosted by Michael I. Jordan at the Foundations of Data Analysis Institute and RISELab. Prior to that, he obtained his PhD from applied mathematics department at University of Washington, working with Emily B. Fox at Mode Lab and Hong Qian. Before that, he obtained his bachelor's degree from
