This tutorial is part of the FoMICS-DADSi Summer School on
Data Assimilation. The tutorial is taking place at the Università della Svizzera italiana (USI) in Lugano, Switzerland on Friday 2018 September 14 and Saturday 2018 September 15.
See this link for other tutorials.
Instructor:
Professor Tamara Broderick
Email: 
Title: Variational Bayes and beyond: Bayesian inference for big data
Abstract: Bayesian methods exhibit a number of desirable properties for modern data analysis---including (1) coherent quantification of uncertainty, (2) a modular modeling framework able to capture complex phenomena, (3) the ability to incorporate prior information from an expert source, and (4) interpretability. In practice, though, Bayesian inference necessitates approximation of a high-dimensional integral, and some traditional algorithms for this purpose can be slow---notably at data scales of current interest. The tutorial will cover modern tools for fast, approximate Bayesian inference at scale. One increasingly popular framework is provided by "variational Bayes" (VB), which formulates Bayesian inference as an optimization problem. We will examine key benefits and pitfalls of using VB in practice, with a focus on the widespread "mean-field variational Bayes" (MFVB) subtype. We will highlight properties that anyone working with VB, from the data analyst to the theoretician, should be aware of. We will cover linear response variational Bayes (LRVB) as one possible correction for some of the potential pitfalls of MFVB and VB -- as well as a tool for assessing sensitivity, or robustness. In addition to VB, we will cover recent data summarization techniques for scalable Bayesian inference that come equipped with finite-data theoretical guarantees on quality. We will motivate our exploration throughout with practical data analysis examples and point to a number of open problems in the field.