Nonparametric Bayes and Exchangeability

This tutorial took place as part of the Bootcamp preceding the MIFODS Workshop on Graphical models, exchangeable models, and graphons. See this link for the latest versions of all tutorials.

Part I: Sunday, August 18, 11:30 AM–12:15 PM
Part II: Wednesday, August 18, 1:45 PM–2:30 PM

  Professor Tamara Broderick


This tutorial introduces nonparametric Bayes (BNP) as a tool for modern data science and machine learning. The tutorial then examines exchangeability as a tool for understanding when to use BNP (and other Bayesian) methods as well as when not to use them. BNP methods are useful in a variety of data analyses---including density estimation without parametric assumptions and clustering models that adaptively determine the number of clusters. We will demonstrate that BNP allows the data analyst to learn more from a data set as the size of the data set grows and see how this feat is accomplished. We will describe one of the most popular BNP models: the Dirichlet process. In the second part, we will look more closely at (infinite) exchangeability, the assumption that a distribution over our data is unchanged by permutations of the data for any finite data size. In particular, we will see how an exchangeability assumption implies the Kingman paintbox characterization in clustering, or partition, models -- and how such an assumption implies the Aldous--Hoover theorem for network, or graph, valued data. We will use these observations to critique models we saw in the first part of the tutorial as well as other models. We will briefly discuss potential solutions.



Working knowledge of Bayesian data analysis. Know how to use Bayes' Theorem to calculate a posterior for both discrete and continuous parametric distributions.

What we won't cover

Gaussian processes are an important branch of nonparametric Bayesian modeling, but we won't have time to cover them here. We'll be focusing on the discrete, or Poisson point process, side of nonparametric Bayesian inference.