A practical guide to running experiments that teach you the most.
Experiments are expensive — in time, money, or participants. Bayesian experimental design asks a simple question before you spend any of them: which experiment will teach me the most? You write down what you currently believe, simulate the data each candidate design could produce, and score the designs by their expected information gain — how much the answer would shrink your uncertainty. Then you run the winner, update, and design the next round with what you just learned.
This page collects the best Learning Bayesian Statistics resources on the topic: the flagship episode with Adam Foster of Microsoft Research on EIG, BALD, EPIG and deep adaptive design, more episodes going deep on experimental design and active learning, and hands-on guides as they appear.
With Adam Foster, Microsoft Research
Which experiment is worth running? Adam Foster explains how to value an experiment before you run it: expected information gain, BALD, EPIG and deep adaptive design — and how to choose the data that reduces uncertainty the most.
Listen to the episode →Bayesian experimental design is the practice of choosing which experiment to run — or which data to collect — so that it teaches you as much as possible. Instead of collecting data passively, you score each candidate design by how much it is expected to reduce your uncertainty, run the winner, update your posterior, and repeat. Done adaptively, each round of data collection reshapes the next experiment.
Expected information gain is the utility that Bayesian experimental design maximizes: before running an experiment, you average over the data it might produce and measure how much your beliefs would move — formally, the expected reduction in posterior entropy, or the expected KL divergence from prior to posterior. The design with the highest EIG is the one you expect to learn the most from.
They are all information-gain criteria aimed at different targets. BALD (Bayesian Active Learning by Disagreement) is EIG about the model parameters — the classic active-learning objective. EPIG (Expected Predictive Information Gain) instead measures what an experiment teaches you about future predictions at the inputs you actually care about, so it avoids spending experiments on parameter regions that never change predictions. Plain EIG sits in between: it depends on the quantity of interest you define.
Start with the featured episode above: Adam Foster (Microsoft Research) explains EIG, BALD, EPIG and deep adaptive design in one sitting. Then try it yourself — simulate data from your prior, score candidate designs by expected information gain, and check how the ranking changes as data comes in.
New to Bayesian modeling itself? Start with the Bayesian workflow hub, explore all episodes or dive into the blog for more applied Bayesian modeling.