What is Expected Improvement?
Expected Improvement is an acquisition function that scores a candidate by the posterior expectation of its positive improvement beyond a defined incumbent or target.
Quick Facts
| Specification | Official Specification |
|---|
How It Works
Average both improvement magnitude and uncertainty
For maximization with posterior Y(x) ~ Normal(mu, sigma^2) and incumbent f*, analytic EI is (mu - f*) Phi(z) + sigma phi(z), where z = (mu - f*) / sigma. The first term rewards promising posterior means and the second rewards uncertainty that can cross the incumbent. The SMT EGO documentation derives the minimization form and places it in the iterative Gaussian Process optimization loop.
Define the incumbent correctly under noise and constraints
With noiseless observations, the best observed feasible value is a common incumbent. Under noise, the maximum raw observation may be a lucky error, so noisy EI variants integrate over uncertainty in latent values or fantasy outcomes. Constraints can weight improvement by feasibility or model objective and constraints jointly. Batch EI values correlated candidates as a set; multiplying independent single-point choices does not reproduce that utility.
Protect acquisition optimization from numerical failure
EI can become extremely small far from plausible improvements, causing floating-point values and gradients to vanish even when mathematical EI is positive. Ament and colleagues analyze this failure and propose LogEI-family transformations with the same or approximately equivalent optima and better numerical behavior. Use maintained implementations, multiple starts, candidate-set checks, and end-to-end regret comparisons rather than assuming the closed-form equation is sufficient.
Key Characteristics
- Measures posterior expected positive gain over an incumbent
- Combines posterior mean advantage with uncertainty
- Has an analytic form for one Gaussian candidate
- Changes meaning with objective direction, noise, and incumbent choice
- Requires joint or approximate treatment for batches and constraints
- Can suffer severe value and gradient underflow in naive implementations
Common Use Cases
- Selecting a next point for noiseless Bayesian Optimization
- Prioritizing candidates likely to beat a current feasible design
- Building a transparent baseline for acquisition-policy comparisons
- Extending sequential search to noisy or constrained objectives
- Choosing joint experiments through qEI or related batch variants
Example
Loading code...Frequently Asked Questions
How does Expected Improvement balance exploration and exploitation?
A candidate can earn EI through a mean above the incumbent, through uncertainty that leaves probability mass above it, or both. The balance is induced by the posterior and optional margin rather than by a fixed random exploration probability.
What should Expected Improvement use as the incumbent?
For noiseless feasible observations, the best observed value is common. With noise or uncertain constraints, a raw maximum can be optimistic. Use a method whose incumbent and latent-value treatment match the statistical model, and document that choice.
Is Expected Improvement always better than UCB or Thompson Sampling?
No. Their utilities and theoretical assumptions differ. EI is a strong baseline, but posterior misspecification, delayed batches, constraints, high dimension, or long-horizon information value can favor another policy. Compare them on repeated equal-budget tasks.
Why can a naive Expected Improvement implementation fail numerically?
When the chance of improvement is tiny, Gaussian tail terms can underflow and gradients can become exactly zero in floating-point arithmetic. Stable LogEI-style formulations and robust acquisition optimizers avoid treating a numerical zero as a mathematical one.
Does maximizing Expected Improvement guarantee a better observation?
No. EI is an expectation under the current surrogate posterior. The next observation can be worse, infeasible, or noisier than expected. Its value is assessed over repeated sequential decisions and total budget, not by requiring every individual trial to improve.