Adversarial Objectives and Divergences

Dive into Deep Learning · §16.2

Adversarial objectives and divergences
one template, two choices · every loss is a divergence · critics estimate the ratio · which objectives keep a gradient

One Template Generates the Classical Objectives

d(p, q) = \sup_{T \in \mathcal{T}} \big\{ E_{x \sim p}[a(T(x))] - E_{x' \sim q}[b(T(x'))] \big\}

Two independent choices:

  • Scoring functions: with an unconstrained critic, nonlinear scores make the supremum pointwise and produce f-divergences.
  • Critic class: with linear scores, restricting the critic to a ball produces integral probability metrics.

A supremum of affine functionals is convex in (p,q), so these objectives have no spurious local minima in distribution space. The pairing objective in the next section is quadratic and does not fit the template.

Every Proper Loss Yields an f-Divergence

Replace \log \sigma by a general score function \ell: V_\ell(D) = E_p[\ell(D)] + E_q[\ell(-D)].

The optimized objective is determined by the Bayes-risk gap between a classifier that observes the sample and one that does not:

\Delta_\ell(p, q) = L(\tfrac12) - E_{x \sim m}\big[L(\eta(x))\big], \qquad \eta = \frac{p}{p + q}

Proposition. \Delta_\ell = D_f(p \,\|\, q) with f(u) = L(\tfrac12) - \tfrac{u+1}{2} L\big(\tfrac{u}{u+1}\big) — convex, f(1) = 0. Proof: substitute u = p/q; concavity survives the perspective.

Duality: the Optimal Critic Is f′(p/q)

Any f-divergence becomes a game (f-GAN):

D_f(p \,\|\, q) \;\geq\; E_p[T] - E_q[f^*(T)]

Fenchel–Young holds with equality at the slope:

T^\star = f'\!\left(\frac{p}{q}\right)

  • The optimal unconstrained critic reports a fixed transform of the density ratio; the loss determines that transform.
  • \operatorname{dom} f^* fixes the output activation; KL row: T^\star = 1 + \lambda, no activation needed.
  • The JS row, reparameterized by T = \log(2\sigma(D)), is exactly the original GAN’s value function.

Constrain the Critic Instead: Integral Probability Metrics

d_{\mathcal{F}}(p, q) = \sup_{h \in \mathcal{F}} \big\{ E_p[h] - E_q[h] \big\}

  • A symmetric class gives a pseudometric, and a metric when the class separates distributions; the supremum does not reduce to p/q.
  • Kernel ball → MMD: supremum analytic for a fixed kernel — no critic, no inner loop; cost O(n^2) per batch; learned features reopen the kernel choice (KID, :numref:sec_dcgan).
  • Lipschitz ball → W_1: Kantorovich–Rubinstein duality; the constraint is enforced by clipping, gradient penalty, or spectral norm.

JS Saturates; Transport and Kernels Keep Slope

Two unit Gaussians at separation d — all three objectives evaluated without sampling error:

JS at d = 8: 0.693054 nats (ceiling log 2 = 0.693147)

JS approaches \log 2 once the overlap vanishes. The distance W_1=d keeps slope one at every separation. MMD retains a slope over distances comparable to its kernel length and becomes insensitive at larger separations.

Four Losses, One Fixed Point

Same generator, same target, four objectives — logistic, least squares, hinge, MMD (no critic):

All four generators recover the three modes, but their exact \mathrm{JS}(p,q_t) traces follow different optimization trajectories.

What Each Critic Estimates

Freeze one imperfect generator with known density; train a fresh critic per loss; compare with the predicted link of the analytic \lambda:

snapshot JS(p, q) = 0.046 nats

The three losses recover the identity, sigmoid, and step transforms of the same ratio. Each estimate degrades where |\lambda| is large and samples are scarce.

The KL Critic Recovers 1 + log(p/q)

Train T on E_p[T] - E_q[e^{T-1}] between two known Gaussians:

The critic follows the analytic parabola where the mixture has appreciable density. The attained bound approaches the exact KL from below, so an imperfect critic underestimates the divergence.

Gradient Quality and Estimation Cost

  • MMD: parametric rate O(n^{-1/2}), any dimension — cheapest estimator, weakest discrepancy.
  • W_1: rate n^{-1/d} — preserves a gradient under separation but converges slowly in high dimensions.
  • Density ratio: sharpest signal, hardest where the ratio is extreme — moments of the ratio can be infinite at finite divergence.

No corner of the template dominates. The objective→divergence map is collected in :numref:sec_mdl-divergence-objective-map.

Recap

  • One template: scoring functions and critic class.
  • Every proper loss → Bayes-risk gap → an f-divergence; the critic reports a fixed link of \lambda in every row.
  • Duality: T^\star = f'(p/q) — adversarial critics do density-ratio estimation.
  • IPMs constrain the critic: MMD analytic with a fixed kernel, W_1 from Lipschitz duality.
  • Dichotomy, computed: ratio-based objectives saturate under separation; geometry-based ones keep a gradient — and cost more to estimate.
  • The next section keeps the log loss and changes the critic to score pairs of samples.