Closed-form noise predictors: the optimal of a Gaussian mixture, with its exact input Jacobian and parameter VJP. A ring of narrow components is a circle relation with an exact score, which makes implicit inference and its backward pass checkable against arithmetic.
Sources: code:
analytic.jlTheory (CT-ML wiki): Statistical Game · Bayesian Inversion
1. The formulas
For under the VP-SDE, is the mixture with means and variance . With , responsibilities and :
The input Jacobian is times ( minus a covariance of the ), so it is symmetric, as must be.
2. Implementation difficulties
- The sign of the covariance term. The first prototype had ; finite differences caught it (error 0.58). The responsibilities’ gradient is , not .
- Log-sum-exp. Responsibilities are computed from shifted log-weights, because at small the exponents are of order .
- Equal weights, isotropic components. Enough for relations along curves; general weights and covariances would add parameters but no new ideas.
3. Why it lives in src/ and not in test/
It is an oracle, the same role GaussianFactor plays in the parent package. It is also a
usable model: a mixture whose means are trained by implicit’s adjoint is an implicit learner
with an exact score, which Backpropagation through Implicit Inference §7 uses to learn a
parabola from a circle.
4. The derivative interface it implements
epsilon_jacobian, epsilon_jacobian_block (the block on the free coordinates, summed over a noise level’s nodes), epsilon_vjp_input and epsilon_vjp_params are defined in this file for every predictor. A
closed-form predictor implements them exactly, as above. A NoisePredictor around a Lux network
gets them from the AD backend in its ad field, through a package extension; without one the
Jacobian falls back to finite differences and the VJP raises an error that names the fix
(backends). The mixture’s gradient and Hessian of are reused by MixtureProx,
the exact proximal operator (proxdm).
Related: implicit, predictor, backends, proxdm, Implicit Diffusion Learners
LP-018 column times and finite weights (2026-10-29)
The generic batched input VJP selects the time column with the corresponding state column,
including scalar times expanded over noise blocks. Passing the full time matrix into each
single-point Jacobian caused a shape error. Mixture constructor weights are scaled by their
maximum before summation, and nonfinite weights are refused. Evidence: test/lp018.jl,
“LP-018.4 generic batched VJP column times” (epsilon = X .* T, matrix and scalar times)
and “LP-018.9 mixture weights avoid finite overflow”. The mixture-specific Jacobian block,
VJP and JVP already select per-column coordinate times.