DiffusionFactorwith the deterministic implicit solver as its inversion:DiffusionFactor(blocks, pred; prox = ImplicitProx(nodes)).invert,factor_messageandlocal_free_energywork unchanged;implicit_solutionreturns the solver’s report andimplicit_factor_pullbackthe backward pass, per channel.
Sources: code:
implicit_factor.jl,factor.jl,implicit.jlTheory (CT-ML wiki): Bayesian Lens · Statistical Game · Open Model
1. How it is wired
DiffusionFactor’sproxfield may be aREDDiffor anImplicitProx._run_prox(f, …)dispatches on the type of the field, soinvert,factor_messageandlocal_free_energyinfactor.jlneeded no change beyond that one line.- The problem is built from the factor exactly as RED-Diff’s is:
assemble_statelays the incoming beliefs into the state vector,precision_vectorreads the polarity, and infinite precisions becomeInfentries (hard clamps) of the solver’s precision vector. - The prior is the warm start, as in
ImplicitLayers’ DEQ factor: its point seeds the solver, and on a multivalued relation it selects the branch. Whether it also anchors the solution is the polarity’s call:default_precision(Unobserved()) == 1is an anchor; precision0is a pure conditional. - The
scoresummand of the graded energy is the fixed-node quadrature of the weighted denoising loss, so the factor’s free energy is deterministic (RED-Diff’s is one Monte-Carlo sample).
2. The backward pass, per channel
implicit_factor_pullback(f, polarity, inputs, π, z̄, ps, st) takes the downstream cotangent
as a NamedTuple over channels and returns cotangents for each channel’s incoming point
(inputs), for the predictor’s parameters (ps), and for each channel’s precision
(precisions). A channel has one precision for all its coordinates, so its gradient is the
sum over the block. A hard clamp’s precision is infinite and has no gradient (zero is returned).
3. Implementation difficulties
- Dispatch on a value parameter.
DiffusionFactor{names,D,P,C}has a value parameter (names, a tuple of symbols). A method onDiffusionFactor{<:Any,<:Any,<:Any,<:ImplicitProx}never matches, because<:Anyconstrains to types; written withwhere {N,D,P}it matched inisabut lost on specificity against the generic method, because the bounds ofDandPwere dropped. Dispatching onf.proxinstead sidesteps both. - Include order.
ImplicitProxmust exist beforeDiffusionFactor’s type bound is evaluated, soanalytic.jlandimplicit.jlare included beforefactor.jl, and the functions that need both live in this separate file. - What is still not fixed. The diffusion model’s own prior is in every message, so a message is a posterior, not a likelihood (The Diffusion Factor §4); §5 divides out the anchor, which is the part that can be divided out.
4. How it is tested
The test set “DiffusionFactor with the implicit solver as its inversion”: branch selection by
the prior, agreement with the bare solver, the message, a deterministic free energy, the
per-channel pullback against finite differences of invert, the dimension check, and the error
for a RED-Diff factor.
The test set “answers as beliefs” covers §5: the energy of a closed-form mixture against the
field, laplace_belief against implicit_laplace, both branches of a circle query as a mixture
whose weights recover a 3:1 reweighting of the data (odds ratio 3.0), and the factor’s Gaussian
message, which times the anchor gives the posterior.
5. Gaussian messages
ImplicitProx(nodes; λ, message = :gaussian) makes the factor return a GaussianBelief on the
concatenated unobserved blocks instead of a DiracBelief:
| call | returns |
|---|---|
invert | the joint Laplace posterior on all unobserved blocks (implicit §5) |
likelihood_message | the same joint belief, with the anchors on those blocks divided out |
factor_message | the same, with the target’s own anchor divided out |
The solver’s query multiplies in an anchor on the target, , from the incoming belief and the polarity’s precision. For a joint target, the anchor is the product of these diagonal anchors on the concatenated blocks. If the message kept it, the variable would count its own belief twice: once inside the message, once when it pools the message with itself. In canonical form the division is a subtraction,
exact for the Laplace approximation. Marginalising onto the target commutes with it, because the
anchor is diagonal and sits on the target’s own coordinates. The result may be improper, a
likelihood that constrains only some directions, which GaussianBelief allows.
What cannot be divided out is the diffusion model’s own prior: the message is the model’s conditional belief about the target given the inputs, not a likelihood (Open Problems in Implicit Diffusion Learning T10). Two more conditions:
- covariances are in data units only with
λ = density_lambda(schedule, nodes); - the answer must be converged and stable; otherwise the call throws rather than sending a meaningless curvature.
6. Cost options
ImplicitProx(nodes; batched = true, linsolve = KrylovSolve()) passes both options on: the
factor’s model evaluates one noise level per network call, and its solve, its backward pass and
its Gaussian message are matrix-free (krylov). The defaults are the old behaviour.
7. Dividing out the anchor, and the prior’s cotangent
For message = :mixture, dividing the anchor out of each component also changes its normaliser: the
message’s log-weight is the posterior’s plus , with the
Gaussian log-partition, so that multiplying by the anchor gives back the posterior’s weights as well
as its components. A component made improper by the division throws; it has no normalised form.
The pullback’s prior field follows assemble_state: only unobserved blocks without an incoming point
read . For a joint prior point their cotangents fill the corresponding concatenated
ranges; for a shared-block point they add. When inputs override all of them the prior’s
cotangent is zero. “Joint diffusion messages (LP-017)” checks joint ranges and cotangents,
Gaussian covariance and mixture pooling; the single-target calls retain their original path.
The joint analogue is checked by “Joint mixture anchor pooling (LP-017)”; removing the
only confining anchor is rejected by “Joint mixture rejects undefined anchor division
(LP-017)“. Tests: “mixture likelihood removes anchor normalization”, “prior cotangent follows assemble_state
dependencies” (against finite differences).
Related: implicit, factor, The Diffusion Factor, Inference Signatures