ProxDM
Proximal diffusion models
ProxDM (Fang et al. 2025) queries the prior through $\operatorname{prox}_{-\lambda\log p_t}$ instead of its score. proximal is the interface, ProxNetwork a learned prox ($v - \sqrt\lambda\,\varepsilon_\theta(v; t, \lambda)$), MixtureProx the exact one for a Gaussian mixture. proxdm_sample is the paper's sampler (PDA and PDA-hybrid), prox_infer the implicit learner's query by half-quadratic splitting, proximal_matching_loss the training loss.
Differentiating through proximal inference
prox_pullback(p, sol, z₀, ρ, t, z̄, ps, st; λ, linsolve) is the adjoint of prox_infer: one solve with $(I - \operatorname{diag}(a)D_{FF})^\top$, where $D$ is the prox's input Jacobian and $a_i = 1/(1+\lambda\rho_i^2)$, gives the cotangents of the clamps, targets, precisions and the prox's parameters. It is also the ChainRulesCore.rrule of prox_infer, so Zygote differentiates through it. MixtureProx has exact derivatives; a ProxNetwork takes them from its ad backend. With linsolve = KrylovSolve() only prox VJPs are used.
In a factor graph
DiffusionFactor(blocks, pred; prox = ProxDM(prox; t, λ)) inverts by prox_infer. The factor then has two parameter trees, (predictor = …, prox = …): the inversion reads the prox's, the free energy's score term the predictor's. implicit_solution and implicit_factor_pullback work as for ImplicitProx, the latter through prox_pullback.
proxdm_sample(…; scheme = :trapezoidal) is a second-order sampler at the same cost as PDA: one prox per step, the explicit score read off the previous prox.
Known gap: the factor's message is a point only (no Laplace Gaussian).
API
VariationalDiffusion.AbstractProximalPredictor — Type
AbstractProximalPredictorAnything that can evaluate $\operatorname{prox}_{-\lambda\log p_t}(v)$ through proximal(p, v, t, λ, ps, st) -> (u, st).
VariationalDiffusion.MixtureProx — Type
MixtureProx(mixture::GaussianMixtureEps; tol = 1e-10, maxiters = 100)The exact proximal operator of $-\log p_t$ for the Gaussian-mixture data distribution of GaussianMixtureEps, by a Newton solve with backtracking on the prox objective. It plays the role a perfectly trained ProxNetwork would, so samplers and proximal inference can be checked against arithmetic. Uses $\nabla\log p_t$ directly (not $\varepsilon/\sigma_t$), so it is valid at $t = 0$. Where $-\log p_t$ is not convex the prox can be multivalued; the solve returns the local minimiser reached from $v$, escaping negative curvature at stationary points. Throws if the iteration or line search fails; no global minimum is promised.
VariationalDiffusion.ProxDM — Type
ProxDM(prox; t = 0.005, λ = 0.01, tol = 1e-9, maxiters = 500, linsolve = DenseSolve(),
nodes = nothing)Proximal inference (prox_infer) as a DiffusionFactor's inversion, in place of REDDiff or ImplicitProx:
f = DiffusionFactor((x = 1, y = 1), pred; prox = ProxDM(MixtureProx(mixture)))
ps, st = LuxCore.setup(rng, f) # (predictor = …, prox = …), two parameter treesprox is the proximal predictor (MixtureProx, a ProxNetwork, or any AbstractProximalPredictor); t and λ are the prox's noise level and the splitting's coupling. Inversion runs prox_infer on the factor's state space with the polarity's precisions (Inf = hard clamp), warm-started at the incoming points, and returns a DiracBelief on the target; implicit_solution returns the report and implicit_factor_pullback the backward pass (through prox_pullback, with linsolve).
Two parameter trees. The factor keeps its NoisePredictor, which the free energy's score summand needs; the prox is a second model with its own parameters. The factor's parameters and states are therefore (predictor = …, prox = …), and the inversion reads only prox. A ProxNetwork must be a Lux layer; a MixtureProx gets (μ = …,).
nodes (a FieldNodes of the factor's dimension) fix the score summand of the free energy, as for ImplicitProx with $\lambda = 1$; by default 8 nodes from Xoshiro(0).
VariationalDiffusion.ProxNetwork — Type
ProxNetwork(model, schedule; input = (v, t, λ) -> (v, t, λ), ad = nothing)A learned proximal operator in ProxDM's parametrisation $f_\theta(v; t, \lambda) = v - \sqrt\lambda\,\varepsilon_\theta(v; t, \lambda)$: the network predicts the normalised residual of the prox, conditioned on two scalars, $t$ and $\lambda$. model is any Lux layer; input adapts $(v, t, \lambda)$ to what it expects. Parameters are the model's. Train it with proximal_matching_loss.
ad is an ADTypes backend (AutoZygote(), …; load DifferentiationInterface and the backend) for the derivatives the adjoint of prox_infer needs: the parameter VJP, which has no other route, and the input VJP, which otherwise falls back to finite differences.
VariationalDiffusion.prox_infer — Method
prox_infer(p, z₀, ρ, t, ps, st; λ = 0.01, z_init = z₀, maxiters = 500, tol = 1e-9,
linsolve = DenseSolve()) -> (ImplicitSolution, st)Deterministic inference for the implicit learner with a proximal prior instead of a score: half-quadratic splitting between the prior's prox at level t and the clamp's prox,
\[u \leftarrow \operatorname{prox}_{-\lambda\log p_t}(w),\qquad w_i \leftarrow \frac{u_i/\lambda + \rho_i^2 z_{0,i}}{1/\lambda + \rho_i^2}\quad(\rho_i<\infty),\qquad w_i \leftarrow z_{0,i}\quad(\rho_i=\infty).\]
A fixed point $(u, w)$ is a stationary point of $-\log p_t(u) + \tfrac1{2\lambda}\lVert u-w\rVert^2 + \tfrac12\lVert P(w - z_0)\rVert^2$ — the relaxed problem whose $\lambda\to0$ limit is the implicit learner's (Deterministic Relaxation.md); this is the plug-and-play / DiffPIR splitting. Hard clamps are exact. Returns w (which satisfies the clamps) as z, with residual $= \lVert u - w\rVert$; stable is not assessed by this solver and is reported true when converged.
Its derivative is prox_pullback, which is also the ChainRulesCore.rrule of this function; linsolve is the linear solver that adjoint uses (the forward pass does not use it).
VariationalDiffusion.prox_jacobian — Method
prox_jacobian(p, v, t, λ, ps, st) -> MatrixThe input Jacobian of the prox at v. Exact for MixtureProx; through ad for a ProxNetwork; central differences (step $10^{-6}(1 + \lVert v\rVert_\infty)$) otherwise.
VariationalDiffusion.prox_pullback — Method
prox_pullback(p, sol, z₀, ρ, t, z̄, ps, st; λ = 0.01, linsolve = DenseSolve())
-> (z₀ = z̄₀, ρ = ρ̄, ps = ps̄)Reverse-mode derivative of prox_infer's answer $w^\star(z_0, \rho, \theta)$ by the adjoint method; λ and t must be the ones inference used. The fixed point of the splitting satisfies, on the coordinates that are not hard-clamped ($F$),
\[w_F = a\odot\operatorname{prox}(w)_F + (1-a)\odot z_{0,F},\qquad a_i = \frac1{1+\lambda\rho_i^2},\]
and $w_C = z_{0,C}$ on the clamped ones. With $D$ the prox's input Jacobian at $w$ and $u = \operatorname{prox}(w)$, solve once
\[(I - \operatorname{diag}(a)\,D_{FF})^\top\mu = \bar z_F,\qquad y = (a\odot\mu \text{ on } F,\ 0 \text{ on } C),\]
and read off every gradient:
| w.r.t. | cotangent |
|---|---|
| parameters $\theta$ | $(\partial_\theta\operatorname{prox})^\top y$ (via prox_vjp_params) |
| hard-clamped inputs $z_{0,C}$ | $\bar z_C + (D^\top y)_C$ |
| soft and free targets $z_{0,F}$ | $(1-a)\odot\mu$ |
| precisions $\rho_F$ | $-2\lambda\rho_F\odot a^2\odot(u_F - z_{0,F})\odot\mu$ |
Only vector-Jacobian products of the prox appear, so linsolve = KrylovSolve() runs GMRES on them and never forms $D$; DenseSolve() forms it (prox_jacobian). Throws if sol did not converge. See proxdm.md §5.
VariationalDiffusion.prox_vjp_input — Method
prox_vjp_input(p, v, t, λ, ps, st, y) -> v̄$(\partial_v \operatorname{prox}_{-\lambda\log p_t}(v))^\top y$. Exact for MixtureProx (the implicit function theorem on the prox's optimality condition: $(I - \lambda\nabla^2\log p_t(u))^{-1}y$); through ad for a ProxNetwork; by central differences otherwise.
VariationalDiffusion.prox_vjp_params — Method
prox_vjp_params(p, v, t, λ, ps, st, y) -> ps̄$(\partial_\theta \operatorname{prox}_{-\lambda\log p_t}(v))^\top y$, shaped like ps. Exact for MixtureProx; through ad for a ProxNetwork.
VariationalDiffusion.proxdm_sample — Method
proxdm_sample(p, schedule, x, ps, st; steps = 50, scheme = :pda, rng = Random.default_rng(),
T = 1.0) -> (x₀, st)ProxDM's sampler (Fang et al., Algorithm 1), started from x (pure noise for the full reverse process). With $\gamma_k = \int_{t_{k-1}}^{t_k}\beta(s)\,ds$ on a uniform grid $0 = t_0 < \dots < t_N = T$:
- PDA (backward Euler): $X_{k-1} = \operatorname{prox}_{-\frac{2\gamma_k}{2-\gamma_k}\log p_{t_{k-1}}}\bigl(\tfrac{2}{2-\gamma_k}(X_k + \sqrt{\gamma_k}\,z_k)\bigr)$, which needs $\gamma_k < 2$;
- PDA-hybrid (
scheme = :hybrid): $X_{k-1} = \operatorname{prox}_{-\gamma_k\log p_{t_{k-1}}}\bigl((1+\tfrac12\gamma_k)X_k + \sqrt{\gamma_k}\,z_k\bigr)$, no step-size limit. - Trapezoidal (
scheme = :trapezoidal, second order): the drift averaged over both ends, $X_{k-1} = X_k + \tfrac{\gamma_k}2\bigl[(\tfrac12X_k + s_k) + (\tfrac12X_{k-1} + s_{k-1})\bigr] + \sqrt{\gamma_k}\,z_k$, i.e. $X_{k-1} = \operatorname{prox}_{-\frac{2\gamma_k}{4-\gamma_k}\log p_{t_{k-1}}}\bigl(\tfrac{(1+\gamma_k/4)X_k + (\gamma_k/2)s_k + \sqrt{\gamma_k}z_k}{1-\gamma_k/4}\bigr)$, which needs $\gamma_k < 4$. The explicit score $s_k = \nabla\log p_{t_k}(X_k)$ costs nothing: the previous prox returned $X_k = \operatorname{prox}_\lambda(v)$, and its optimality condition gives $s_k = (X_k - v)/\lambda$. The first step is a hybrid step, which supplies the first score. Still one prox call per step.
PDA and hybrid follow from discretising the drift of the reverse VP-SDE at the new point (hybrid: the score term only), which turns each step into a proximal step; they are first order in the step (the spread converges like $1/N$), the trapezoidal scheme second order. Unlike score-based samplers the last step denoises after adding noise, so no final denoising step is needed. hybrid = true is the old spelling of scheme = :hybrid.
VariationalDiffusion.proximal — Function
proximal(p, v, t, λ, ps, st) -> (u, st)$\operatorname{prox}_{-\lambda\log p_t}(v) = \arg\min_u \tfrac12\lVert u-v\rVert^2-\lambda\log p_t(u)$: a MAP denoiser at level t with regularisation λ. Its optimality condition is $u - v = \lambda\nabla\log p_t(u)$ — a backward (implicit) gradient step.
VariationalDiffusion.proximal_matching_loss — Method
proximal_matching_loss(p::ProxNetwork, x_t, t, λ, ε, ζ, ps, st) -> (ℓ, st)One sample of ProxDM's training objective (their Eq. 9): $\ell_{PM}\bigl(\varepsilon_\theta(x_t + \sqrt\lambda\,\varepsilon; t, \lambda), \varepsilon; \zeta\bigr)$ with $\ell_{PM}(a, b; \zeta) = 1 - \exp\bigl(-\lVert a-b\rVert^2/(d\zeta^2)\bigr)$. As $\zeta\to0$ its minimiser is the prox (a MAP denoiser), where the squared loss would give the MMSE denoiser. Shrink $\zeta$ during training. x_t is a sample of $p_t$, ε a fresh standard normal.