implementation

Opt-in first-order parameter adjoints of a converged BP message fixed point. A dense reference solve and an optional matrix-free GMRES solve share the same message equation, cuts and fixed-point checks. Ordinary inference is unchanged.

Sources: original derivation from the message fixed-point equation; code: graph_adjoint.jl, gradient_couplings.jl, ext/MyceliumLearningKrylovExt.jl; tests: lib/MyceliumLearning.jl/test/graph_adjoint.jl, graph_adjoint_devices.jl.

Theory (CT-ML wiki): Statistical Game

Equation and API (2026-10-28)

free_energy_gradient(store, g, ps, st; ad=AutoZygote(),
    schedule=flooding_schedule(g), adjoint=:implicit,
    maxsweeps=1000, tol=1e-10, damping=0.0,
    linear_solver=:dense, linear_tol=1e-10, linear_maxiter=1000)

parameter_step forwards the same keywords and retains its deterministic three-tuple return. The input store is untouched. Reference inference starts from empty messages at the look-ahead parameters; nsweeps only controls the unrolled driver. The default remains adjoint=:unrolled, and all-diagonal calls without the opt-in use the local fixed-belief derivative (fixed-cavity before LP-020).

Let concatenate every legal directed message, the active parameters, and the scalarised local free energies plus Bethe variable corrections. The map computes all candidate messages from the same input store. It uses the ordinary factor and variable exclusions and damps only outgoing factor messages, when an old message exists and the belief family supports damping. The chosen reference schedule finds a root; it does not select a different fixed-point equation. Sequential, tree, forward/backward, flooding and residual reference schedules are accepted. Static lists must cover every legal message.

At a differentiable root ,

The implementation differentiates with and held constant. It never differentiates convergence tests or the reference iterations. This is a local derivative of the selected root branch. Differentiability and invertibility of are required. A contraction is sufficient for local invertibility and iteration convergence; convergence by itself does not prove either differentiability or invertibility. No spectral-radius bound or uniqueness guarantee is computed.

Boundaries and representations

Exact and Pathwise couplings are supported; Pathwise still requires Gaussian, Dirac or uninformative messages. A diagonal/detached attachment replaces each of its legal directed update rows by their constant reference coordinates. Hence : paths through those rows disappear from the linear system. Downstream columns can still read the fixed boundary values. Frozen or intrinsically nonlearnable parameter coordinates are pinned, preserving input message derivatives through their factors. The mixed system is the fixed-point counterpart of the cut replay. It agrees with its limiting gradient when the differentiated replay converges; invertibility alone does not ensure that limit. This need not be the ordinary derivative of the converged primal objective with no cuts.

The coordinate adapter supports CPU real floating Gaussian canonical parameters (Vector information vector, dense Matrix or vector-backed Diagonal precision), numeric scalar/CPU-array Diracs, and coordinate-free Trivial messages. Dense precisions use all matrix entries; diagonal precisions use their diagonal only. Message families, shapes, polarity choices and support must be locally stable. There is no device coordinate adapter, implicit score estimator, state derivative, higher-order rule, arbitrary outer-AD rule, or differentiation of graph structure.

Incoming state and every state returned by a message update must be recursively empty. Stateful factors would require either a larger fixed-point equation including state or a specified fixed-state map; silently dropping state is not such a specification. Reference differences (ad=nothing), DI ForwardDiff and Zygote are the tested backend choices. Unsupported backends, messages, states and solver choices throw ArgumentError. Krylov requires explicitly loading Krylov.jl; Mycelium has no dependency on ImplicitLayers.

Validation, cost and difficulties

The reference loop verifies convergence and then checks every ordinary synchronous BP row before applying derivative cuts. This avoids trusting the ordinary one-sweep TreeSchedule report for damped or manually assembled schedules. Both linear solvers check the original adjoint residual against linear_tol * max(1,norm(rhs)); singular dense systems, failed GMRES solves and nonfinite results fail. These checks are numerical evidence, not a condition-number estimate. Tight finite-difference linear tolerances can fail due to cancellation.

For message coordinates, dense assembly stores entries and dense LU costs . GMRES stores Krylov vectors and applies with a backend gradient; ForwardDiff obtains that gradient with forward seeds, so choosing it does not give reverse-mode cost. Reference inference, AD workspace and functional store copies add to these costs. Each map evaluation currently copies edge-sized vectors while rebuilding stores, so this implementation is a checked baseline rather than an allocation claim. Its differentiation memory does not grow with the reference sweep count.

The full-suite ambiguity check caught an overly broad mul! method on the new operator: it intersected LinearAlgebra’s multiplication by an AbstractQ. The operator now restricts both operands to vectors, which is the GMRES contract. Its advertised element type follows the adjoint RHS, including promotion by CPU scalar objective constants; message coordinates themselves are unchanged.

The Gaussian fixtures are independent unnormalised unary/pair potentials, including a correlated two-dimensional dense-precision tree oracle. On a tree, the Bethe value and gradient agree with the dense global Gaussian and converged exact replay. On the loop, references reconverge BP at each parameter perturbation; the test measures convergence of the unrolled gradient to this adjoint. Tests also cover cuts, freezing, damping, parameter steps, Diracs and failure paths. Device replay uses diagonal Gaussian fixtures with scalar indexing disabled; device implicit calls explicitly fail for both linear solvers. See LP-002’s dated batch C section for run measurements and verification results.

Fully cut factors at fixed beliefs (LP-021, 2026-11-02)

The objective of the coupled drivers (_graph_scalar, used by the unrolled replay and by the implicit adjoint’s loss) evaluated every factor’s local free energy at its messages. A factor whose edges are all diagonal or detached has constant messages, so its parameter derivative was the fixed-message one, which LP-020 showed is not the free-energy gradient: with default couplings adjoint = :implicit was 0.58 off on a three-variable tree, and one ExactCoupling edge anywhere switched every fully cut factor from the default driver’s fixed-belief gradient to this one. _frozen_beliefs now evaluates factor_belief for those factors at the reference messages, outside AD, and _graph_scalar differentiates belief_free_energy at that belief. Factors with a derivative path, and factors without factor_belief, are unchanged. test/lp021.jl checks the all-diagonal, one-exact-edge and all-exact cases against central differences.

Mixed cuts: couplings only in a replay (LP-021 option 3, 2026-11-04)

At a fixed point the free-energy gradient is the envelope one, whatever the couplings; a coupling can only change the derivative of a computation that is not at its fixed point, or of another loss. So free_energy_gradient without schedule uses the envelope driver for graphs with diagonal, exact or pathwise couplings (_deterministic_couplings; before, non-diagonal edges required a replay), and adjoint = :implicit returns the envelope gradient at its converged fixed point when the graph is mixed (_mixed_cuts: a cut and a differentiated edge, both carrying messages that depend on the parameters; a data factor’s edges and an observed variable’s do not count). On a graph that is not mixed the implicit adjoint is kept; it equals the envelope gradient there (test/lp021.jl, test/graph_adjoint.jl). A replay with schedule still differentiates the replayed sweeps, and warns on a mixed graph: holding a parameter-dependent message fixed next to differentiated ones gave errors of 0.18–0.73 against the free-energy gradient, at convergence (the LP-021 note has the table). Score-function couplings and StraightThroughFactor keep the replay semantics.