Lenticulum

The user-facing package: Gaussian beliefs and the linear-Gaussian factors built on them. If you want to try the framework on something, start here — everything in this package has a closed form, so you can check the answers.

Gaussian beliefs

GaussianBelief stores a Gaussian in canonical form — information vector η = Λμ and precision Λ = Σ⁻¹ — rather than as a mean and covariance. Two reasons:

  1. Pooling becomes addition. Combining two beliefs adds their canonical parameters, which is exact, associative, commutative and total.
  2. Improper beliefs are representable. A factor → variable message is a likelihood, and a likelihood constraining only some directions has a singular Λ. The moment form cannot write that down at all.
g = Gaussian([1.0], fill(2.0, 1, 1))   # from mean and covariance
belief_mean(g), belief_cov(g)           # back again
uninformative(1)                        # η = 0, Λ = 0
isproper(g)                             # is Λ positive definite?

belief_mean on an improper belief throws rather than returning a plausible-looking pseudo-mean.

The factors

factorischannels
GaussianFactor$p(y \mid x) = \mathcal{N}(y; Ax+b, Q)$two, named
GaussianPrior$\mathcal{N}(\mu, \Sigma)$ on one channelone
LinearConstraintFactor$0 = \sum_i A_i x_i - c + \varepsilon$any number
HardConstraintFactor$\sum_i A_i x_i = c$ exactly; messages are ExtendedGaussiansany number
SwitchingConstraintFactor$0 = \sum_i A_i x_i - b_k + \varepsilon$, outcome $k$ of weight $w_k$any number, plus the outcome
RegimeFactor$0 = \sum_i A_{i,k} x_i - b_k + \varepsilon_k$, $\varepsilon_k \sim \mathcal{N}(0, Q_k)$: coefficients and noise per outcomeany number, plus the outcome
NonlinearFactor$p(y \mid x) = \mathcal{N}(y; f(x), Q)$, $f$ giventwo, named
LikelihoodFactor$p(y \mid x) = L(y \mid Ax + b)$, $L$ non-Gaussian (counts, ordered categories)two, named

GaussianFactor is bidirectional: both (x observed, y unobserved) and the reverse, each exact. By default parameters are (A, b) and noise is fixed. Opt-in learn_noise=true (full covariance) or :diagonal adds positive-precision noise coordinates; train them with normalised free energy, not the residual norm alone. GaussianPrior supports the same covariance-learning options with its mean fixed. lib/Lenticulum.jl/test/learnable_noise.jl checks these parameter derivatives against references.

LinearConstraintFactor is the acausal generalisation — one equation over n channels, none of them distinguished, with one supported polarity per channel. It subsumes GaussianFactor exactly (same messages, same energy) and is what you want for conservation laws, balance equations and anything written as a relation rather than an assignment:

# Kirchhoff's current law at a three-way node: i₁ + i₂ + i₃ = 0
f  = LinearConstraintFactor((i1 = 1, i2 = 1, i3 = 1), 1; noise = 1e-8)
ps = (A = (i1 = ones(1,1), i2 = ones(1,1), i3 = ones(1,1)), c = [0.0])

The noise = 1e-8 there stands in for a relation that holds exactly. HardConstraintFactor holds it exactly: its messages are ExtendedGaussians (LenticulumCore), hard along what the equation fixes and flat along what it cannot see, so HardConstraintFactor((i1 = 1, i2 = 1, i3 = 1), 1) with the same parameters is Kirchhoff's law itself. Its free energy is the finite limit of the noisy relation's as the noise vanishes (hard_constraint.md, "Free energy"); on trees the Bethe free energy of a graph with it is $-\log Z$.

SwitchingConstraintFactor lets one discrete outcome choose the right-hand side: a measurement of the linear observable $\sum_i A_i x_i$. It is the smallest factor that leaves the Gaussian class. Its message to a continuous channel is a mixture of $K$ Gaussians, its message to the outcome a CategoricalBelief, and its local free energy is closed form. With K = 1 it is a LinearConstraintFactor. Both constraint factors also have a mean-field message (Mycelium.mean_field_message), used when their node is added with rule = MeanField(), and an exact message for mixture inputs (Mycelium.mixture_message), used by rule = MixtureReduction(M); on rule = ExpectationPropagationRule() a switching node sends Gaussian sites instead of mixtures. In a plate, tied switching factors send their outcome messages as one block and evaluate their free energy in one call (FrozenSwitchingBeliefs). RegimeFactor lets the outcome choose the coefficients and the noise as well (mixtures of regressions, of factor analysers, Gaussian mixtures with one covariance per component). It is exact inside a Mycelium cluster node and is a one-factor cluster outside one. The tutorial mixtures from switching factors works through both. The design and the experiments behind it are in the vault notes Switching Constraint Factors (the theory) and Composing Stochastic Factors (the experiments):

# a three-component mixture in the plane, z = b_k + ε
f  = SwitchingConstraintFactor((z = 2,), 3, 2; noise = 0.1)
ps = (A = (z = Matrix(1.0I, 2, 2),), b = [-1.0 1.0 0.0; 0.0 0.0 2.0], logw = zeros(3))

NonlinearFactor takes a hand-written function (a measurement model, known dynamics) and approximates its messages through a local linear model: the Jacobian (Linearization(), the extended Kalman filter) or sigma points (Unscented(), the unscented Kalman filter). Backward messages are linearised at the cavity belief, and iterations > 1 relinearises the posterior in invert, which is Gauss–Newton on the MAP:

f = NonlinearFactor(x -> [hypot(x[1], x[2])], 2 => 1; noise = 0.01, method = Unscented())

LikelihoodFactor observes a Gaussian latent through a non-Gaussian likelihood of a linear predictor $z = Ax + b$: PoissonLikelihood (counts, log link), BivariatePoissonLikelihood (two counts with a common shock) or OrdinalProbitLikelihood (ordered categories; one cutpoint is binary probit), or any AbstractLikelihood you define. The message to $x$ is Gaussian, by Laplace() or ExpectationPropagation() (adaptive Gauss–Hermite moments) at the cavity; the message to $y$ is the predictive, a CategoricalBelief over the outcomes:

f = LikelihoodFactor(OrdinalProbitLikelihood([-1.0, 1.0]), 2; A = [1.0 -1.0],
                     method = ExpectationPropagation())
p = predictive(f, Gaussian([0.3, 0.0], [1.0 0.2; 0.2 0.5]))
probabilities(p)            # P(y = 1), P(y = 2), P(y = 3)

Known gaps

  • Covariance learning is opt-in for GaussianFactor/GaussianPrior; other factor families retain their own noise restrictions.
  • A loopy graph of constraint factors cannot bootstrap without a weak prior on every variable.

API

Lenticulum.Lenticulum — Module
Lenticulum

Implicit machine learning on factor graphs: learning relations $R_\theta \subseteq X_1\times\cdots\times X_n$ rather than functions.

This package is the user-facing layer, mirroring Lux.jl. It sits on

  • LenticulumCore — what a factor is (a parameterized statistical game);
  • Mycelium — how factors are wired and in what order they talk.

What is here so far

The linear-Gaussian factor and its beliefs: the first factor in the project that implements the entire interface with nothing stubbed — a real open model, an exact Bayesian inversion, a genuine vector energy and entropy, bidirectional polarity, and learnable parameters.

Its purpose is to be a test oracle. Every quantity it produces has a closed form, so the framework's claims can be checked against arithmetic instead of against itself. Two of them turned out to be wrong; see The Gaussian Factor.md §"What this caught".

  • beliefs.jl — projections onto GaussianBelief (moment_match; reduce_mixture is re-exported from Mycelium) and kl_divergence. The type itself is in LenticulumCore and its message rules (pooling is addition of canonical parameters) in Mycelium; both are re-exported here.
  • gaussian.jl — GaussianFactor and GaussianPrior.
  • constraint.jl — LinearConstraintFactor, the acausal n-ary sibling of GaussianFactor: one equation $0 = \sum_i A_i x_i - c + \varepsilon$ over any number of channels, none of them distinguished. This is ModelingToolkit's 0 ~ ... equation as a statistical game; see ModelingToolkit as an Acausal Relation.md.
  • nonlinear.jl — NonlinearFactor: a given $f$ with Gaussian noise, by linearisation or the unscented transform.
  • likelihood.jl — LikelihoodFactor: a Gaussian latent observed through a non-Gaussian likelihood (counts, ordered categories), with Laplace and EP messages and a predictive over discrete outcomes; see likelihood.md.
  • switching.jl — SwitchingConstraintFactor: a linear constraint whose right-hand side is chosen by a discrete outcome, closed form in every direction; see switching.md.
source
Lenticulum.AbstractLikelihood — Type
AbstractLikelihood

A likelihood $L(y \mid z)$ of an observation y (a vector of length obsdim) given a linear predictor z (a vector of length npredictors). A new likelihood defines

  • npredictors(L), obsdim(L);
  • loglikelihood(L, y, z);
  • optionally loglikelihood_derivatives(L, y, z) -> (ℓ, ∇ℓ, ∇²ℓ) in z (default: central differences of loglikelihood);
  • for the predictive (the forward message), outcomes(L), the finite list of values of y, and optionally outcome_logprobs(L, z), their log-probabilities given z (default: loglikelihood at each outcome).

Every factor rule then works for it: see LikelihoodFactor.

source
Lenticulum.BivariatePoissonLikelihood — Type
BivariatePoissonLikelihood(; maxcount = 20)

Two dependent counts by a common shock: $y_1 = w_1 + w_3$, $y_2 = w_2 + w_3$ with $w_i \sim \mathrm{Poisson}(\lambda_i)$ independent, so $\operatorname{Cov}(y_1, y_2) = \lambda_3$. The predictor is $z = (\log\lambda_1, \log\lambda_2, \log\lambda_3)$. Derivatives are exact, through the posterior of the shared part $w_3$ given $y$. Not log-concave in general, so a message's precision can be indefinite. The predictive is over $\{0, \dots, \texttt{maxcount}\}^2$, the top value meaning "or more".

source
Lenticulum.ExpectationPropagation — Type
ExpectationPropagation(; order = 20, iterations = 100, tol = 1e-10)

Approximate the posterior $\propto$ cavity × likelihood by the Gaussian with its mean and covariance (moment matching: the minimiser of $\mathrm{KL}(p \Vert q)$, the projection step of EP). The moments are computed by adaptive Gauss–Hermite quadrature, order points per predictor dimension, centred and scaled at the Laplace approximation (found as for Laplace with iterations and tol); in closed form for OrdinalProbitLikelihood. The tensor grid has $\texttt{order}^k$ points, so this is for few predictors ($k \le 4$).

source
Lenticulum.FrozenGaussianBelief — Type
FrozenGaussianBelief

A Gaussian-family factor's own belief $b_c$, evaluated once and then held fixed while the parameters move: the point channels by value, the others as one joint Gaussian (mean m, covariance Σ, in the order of free) with entropy H. For the switching factor m has one column per outcome and q holds the outcome posterior; otherwise q is nothing.

Returned by Mycelium.factor_belief and read by Mycelium.belief_free_energy, whose parameter derivative at a fixed b is the envelope gradient of the Bethe free energy.

source
Lenticulum.FrozenGaussianBeliefs — Type
FrozenGaussianBeliefs

The factor beliefs of n tied replicas held fixed, as FrozenGaussianBelief column for column: the point channels as d × n matrices, the free channels' joint means as the columns of M, and one covariance Σ and entropy H shared by every replica. Built from message blocks whose Gaussian inputs share one precision, which is what makes Σ shared.

source
Lenticulum.FrozenSwitchingBeliefs — Type
FrozenSwitchingBeliefs

The factor beliefs of n tied switching replicas held fixed: the point channels as d × n matrices, the free channels' joint means per outcome and replica (M[:, k, i]), one covariance Σ and conditional entropy $H(x \mid k)$ shared by every replica, and the outcome posteriors as the columns of q. Built from message blocks whose Gaussian inputs share one precision; the batched counterpart of the FrozenGaussianBelief that factor_belief returns.

source
Lenticulum.GaussianFactor — Type
GaussianFactor(nin => nout; noise = 0.1, learn_noise = false,
               noise_prior = nothing, eigenvalue_floor = 1e-6,
               channels = (:x, :y), init_weight, init_bias)

The linear-Gaussian conditional $p(y \mid x) = \mathcal{N}(y;\ Ax + b,\ Q)$.

noise is the initial covariance Q: a matrix, a vector of variances, or a scalar variance. By default it is fixed and parameters are (A, b). With learn_noise=true (or :full), ps.noise stores a precision Cholesky's log_diagonal and packed strict lower entries; :diagonal stores only log_precision. The initial covariance must be diagonal in diagonal mode. Use noise_covariance to read the current value.

Train noise with free_energy / local_free_energy, which include the Gaussian normaliser. The residual energy and its quadratic scalar_energy omit it: minimising those alone inflates noise without bound. An optional WishartPrecisionPrior (full) or LogNormalVariancePrior (diagonal) adds its negative log density to free energy. Otherwise covariance eigenvalues are clipped at eigenvalue_floor. The hard floor is continuous with a kink; its derivative is zero in saturated directions. See noise.md and gaussian.md for density coordinates and prior multiplicity.

Parameters are produced by initialparameters exactly as a Lux layer's are: the factor is a description, ps is an inhabitant.

Bidirectional: both (x observed, y unobserved) and (x unobserved, y observed) are supported, and each is exact. That is what makes it a relation rather than a function, and it is the smallest non-trivial witness that assemble — the operation with no Lux counterpart — does real work.

source
Lenticulum.GaussianPrior — Type
GaussianPrior(channel, μ, Σ; learn_noise = false, noise_prior = nothing, eigenvalue_floor = 1e-6)

The prior factor $\pi : 1 \multimap X$ of AutoBayes Remark 24: emits $\mathcal{N}(\mu,\Sigma)$ and charges energy $-\log p_\pi(x)$. Its game-level entropy is the entropy of the factor belief after pooling with incoming information (zero for a point observation).

Without it the composite free energy is an open free energy, missing the $-\log p_\pi(x)$ term. With it, $F^{c\pi}(\ast, y) = \mathrm{VFE}(c,c')(\pi,y)$, and when the inversion is exact that equals $-\log p_{c_*\pi}(y)$ on the nose. That identity is the project's main correctness test.

The mean μ stays fixed. Covariance learning is opt-in with the same learn_noise, noise_prior and eigenvalue_floor options as GaussianFactor. Its existing pointwise energy is the normalised negative log-density, including the normaliser for both fixed and learned covariance. A noise hyperprior is charged only in free energy.

source
Lenticulum.HardConstraintFactor — Type
HardConstraintFactor(dims::NamedTuple, m::Int; noise = 0,
                     init_coeff = (rng, m, n) -> randn(rng, m, n) ./ sqrt(n),
                     init_constant = (rng, m) -> zeros(m))

The linear relation $\sum_i A_i x_i = c$ among the channels dims (names and dimensions), with m scalar equations, imposed exactly. Parameters (A, c) as for LinearConstraintFactor, and the same polarities: any one channel Unobserved(), each other Observed() or Latent().

Its messages are ExtendedGaussians: hard along what the equation fixes, flat along what it cannot see. An optional noise (a positive semidefinite m × m covariance, or a scalar) softens the relation in the directions where it is positive; with a positive definite noise the messages are those of LinearConstraintFactor with that noise.

Inputs may be GaussianBeliefs (also improper), DiracBeliefs, ExtendedGaussians, or carry no information (TrivialBelief(), no message, a latent channel). Mixtures are refused.

Messages only: the factor has no free energy (a relation without noise has no finite one in closed form) and is not learnable through the free energy; local_free_energy throws.

# Kirchhoff's current law at a node, exactly: i₁ + i₂ + i₃ = 0
f  = HardConstraintFactor((i1 = 1, i2 = 1, i3 = 1), 1)
ps = (A = (i1 = ones(1, 1), i2 = ones(1, 1), i3 = ones(1, 1)), c = [0.0])
source
Lenticulum.Laplace — Type
Laplace(; iterations = 100, tol = 1e-10)

Approximate the posterior $\propto$ cavity × likelihood by the Gaussian at its mode with the curvature there (Newton's method, at most iterations steps, stopping when a step is below tol). The message is that Gaussian divided by the cavity: precision $-A^\top\nabla^2\ell\,A$ at the mode. Works for any predictor dimension.

source
Lenticulum.LikelihoodFactor — Type
LikelihoodFactor(lik, nin; A = nothing, b = nothing, channels = (:x, :y),
                 method = Laplace(), x0 = nothing)

A Gaussian latent $x \in \mathbb R^{n}$ observed through a non-Gaussian likelihood of the linear predictor:

\[p(y \mid x) = L(y \mid Ax + b),\]

lik an AbstractLikelihood (PoissonLikelihood, BivariatePoissonLikelihood, OrdinalProbitLikelihood, or one of your own). A is npredictors(lik) × nin (default the identity when the sizes agree) and b the offset (default zero); both fixed.

  • Backwards (target $x$, $y$ observed as a DiracBelief): the Gaussian message $q / \text{cavity}$, where the cavity is the prior argument (the other messages into $x$) and $q$ its product with the likelihood, approximated by method: Laplace or ExpectationPropagation. With no proper cavity, the second-order expansion of $\log L$ at x0 (one Newton step from there), or an uninformative message without x0.
  • Forwards (target $y$): the predictive $P(y) = \mathbb E[L(y \mid Ax+b)]$ under the belief on $x$, a CategoricalBelief over outcomes(lik) (labelled by them), by Gauss–Hermite quadrature (method.order points per dimension for EP, 20 for Laplace).

No parameters (A, b are fixed). See likelihood.md.

source
Lenticulum.LinearConstraintFactor — Type
LinearConstraintFactor(dims::NamedTuple, m::Int; noise = 0.1, output = nothing, output_coeff = I,
                       init_coeff, init_constant)

The soft linear relation $0 = \sum_i A_i x_i - c + \varepsilon$, $\varepsilon \sim \mathcal{N}(0, Q)$ — equivalently the density

\[f(x_1,\ldots,x_n) \;\propto\; \exp\Bigl(-\tfrac12 \bigl\|\textstyle\sum_i A_i x_i - c\bigr\|^2_{Q^{-1}}\Bigr)\]

dims names the channels and their dimensions, m is the dimension of the residual (the number of scalar equations). Q is a fixed hyperparameter, exactly as in GaussianFactor; parameters are (A, c), with A a NamedTuple keyed by channel.

Acausal: no channel is privileged. Every channel can be solved for — "solve this equation for x_t, given the others" — and every one of these messages is exact; the others may also be latent (supported_polarities). As $Q \to 0$ the factor becomes the hard relation $\{x : \sum_i A_i x_i = c\}$; a finite Q is the soft relation, which is what makes it a statistical game rather than a set.

# Kirchhoff's current law at a three-way node: i₁ + i₂ + i₃ = 0
f  = LinearConstraintFactor((i1 = 1, i2 = 1, i3 = 1), 1; noise = 1e-8)
ps = (A = (i1 = ones(1,1), i2 = ones(1,1), i3 = ones(1,1)), c = [0.0])

Output channel (LP-022). The factor is normalised in its residual, not in its channels, so with every coefficient learnable it is not a density of any of them: $A = 0, c = 0$ puts every point on the relation, and scaling $A$ and the noise together lowers $-\log f$ without bound. With output = :y (dims[:y] == m) the coefficient of y is fixed at output_coeff (the identity, or an invertible m × m matrix) and is not a parameter (ps.A omits it): the factor is then the normalised conditional density of y given the other channels, up to the constant $|\det A_y|$. Declare it whenever y carries data and the other coefficients are learned.

See constraint.md for the message algebra and the honest list of what is missing.

source
Lenticulum.LinearConstraintModel — Type
LinearConstraintModel(factor, polarity)

The open model the polarity selects: "solve this equation for the unobserved channel". Pure — a linear relation with additive Gaussian noise hides nothing.

source
Lenticulum.LinearGaussianModel — Type
LinearGaussianModel(factor, polarity)

The open model $c : X \nrightarrow Y$ of AutoBayes Definition 1, in the direction the polarity selects. Pure ($\llbracket c \rrbracket \cong 1$): a linear-Gaussian conditional hides nothing.

source
Lenticulum.Linearization — Type
Linearization(; iterations = 1)

Approximate a nonlinear factor's messages by its first-order Taylor expansion at the expansion point: the extended Kalman filter's rule. iterations > 1 relinearises invert's backward direction at the posterior mean, which is Gauss–Newton on the MAP (the iterated EKF). likelihood_message returns the final site used by this update.

source
Lenticulum.LogNormalVariancePrior — Type
LogNormalVariancePrior(logmean, logstd)

Independent log-normal densities on diagonal variances. Arguments are scalars or vectors of the noise dimension; logstd must be positive. The negative log density (including the log-variance term) is charged in free energy. MAP is in variance coordinates, without a Jacobian for the trainable log-precision coordinates.

source
Lenticulum.NonlinearFactor — Type
NonlinearFactor(f, nin => nout; noise = 0.1, channels = (:x, :y), method = Linearization(),
                jacobian = nothing, x0 = nothing)

The nonlinear Gaussian conditional $p(y \mid x) = \mathcal N(y;\ f(x),\ Q)$ for a given function f (a measurement model, known dynamics): the factor that turns Lenticulum's graphs into extended or unscented Kalman filters and smoothers, and that sits beside a learned factor in one graph.

  • noise is Q (a matrix, a vector of variances, or a scalar variance), as in GaussianFactor.
  • method is Linearization (EKF) or Unscented (UKF).
  • jacobian(x) is $\partial f/\partial x$ (an nout × nin matrix); without it, central differences. The package depends on no AD backend; pass e.g. x -> ForwardDiff.jacobian(f, x) for an exact one.
  • x0 is the expansion point used when a backward message is asked for before anything is known about x (no cavity belief). Without it such a message is uninformative.

Both polarities are supported, like GaussianFactor. It has no parameters (f is fixed). Messages are approximate (SolverInversion marks that), exact when f is affine.

source
Lenticulum.OrdinalProbitLikelihood — Type
OrdinalProbitLikelihood(cutpoints; scale = 1)

An ordered category $y \in \{1, \dots, K\}$ from one predictor: $y = k$ when $z + \varepsilon \in (c_{k-1}, c_k]$, $\varepsilon \sim \mathcal N(0, \sigma^2)$, with $c_0 = -\infty$ and $c_K = +\infty$:

\[P(y = k \mid z) = \Phi\Bigl(\frac{c_k - z}{\sigma}\Bigr) - \Phi\Bigl(\frac{c_{k-1} - z}{\sigma}\Bigr).\]

cutpoints are $c_1 < \dots < c_{K-1}$; one cutpoint is binary probit. Log-concave. The EP moments and the predictive are in closed form.

source
Lenticulum.PoissonLikelihood — Type
PoissonLikelihood(k = 1; maxcount = 20)

$k$ independent counts, $y_j \sim \mathrm{Poisson}(e^{z_j})$: the log link, one predictor per count. Log-concave in z, so its Laplace and EP messages always have a positive semidefinite precision. The predictive is over $\{0, \dots, \texttt{maxcount}\}^k$, the top value meaning "maxcount or more".

source
Lenticulum.RegimeFactor — Type
RegimeFactor(dims::NamedTuple, K::Int, m::Int; switch = :k, noise = 0.1, learn_noise = false,
             output = nothing, output_coeff = I, init_coeff, init_offsets)

The switching soft linear relation whose coefficients and noise depend on the outcome $k \in \{1, \dots, K\}$ (a regime):

\[f(x_1, \ldots, x_n, k) \;=\; w_k\, \mathcal N\Bigl(\textstyle\sum_i A_{i,k} x_i - b_k;\ 0,\ Q_k\Bigr).\]

dims names the continuous channels, m is the residual dimension, the outcome is the channel switch. The parameters are (A, b, logw): A a NamedTuple of m × dims[i] × K arrays, b the m × K right-hand sides, logw the unnormalised log-weights. noise is shared by the regimes as a scalar, a real vector (a diagonal) or an m × m matrix, or one per regime as a vector of K matrices or an m × m × K array. With learn_noise = true the noise is a parameter L too (m × m × K: $Q_k = L_k L_k^\top$, $L_k$ lower triangular with the exponential of L[:, :, k]'s diagonal on its diagonal), initialised at noise.

output = :y (dims[:y] == m) fixes the coefficient of y at output_coeff in every regime and drops it from the parameters: the factor is then a normalised conditional density of y given the other channels, a mixture of regressions. Without it, learning every coefficient (above all with learn_noise) has no lower bound: scaling A and L together lowers $-\log f$ (LP-022).

Mixtures of regressions (E1's crossing lines: A_{z,k} = [-a_k\ 1]), Gaussian mixtures with one covariance per component, mixtures of factor analysers ($z = W_k x + \mu_k + \varepsilon_k$ with a Gaussian x) and switching linear dynamics are instances. The factor is exact inside a Mycelium.cluster node. Outside one it sends exact messages and has its exact Bethe free energy as a one-factor cluster (Mycelium.as_cluster), which needs every continuous input but the target proper or observed, and a message to a continuous channel whose regimes give improper components only if they share their precision.

See regime.md.

source
Lenticulum.SwitchingConstraintFactor — Type
SwitchingConstraintFactor(dims::NamedTuple, K::Int, m::Int; switch = :k, noise = 0.1,
                          output = nothing, output_coeff = I, init_coeff, init_offsets)

The switching soft linear relation $0 = \sum_i A_i x_i - b_k + \varepsilon$, $\varepsilon \sim \mathcal N(0, Q)$, with a discrete outcome $k \in \{1, \dots, K\}$ of weight $w_k$:

\[f(x_1, \ldots, x_n, k) \;=\; w_k\, \mathcal N\Bigl(\textstyle\sum_i A_i x_i - b_k;\ 0,\ Q\Bigr).\]

dims names the continuous channels and their dimensions, m is the residual dimension, and the outcome is the channel switch. Q is a fixed hyperparameter, as in LinearConstraintFactor; the parameters are (A, b, logw): A a NamedTuple keyed by channel and shared by the outcomes, b the m × K matrix of right-hand sides, and logw the unnormalised log-weights (w = softmax(logw)).

Every channel, the outcome included, can be the target of an exact message, with the others observed or latent. With K = 1 it is a LinearConstraintFactor with c = b₁, message for message.

# a two-component mixture on a line in the plane: z = b_k + ε
f  = SwitchingConstraintFactor((z = 2,), 2, 2; noise = 0.1)
ps = (A = (z = Matrix(1.0I, 2, 2),), b = [-1.0 1.0; 0.0 0.0], logw = zeros(2))

output = :y fixes the coefficient of y (dims[:y] == m) at output_coeff and drops it from the parameters, so that the factor is a normalised conditional density of y, as for LinearConstraintFactor (LP-022); declare it when y carries data and A is learned.

See switching.md for the messages, the free energy and what is not done.

source
Lenticulum.Unscented — Type
Unscented(; α = 1.0, β = 2.0, κ = 0.0, iterations = 1)

Approximate a nonlinear factor's messages by the unscented transform: $2n+1$ sigma points of the expansion belief, pushed through $f$, fitted by a linear model with a residual covariance (statistical linear regression). Exact for linear $f$, and second-order accurate in the mean for smooth $f$; it needs a covariance at the expansion point (from the incoming belief), and falls back to linearisation at a point. α, β, κ are the usual scaling parameters (Wan & van der Merwe 2000); iterations as for Linearization (posterior linearisation).

The sigma points lie at $m \pm \alpha\sqrt{n + \kappa}$ standard deviations. The default α = 1 spreads them over the belief, so the regression sees the nonlinearity at the scale of the uncertainty and the centre weight is not negative. Wan & van der Merwe's α = 1e-3 puts them a thousandth of a standard deviation out, which makes the transform a local, second-order linearisation with weights of order $10^6$ (review of 2026-11-10).

source
Lenticulum.WishartPrecisionPrior — Type
WishartPrecisionPrior(df, scale)

Wishart density on the precision, with degrees of freedom df and SPD scale matrix scale (mean precision df * scale). Its negative log density is charged once per local free-energy evaluation, with no parameter-coordinate Jacobian. Thus MAP refers to density in precision coordinates. Requires df > dimension - 1.

source
Lenticulum.kl_divergence — Method
kl_divergence(q::GaussianBelief, p::GaussianBelief) -> Real

$D_{KL}(q \,\|\, p)$. Both must be proper. Used to measure the laxness quantities the framework keeps insisting should be reported rather than hidden — see Composition of Bayesian Lenses.md Remark 16.

source
Lenticulum.linearize — Method
linearize(f::NonlinearFactor, m, S) -> (A, b, Ω)

The local linear model $y \approx Ax + b + \nu$, $\nu\sim\mathcal N(0, \Omega + Q)$, of the factor around the belief $\mathcal N(m, S)$ on $x$: the Jacobian at $m$ ($\Omega = 0$) for Linearization; the statistical linear regression of $f$ on the sigma points of $\mathcal N(m, S)$ for Unscented, whose $\Omega$ is the part of $f$'s spread the line does not explain. Every message of the factor is GaussianFactor's message for $(A, b, Q + \Omega)$.

source
Lenticulum.log_evidence — Method
log_evidence(f::LikelihoodFactor, cavity, y) -> Real

$\log \int \text{cavity}(x)\, L(y \mid Ax+b)\, dx$, the log normaliser of the tilted distribution, as the factor's method approximates it (the Laplace evidence, or the quadrature of EP). The quantity whose gradient fits hyperparameters by evidence.

source
Lenticulum.moment_match — Method
moment_match(b) -> GaussianBelief

The Gaussian with the same mean and covariance as b: the minimiser of $\mathrm{KL}(b \,\|\, q)$ over Gaussians $q$, i.e. the projection step of expectation propagation. Defined for a SampleBelief (weighted sample mean and covariance; vector or matrix samples), a MixtureBelief of Gaussians (total mean and covariance), and a Gaussian (itself).

source
Lenticulum.noise_covariance — Method
noise_covariance(f::Union{GaussianFactor,GaussianPrior}, ps)

Current covariance reconstructed from opt-in precision parameters (including the eigenvalue floor when no noise prior is supplied). Fixed-noise factors return their original covariance. The initial covariance on the struct is never updated.

source
Lenticulum.outcomes — Function
outcomes(L) -> Vector

The values of y the predictive is a distribution over, in order. For count likelihoods the largest value of each coordinate stands for "that many or more", so the predictive keeps the whole mass.

source
Lenticulum.partial_sums — Method
partial_sums(g, ps, sum::Symbol, switches, groups; noise = 1e-6, prefix = :part)
    -> (g′, ps′, clusters)

E7's block wiring as an edit of g. sum is a LinearConstraintFactor (residual $\sum_c A_c x_c + c$) fed by the switching factors switches; groups lists groups of indices into switches, as Mycelium.overlap_groups(switch_overlaps(g, ps, sum, switches), B) returns them. For each group of two or more switches, their channels of sum are moved to a new LinearConstraintFactor :<prefix>j with residual $\sum_{l \in b} A_l u_l - s_j$ and noise noise·I (the coefficients $A_l$ copied from ps), and sum reads the partial sum $s_j$ (variable :<prefix>j_sum) with coefficient $I$ instead. As noise → 0 the model is unchanged.

Returns the edited graph, its parameters (the new factors' entries added, sum's replaced), and the groups :<prefix>j_block => [:<prefix>j, switches...] to pass to Mycelium.cluster. Singleton groups are left as they are. A sum whose parameters are tied to another node is refused: its coefficients would change for both.

source
Lenticulum.predictive — Method
predictive(f::LikelihoodFactor, belief) -> CategoricalBelief

$P(y = o) = \mathbb E\,L(o \mid Ax + b)$ for each o in outcomes(f.lik), under a GaussianBelief or a DiracBelief on $x$, labelled by the outcomes. Gauss–Hermite quadrature over the range of $\operatorname{Cov}(Ax+b)$; in closed form for OrdinalProbitLikelihood and for a point.

source
Lenticulum.regime_covariances — Method
regime_covariances(f::RegimeFactor, ps) -> Array

The noise covariances $Q_k$ as an m × m × K array: the fixed ones, or $L_k L_k^\top$ from the parameter L with learn_noise = true.

source
Lenticulum.residual — Function
residual(f::NonlinearFactor, x, y) -> Vector

$r = y - f(x)$, the factor's vector energy.

source
Lenticulum.residual — Method
residual(f, x, y, ps) -> Vector

$r = y - Ax - b$: the pointwise vector energy of Scalar and Multivariate Energy.md, in the residual space where the observation noise lives (Algebraic versus Geometric Distance.md §"what the energy space is").

Q⁻¹ is the noise precision, hence the natural scalarisation's metric — derived, not chosen.

source
Lenticulum.residual — Method
residual(f::LinearConstraintFactor, vals::NamedTuple, ps) -> Vector

$r = \sum_i A_i x_i - c$, the pointwise vector energy, in the space where the equation's own noise lives.

Note the signature: a NamedTuple of all channel values, with no input/output split, because the factor has none. See constraint.md §3 for why LenticulumCore.energy's (x, a, y) signature cannot express this directly.

source
Lenticulum.residual — Method
residual(f::SwitchingConstraintFactor, vals::NamedTuple, ps) -> Vector

$r = \sum_i A_i x_i - b_k$ for the outcome vals[f.switch] (an index in 1:K).

source
Lenticulum.residual_statistics — Method
residual_statistics(f, msgs, ps) -> (r̄, Cov_r)

The mean and covariance of the residual $r = y - Ax - b$ under the factor's own belief $b_c \propto f_c \prod_i \mu_{i\to c}$.

These are what make the two energies comparable on this factor:

\[\underbrace{\mathbb{E}\bigl[\tfrac12 r^\top Q^{-1} r\bigr]}_{\text{scalar (exact)}} \;-\; \underbrace{\tfrac12 \bar r^\top Q^{-1}\bar r}_{\text{multivariate (lax)}} \;=\; \tfrac12\operatorname{tr}\bigl(Q^{-1}\operatorname{Cov}(r)\bigr)\]

which is precisely the Jensen gap $\tfrac12\operatorname{tr}\operatorname{Cov}$ of Scalar and Multivariate Energy.md §5, here in closed form on a real model. Asserted in the test suite.

source
Lenticulum.residual_statistics — Method
residual_statistics(f::LinearConstraintFactor, msgs, ps) -> (r̄, Cov_r, H)

Mean and covariance of $r = \sum_i A_i x_i - c$ under the factor's own belief $b_c \propto f_c \prod_i \mu_{i\to c}$, plus that belief's entropy.

Dirac channels are substituted out rather than represented: a clamped channel moves its contribution into the constant, $c \mapsto c - A_j v_j$, and drops out of the joint. That is what keeps the joint precision finite, and it is the n-ary version of the three hand-written cases in gaussian.jl. With every channel clamped the residual is deterministic and the entropy is zero, which falls out of the same code rather than needing its own branch.

A Gaussian-mixture message is collapsed to its moments first, as factor_message collapses it, so the belief is the one the outgoing messages were computed from (assumed-density filtering), not the exact mixture belief.

source
Lenticulum.switch_overlaps — Method
switch_overlaps(g, ps, sum::Symbol, switches) -> Matrix

The pairwise overlaps of switching factors feeding one linear factor sum, for grouping them into cluster nodes (Mycelium.overlap_groups, E7's rule). Switch $l$ sets its channel $u_l$ ($A_{u,l} u_l = b_{l,k} + \varepsilon_l$, $A_{u,l}$ square and invertible), which enters sum as $A_{\mathrm{sum},l}\, u_l$; its effect on the sum's residual is the matrix of directions $V_l = A_{\mathrm{sum},l} A_{u,l}^{-1} (b_{l,k} - b_{l,j})_{j < k}$ (every pair of outcomes). In the metric of the noise of the sum and the listed switches (other inputs of sum are not counted) $R = Q_{\mathrm{sum}} + \sum_l A_{\mathrm{sum},l} A_{u,l}^{-1} Q_l A_{u,l}^{-\top} A_{\mathrm{sum},l}^\top$, the overlap of two switches is the largest $\lvert\cos\rvert$ between a direction of one and a direction of the other; the diagonal is 1. For binary switches feeding $z = \sum_l W_l k_l + \varepsilon$ this is E7's $\lvert G_{lm}\rvert / \sqrt{G_{ll} G_{mm}}$, $G = W^\top R^{-1} W$.

source
Lenticulum.switch_statistics — Method
switch_statistics(f::SwitchingConstraintFactor, msgs, ps)
    -> (q, r̄, Cov_r, H_x)

The factor's own belief $b_c(x, k) \propto f(x, k)\, \mu_k(k) \prod_i \mu_i(x_i)$, summarised: the outcome posterior q, the per-outcome residual means $\bar r_k$ (columns), the residual covariance (one for all outcomes) and the entropy $H(x \mid k)$ (also one for all outcomes). Dirac channels are substituted out, as in residual_statistics. A Gaussian-mixture message is collapsed to its moments first.

source
LenticulumCore.energy — Method
LenticulumCore.energy(f::LikelihoodFactor, x, a, y, ps, st)

The negative log-likelihood $-\log L(y \mid Ax + b)$, as a one-element vector.

source
LenticulumCore.energyspace — Method
energyspace(f::GaussianFactor)

$E_c = \mathbb{R}^3$, graded as (fit, complexity, negentropy):

summandisrole
fit$\mathbb{E}_{b_c}[\tfrac12 r^\top Q^{-1} r]$how badly the residual misses
complexity$\tfrac12\log\det(2\pi Q)$the log-normaliser; penalises variance inflation
negentropy$-H(b_c)$the entropy term of the free energy

The grading is not decoration: it is the fit/complexity/entropy decomposition every Gaussian model has, made visible for free by $E_G = \bigoplus_f E_f$. Summing it recovers the exact free energy, so the scalarisation is IdentityScalarisation on each summand and is therefore linear — hence composition is strict, per Scalar and Multivariate Energy.md §5.

This is the space of the free energy. The pointwise energy is the residual, in its own space: see pointwise_energyspace below.

The alternative grading — keep the residual $r$ as a vector with SquaredNorm(Q⁻¹) — is lax, and by exactly $\tfrac12\operatorname{tr}(Q^{-1}\operatorname{Cov}(r))$. residual_statistics exposes both sides of that identity; see gaussian.md §2.

source
LenticulumCore.likelihood_message — Method
LenticulumCore.invert(lens, π, inputs, ps, st) -> (belief, st)

The approximate posterior $\pi \cdot \mu_{c\to\text{target}}$: backwards, the Laplace or EP approximation of $\pi(x) L(y \mid Ax+b)$; forwards, the predictive pooled with $\pi$.

source
LenticulumCore.likelihood_message — Method
LenticulumCore.invert(lens, π, inputs, ps, st) -> (belief, st)

The exact Bayesian inversion $c^\dagger_\pi(y)$ of AutoBayes Definition 10 — the posterior, including the prior.

[!warning] This is not the belief-propagation message $c^\dagger_\pi(y)$ is the posterior; a BP message is the likelihood, with the prior divided out. If a factor sent the posterior, a variable of degree $d$ would count the prior $d$ times. In canonical form the two differ by one subtraction, and Mycelium.factor_message returns the likelihood while this returns the posterior.

The relation between them is exactly combine(π, message) == posterior, which is asserted in the test suite. On a chain (every variable of degree 2) the distinction is invisible, which is why it does not appear in the paper. See gaussian.md §4.

source
LenticulumCore.likelihood_message — Method
LenticulumCore.likelihood_message(lens, π, inputs, ps, st) -> (site, st)

The final approximate likelihood site; invert combines it with π to obtain the approximate posterior $\pi \cdot \mu_{c\to\text{target}}$. Backwards, the likelihood is relinearised at the current posterior mean method.iterations times (with the posterior's covariance for the unscented transform): Gauss–Newton on the MAP, the iterated EKF / iterated posterior linearisation. One iteration is the plain EKF / UKF update.

source
LenticulumCore.pointwise_energyspace — Method
pointwise_energyspace(f::GaussianFactor), pointwise_scalarisation(f::GaussianFactor)

The pointwise energy is the residual $r \in \mathbb{R}^{n_{out}}$, signed, scalarised by SquaredNorm(Q⁻¹): $\tfrac12 r^\top Q^{-1} r$, zero at a perfect fit. Its expectation under the factor belief is the fit summand of the free energy (the graded energyspace above); the complexity $\tfrac12\log\det(2\pi Q)$ is the normaliser it leaves out, a constant while Q is fixed. The residual is the same in both polarities, so neither depends on one.

source
LenticulumCore.supported_polarities — Method
LenticulumCore.supported_polarities(f::LinearConstraintFactor)

Every assignment with exactly one Unobserved() channel and each other channel Observed() or Latent(): $n \cdot 2^{n-1}$ of them, the set supports_polarity accepts (L-SPEC-001). The $n$ with every other channel observed are the ones that matter most: one per channel, and none of them is the factor's "real" direction. The equation was never written in a direction. This is the enumerated form of "a relation, not a function".

Contrast GaussianFactor, which has exactly two regardless of anything. Above 6 channels the list throws, as the contract permits; hasnopolarity and isunidirectional answer without it.

source
LenticulumCore.supports_polarity — Method
LenticulumCore.supports_polarity(f::SwitchingConstraintFactor, p)

Any polarity over exactly this channel set, in any order, with exactly one Unobserved() channel. A Latent() outcome is marginalised under the factor's own weights; a Latent() continuous channel is integrated out under a flat prior, as for LinearConstraintFactor. Either way the channel is a graph variable the message sums or integrates over, not a latent space of the open model, which stays pure.

source
LenticulumCore.supports_polarity — Method
LenticulumCore.supports_polarity(f::LinearConstraintFactor, p)

Any polarity over exactly this channel set, in any order, with exactly one Unobserved() channel.

A Latent() channel is integrated out under a flat prior (see Mycelium.factor_message below): the message keeps what the equation says in the directions that channel cannot reach, and is uninformative when its A has full row rank. A latent channel is a legal modelling choice, and "this equation tells you nothing more about $x_t$ once $x_j$ is free" is a correct answer, not an error.

source
Mycelium.factor_belief — Method
Mycelium.factor_belief(f::GaussianFactor, msgs, ps, st) -> FrozenGaussianBelief
Mycelium.belief_free_energy(f::GaussianFactor, b, ps, st)

The factor belief, and the local free energy at a fixed one: (fit, complexity, negentropy) with the residual $r = y - Ax - b$ averaged under the frozen b, so fit and complexity move with the parameters and negentropy does not. Agrees with local_free_energy at the belief it was evaluated at. See FrozenGaussianBelief.

source
Mycelium.factor_belief — Method
Mycelium.factor_belief(f::GaussianPrior, msgs, ps, st) -> FrozenGaussianBelief
Mycelium.belief_free_energy(f::GaussianPrior, b, ps, st)

As for GaussianFactor, with residual $x - \mu$; only a learnable noise moves.

source
Mycelium.factor_belief — Method
Mycelium.factor_belief(f::LinearConstraintFactor, msgs, ps, st) -> FrozenGaussianBelief
Mycelium.belief_free_energy(f::LinearConstraintFactor, b, ps, st)

The factor belief, and the local free energy at a fixed one, with $r = \sum_i A_i x_i - c$ averaged under b; it agrees with local_free_energy at the belief it was evaluated at. Its parameter derivative is the graph gradient's (LP-020). See FrozenGaussianBelief.

source
Mycelium.factor_belief — Method
Mycelium.factor_belief(f::SwitchingConstraintFactor, msgs, ps, st) -> FrozenGaussianBelief
Mycelium.belief_free_energy(f::SwitchingConstraintFactor, b, ps, st)

The factor belief $b_c(x, k)$, held fixed: the outcome posterior q, one mean per outcome and the shared covariance. The local free energy at a fixed b averages $r_k = \sum_i A_i x_i - b_k$ and $-\log w_k$ under it, so A, b and logw all move and the entropy $H(q) + H(x \mid k)$ does not. Agrees with local_free_energy at the belief it was evaluated at (LP-020).

source
Mycelium.factor_beliefs — Method
Mycelium.factor_beliefs(fs::AbstractVector{<:SwitchingConstraintFactor}, msgs::NamedTuple, ps, st)
Mycelium.belief_free_energies(fs::AbstractVector{<:SwitchingConstraintFactor}, b::FrozenSwitchingBeliefs, ps, st)
Mycelium.local_free_energies(fs::AbstractVector{<:SwitchingConstraintFactor}, msgs::NamedTuple, ps, st)

The free energy of a plate row of tied switching factors on message blocks (L-ADR-0010): every continuous channel a point block, or a Gaussian block with one shared precision, trivial or absent, and the outcome a categorical block, trivial or absent. Then the factor belief's precision, covariance and $H(x \mid k)$ are computed once, and each replica costs its means and its outcome posterior $q_i$, as switch_statistics computes them one replica at a time. factor_beliefs returns nothing otherwise, and the energies go replica by replica.

source
Mycelium.factor_message — Method
Mycelium.factor_message(f::GaussianFactor, target, polarity, inputs, prior, ps, st)

The factor → variable message: the likelihood contribution, in canonical form.

Forward ($x$ observed, target $y$), for an incoming $\mathcal{N}(m, S)$ on $x$:

\[\mu_{c\to y} = \mathcal{N}\bigl(Am + b,\ ASA^\top + Q\bigr)\]

Backward ($y$ observed, target $x$), for an incoming $\mathcal{N}(m_y, S_y)$ on $y$, writing $R = Q + S_y$:

\[\mu_{c\to x} = \mathcal{N}^{-1}\bigl(A^\top R^{-1}(m_y - b),\ A^\top R^{-1}A\bigr)\]

Note the backward message is generally improper — $A^\top R^{-1} A$ is rank-deficient whenever A is not full column rank, because a likelihood constrains only the directions the model can see. It is then an ExtendedGaussian, flat along $\operatorname{null}(A)$ (a declared-improper GaussianBelief under ForwardDiff); otherwise a GaussianBelief.

The prior argument is deliberately unused: including it would double-count. See LenticulumCore.invert above.

source
Mycelium.factor_message — Method
Mycelium.factor_message(f::LikelihoodFactor, target, polarity, inputs, prior, ps, st)

Backwards (target $x$): the Gaussian site of LikelihoodFactor's method at the cavity prior. The cavity is used to place the approximation, never multiplied in, so nothing is counted twice. Forwards (target $y$): the predictive CategoricalBelief.

source
Mycelium.factor_message — Method
Mycelium.factor_message(f::LinearConstraintFactor, target, polarity, inputs, prior, ps, st)

The factor → variable message, in canonical form. Writing $d = c - \sum_{i \ne t} A_i m_i$ and $R = Q + \sum_{i \ne t} A_i S_i A_i^\top$ for incoming $\mathcal{N}(m_i, S_i)$:

\[\mu_{c\to t} \;=\; \mathcal{N}^{-1}\bigl(A_t^\top R^{-1} d,\ A_t^\top R^{-1} A_t\bigr)\]

One formula for every direction — the target channel appears only as "which $A_i$ is $A_t$". Specialising to two channels with $A_x = -A$, $A_y = I$, $c = b$ reproduces both of GaussianFactor's messages exactly, forward and backward.

The message is improper whenever $A_t$ is not full column rank — an equation constrains only the directions it can see, and with fewer equations than unknowns ($m < \dim x_t$) that is the normal case, not an edge case. Such a message is an ExtendedGaussian, flat along the directions the equation cannot see; a message that is not flat anywhere is a GaussianBelief.

A non-target channel that carries no information (no message, TrivialBelief(), a Latent() channel) is integrated out under a flat prior, the limit $\tau \to \infty$ of $\mathcal N(0, \tau \Sigma_0)$: $R^{-1}$ becomes $R^{-1} - R^{-1} G (G^\top R^{-1} G)^{+} G^\top R^{-1}$, with $G$ stacking those channels' $A$'s, the same limit for every $\Sigma_0 \succ 0$. An improper Gaussian input splits into a proper part on the range of its precision, which enters $d$ and $R$, and a flat part on its null space $N$, which adds $A_i N$ to $G$. The equation then still constrains the directions $G$ cannot reach. The message is TrivialBelief() when nothing is left, as when $G$ has full row rank, and an ExtendedGaussian whenever the (projected) $A_t$ is rank-deficient (under ForwardDiff a declared-improper GaussianBelief(η, Λ, true) instead) (constraint.md §4.1). Under ForwardDiff the flat case is differentiable: when only $c$, the inputs' information vectors or means are dual, the flat directions do not move (a flagged input is then split on its precision's values, gaussian.md, "Dual inputs on the flat path"); when they move (dual coefficients, or a flat input whose precision is dual) the derivative is taken at constant rank, and a tangent that changes the number of flat directions is an ArgumentError (gaussian.md, "Flat directions that move").

source
Mycelium.factor_message — Method
Mycelium.factor_message(f::NonlinearFactor, target, polarity, inputs, prior, ps, st)

The factor → variable message, as for GaussianFactor but through the local linear model of linearize:

  • forwards (target $y$), around the incoming belief $\mathcal N(m, S)$ on $x$: $\mathcal N(Am + b,\ ASA^\top + \Omega + Q)$; for a point input $x$, exactly $\mathcal N(f(x), Q)$;
  • backwards (target $x$), around the cavity prior (the other messages into $x$), or x0: the likelihood $\mathcal N^{-1}(A^\top R^{-1}(m_y - b),\ A^\top R^{-1}A)$, $R = Q + \Omega + S_y$. With no expansion point it is uninformative.

Backwards, method.iterations > 1 repeats linearisation at the current posterior, using the same final site as likelihood_message and invert.

The prior is used only as the point to linearise at, never multiplied into the returned site, so nothing is counted twice.

source
Mycelium.factor_message — Method
Mycelium.factor_message(f::SwitchingConstraintFactor, target, polarity, inputs, prior, ps, st)

With $d_k = b_k - \sum_{i \ne t} A_i m_i$ and $R = Q + \sum_{i \ne t} A_i S_i A_i^\top$ for incoming $\mathcal N(m_i, S_i)$, and $\pi_k$ the incoming outcome belief (flat if latent):

  • to the outcome: $P(k) \propto w_k\, \mathcal N(d_k; 0, R)$, a CategoricalBelief;
  • to a continuous channel $t$: $\sum_k \pi_k w_k\, \mathcal N(A_t x_t; d_k, R)$, a MixtureBelief of the canonical Gaussians $\mathcal N^{-1}(A_t^\top R^{-1} d_k, A_t^\top R^{-1} A_t)$ with the per-outcome constants in the weights. All components share one precision; they are improper when $A_t$ is not full column rank, declared so (GaussianBelief(η, Λ, true)), and weighted as the unnormalised exponentials that Mycelium's mixture product expects. A single possible outcome gives a GaussianBelief.

A Gaussian-mixture input is collapsed to its moments first, as in LinearConstraintFactor (assumed-density filtering; gaussian.md, "Mixture inputs"). An uninformative continuous input (no message, TrivialBelief(), Latent()) is integrated out under a flat prior, as in LinearConstraintFactor, and so is an improper Gaussian input along its null space. $R^{-1}$ is replaced by the limit that projects out the range of those flat directions under the $A$'s, so only the part of $d_k$ they cannot reach separates the outcomes. When they cover every residual direction nothing is left: the outcome gets the factor's own weights and a continuous target TrivialBelief().

source
Mycelium.factor_message — Method
Mycelium.factor_message(f::HardConstraintFactor, target, polarity, inputs, prior, ps, st)

The message to channel t: the joint of the other inputs, as extended Gaussians, pushed forward to $r = c - \sum_{i \ne t} A_i x_i + \varepsilon$ ($\varepsilon \sim \mathcal N(0, Q)$, Q the noise, zero by default), then the likelihood of $x_t$ under $A_t x_t = r$:

\[\mu_{f\to t} \;=\; \operatorname{preimage}\bigl(\operatorname{pushforward}(\textstyle\bigotimes_{i\ne t} \mu_{i\to f},\ -[A_i]_{i \ne t},\ c,\ Q),\ A_t\bigr).\]

An ExtendedGaussian: hard where the equation and the inputs fix $x_t$, flat along $\operatorname{null}(A_t)$ and wherever an uninformative input reaches.

source
Mycelium.factor_messages — Method
Mycelium.factor_messages(fs::AbstractVector{<:LinearConstraintFactor}, target, polarity, inputs, priors, ps, st)

The messages of tied replicas (a plate). When the replicas have one noise covariance and every replica's inputs have the same covariances and flat directions (points, or Gaussians of one covariance, as with data and a shared prior), the metric, its factorisation and the rank test depend on the parameters only and are computed once; each replica then costs one matrix-vector product. The messages are those of factor_message, term for term. Otherwise each replica is computed on its own.

source
Mycelium.factor_messages — Method
Mycelium.factor_messages(fs::AbstractVector{<:LinearConstraintFactor}, target, polarity, inputs::NamedTuple, priors, ps, st)

The same messages from message blocks (a plate sweep on blocks, L-ADR-0010): inputs holds one BeliefBlock per channel. When the replicas share the noise covariance and each input block is points, trivial, or proper Gaussians with one shared precision, the shared work is done once and the result is one GaussianBlock with a shared precision: $H = K(c\,\mathbf 1^\top - \sum_n A_n M_n)$, with $M_n$ the input means as columns. Otherwise each replica's column goes through the method above.

source
Mycelium.factor_messages — Method
Mycelium.factor_messages(fs::AbstractVector{<:SwitchingConstraintFactor}, target, polarity, inputs::NamedTuple, priors, ps, st)

The messages of tied replicas from message blocks. To the outcome, when the replicas share the noise covariance and each continuous input block is points, trivial, or proper Gaussians with one shared precision (as LinearConstraintFactor's block method), the metric is factorised once and the result is one CategoricalBlock: $\log P_i(k) = \log w_k - \tfrac12 \lVert T(b_k - s_i)\rVert^2$ up to normalisation, with $s_i = \sum_n A_n m_{n,i}$. A mixture message to a continuous channel, and any other input, goes replica by replica through factor_message.

source
Mycelium.factor_residual_moments — Method
Mycelium.factor_residual_moments(f::LinearConstraintFactor, msgs, ps) -> (mean, cov)

The mean and full covariance of the residual under the factor's belief, from residual_statistics. Mixture messages are collapsed to their moments first. Used by Mycelium's SampleBuffer.

source
Mycelium.factor_residual_moments — Method
Mycelium.factor_residual_moments(f::SwitchingConstraintFactor, msgs, ps) -> (mean, cov)

The mean and full covariance of the residual $r = \sum_i A_i x_i - b_k$ under the factor's belief, over the outcome as well: $\bar r = \sum_k q_k \bar r_k$ and $\operatorname{Cov} r = \operatorname{Cov}(r \mid k) + \sum_k q_k (\bar r_k - \bar r)(\bar r_k - \bar r)^\top$, from switch_statistics. Used by Mycelium's SampleBuffer.

source
Mycelium.local_free_energy — Method
Mycelium.local_free_energy(f::GaussianFactor, msgs, ps, st)

The factor's Bethe contribution $F_c = U_c - H(b_c)$, as a graded energy (fit, complexity, negentropy), where

\[b_c(x,y) \;\propto\; \mathcal{N}(y;\,Ax+b,\,Q)\ \mu_{x\to c}(x)\ \mu_{y\to c}(y)\]

Note msgs are the incoming messages $\mu_{i\to c}$, not the variable marginals. That is what the Bethe formula asks for, and getting it wrong is the bug recorded in free_energy.md §7.

source
Mycelium.local_free_energy — Method
Mycelium.local_free_energy(f::LikelihoodFactor, msgs, ps, st)

The factor's Bethe contribution, graded (fit, complexity, negentropy), with the factor belief replaced by its Gaussian approximation $q$ (cavity = the incoming message on $x$, times the site): fit is $\mathbb E_q[-\log L(y \mid Ax+b)]$ by Gauss–Hermite quadrature, negentropy is $-H(q)$, complexity is 0 (the likelihood is normalised in $y$). For a point $x$ it is $-\log L(y \mid Ax+b)$ exactly. Approximate as the messages are.

With y unobserved, fit = E_x[H(L(⋅|Ax+b))] and negentropy = -fit - H(x). The conditional entropy sums all ordinal categories exactly; its expectation over a Gaussian input uses Gauss–Hermite quadrature (an approximation, method.order for EP, 20 for Laplace). The count likelihoods truncate their support (not the lumped predictive grid) with geometric tail bounds and hard work limits that throw. Likelihoods without a _conditional_entropy method throw.

source
Mycelium.local_free_energy — Method
Mycelium.local_free_energy(f::NonlinearFactor, msgs, ps, st)

The factor's Bethe contribution, graded (fit, complexity, negentropy), computed for the local linear model around the incoming belief on $x$ (its point, if it is a DiracBelief): exact for affine $f$, the usual EKF / UKF approximation of the evidence otherwise.

source
Mycelium.local_free_energy — Method
Mycelium.local_free_energy(f::SwitchingConstraintFactor, msgs, ps, st)

$F_c = \mathbb E_{b_c}[-\log f] - H(b_c)$, exact for Gaussian, Dirac and categorical messages, since $H(b_c) = H(q) + H(x \mid k)$. Graded as (fit, complexity, negentropy): fit is $\sum_k q_k\, \tfrac12 \mathbb E\|r_k\|^2_{Q^{-1}}$, complexity is $\tfrac12 \log\det 2\pi Q - \sum_k q_k \log w_k$, and negentropy is $-H(q) - H(x \mid k)$.

source
Mycelium.local_free_energy — Method
Mycelium.local_free_energy(f::HardConstraintFactor, msgs, ps, st)

The factor's Bethe term at its incoming messages. Points and the hard directions of incoming extended Gaussians are substituted out, leaving the relation $\tilde G s = \tilde c$ on the remaining coordinates $s$ (orthonormal along each input's support).

  • Without noise (noise = 0), the limit of LinearConstraintFactor with noise $\varepsilon I$ as $\varepsilon \to 0$, whose divergent parts cancel:

    \[F_f \;=\; \tfrac12 \log\det\bigl(\tilde G \tilde G^\top\bigr) \;-\; H_R(b_f),\]

    with $H_R$ the entropy of the factor belief relative to Lebesgue measure on the solution set $R$ of the relation (complexity and negentropy; fit is 0). It needs $\tilde G$ of full row rank: redundant equations, or equations on directions an input already fixes exactly, have an infinite free energy, and throw. So does a factor belief that is flat along $R$.

  • With a positive definite noise, LinearConstraintFactor's free energy.

A noise that is singular but not zero is not implemented. See hard_constraint.md, "Free energy".

source
Mycelium.mean_field_energy — Method
Mycelium.mean_field_energy(f::LinearConstraintFactor, marginals, ps, st)

The average energy of a MeanField() node, $\mathbb E_{\prod_i q_i}[-\log f]$ with $r = \sum_i A_i x_i - c$ under independent marginals: fit is $\tfrac12(\bar r^\top Q^{-1}\bar r + \operatorname{tr}(Q^{-1}\sum_i A_i \Sigma_i A_i^\top))$, complexity the normaliser, and negentropy zero, since the variables' entropies are counted at the variables. Marginals must be points or proper Gaussians (mixtures by their moments, which is exact for this quadratic energy).

source
Mycelium.mean_field_energy — Method
Mycelium.mean_field_energy(f::SwitchingConstraintFactor, marginals, ps, st)

The average energy of a MeanField() node under independent marginals on the continuous channels and the outcome $q$: $\sum_k q_k \bigl(\tfrac12 \mathbb E\lVert r_k\rVert^2_{Q^{-1}} - \log w_k\bigr)$ plus the normaliser, with no entropy (Riegler et al. 2013, eq. 17). The outcome's marginal must be a categorical or a point; the continuous ones as for LinearConstraintFactor.

source
Mycelium.mean_field_message — Method
Mycelium.mean_field_message(f::LinearConstraintFactor, target, polarity, beliefs, ps, st)

The message of a MeanField() node: $\exp \mathbb E_q[\log f]$ as a function of $x_t$, over the other channels' current marginals $q_i$. $\log f$ is quadratic, so only their means enter, and the message is $\mathcal N^{-1}(A_t^\top Q^{-1} \bar d, A_t^\top Q^{-1} A_t)$ with $\bar d = c - \sum_{i \ne t} A_i\, \mathbb E_q[x_i]$: the belief-propagation message with every input replaced by a point mass at its mean. A mixture marginal enters by its mean. A channel without a marginal yet is integrated out under a flat prior, as in factor_message, and so is an improper marginal along the directions where it is flat; its mean elsewhere enters as usual.

source
Mycelium.mean_field_message — Method
Mycelium.mean_field_message(f::SwitchingConstraintFactor, target, polarity, beliefs, ps, st)

The message of a MeanField() node, $\exp \mathbb E_q[\log f]$, over the other channels' current marginals. With $\bar s = \sum_{i \ne t} A_i\, \mathbb E_q[x_i]$ and $q$ the outcome's marginal:

  • to a continuous channel: $\mathcal N^{-1}(A_t^\top Q^{-1}(\bar b - \bar s),\ A_t^\top Q^{-1} A_t)$ with $\bar b = \sum_k q_k b_k$;
  • to the outcome: $P(k) \propto w_k \exp(-\tfrac12 (b_k - \bar s)^\top Q^{-1} (b_k - \bar s))$.

Only means enter, since $\log f$ is quadratic in the continuous channels for each outcome and the covariance terms do not depend on $x_t$ or $k$. An outcome without a marginal yet counts with the factor's weights, which is where the iteration starts. A continuous channel without a proper marginal yet is integrated out under a flat prior, as in factor_message, with $Q^{-1}$ in place of $R^{-1}$; so is an improper marginal along the directions where it is flat.

A rank-deficient precision is declared improper, as in factor_message.

source
Mycelium.mixture_message — Method
Mycelium.mixture_message(f::SwitchingConstraintFactor, target, polarity, inputs, prior, ps, st)
Mycelium.mixture_message(f::LinearConstraintFactor, target, polarity, inputs, prior, ps, st)

The exact message when inputs are Gaussian mixtures, for MixtureReduction() nodes: each mixture input $\sum_j v_{ij}\, \mathcal N(m_{ij}, S_{ij})$ is expanded instead of collapsed. For a combination $c$ of components, $R_c = Q + \sum_{i \ne t} A_i S_{i c_i} A_i^\top$ and $d_{kc} = b_k - \sum_{i \ne t} A_i m_{i c_i}$, and the combination carries the evidence $\prod_i v_{i c_i}\, \mathcal N(d_{kc}; 0, R_c)$, with the determinant of $R_c$, which now differs between combinations:

  • to the outcome: $P(k) \propto w_k \sum_c \prod_i v_{i c_i}\, \mathcal N(d_{kc}; 0, R_c)$;
  • to a continuous channel: the mixture over $(k, c)$ of $\mathcal N(A_t x_t; d_{kc}, R_c)$, weighted by $\pi_k w_k \prod_i v_{i c_i}$, one component each (a GaussianBelief if only one).

Components are normalised when proper and weighted as unnormalised exponentials when not, as in factor_message. Flat directions of non-mixture inputs are integrated out as there; the diverging normaliser is the same for every combination. The components of a mixture input must be proper Gaussians. A LinearConstraintFactor is the case $K = 1$, $b_1 = c$. Without mixture inputs the message is factor_message's.

source
Mycelium.parameter_free_energy — Method
Mycelium.parameter_free_energy(f::GaussianFactor, msgs, ps, st)

The Bethe term $U_c - H(b_c)$ with the factor belief over every block that carries a belief: $b_c(z) \propto \mathcal N(r(z); 0, Q) \prod_k \mu_{k \to c}(z_k)$, Gaussian in $z$. Same grading and entropy convention as local_free_energy (points carry no entropy; Gaussians relative to Lebesgue measure).

source
Mycelium.parameter_message — Method
Mycelium.parameter_message(f::GaussianFactor, target, polarity, msgs, ps, st)

The exact likelihood message to target ∈ (:A, :b, the output channel) when A and/or b are exposed, for a point input $x_0$. Writing $r = \sum_k C_k z_k$ over the blocks $z = (y, \operatorname{vec}A, b)$ with $C = (I, -x_0^\top \otimes I, -I)$, the other blocks' means and covariances give $s = \sum_{k \ne t} C_k m_k$ and $V = Q + \sum_{k \ne t} C_k S_k C_k^\top$, and the message is $\mathcal N^{-1}(-C_t^\top V^{-1} s,\ C_t^\top V^{-1} C_t)$. To :A it has rank nout when $x_0 \ne 0$ (improper unless nin = 1) and rank 0 when $x_0 = 0$; to the output it is the predictive $\mathcal N(\bar A x_0 + \bar b,\ K S_A K^\top + S_b + Q)$.

A block without information, whose coefficient has full row rank, leaves the residual unconstrained, so the message is TrivialBelief(); this is exact, not a fallback. An improper belief on a block is used where the factor sees only its proper directions; otherwise the message throws.

source

Exact Gaussian game composition (LP-009)

ComposedFactor(first,second) supports two GaussianFactors sharing an intermediate channel of matching dimension. The internal channel is integrated out; endpoint game/free energy equals the collapsed GaussianFactor with covariance B*Q₁*B' + Q₂, including propagated first-stage noise. Both endpoint polarities, messages and Gaussian pushforwards are exact. Component noise hyperpriors and other compositions are unsupported. Independent TensorFactor games use NamedTuple products keyed by parts, preserving each branch's channel game and grading; scalar free energies add. Correlated beliefs are refused.

Composite parameter gradients

Gaussian ComposedFactor(a,b; coupling) is a collapsed endpoint game. Its historical diagonal, exact and analytic pathwise gradients all differentiate that collapsed law; there is no exposed internal message to cut. Score coupling currently requires Dirac input/endpoint beliefs, explicit rng and nsamples >= 2, and a fixed scalar baseline. It samples the exact internal posterior and returns MyceliumLearning.GradientEstimate with componentwise standard errors. The baseline equal to the reference endpoint NLL cancels the posterior score noise. ForwardDiff/Zygote and reference differences support the analytic gradients; the score estimator uses the documented schedule-backend whitelist. An explicit Mycelium graph exposes internal boundaries when different cuts are needed.

The private static-replay message adapters for GaussianFactor, GaussianPrior and GaussianComposite carry the selected AD backend context while retaining their existing functional inverse formulas. This preserves CPU ForwardDiff/Zygote marginal-likelihood gradients when ImplicitLayers replay selects its IFT backend. It adds no dense device Gaussian factorisation; the JLArrays diagonal-message subset is described under Mycelium.