Canonical Gaussian beliefs, their moment accessors and properness checks.
Sources: code:
gaussian_belief.jlTheory: Gaussian Belief · Beliefs · Belief Algebra
Implements: the Gaussian belief type in canonical form, with , , and its accessors.
1. What is in the file
| name | what |
|---|---|
GaussianBelief(η, Λ), GaussianBelief(η, Λ, improper) | the type; Λ may be singular (a likelihood message), and improper = true says so whatever rounding does to Λ |
Gaussian(μ, Σ) | from moments; Σ must be positive definite |
uninformative(n) | , : the identity for pooling |
isproper, belief_mean, belief_cov, logpartition | require a proper belief, and say so |
dimension | extends the package’s dimension |
2. Why it is here
It used to live in the top-level Lenticulum package, where no lib/ package could reach it.
So the equilibrium and diffusion factors returned points even where they could compute a
Gaussian, and four notes recorded the same wall (The Equilibrium Family §5). It moved here,
next to the other five belief types, in October 2026. The split is now:
| layer | Gaussian code |
|---|---|
LenticulumCore (gaussian_belief.jl) | the type and its accessors |
Mycelium (gaussian.jl) | the message rules: combine, densities, entropy, distance, damping, combine_with_overlap; mixture reduction (reduce_mixture, since P17) |
Lenticulum (beliefs.jl) | projections: moment_match, and kl_divergence |
Lenticulum re-exports every name, so user code did not change. The cost is one dependency,
the LinearAlgebra standard library.
3. A fix that came with the move
VariationalDiffusion and ImplicitLayers read a prior’s point through
Mycelium.belief_mean, a function that never existed. The call sat in a try block, so every
Gaussian prior was silently treated as having no point. It now calls
LenticulumCore.belief_mean, and a Gaussian prior’s mean is the anchor and warm start, as
documented. This is tested in VariationalDiffusion’s “answers as beliefs” test set.
Related: open_model, gaussian, messages
Device diagonal beliefs (2026-10-27)
For device Diagonal precision/covariance, properness is an all-positive finite
check, mean is elementwise division, covariance is reciprocal diagonal and logdet
is the sum of logarithms. The diagonal log-partition uses typed constants and an
elementwise quadratic form, preserving Float32 in values and both tested derivatives.
No dense factorisation or host array copy is needed.
Dense device Gaussian precision/covariance gives an ArgumentError rather than
silently entering the legacy CPU Matrix path. CPU factorisation behavior is
unchanged. Hardware dense Gaussian algebra is not claimed.
Device mean/solve paths use elementwise operations; the covariance reverse rule
returns structural diag cotangents. A dense precision cotangent conflicted with direct diagonal reads in
Zygote when the same belief contributed moments and variance terms. The analytic
rules are , and
, elementwise.
Tests in lib/ImplicitLayers.jl/test/device_arrays.jl compare Gaussian graph
expectation derivatives with closed forms and CPU references. Singular diagonal
beliefs are still valid likelihood messages, but moment/density requests reject them.
Batch C graph polarity check (2026-10-28)
A Gaussian-consuming device factor exposed an ambiguity between GPUArrays and
LinearAlgebra/SparseArrays when isuninformative reduced over the whole device-backed
Diagonal wrapper. The GPUArraysCore extension now checks its stored diagonal
and information vector directly. This is equivalent to checking the full diagonal
matrix, and keeps CPU behavior unchanged. Mycelium/test/graph_adjoint_devices.jl
checks zero/precision-only/information-only beliefs, then tree/loopy Gaussian
schedule replay with scalar indexing disabled. Earlier terminal-only device
Gaussian fixtures did not exercise this polarity-resolution path.
Symmetry is checked (2026-10-29, LP-019 item 3)
Symmetric(M) reads one triangle, so Gaussian(μ, [1 0.2; 10 1]) was accepted as
[1 0.2; 0.2 1], and the same matrix as a precision passed isproper although its symmetric
part has eigenvalues −4.1 and 6.1. Gaussian, the precision factorisation behind the moment
accessors, and isproper now check
first: rounding-level asymmetry, as left by , passes. GaussianBelief(η, Λ)
itself still accepts any square Λ, since it may be a likelihood; an asymmetric one is simply
not proper. Tested in test/lp019.jl.
After review (2026-10-30): a tolerance of still admitted
[1 0.9999; 1.0002 1], whose upper triangle is definite and whose symmetric part is not. The
tolerance is now of the element type, and definiteness is checked on, and
factorisations use, the symmetric part (_symmetric_part; the form
overflowed for entries near floatmax).
Second review (2026-10-30): the tolerance is again for every element type. With definiteness on the symmetric part it only refuses gross asymmetry, and a type-dependent tolerance had changed under ForwardDiff duals and rejected LP-002’s finite-difference adjoint perturbations. The symmetric part is , exact for symmetric (a subnormal diagonal survives) and without overflow.
Declared improper (2026-11-02)
isproper used to be isposdef of the symmetric part, and nothing else. A precision of rank
built as is positive definite after rounding often enough. Mycelium’s mixture
product decides from isproper whether a component is a distribution or a likelihood, and the
two weightings differ by , which for a rank-deficient is rounding noise.
The switching factor had worked around this by pushing the null-space eigenvalues of its
components below zero (switching.md §6).
A third field, improper::Bool, now carries what the producer knows. With improper = true,
isproper is false and belief_mean, belief_cov, logpartition and the other accessors
refuse the belief, whatever the eigenvalues. The default is false, and then nothing changes:
properness is read off Λ, so the more than a hundred existing constructions are untouched.
Where the flag survives:
Mycelium.combinekeeps it only when the other operand has zero precision. Two improper sites can sum to a proper belief (two rank-one measurements of a plane), so a sum with anything informative goes back to being read offΛ.- Damping drops it, for the same reason.
stop_gradient(the ForwardDiff extension) keeps it.
The flag is a Bool field, not a type parameter, so the type’s parameters, the cotangents of
the ChainRules extension and the device dispatch are unchanged. isproper is marked
non-differentiable. The tests are in Mycelium’s test/overlap.jl (“A declared-improper
Gaussian is improper whatever the rounding”): the flag beats a precision that factorises after
rounding, it survives pooling only with zero precision, and the mixture product weights a
declared site as a likelihood, so two sites with one precision differ by the exact log ratio.
What the flag does not do: it cannot make a dense Float64 precision hold a condition number above . A proper prior weaker than along a site’s null direction (the case the 2026-11-01 review raised) gives such a precision when the two are pooled, and that fails with or without the flag.
Per-message costs (2026-11-12, I18)
Every Gaussian message reads the moments of its input, so the accessors here run several times per message. Three changes make them cheaper without changing a result:
_nearly_symmetrichas a loop method for a denseFloat32orFloat64matrix, without the two copies of the generic method (72 → 34 ns at dimension 2). It returns false wherever someA[i,j] - A[j,i]is NaN (a NaN entry, an infinite diagonal entry, equal infinities in mirrored places), where the generic method gets NaN frommaximum. The differences are taken in the element type, asA - A'takes them._symmetric_partis one fused broadcast,A .+ (A' .- A) ./ 2: the same operations per element as before, so the same bits, with one array instead of three (99 → 64 ns)._mean_cov(b)gives(belief_mean(b), belief_cov(b))from one factorisation of Λ instead of two;_momentsof the relation factors (Lenticulum’sgaussian.jl) uses it (0.64 → 0.34 μs). It is defined forArray-backed beliefs only; others, the device ones among them, go to the two functions. Reverse mode does not see the fused code: the ChainRules rule calls the rules of the two functions (rrule_via_ad) and adds their cotangents, so Zygote’s gradients are those of before. ForwardDiff and Enzyme go through the fused code, the same operations as the two functions.
Times from one machine, medians of Chairmarks @b (a scratch script, not in the repository).
Tested in test/i18.jl: the loop against the generic method on 6 000 generated matrices with
NaN, ±Inf, signed zeros, subnormals and near-threshold asymmetry; the broadcast bit for bit;
_mean_cov for equality, its errors, and its Zygote and ForwardDiff gradients against the
two functions. The effect on a sweep is in Mycelium’s passing.md, “After I18”.
_spd_inv (P23, 2026-11-16)
_spd_inv(M, name) is inv(_chol(M, name)), with _chol’s checks and messages, and an rrule in
LenticulumCoreChainRulesCoreExt: Zygote has no rule for inv(::Cholesky). The pullback is the
symmetric part of (), since _chol factors :
only the symmetric part of a tangent of reaches the result. Gaussian(μ, Σ) uses it, and
Lenticulum uses it wherever it inverted a Cholesky factor. The same extension marks _complement
(a basis of the complement of an extended Gaussian’s hard directions) non-differentiable.
Test: Lenticulum test/zygote_cavity.jl, against ForwardDiff with a non-symmetric cotangent.