LenticulumCore
Defines what a factor is. No graphs, no message passing, no concrete factors — just the interface every factor implements and the types it traffics in.
This is the analogue of LuxCore: a small package that other things agree on. If you are writing a new factor, this is the interface you implement.
What is in here
| group | types |
|---|---|
| the factor interface | AbstractLenticulumFactor, channels, supports_polarity, assemble |
| channels and roles | Channel, Polarity, Observed, Unobserved, Latent |
| beliefs | AbstractBelief, DiracBelief, SampleBelief, TrivialBelief, AbstractPotentialBelief, LogPotentialBelief |
| the forward half | AbstractOpenModel, forward, pushforward, logdensity |
| the backward half | BayesianLens, invert, and the inversion kinds below |
| energies | AbstractEnergySpace, GradedEnergy, scalarisations |
LogPotentialBelief(logφ) is an improper likelihood message with log-density logφ(x) up to an additive constant. It has no sampling, mean, or entropy contract. Mycelium pools it with particles by importance weighting, and with another potential by adding log-densities. Richer likelihoods can subtype AbstractPotentialBelief to expose gradients, parametric structure, or batched evaluation.
Writing a factor
A factor must provide LuxCore.initialparameters / initialstates (exactly as a Lux layer does), plus:
LenticulumCore.channels(f) # the named ports
LenticulumCore.supports_polarity(f, p) # can it be run this way?
LenticulumCore.supported_polarities(f) # ...and list the ways
LenticulumCore.assemble(f, p, ps, st) # build the lens for one polarity
LenticulumCore.energy(f, x, a, y, ps, st)assemble is the operation with no Lux counterpart. A Lux layer is a forward/backward pair; a factor only becomes one once you have chosen a direction.
The inversion kinds
assemble returns a BayesianLens pairing a model with an inversion, and the inversion kind says how the backward direction is computed:
| kind | means | used by |
|---|---|---|
ExactInversion | exact model inversion, up to numerical solve tolerance | GaussianFactor; symmetric fixed-step NeuralODEFactor on matching grids |
SolverInversion | approximate numerical inversion | DEQFactor; explicit/adaptive NeuralODEFactor |
ProximalInversion | a proximal / optimisation step | DiffusionFactor |
AmortisedInversion | a separate trained network | RatioFactor |
TrivialInversion | there is nothing to invert | LuxFactor, priors |
Nothing requires an inversion to be exact. An approximate one is legal; the free energy is what records the cost.
Parameter derivatives
using LenticulumCore, DifferentiationInterface, ADTypes, ForwardDiff
ad = AutoForwardDiff() # or load Zygote and use AutoZygote()
J = energy_jacobian(f, x, a, y, ps, st; wrt=:ps, ad)
G = gauss_newton_metric(f, x, a, y, ps, st; wrt=:ps, ad)
grad = energy_gradient(f, x, a, y, ps, st; ad)
grad_free = free_energy_gradient(f, prior, observation, ps, st; ad)J has one column per flattened parameter (depth-first fields, column-major arrays). Scalar gradients have the same numeric tree structure as ps. Omitting ad keeps central finite differences. DI is a weak dependency and the user loads the backend. parameter_gradient(loss, ps; ad) supports custom scalar objectives. State/data are held fixed; no estimator for stochastic expectations is supplied. ForwardDiff, Zygote and Enzyme are tested against closed forms; other backends depend on their support for the chosen model. AD through finite-difference Sampson localisation is not covered.
Gradient semantics and remaining gaps
All four built-in gradient couplings are accepted. Gradient evaluation requires an implemented family/domain. Mycelium supplies finite static graph replay and terminal score estimates; Lenticulum supplies collapsed Gaussian composite gradients.
isfrozen=truepins parameter updates while preserving input derivatives. Graph-local subset selection isfactor!(...; freeze).islearnable=falseis the intrinsic optimiser-candidate trait; it also preserves input derivatives.stop_gradienthas ForwardDiff/ChainRules zero-derivative rules.straight_through(forward,surrogate,x)uses the forward value and surrogate input Jacobian with those backends; differences of the forward function do not implement it.SampleBeliefhas nobelief_logdensity, so two of them cannot be pooled. SeeAdversarialFactorsfor a route around this.GaussianBelieflives here too (canonical form,gaussian_belief.jl); its message rules are inMycelium. So far onlyVariationalDiffusion's implicit solver emits one (ImplicitProx(…; message = :gaussian)); the equilibrium factors still return points.
API
LenticulumCore.LenticulumCoreLenticulumCore.AbstractBayesianLensLenticulumCore.AbstractBeliefLenticulumCore.AbstractChannelPolarityLenticulumCore.AbstractEnergySpaceLenticulumCore.AbstractGradientCouplingLenticulumCore.AbstractInversionLenticulumCore.AbstractLenticulumContainerFactorLenticulumCore.AbstractLenticulumFactorLenticulumCore.AbstractLenticulumWrapperFactorLenticulumCore.AbstractOpenModelLenticulumCore.AbstractPointScalarisationLenticulumCore.AbstractPotentialBeliefLenticulumCore.AbstractScalarisationLenticulumCore.AmortisedInversionLenticulumCore.BayesianLensLenticulumCore.CategoricalBeliefLenticulumCore.ChannelLenticulumCore.ComposedFactorLenticulumCore.ComposedLensLenticulumCore.DiagonalCouplingLenticulumCore.DiracBeliefLenticulumCore.EuclideanEnergySpaceLenticulumCore.ExactCouplingLenticulumCore.ExactInversionLenticulumCore.ExtendedGaussianLenticulumCore.FreeEnergyBoundEstimateLenticulumCore.FreeEnergyUpperBoundLenticulumCore.GaussianBeliefLenticulumCore.GaussianBeliefLenticulumCore.GradedEnergyLenticulumCore.GradedEnergySpaceLenticulumCore.GradedScalarisationLenticulumCore.IdentityScalarisationLenticulumCore.LatentLenticulumCore.LogPotentialBeliefLenticulumCore.MixtureBeliefLenticulumCore.ObservedLenticulumCore.OpenModelResultLenticulumCore.PathwiseCouplingLenticulumCore.PolarityLenticulumCore.ProximalInversionLenticulumCore.SampleBeliefLenticulumCore.SampsonNormLenticulumCore.ScalarEnergySpaceLenticulumCore.ScoreFunctionCouplingLenticulumCore.SolverInversionLenticulumCore.SquaredNormLenticulumCore.TensorFactorLenticulumCore.TensorLensLenticulumCore.TrivialBeliefLenticulumCore.TrivialInversionLenticulumCore.UnobservedLenticulumCore.WeightedSumLenticulumCore.GaussianLenticulumCore.as_beliefLenticulumCore.as_distributionLenticulumCore.assembleLenticulumCore.assembleLenticulumCore.belief_covLenticulumCore.belief_meanLenticulumCore.belief_meanLenticulumCore.belief_meanLenticulumCore.bernoulliLenticulumCore.chain_rule_defectLenticulumCore.channel_precisionLenticulumCore.channelsLenticulumCore.composeLenticulumCore.compose_energyLenticulumCore.compose_entropyLenticulumCore.compose_free_energyLenticulumCore.default_precisionLenticulumCore.energyLenticulumCore.energyLenticulumCore.energyLenticulumCore.energyLenticulumCore.energy_gradientLenticulumCore.energy_jacobianLenticulumCore.energyspaceLenticulumCore.energyspaceLenticulumCore.energyspaceLenticulumCore.entropyLenticulumCore.estimate_free_energy_upper_boundLenticulumCore.extendedLenticulumCore.flat_directionsLenticulumCore.forwardLenticulumCore.free_energyLenticulumCore.free_energyLenticulumCore.free_energyLenticulumCore.free_energyLenticulumCore.free_energy_gradientLenticulumCore.free_energy_gradientLenticulumCore.free_energy_upper_boundLenticulumCore.gauss_newton_metricLenticulumCore.hard_directionsLenticulumCore.hasnopolarityLenticulumCore.invertLenticulumCore.invertLenticulumCore.invertLenticulumCore.invert_is_extrinsicLenticulumCore.isexactLenticulumCore.isfrozenLenticulumCore.islearnableLenticulumCore.islinearLenticulumCore.ispartitionLenticulumCore.isproperLenticulumCore.ispureLenticulumCore.isunidirectionalLenticulumCore.isuninformativeLenticulumCore.jensen_gapLenticulumCore.latentspaceLenticulumCore.likelihood_messageLenticulumCore.likelihood_messageLenticulumCore.likelihood_messageLenticulumCore.likelihood_messageLenticulumCore.likelihood_messageLenticulumCore.localiseLenticulumCore.logdensityLenticulumCore.logpartitionLenticulumCore.marginaliseLenticulumCore.mixture_weightsLenticulumCore.oplusLenticulumCore.parameter_gradientLenticulumCore.pointwise_energyspaceLenticulumCore.pointwise_energyspaceLenticulumCore.pointwise_energyspaceLenticulumCore.pointwise_scalarisationLenticulumCore.polaritiesLenticulumCore.preimageLenticulumCore.probabilitiesLenticulumCore.pushforwardLenticulumCore.pushforwardLenticulumCore.pushforwardLenticulumCore.pushforwardLenticulumCore.sampson_dampingLenticulumCore.scalar_energyLenticulumCore.scalar_free_energyLenticulumCore.scalar_free_energyLenticulumCore.scalarisationLenticulumCore.scalariseLenticulumCore.selectLenticulumCore.setupLenticulumCore.stop_gradientLenticulumCore.straight_throughLenticulumCore.support_entropyLenticulumCore.supported_polaritiesLenticulumCore.supported_polaritiesLenticulumCore.supported_polaritiesLenticulumCore.supported_polaritiesLenticulumCore.supported_polaritiesLenticulumCore.supported_polaritiesLenticulumCore.supports_polarityLenticulumCore.supports_polarityLenticulumCore.supports_polarityLenticulumCore.uninformative
Abstract types
LenticulumCore.AbstractBayesianLens — Type
abstract type AbstractBayesianLensA Bayesian lens $(c, c')$ (AutoBayes, Definition 9): an AbstractOpenModel paired with an AbstractInversion.
LenticulumCore.AbstractBelief — Type
abstract type AbstractBeliefAn element of $\mathcal{P}X$: a distribution over a channel's space.
Beliefs are what flows forward along the graph (as priors, propagated by pushforward) and what inversions consume. Concrete subtypes decide the representation: a Dirac, a particle set, an exponential-family natural parameter, a Gaussian.
LenticulumCore.AbstractEnergySpace — Type
abstract type AbstractEnergySpaceThe space $E_c$ in which a factor's vector energy and entropy take values.
AutoBayes fixes $E_c = [0,\infty]$ and composes energies by addition. Lenticulum keeps the vector and composes by direct sum, recovering the paper's law under scalarisation. See Scalar and Multivariate Energy.md.
LenticulumCore.AbstractGradientCoupling — Type
abstract type AbstractGradientCouplingPer-boundary Jacobian semantics (AutoBayes Definition 29). Mycelium's parameter driver uses fixed messages for diagonal edges and explicitly replays a finite static schedule for non-diagonal edges. A path survives only when every crossed boundary permits it. Detach overrides all couplings. See gradient_couplings.md for supported domains.
LenticulumCore.AbstractInversion — Type
abstract type AbstractInversionThe backward half $c'$ of a Bayesian lens (AutoBayes, Definition 9): a rule that turns a prior $\pi \in \mathcal{P}X$ and an observation $y \in Y$ into a belief over $X \times \llbracket c \rrbracket$.
Nothing requires an inversion to be exact. Exact, amortised, mean-field, solver-based and diffusion-based inversions are all legal inhabitants; the quality of the choice is what the free energy measures.
LenticulumCore.AbstractLenticulumContainerFactor — Type
abstract type AbstractLenticulumContainerFactor{factors} <: AbstractLenticulumFactorA factor built from several sub-factors named by the tuple of symbols factors. Mirrors LuxCore.AbstractLuxContainerLayer: parameters and states are assembled into a NamedTuple keyed by factors.
This is the [Para] composition of Definition 28: parameter spaces multiply, and the product is realised as a NamedTuple.
LenticulumCore.AbstractLenticulumFactor — Type
abstract type AbstractLenticulumFactorA factor: a parameterized statistical game (AutoBayes, Definition 27).
This is Lenticulum's analogue of LuxCore.AbstractLuxLayer, and the parameter/state interface is inherited verbatim: implementors must provide
initialparameters(rng, factor)initialstates(rng, factor)
and additionally, for the statistical-game structure,
channels– the named ports of the factorsupports_polarity– which input/output splits it can answerassemble– produce a Bayesian lens for a chosen polarityenergyspace– the energy space $E_c$energy– the vector energy $\mathbf{l}^c$entropy– the vector entropy $\mathbf{H}^c$
Optionally scalarisation, parameterlength, statelength, display_name.
Unlike a Lux layer, a factor has no fixed direction: assemble(factor, polarity) constructs the parametric lens on demand. See Channels and Polarity.md.
LenticulumCore.AbstractLenticulumWrapperFactor — Type
abstract type AbstractLenticulumWrapperFactor{factor} <: AbstractLenticulumFactorA factor wrapping exactly one sub-factor, whose parameters and states are not nested under an extra name. Mirrors LuxCore.AbstractLuxWrapperLayer.
LenticulumCore.AbstractOpenModel — Type
abstract type AbstractOpenModelAn open model $c : X \nrightarrow Y$ (AutoBayes, Definition 1): a measure kernel $X \rightsquigarrow \llbracket c \rrbracket \times Y$ together with its latent space $\llbracket c \rrbracket$.
The latent space is the reason composition of open models needs no integration; see open_model.md.
LenticulumCore.AbstractScalarisation — Type
abstract type AbstractScalarisationA map $\sigma : E \to \mathbb{R}$ collapsing a vector energy to a loss.
Must satisfy σ(0) == 0, non-negativity on the cone, and monotonicity. The critical trait is islinear: a linear σ commutes with the expectation in the chain rule and so composition is strict; a merely convex σ makes it lax, with the gap given by a Jensen term (a variance, for a squared norm).
LenticulumCore.DiagonalCoupling — Type
DiagonalCoupling()Default: hold messages fixed in graph parameter gradients. This preserves the legacy fixed-cavity derivative, including any local posterior recomputation within a factor; it is not generally a fixed-posterior EM derivative.
LenticulumCore.ExactCoupling — Type
ExactCoupling()Keep the full derivative of the specified finite inference schedule, including message pooling, inversions and the Bethe entropy correction. Exact refers to the iteration derivative; it does not promise exact inference on a loopy graph.
LenticulumCore.PathwiseCoupling — Type
PathwiseCoupling()Keep derivatives of Gaussian/Dirac message representations. Gaussian expectations are propagated analytically through moments (Rao–Blackwellised reparametrisation); no numerical sampler is differentiated. Other informative families throw.
LenticulumCore.ScoreFunctionCoupling — Type
ScoreFunctionCoupling(; baseline = nothing)REINFORCE for sampled terminal Gaussian/categorical messages, with an optional fixed scalar baseline. Graph gradients require explicit rng and nsamples >= 2 and return a GradientEstimate with a componentwise standard error. Bidirectional sampling throws.
Channels and polarity
LenticulumCore.AbstractChannelPolarity — Type
abstract type AbstractChannelPolarityThe role a channel plays in one particular use of a factor.
LenticulumCore.Channel — Type
Channel{name,S}(space)A named port of a factor, carrying the space its values live in.
LenticulumCore.Latent — Type
Latent()AutoBayes' latent space $\llbracket c \rrbracket$; the note's $P_{latent}$. Internal scratch that composition hid. Either revealed (a free retyping) or marginalised (expensive).
LenticulumCore.Observed — Type
Observed()AutoBayes' observed space $Y$; the note's $P_{in}$. Clamped to data. The posterior does not range over this channel.
LenticulumCore.Polarity — Type
Polarity(assignment::NamedTuple, [precisions::NamedTuple])
Polarity(; channel = polarity, ...)An assignment of an AbstractChannelPolarity to each channel of a factor, together with a precision $\rho$ per channel.
Because the assignment is a NamedTuple, the channel names live in the type, so an assembled lens specialises at compile time rather than dispatching through a Dict.
Polarity(; x = Observed(), y = Unobserved())LenticulumCore.Unobserved — Type
Unobserved()AutoBayes' unobserved space $X$; the note's $P_{out}$. This is what inference solves for, and what the inversion produces a belief over.
LenticulumCore.assemble — Function
assemble(factor, p::Polarity, ps, st) -> (lens, st)Produce the AbstractBayesianLens realising factor under polarity p.
This is the operation that has no counterpart in Lux, and the reason a factor cannot be a Lux layer: a Lux layer is a lens, whereas a factor only becomes one once a direction is chosen. See Lux as a Parametric Lens.md.
LenticulumCore.channel_precision — Method
channel_precision(p::Polarity, name::Symbol)The weight $\rho$ attached to channel name.
LenticulumCore.channels — Function
channels(factor) -> Tuple{Vararg{Channel}}The named ports of factor. Must be implemented by every factor.
LenticulumCore.default_precision — Method
default_precision(p::AbstractChannelPolarity)The default weight $\rho$ for a channel of this polarity, as in $P = \rho_{in}P_{in} + \rho_{out}P_{out} + \rho_{latent}P_{latent}$.
Observed defaults to Inf (a hard clamp — a categorical cup); the others to 1. A finite Observed precision is a soft clamp, which is what makes noisy observations and annealed conditioning expressible.
LenticulumCore.hasnopolarity — Method
hasnopolarity(factor) -> BoolWhether the factor supports no polarity. Defaults to isempty(supported_polarities(factor)); override this and isunidirectional when a factor cannot cheaply list its support. Graph sinks use this predicate.
LenticulumCore.ispartition — Method
ispartition(p::Polarity)Check $P_{in} + P_{out} + P_{latent} = \mathrm{Id}$ with pairwise-zero products: every channel gets exactly one polarity. Guaranteed by the NamedTuple representation, so this is a total function returning true; kept as a named predicate because the corresponding check on a graph-wide polarity is not trivial.
LenticulumCore.isunidirectional — Method
isunidirectional(factor) -> BoolWhether factor admits exactly one polarity, i.e. has a fixed input/output split and is therefore an explicit rather than an implicit factor.
Non-learnable structural nodes are usually unidirectional: a data source only emits, a loss only absorbs. Capped factors must override this predicate to avoid enumeration.
LenticulumCore.polarities — Method
polarities(factor)Lazily iterate the default-precision Observed/Unobserved/Latent assignments over channels(factor) accepted by supports_polarity. Roles cycle in that order, with the first channel changing fastest. No set of size 3^n is materialised, although complete traversal still takes exponential work. Assignments have heterogeneous types, so this iterator is intentionally type-unstable. Its order need not match supported_polarities.
LenticulumCore.select — Method
select(p::Polarity, ::Type{P}) where {P<:AbstractChannelPolarity}The tuple of channel names carrying polarity P. This is the diagonal selection matrix $P_{in}$ / $P_{out}$ / $P_{latent}$ as a tuple of Symbols.
LenticulumCore.supported_polarities — Method
supported_polarities(factor) -> Tuple{Vararg{Polarity}}The polarities factor can answer, enumerated.
The list must equal the set of default-precision Observed/Unobserved/Latent assignments accepted by supports_polarity when finite and listable. The default is empty, matching supports_polarity's conservative false.
A factor may throw an informative ArgumentError above a size cap instead of returning an incomplete list. It must then override hasnopolarity and isunidirectional to answer without enumeration. Callers can use those structural predicates, supports_polarity for a particular assignment, or polarities for lazy traversal. The default structural predicates retain their list-based behaviour.
A factor with exactly one supported polarity is unidirectional: it can be evaluated in one direction only, like an ordinary Lux layer. See isunidirectional.
LenticulumCore.supports_polarity — Method
supports_polarity(factor, p::Polarity) -> BoolWhether factor can answer the input/output split p. This predicate is the authority; answering it must not require enumerating the supported assignments.
This predicate replaces Lux's acyclicity restriction. A Lux Chain is well-formed iff the wiring is a DAG; a Lenticulum graph is well-formed iff every scheduled message uses a polarity its factor supports. The default is conservative: a factor supports nothing until it says otherwise.
Beliefs and open models
LenticulumCore.AbstractPotentialBelief — Type
AbstractPotentialBelief <: AbstractBeliefAn improper, unnormalised likelihood potential known through its log-density. Mycelium.belief_logdensity(b, x) is defined up to an additive constant independent of x. Intended for factor → variable messages: a potential has no sampling, mean, or entropy contract. Richer likelihood types may subtype this abstraction to provide gradients, parametric structure, or batched evaluation.
LenticulumCore.CategoricalBelief — Type
CategoricalBelief(p; labels = nothing)A distribution over a finite set $\{1, \dots, K\}$ (or over labels), stored as normalised log-probabilities logp. p is any non-negative vector with a positive sum; it is normalised. Pooling two categorical beliefs is elementwise multiplication and renormalisation: exact, associative, commutative, and always defined unless the two have disjoint support (Belief Algebra.md, Categorical Belief.md). bernoulli is the two-state case.
LenticulumCore.DiracBelief — Type
DiracBelief(value)A point mass. What a clamped (observed) channel carries, and what a categorical cup produces (see Copiers Cups and Caps.md). Inversions against a Dirac prior typically trivialise — which is exactly AutoBayes' Example 4: supervised learning is the case where the cup has collapsed the posterior.
LenticulumCore.LogPotentialBelief — Type
LogPotentialBelief(logφ)A likelihood potential wrapping a callable logφ(x). Evaluation through Mycelium.belief_logdensity returns this log-potential, up to an additive constant. This is an improper message, with no sampling, mean, or entropy; see AbstractPotentialBelief for the extension contract.
LenticulumCore.MixtureBelief — Type
MixtureBelief(components, weights = equal)A weighted mixture $\sum_k w_k\, b_k$ of beliefs of one type, with normalised log-weights logw. It is how a belief holds several answers at once, e.g. both branches of a relation. Its density is the weighted sum of the components' densities; its product with another belief is again a mixture, with weights scaled by each component's overlap with that belief (Mixture Belief.md).
LenticulumCore.OpenModelResult — Type
OpenModelResult(observed, latent)The result of running an open model forward: a value in $Y$ together with the value in $\llbracket c \rrbracket$ that composition would otherwise have destroyed.
This is the exact analogue of an autodiff tape entry. Reverse-mode AD caches activations so the backward pass can use them; an open model caches $\llbracket c \rrbracket$ so the inversion can. Same trade of memory for tractability, under the same chain rule.
LenticulumCore.SampleBelief — Type
SampleBelief(samples, [weights])A particle representation of $\pi \in \mathcal{P}X$. The default fallback whenever no conjugate structure is available, which is most of the time.
LenticulumCore.TrivialBelief — Type
TrivialBelief()The unique belief on the one-point space $1$. The prior argument of a factor with $X \cong 1$ — a prior distribution, in AutoBayes' terminology, has this as its input.
LenticulumCore.bernoulli — Method
bernoulli(p; labels = [false, true]) -> CategoricalBeliefA two-state belief: probability p of the second state (true).
LenticulumCore.forward — Function
forward(model, x, ps, st) -> (OpenModelResult, st)Sample the kernel $c : X \rightsquigarrow \llbracket c \rrbracket \times Y$ at x.
LenticulumCore.isexact — Method
isexact(x) -> BoolWhether a pushforward or an inversion is exact rather than approximate. Defaults to false because approximation is the normal case and silence should not imply exactness.
LenticulumCore.ispure — Method
ispure(model) -> Bool$\llbracket c \rrbracket \cong 1$: an ordinary kernel $X \rightsquigarrow Y$.
Purity is not preserved by composition — that is precisely why the latent space exists. Composing two pure models yields a model whose latent space is the intermediate space.
LenticulumCore.latentspace — Function
latentspace(model)
observedspace(model)
unobservedspace(model)The three spaces $\llbracket c \rrbracket$, $Y$, $X$ of AutoBayes Definition 1.
LenticulumCore.logdensity — Function
logdensity(model, x, a, y, ps, st) -> (Real, st)$\log p_c(a, y \mid x)$. Needed whenever the energy is a negative log-likelihood, which is the default choice in every example of AutoBayes' Appendix A.
LenticulumCore.mixture_weights — Method
mixture_weights(m::MixtureBelief) -> VectorThe mixture weights, $\exp$ of the stored log-weights.
LenticulumCore.probabilities — Method
probabilities(b::CategoricalBelief) -> VectorThe probabilities of the states, $\exp$ of the stored log-probabilities.
LenticulumCore.pushforward — Function
pushforward(model, π::AbstractBelief, ps, st) -> (AbstractBelief, st)The pushforward prior $c_*\pi$.
Computing $c_*\pi$ is marginalisation, and is about as costly as exact inversion. AutoBayes' closing discussion names belief propagation and variational message passing as the remedy, and states that they fit in the framework — that is Mycelium.jl's job. A LenticulumCore implementation may legitimately return an approximate belief, but it should say so via isexact.
LenticulumCore.GaussianBelief — Type
GaussianBelief(η, Λ)
GaussianBelief(η, Λ, improper::Bool)A Gaussian in canonical form: information vector η = Λμ and precision Λ = Σ⁻¹.
Λ may be singular or zero, and that is not an error: a factor → variable message is a likelihood, not a distribution, and a likelihood that constrains only some directions has a rank-deficient precision. The moment form cannot represent this at all, which is the second reason for the canonical parametrisation.
Whether a belief is proper is otherwise read off Λ (isproper), and a precision of rank r < n is positive definite after rounding often enough. A factor that knows its message is rank-deficient says so with improper = true: isproper is then false whatever the rounding, and the mixture product weights it as a likelihood. The flag survives pooling only with an uninformative belief, since a sum of improper sites can be proper.
Use Gaussian to build one from a mean and covariance.
LenticulumCore.Gaussian — Method
Gaussian(μ, Σ)
Gaussian(μ::Real, σ²::Real)A Gaussian from its moments. Converts to canonical form immediately; Σ must be positive definite.
LenticulumCore.belief_cov — Method
belief_cov(b) -> Matrix$\Sigma = \Lambda^{-1}$.
LenticulumCore.belief_mean — Method
belief_mean(b) -> Vector$\mu = \Lambda^{-1}\eta$. Errors on an improper belief, rather than returning a least-squares pseudo-mean that would be silently wrong.
Named belief_mean rather than extending Statistics.mean to avoid a fifth name collision in this project — see Mycelium.md §2.
LenticulumCore.belief_mean — Method
belief_mean(b::SampleBelief) -> Vector
belief_cov(b::SampleBelief) -> MatrixSelf-normalised importance-sampling estimates of the first two moments: the weighted mean and the weighted covariance $\sum_i w_i (x_i - \bar x)(x_i - \bar x)^\top$ with the weights normalised to sum to one (uniform without weights). Samples are the columns of a matrix or the elements of a vector (vectors or scalars). Defined here, beside the type (P18; they were in AdversarialFactors, ratio.jl, and existed only with it loaded). See open_model.md, "Sample moments".
LenticulumCore.isproper — Method
isproper(b::GaussianBelief) -> BoolWhether the belief is a normalisable distribution rather than a likelihood: false if it was built with improper = true, otherwise whether Λ is positive definite. belief_mean, belief_cov, variable_entropy and belief_logdensity all require this.
LenticulumCore.isuninformative — Method
isuninformative(b::AbstractBelief) -> BoolWhether b carries zero precision / information (TrivialBelief() or a GaussianBelief with Λ == 0 and η == 0). Used by polarity resolution to distinguish informative incoming messages from identity placeholders.
LenticulumCore.logpartition — Method
logpartition(b) -> Real$\log \int \exp(-\tfrac12 x^\top\Lambda x + \eta^\top x)\,dx = \tfrac12\eta^\top\Lambda^{-1}\eta - \tfrac12\log\det\Lambda + \tfrac n2\log 2\pi$.
LenticulumCore.uninformative — Method
uninformative(n)
uninformative(::Type{T}, n)The improper uniform belief on $\mathbb{R}^n$: η = 0, Λ = 0. The identity for Mycelium.combine, and the canonical-form spelling of LenticulumCore.TrivialBelief.
LenticulumCore.ExtendedGaussian — Type
ExtendedGaussian(μ, Σ, D)
ExtendedGaussian(μ, Σ)A Gaussian N(μ, Σ) made flat along the subspace spanned by the columns of D: the law of μ + ε + d with ε ~ N(0, Σ) and d uniform on span(D) (an extended Gaussian distribution, Stein & Samuelson). Σ may be singular: a direction in null(Σ) and not in span(D) is hard, a linear constraint the belief imposes exactly.
The constructor normalises the representative: the columns of D are made orthonormal (and dependent ones dropped), and μ and Σ are projected onto the complement of span(D). Σ must be positive semidefinite up to rounding.
Operations: pushforward(b, M, c, Q) (exact through affine maps, with optional added noise), preimage(b, M, c) (the likelihood of x when b is about Mx + c), marginalise, conversion from and to GaussianBelief (extended, GaussianBelief(b)), flat_directions, hard_directions, support_entropy. The product of two beliefs is Mycelium's combine. A belief with flat directions has no density and no entropy; one with hard directions has them only relative to the measure on its support (support_entropy).
LenticulumCore.GaussianBelief — Method
GaussianBelief(b::ExtendedGaussian)The canonical form of an extended Gaussian without hard directions: Λ = Σ⁺, η = Λμ, declared improper when b is flat anywhere. A hard direction has infinite precision and no canonical form: that is an ArgumentError (keep the ExtendedGaussian, or use a DiracBelief when every direction is hard).
LenticulumCore.belief_mean — Method
belief_mean(b::ExtendedGaussian)
belief_cov(b::ExtendedGaussian)The mean μ and covariance Σ of an extended Gaussian that is flat nowhere (hard directions allowed: they have variance zero). Along a flat direction there is no mean, which is an ArgumentError.
LenticulumCore.extended — Method
extended(b) -> ExtendedGaussian
extended(b, n)The belief as an ExtendedGaussian:
- a proper
GaussianBelief: its mean and covariance, nowhere flat; - an improper or singular one: flat along
null(Λ), with meanΛ⁺ηand covarianceΛ⁺onrange(Λ)(ηmust lie inrange(Λ): a component alongnull(Λ)isexp(ηᵀx)there, which no limit of Gaussians gives;Λmust be positive semidefinite to rounding); - a
DiracBelief: its value, every direction hard; TrivialBelief()(needs the dimensionn): flat everywhere.
η and Λ may be first-order ForwardDiff duals (or tracked by Zygote): the split is taken at the value and moves at constant rank (extended_gaussian.md, "Derivatives").
LenticulumCore.flat_directions — Method
flat_directions(b::ExtendedGaussian) -> MatrixAn orthonormal basis of the subspace along which b is flat (says nothing).
LenticulumCore.hard_directions — Method
hard_directions(b::ExtendedGaussian) -> MatrixAn orthonormal basis of the directions in which b has no variance and is not flat: the linear constraints Hᵀx = Hᵀμ it imposes exactly.
LenticulumCore.marginalise — Method
marginalise(b::ExtendedGaussian, idx) -> ExtendedGaussianThe law of the coordinates idx (a vector of indices or a range): the pushforward through the selection matrix.
LenticulumCore.preimage — Function
preimage(b::ExtendedGaussian, M, c = 0) -> ExtendedGaussianThe likelihood $x \mapsto b(Mx + c)$ as an extended Gaussian on the domain of M: what b says about x when it is a belief about y = Mx + c. Hard where Mx + c would have to lie in a hard direction of b, Gaussian with precision $M^\top \Sigma^+ M$ where b is Gaussian, flat along null(M) and along the preimage of b's flat directions. Exact for any M. With pushforward it gives the message of a linear relation: preimage(pushforward(...), Aₜ).
LenticulumCore.pushforward — Function
pushforward(b::ExtendedGaussian, M, c = 0, Q = 0) -> ExtendedGaussianThe law of M x + c + ε for x ~ b and independent ε ~ N(0, Q): mean Mμ + c, covariance MΣMᵀ + Q, flat along M·span(D). Exact for any matrix M, also rank-deficient or rectangular.
LenticulumCore.support_entropy — Method
support_entropy(b::ExtendedGaussian) -> RealThe differential entropy of b relative to Lebesgue measure on its support, the affine subspace $\mu + \operatorname{range}(\Sigma)$, in orthonormal coordinates:
\[H \;=\; \tfrac r2\,(1 + \log 2\pi) \;+\; \tfrac12 \textstyle\sum_{i=1}^{r} \log \lambda_i ,\]
over the $r$ nonzero eigenvalues of $\Sigma$. Without hard directions it is the Gaussian entropy; a point (every direction hard) has entropy 0, the convention for clamped variables. It is the finite part of the entropy of $\mathcal N(\mu, \Sigma + \varepsilon I)$ as $\varepsilon \to 0$, after removing $\tfrac{n-r}{2}\log(2\pi e\varepsilon)$; the Bethe free energy of a graph with hard constraints is finite with it (extended_gaussian.md, "Entropy"). A belief with flat directions has no entropy (it is improper): ArgumentError.
LenticulumCore.as_belief — Function
as_belief(d) -> AbstractBeliefThe inverse of as_distribution, available when ExponentialFamily.jl is loaded. Any univariate or multivariate normal (from Distributions.jl or ExponentialFamily.jl, in any parametrisation) becomes a GaussianBelief through its weighted mean and precision; a PointMass a DiracBelief; Categorical and Bernoulli a CategoricalBelief; a MixtureModel a MixtureBelief; a SampleList a SampleBelief.
LenticulumCore.as_distribution — Function
as_distribution(b::AbstractBelief; univariate = false)The belief as a distribution of the Julia ecosystem, available when ExponentialFamily.jl is loaded (RxInfer.jl loads it):
| belief | distribution |
|---|---|
GaussianBelief(η, Λ) | MvNormalWeightedMeanPrecision(η, Λ), exactly; with univariate = true and dimension 1, NormalWeightedMeanPrecision |
DiracBelief | PointMass |
CategoricalBelief | Categorical (labels are dropped) |
MixtureBelief | MixtureModel of the converted components |
SampleBelief | SampleList |
An improper Gaussian (a likelihood message) converts too: RxInfer's canonical form allows a singular precision. TrivialBelief has no counterpart and throws.
Bayesian lenses
LenticulumCore.AmortisedInversion — Type
AmortisedInversion(net)$c'$ realised by a learned network — a VAE encoder. net is an AbstractLuxLayer, so the inversion carries its own parameters, independent of the forward kernel's.
That a factor has two independently parametrised halves is the structural reason a factor cannot be a Lux layer.
LenticulumCore.BayesianLens — Type
BayesianLens(model, inversion)The pair $(c, c')$ of AutoBayes Definition 9.
Nothing constrains inversion to be exact. Exact posteriors, amortised encoders, mean-field families, root-finding solvers and diffusion denoisers are all legal, and their relative quality is precisely what the free energy measures.
LenticulumCore.ComposedLens — Type
ComposedLens(first, second)$(d, d') \diamond (c, c')$ of AutoBayes Definition 12. Priors propagate forward by pushforward; corrections propagate backward by sampling the inversions in reverse order.
LenticulumCore.ExactInversion — Type
ExactInversion()$c^\dagger$ of AutoBayes Definition 10: Bayes' law applied to the model's own kernel. Available only when the model declares a tractable posterior.
LenticulumCore.ProximalInversion — Type
ProximalInversion(prox)$c'$ realised by a proximal operator on an energy — the diffusion family (RED-Diff, ProxDM). See lib/VariationalDiffusion.jl and ImplicitREDDiff.md.
LenticulumCore.SolverInversion — Type
SolverInversion(solver)$c'$ realised by root-finding on a residual: Implicit Learners.md's algebraic and equilibrium families. The backward pass is the implicit function theorem, which needs the Jacobian of the vector energy — see Scalar and Multivariate Energy.md §6.
A solver that stopped early is simply an inexact inversion, and the loss records the cost. That is a far better failure mode than a divergent unroll.
LenticulumCore.TensorLens — Type
TensorLens(parts::Tuple)$(c,c') \otimes (d,d')$ of AutoBayes Definition 15.
Parallel composition of inversions is lax: each branch only ever sees the marginal of a joint prior, so correlations between branches are discarded and the composite inversion is mean-field. The discrepancy is the mutual information between the branches (Remark 26). Track and report it; do not pretend it is zero.
LenticulumCore.TrivialInversion — Type
TrivialInversion()The inversion of a prior $\pi : 1 \nrightarrow X$: there is nothing to infer.
AutoBayes Remark 24 uses exactly this to turn a prior into a statistical game with $l^\pi = -\log p_\pi$ and $H^\pi \equiv 0$, whose composite with c has the true variational free energy as its loss. The prior is a factor, not part of the model.
LenticulumCore.compose — Method
compose(c::AbstractBayesianLens, d::AbstractBayesianLens)Build $d \diamond c$. Argument order follows the wiring (c then d), not the mathematical notation $d \diamond c$.
LenticulumCore.invert — Function
invert(lens, π::AbstractBelief, y, ps, st) -> (AbstractBelief, st)Apply $c'_\pi$ to the observation y, returning a belief over $X \times \llbracket c \rrbracket$.
Note the return type reconstructs the latent space too, not just $X$. That is what the next factor upstream will consume, and dropping it breaks the chain rule.
LenticulumCore.invert_is_extrinsic — Method
invert_is_extrinsic(lens) -> BoolWhether invert(lens, π, ...) already returns an extrinsic message, one that does not multiply in the cavity π (it may use π as a warm start), so that likelihood_message can default to it. false by default; true for lenses with TrivialInversion.
LenticulumCore.likelihood_message — Method
likelihood_message(lens, π::AbstractBelief, inputs, ps, st) -> (AbstractBelief, st)The extrinsic likelihood (or site) message from lens given clamped/incoming inputs and cavity reference π on the unobserved channel(s).
Unlike invert, which returns the posterior $q(x) \propto \pi(x)\,\ell(y\mid x)$, likelihood_message returns the likelihood factor $\ell(y\mid x)$ with π divided out (π is used only as a linearisation, tilting, or warm-start point when the inversion is approximate). Belief propagation (Mycelium.factor_message) calls likelihood_message so that a variable of degree $d$ does not count its prior $d$ times.
Defaults to invert(lens, π, inputs, ps, st) only for lenses that declare invert_is_extrinsic (point and particle lenses, whose invert does not multiply in π); any other lens must implement likelihood_message, and the default throws. Until 2026-11-04 the default applied to every lens, so a lens whose invert returns a posterior silently counted π twice in belief propagation (L-ADR-0006; review of 2026-11-04).
Energies and scalarisations
AbstractPointScalarisation represents a field of metrics. scalar_energy calls localise(σ, f, x, a, y, ps, st) before applying it. SampsonNorm localises to a SquaredNorm from the pushed-forward noise covariance; factors with a residual gauge supply sampson_damping transforming as B*R*B'. Singular covariance throws instead of using a pseudoinverse. Graded scalarisations containing point-dependent parts must have their parts localised in their own contexts first. jensen_gap requires a fixed scalarisation; the factor form of gauss_newton_metric freezes the localised metric.
LenticulumCore.AbstractPointScalarisation — Type
AbstractPointScalarisation <: AbstractScalarisationA field of scalarisations depending on a factor, its point and parameters. Call localise before scalarise. It is not a fixed map on energy space, so jensen_gap does not apply.
LenticulumCore.EuclideanEnergySpace — Type
EuclideanEnergySpace(n)$E = \mathbb{R}^n$. The space a vector residual $r_\theta$ lands in; n must match the number of unobserved coordinates for the implicit function theorem to apply.
LenticulumCore.GradedEnergy — Type
GradedEnergy(parts::NamedTuple)An element of a GradedEnergySpace: the per-factor breakdown of a composite energy, before it is collapsed to a number.
Per-factor loss attribution is not a logging feature bolted on afterwards — it is what the composite energy is.
LenticulumCore.GradedEnergySpace — Type
GradedEnergySpace(parts::NamedTuple)The direct sum $E_G = \bigoplus_{f} E_f$ graded by factor name.
This is what a composite factor's energy space is: composition does not add, it files the summands away under their factors' names — exactly as an open model files intermediate values into $\llbracket c \rrbracket$ rather than integrating them out.
LenticulumCore.GradedScalarisation — Type
GradedScalarisation(parts::NamedTuple)$\sigma_{dc}(e_c, e_d) = \sigma_c(e_c) + \sigma_d(e_d)$, the scalarisation induced on a direct sum. Additivity across summands is automatic; linearity is inherited from the parts.
LenticulumCore.IdentityScalarisation — Type
IdentityScalarisation()$\sigma = \mathrm{id}$ on ScalarEnergySpace. Linear. Recovers AutoBayes' Definition 22 verbatim.
LenticulumCore.SampsonNorm — Type
SampsonNorm(; wrt = :y, Σ = nothing, λ = 0)Point-dependent residual scalarisation: localises to SquaredNorm(inv(A)), where A = J * Σ * J' + λ * sampson_damping(f, x, a, y, ps, st) and J = energy_jacobian(f, x, a, y, ps, st; wrt). Σ = nothing means identity noise covariance in data space; otherwise it must be a finite real square PSD matrix. wrt is :x or :y; damping λ must be finite and nonnegative.
Localisation throws ArgumentError unless A is numerically positive definite: λmin(A) > size(A,1) * eps(T) * λmax(A), where T is its floating element type. There is no pseudoinverse. Identity damping is the default; gauge-invariant damping requires a factor override of sampson_damping. The reference finite-difference Jacobian is not differentiable; factors may supply an exact Jacobian method.
LenticulumCore.ScalarEnergySpace — Type
ScalarEnergySpace()$E = \mathbb{R}$, $K = [0,\infty)$. Recovers AutoBayes exactly when paired with IdentityScalarisation.
LenticulumCore.SquaredNorm — Type
SquaredNorm(M = nothing; check::Bool = true)$\sigma(e) = \tfrac12 \|e\|_M^2$. Convex, not linear.
The natural scalarisation for a residual factor: $r_\theta(x) \approx 0$ iff $\sigma(r_\theta(x)) \approx 0$. Its differential is $\mathrm{d}\sigma(e) = Me$, so the scalar gradient is $J^\top M r$ and the Gauss–Newton metric is $J^\top M J$ — both of which need the Jacobian of the vector energy, which is exactly what is lost by scalarising early.
When check = true (the default), validates that M is square, symmetric, and positive semidefinite. Pass check = false when M is derived from an already-validated SPD matrix (e.g. inv(Symmetric(f.Q))) or inside an AD trace.
LenticulumCore.WeightedSum — Type
WeightedSum(λ)$\sigma_\lambda(e) = \langle \lambda, e\rangle$ for λ in the dual cone. Linear.
This is where β-VAE weights, KL annealing schedules, curriculum weights and the precisions $\rho$ of ImplicitREDDiff.md live. Because the energy is kept as a vector, λ can be changed after the graph is composed — with a scalar energy each λ is a different graph.
LenticulumCore.energy — Function
energy(factor, x, a, y, ps, st) -> (𝐥, st)The vector energy $\mathbf{l}^c(x, a, y)$, an element of pointwise_energyspace(factor).
Pointwise: it is evaluated at samples drawn from the inversion, so it needs no normalisation. That is why energies compose by direct sum with no expectation involved.
It need not be non-negative: a residual is signed, and "small" means close to 0 under the factor's scalarisation, not low in a cone (Scalar and Multivariate Energy.md §8).
LenticulumCore.energy_gradient — Method
energy_gradient(f, x, a, y, ps, st; ad = nothing, ε = 1e-6, kwargs...)Parameter gradient of first(scalar_energy(...; kwargs...)), shaped like ps. Scalarisation and polarity keywords have exactly scalar_energy's meaning, including any parameter dependence of a point scalarisation. The incoming state is held fixed.
LenticulumCore.energy_jacobian — Method
energy_jacobian(factor, x, a, y, ps, st; wrt = :x, ε = 1e-6, ad = nothing) -> MatrixThe Jacobian $J = \partial \mathbf{l}^c / \partial u$ of the pointwise vector energy energy with respect to wrt (:x, :y, or :ps). Parameter columns follow depth-first tuple field order and column-major array order. This is the reason the pointwise energy is kept vector-valued (L-ADR-0002, LP-002): from J and a SquaredNorm metric $M$, gauss_newton_metric forms $J^\top M J$ without second derivatives.
This is a reference implementation by central finite differences (step ε (1 + |u_j|)), so that LenticulumCore needs no AD dependency: accurate to about √eps, one energy evaluation per coordinate and side, and not differentiable itself. Pass ad=AutoForwardDiff() or another ADTypes backend to use the optional DifferentiationInterface extension instead. Load that package and the backend first.
LenticulumCore.energyspace — Method
energyspace(factor[, polarity]) -> AbstractEnergySpaceThe space $E_c$ of factor's vector free energy $\mathbf{F}^c$, the one that composes by direct sum and that a graph's free energy is graded by. Defaults to ScalarEnergySpace(), and to the polarity-free space when a polarity is given.
The pointwise energy returned by energy lives here too unless the factor says otherwise with pointwise_energyspace.
LenticulumCore.entropy — Function
entropy(factor, π, y, ps, st) -> (𝐇, st)The vector entropy $\mathbf{H}^c(\pi, y)$, an element of the same space.
Note the argument type: unlike energy, this eats a distribution π, not a sample. That type difference is exactly why entropies chain (averaged under the downstream inversion, evaluated at the pushforward prior) while energies merely add.
Usually the entropy is scalar but must be told which coordinate of $E_c$ it regularises; returning h * u for a fixed direction u is the idiomatic way to say so. For the graded (fit, complexity, negentropy) convention, entropy has coordinates (0, 0, H): free_energy subtracts this vector, producing negentropy = -H.
LenticulumCore.free_energy_gradient — Method
free_energy_gradient(f, π, y, ps, st; ad = nothing, ε = 1e-6, kwargs...)Parameter gradient of first(scalar_free_energy(...; kwargs...)), shaped like ps. Beliefs and incoming state are held fixed. This differentiates the family-defined free energy, including expectations if that implementation differentiates them; it does not supply an estimator for a stochastic inversion.
LenticulumCore.gauss_newton_metric — Method
gauss_newton_metric(σ::SquaredNorm, J::AbstractMatrix) -> Symmetric
gauss_newton_metric(factor, x, a, y, ps, st; wrt = :x, polarity = nothing, σ, ε = 1e-6) -> SymmetricThe Gauss–Newton / Fisher pull-back metric $J^\top M J$ at (x, a, y), where $J$ is energy_jacobian and $M$ is the metric of factor's SquaredNorm pointwise scalarisation (I when M === nothing). Always symmetric positive semidefinite whenever $M \succeq 0$. The factor form first localises σ, so it accepts SampsonNorm. This freezes the metric at the point: it ignores derivatives of the metric and is not the full Hessian of the point-dependent scalar energy. The pull-back wrt may differ from SampsonNorm.wrt, which defines the noise space.
LenticulumCore.islinear — Function
islinear(σ::AbstractScalarisation) -> BoolWhether $\sigma$ commutes with expectation.
This trait is load-bearing, not decorative. With a linear σ the composite scalar loss may be accumulated eagerly factor by factor, because $\sigma(\mathbb{E}[\cdot]) = \mathbb{E}[\sigma(\cdot)]$. With a convex σ it must be deferred until the vector loss is assembled, otherwise you silently compute $\mathbb{E}[\sigma(\mathbf{F})]$ when you asked for $\sigma(\mathbb{E}[\mathbf{F}])$. The two differ by jensen_gap.
LenticulumCore.jensen_gap — Method
jensen_gap(σ, samples) -> Real$\mathbb{E}[\sigma(e)] - \sigma(\mathbb{E}[e])$ over a collection of energy samples: the exact discrepancy between the scalar chain rule (AutoBayes Theorem 23) and the multivariate one.
Only defined for fixed scalarisations; point-dependent parts throw even for empty samples. Zero for a linear σ; for SquaredNorm() it equals $\tfrac12 \operatorname{tr} \operatorname{Cov}(e)$, the posterior variance of the upstream factor's loss. Report it — it is the same kind of object as the mutual information that measures the laxness of the tensor in AutoBayes' Remark 26.
LenticulumCore.localise — Method
localise(σ, f, x, a, y, ps, st) -> AbstractScalarisationReturn an ordinary scalarisation at this point, using the incoming state st. Existing fixed scalarisations return themselves (===). Point-dependent subtypes must implement this method. Graded scalarisations containing point-dependent parts are rejected: each part needs its own factor and arguments, which cannot in general be recovered from the enclosing factor's context (including tensor/composed factors). Localise the parts with their own contexts before constructing a graded scalarisation.
LenticulumCore.oplus — Function
⊕(a::AbstractEnergySpace, b::AbstractEnergySpace)
oplus(a, b)Direct sum of energy spaces. Named summands are merged; unnamed ones are wrapped under :left / :right.
LenticulumCore.parameter_gradient — Method
parameter_gradient(loss, ps; ad = nothing, ε = 1e-6)Differentiate scalar loss(ps) and return the numeric parameter tree's structure. Arrays use column-major order; tuples/NamedTuples use depth-first field order. Empty subtrees are preserved. State and data captured by loss are held fixed. No state is committed by this operation. ad=nothing uses central differences; an ADTypes object selects the optional DifferentiationInterface extension and a user-loaded backend. Only numeric scalar/array leaves and tuple/NamedTuple containers are supported.
LenticulumCore.pointwise_energyspace — Method
pointwise_energyspace(factor[, polarity]) -> AbstractEnergySpaceThe space the pointwise energy $\mathbf{l}^c$ of energy lands in. Defaults to energyspace(factor): one space for both, as in AutoBayes.
A factor overrides it when its pointwise energy is finer than its free energy, e.g. a residual $r \in \mathbb{R}^k$ (EuclideanEnergySpace(k)): signed coordinate by coordinate, with the minimum at $r = 0$ rather than at the bottom of a cone. The Jacobian of that vector is what Gauss–Newton steps, the Fisher metric and the implicit function theorem need (Scalar and Multivariate Energy.md §6).
LenticulumCore.pointwise_scalarisation — Method
pointwise_scalarisation(factor[, polarity]) -> AbstractScalarisationThe default scalarisation of the pointwise energy, used by scalar_energy. Defaults to scalarisation(factor); a residual factor returns a norm such as SquaredNorm(Q⁻¹).
LenticulumCore.sampson_damping — Method
sampson_damping(f, x, a, y, ps, st) -> AbstractMatrixResidual-space damping covariance for SampsonNorm, by default the identity of size size(energy_jacobian(f, x, a, y, ps, st), 1). A factor with a residual gauge r → B*r overrides this with a PSD matrix transforming as R → B*R*B', e.g. ps.Θ * ps.Θ' for r = Θ*v(z). Identity damping is not invariant under general B.
LenticulumCore.scalar_energy — Method
scalar_energy(factor, x, a, y, ps, st; polarity = nothing, σ) -> (Real, st)AutoBayes' $l^c$: the scalarisation of energy. Provided so that a factor written against the paper's interface needs no vector machinery.
σ defaults to pointwise_scalarisation(factor), or to pointwise_scalarisation(factor, polarity) when a polarity is given; pass another to scalarise the same energy differently (a weighted norm, other weights), as norm(x, p) takes p. Calls localise with the incoming st before evaluating and scalarising the energy.
LenticulumCore.scalarisation — Method
scalarisation(factor[, polarity]) -> AbstractScalarisationThe $\sigma_c$ that turns factor's vector free energy into a number. Defaults to the identity, which makes the factor behave exactly as in AutoBayes, and to the polarity-free choice when a polarity is given.
A scalarisation can carry parameters (the weights of WeightedSum, the metric of SquaredNorm); this is the factor's default, and scalar_free_energy and scalar_energy accept another as the keyword σ.
LenticulumCore.scalarise — Method
scalarise(σ::AbstractScalarisation, e) -> RealApply $\sigma$ to a (possibly graded) energy.
LenticulumCore.stop_gradient — Method
stop_gradient(x)Identity on values, zero derivative. ChainRules and ForwardDiff extensions implement the AD rule. Reference differences require a fixed reference value at the boundary.
Statistical games
LenticulumCore.ComposedFactor — Type
ComposedFactor(first, second; coupling = DiagonalCoupling())$d \diamond c$ of AutoBayes Definitions 22 and 28.
The composite's parameter space is the product $\Phi \times \Theta$, realised as the NamedTuple (first = ..., second = ...) — literally the same data structure Lux uses for Chain.
Lenticulum implements a pair of GaussianFactors as a collapsed endpoint game, integrating out the internal channel. Its energy space is that collapsed Gaussian's, distinct from the generic path-energy pairs of compose_energy / compose_free_energy.
coupling selects the composite parameter-gradient rule. Non-diagonal rules require an implemented family and otherwise throw at gradient evaluation. Lenticulum's collapsed Gaussian pair supports analytic exact/pathwise gradients and an explicit posterior score estimator; its historical diagonal endpoint gradient already marginalises the internal channel. A collapsed endpoint game is distinct from a graph exposing that wire.
LenticulumCore.FreeEnergyBoundEstimate — Type
FreeEnergyBoundEstimate(value, standard_error, nsamples)A Monte Carlo estimate of a variational upper bound. The expectation is a bound; the sampled value need not lie above negative log evidence. No confidence bound is implied.
LenticulumCore.FreeEnergyUpperBound — Type
FreeEnergyUpperBound(value)An evaluated variational upper bound on scalar negative log evidence. This is deliberately neither a number nor an energy: use .value explicitly. It cannot enter scalarise or scalar_free_energy accidentally. The model and variational law belong to the producing API.
LenticulumCore.TensorFactor — Type
TensorFactor(parts::NamedTuple)$c \otimes d$ of AutoBayes Definition 25. Both energies and entropies add across summands, because in parallel there is no upstream/downstream.
Exact games accept only explicit independent products: π and y are NamedTuples keyed by parts, whose entries supply each component's complete channel game. Only declared exact linear-Gaussian components are supported. Correlated joint beliefs throw, so no mean-field projection is silently applied. Tensor graph wiring/inversion is not supplied.
LenticulumCore.chain_rule_defect — Method
chain_rule_defect(σ, 𝐅c_samples)How far AutoBayes' Theorem 23 and the multivariate chain rule disagree across one edge:
\[F^{dc} - \sigma_{dc}(\mathbf{F}^{dc}) = \mathbb{E}[\sigma_c(\mathbf{F}^c)] - \sigma_c(\mathbb{E}[\mathbf{F}^c]) \ \ge 0\]
Zero for linear σ. For SquaredNorm it is $\tfrac12\operatorname{tr}\operatorname{Cov}(\mathbf{F}^c)$: how much the downstream posterior disagrees with itself about what the upstream factor should be doing.
LenticulumCore.compose_energy — Method
compose_energy(𝐥c, 𝐥d)The composite vector energy of AutoBayes Definition 22, adapted:
\[\mathbf{l}^{dc}(x,a,y,b,z) = (\mathbf{l}^c(x,a,y),\ \mathbf{l}^d(y,b,z))\]
The paper writes + here; we write a pair. Applying a GradedScalarisation to the pair recovers the paper's + exactly, and does so strictly — the direct sum is where no information is lost. Laxness enters only through compose_entropy's expectation.
LenticulumCore.compose_entropy — Method
compose_entropy(𝐇c_samples, 𝐇d)The composite vector entropy of AutoBayes Definition 22, adapted:
\[\mathbf{H}^{dc}(\pi,z) = \Bigl(\mathbb{E}_{(y,b)\sim d'_{c_*\pi}(z)}[\mathbf{H}^c(\pi,y)],\ \mathbf{H}^d(c_*\pi,z)\Bigr)\]
𝐇c_samples are evaluations of $\mathbf{H}^c(\pi, y)$ at samples (y,b) drawn from the downstream inversion at the pushforward prior. Getting those two adjectives right is the whole content of the definition, and the commonest place to go wrong.
LenticulumCore.compose_free_energy — Method
compose_free_energy(𝐅c_samples, 𝐅d)The multivariate chain rule:
\[\mathbf{F}^{dc}(\pi,z) = \Bigl(\mathbb{E}_{(y,b)\sim d'_{c_*\pi}(z)}[\mathbf{F}^c(\pi,y)],\ \mathbf{F}^d(c_*\pi,z)\Bigr)\]
Compare AutoBayes Theorem 23, which is this composed with a scalarisation. The two agree strictly iff σ is linear; otherwise they differ by jensen_gap, and the multivariate one is the smaller (Jensen).
LenticulumCore.free_energy — Function
free_energy(factor, π::AbstractBelief, y, ps, st) -> (𝐅, st)The vector loss
\[\mathbf{F}^c(\pi, y) = \mathbb{E}_{(x,a)\sim c'_\pi(y)}[\mathbf{l}^c(x,a,y)] - \mathbf{H}^c(\pi, y)\]
an element of energyspace(factor) — not a number. Collapse it with scalar_free_energy when you want a loss.
Concrete families may also implement free_energy(lens, π, inputs, ps, st) for an assembled AbstractBayesianLens: this evaluates the expectation under that inversion. Its domain and prior cost must be specified by the family; no generic lens game is assumed. For DEQ/NeuralODE, the factor game scores incoming points, while the assembled lens game scores the solved point plus an anchor cost. Arbitrary point scoring uses energy or scalar_energy (L-ADR-0003), not a keyword on free_energy.
LenticulumCore.isfrozen — Method
isfrozen(factor) -> BoolWhether factor's parameters exist but are pinned for this run.
Defaults to false. True pins all parameter updates for this run, excludes them from optimiser state, and preserves input cotangents. Unlike islearnable, it does not remove a factor from learnable_factors. Mycelium also accepts factor!(...; freeze=(:encoder,)), a tuple of paths, or freeze=:all for graph-local selection.
LenticulumCore.islearnable — Method
islearnable(factor) -> BoolWhether factor contributes trainable parameters.
Defaults to LuxCore.parameterlength(factor) > 0. Override to false when a factor intrinsically contributes no trainable parameters. Mycelium's learnable_factors uses this trait as the candidate list. Parameters of nonlearnable factors are fixed in graph parameter gradients, while input derivatives still propagate through their computations.
See also isfrozen.
LenticulumCore.scalar_free_energy — Method
scalar_free_energy(factor, π, y, ps, st; polarity = nothing, σ) -> (Real, st)$F^c = \sigma_c \circ \mathbf{F}^c$: AutoBayes' Definition 20 loss.
σ defaults to scalarisation(factor), or to scalarisation(factor, polarity) when a polarity is given; pass another to weigh the summands differently without rebuilding the factor. Also accepts an assembled lens when its family implements free_energy and scalarisation. The lens already fixes a polarity; omit the polarity override in that case.
LenticulumCore.setup — Method
setup(rng, factor) -> (ps, st)As LuxCore.setup. Provided so that factors and Lux layers are set up identically.
LenticulumCore.straight_through — Method
straight_through(forward, surrogate, x)Evaluate forward(x) with the input Jacobian of surrogate(x). Supported by ForwardDiff and ChainRules reverse AD; finite differences of this primal do not compute the surrogate derivative. The callables are fixed, parameterless functions.
Everything else
LenticulumCore.LenticulumCore — Module
LenticulumCoreCore abstractions for Lenticulum.jl: factors as parameterized statistical games.
LuxCore.jl is to Lux.jl as LenticulumCore.jl is to Lenticulum.jl, and the parameter/state interface is deliberately identical. The difference is what a node is:
| Lux.jl | Lenticulum.jl | |
|---|---|---|
| node | parametric lens (Cruttwell et al., Def. 2.5) | parameterized statistical game (AutoBayes, Def. 27) |
| category | Para(Lens(C)) | Para(StatGame) |
| direction | fixed at construction | chosen per call, via a Polarity |
| wiring | directed acyclic graph | any weakly-connected digraph |
| backward pass | reverse derivative | Bayesian inversion + free-energy accumulation |
A factor carries four pieces (AutoBayes Definition 20): a forward kernel c, an inversion c', an energy and an entropy. Lenticulum keeps the energy and entropy vector-valued, with a separate scalarisation, so that Jacobians — and hence the Gauss–Newton and Fisher metrics, and the implicit function theorem — survive composition. See Scalar and Multivariate Energy.md.
Concept notes live in vault/; per-file implementation notes sit next to each source file.
Lenticulum extends ComposedFactor for a pair of GaussianFactors by integrating out the shared channel. It exposes the endpoint channels and exact collapsed Gaussian messages, entropy and game/graph free energy. Other compositions throw with an exact-pushforward reason. TensorFactor combines independent linear-Gaussian games: π and y must be NamedTuples keyed exactly like parts; their entries describe each component's complete channel game. Results retain the branch grading and scalarise to the sum. Correlated joint beliefs and unsupported components throw. Tensor graph wiring/inversion remains absent.
The helpers compose_energy, compose_entropy and compose_free_energy retain their value-level path-energy semantics, distinct from a collapsed Gaussian marginal game. FreeEnergyUpperBound and FreeEnergyBoundEstimate are separate wrappers for variational bounds and estimates; neither enters exact free_energy or scalarise. Diffusion supplies the explicitly normalised finite-chain instance (Factors).
Device parameter derivatives and beliefs
The optional GPUArraysCore extensions support JLArrays Float32/Float64 parameter array leaves with Zygote and directional ForwardDiff through the existing parameter_gradient, energy_gradient and free_energy_gradient APIs. Direct ForwardDiff vector APIs retain their upstream scalar-seeding limitation. Device finite differences and Enzyme/Mooncake parameter AD are unsupported and throw. Functional device flattening preserves empty subtrees and all-empty device arrays without selecting host concatenation. CPU arithmetic is unchanged.
GaussianBelief(η, Diagonal(λ)) and Gaussian(μ, Diagonal(σ²)) support diagonal device moment/density/entropy and log-partition operations without factorisation. Dense device covariance/precision is rejected explicitly. See the JLArrays tests in lib/ImplicitLayers.jl/test/device_arrays.jl; no hardware GPU support is asserted.
Sampson localisation derivative boundary (LP-002 batch C)
The open item means differentiating the total varying-metric energy $\frac12r^\top A^{-1}r$, including the covariance derivative, not changing the frozen-metric gauss_newton_metric. With $w=A^{-1}r$, its differential is $w^\top dr-\frac12w^\top(dA)w$. Localisation still uses the default mutable central-difference Jacobian unless the factor overrides it; supplying ad to an outer parameter derivative does not replace that inner algorithm. Generic AD through finite-difference localisation remains unsupported. The probe lib/ImplicitLayers.jl/examples/lp002_metric_divergence_limits.jl uses $r=a y^2-b$ and its analytic scalar energy/gradient to separate reference cancellation error from backend failures. LP-002 records the measured results. No Sampson metric, rank threshold or Gauss–Newton default changes in batch C.