LenticulumCore

abstract type AbstractBayesianLens

A Bayesian lens $(c, c')$ (AutoBayes, Definition 9): an AbstractOpenModel paired with an AbstractInversion.

abstract type AbstractBelief

An element of $\mathcal{P}X$: a distribution over a channel's space.

Beliefs are what flows forward along the graph (as priors, propagated by pushforward) and what inversions consume. Concrete subtypes decide the representation: a Dirac, a particle set, an exponential-family natural parameter, a Gaussian.

abstract type AbstractEnergySpace

The space $E_c$ in which a factor's vector energy and entropy take values.

AutoBayes fixes $E_c = [0,\infty]$ and composes energies by addition. Lenticulum keeps the vector and composes by direct sum, recovering the paper's law under scalarisation. See Scalar and Multivariate Energy.md.

abstract type AbstractGradientCoupling

How an edge of the graph handles the terms AutoBayes' Definition 29 drops.

Definition 29 composes gradients block-diagonally; the true Jacobian has two extra blocks, because the parameter of one factor moves the sampling distribution and the pushforward prior seen by the other. The paper calls the resulting assignment "lax" and says it "can be accounted for mechanistically by an implementation". This type is that mechanism: each edge declares which correction it applies, and the choice is exactly the paper's "different semantics functors".

See DiagonalCoupling, PathwiseCoupling, ScoreFunctionCoupling, ExactCoupling.

abstract type AbstractInversion

The backward half $c'$ of a Bayesian lens (AutoBayes, Definition 9): a rule that turns a prior $\pi \in \mathcal{P}X$ and an observation $y \in Y$ into a belief over $X \times \llbracket c \rrbracket$.

Nothing requires an inversion to be exact. Exact, amortised, mean-field, solver-based and diffusion-based inversions are all legal inhabitants; the quality of the choice is what the free energy measures.

abstract type AbstractLenticulumContainerFactor{factors} <: AbstractLenticulumFactor

A factor built from several sub-factors named by the tuple of symbols factors. Mirrors LuxCore.AbstractLuxContainerLayer: parameters and states are assembled into a NamedTuple keyed by factors.

This is the [Para] composition of Definition 28: parameter spaces multiply, and the product is realised as a NamedTuple.

abstract type AbstractLenticulumFactor

A factor: a parameterized statistical game (AutoBayes, Definition 27).

This is Lenticulum's analogue of LuxCore.AbstractLuxLayer, and the parameter/state interface is inherited verbatim: implementors must provide

• initialparameters(rng, factor)
• initialstates(rng, factor)

and additionally, for the statistical-game structure,

• channels – the named ports of the factor
• supports_polarity – which input/output splits it can answer
• assemble – produce a Bayesian lens for a chosen polarity
• energyspace – the energy space $E_c$
• energy – the vector energy $\mathbf{l}^c$
• entropy – the vector entropy $\mathbf{H}^c$

Optionally scalarisation, parameterlength, statelength, display_name.

Unlike a Lux layer, a factor has no fixed direction: assemble(factor, polarity) constructs the parametric lens on demand. See Channels and Polarity.md.

abstract type AbstractLenticulumWrapperFactor{factor} <: AbstractLenticulumFactor

A factor wrapping exactly one sub-factor, whose parameters and states are not nested under an extra name. Mirrors LuxCore.AbstractLuxWrapperLayer.

abstract type AbstractOpenModel

An open model $c : X \nrightarrow Y$ (AutoBayes, Definition 1): a measure kernel $X \rightsquigarrow \llbracket c \rrbracket \times Y$ together with its latent space $\llbracket c \rrbracket$.

The latent space is the reason composition of open models needs no integration; see open_model.md.

abstract type AbstractScalarisation

A map $\sigma : E \to \mathbb{R}$ collapsing a vector energy to a loss.

Must satisfy σ(0) == 0, non-negativity on the cone, and monotonicity. The critical trait is islinear: a linear σ commutes with the expectation in the chain rule and so composition is strict; a merely convex σ makes it lax, with the gap given by a Jensen term (a variance, for a squared norm).

DiagonalCoupling()

Keep only the block-diagonal terms of Definition 29; i.e. stop-gradient through the sampling path and through the pushforward prior. Cheapest, biased.

ExactCoupling()

The conjugate/analytic case: the full Jacobian is available in closed form and no term is dropped. Composition of gradients is then strictly functorial across this edge.

PathwiseCoupling()

Differentiate through the sampler (the reparametrisation trick). Requires the inversion to be reparametrisable. Unbiased, low variance.

ScoreFunctionCoupling(baseline=nothing)

REINFORCE: recover the dropped term as E[F ∇ log q], optionally with a control variate. Unbiased, high variance, works for discrete inversions.

abstract type AbstractChannelPolarity

The role a channel plays in one particular use of a factor.

Channel{name,S}(space)

A named port of a factor, carrying the space its values live in.

Latent()

AutoBayes' latent space $\llbracket c \rrbracket$; the note's $P_{latent}$. Internal scratch that composition hid. Either revealed (a free retyping) or marginalised (expensive).

Observed()

AutoBayes' observed space $Y$; the note's $P_{in}$. Clamped to data. The posterior does not range over this channel.

Polarity(assignment::NamedTuple, [precisions::NamedTuple])
Polarity(; channel = polarity, ...)

An assignment of an AbstractChannelPolarity to each channel of a factor, together with a precision $\rho$ per channel.

Because the assignment is a NamedTuple, the channel names live in the type, so an assembled lens specialises at compile time rather than dispatching through a Dict.

Polarity(; x = Observed(), y = Unobserved())

Unobserved()

AutoBayes' unobserved space $X$; the note's $P_{out}$. This is what inference solves for, and what the inversion produces a belief over.

assemble(factor, p::Polarity, ps, st) -> (lens, st)

Produce the AbstractBayesianLens realising factor under polarity p.

This is the operation that has no counterpart in Lux, and the reason a factor cannot be a Lux layer: a Lux layer is a lens, whereas a factor only becomes one once a direction is chosen. See Lux as a Parametric Lens.md.

channel_precision(p::Polarity, name::Symbol)

The weight $\rho$ attached to channel name.

channels(factor) -> Tuple{Vararg{Channel}}

The named ports of factor. Must be implemented by every factor.

default_precision(p::AbstractChannelPolarity)

The default weight $\rho$ for a channel of this polarity, as in $P = \rho_{in}P_{in} + \rho_{out}P_{out} + \rho_{latent}P_{latent}$.

Observed defaults to Inf (a hard clamp — a categorical cup); the others to 1. A finite Observed precision is a soft clamp, which is what makes noisy observations and annealed conditioning expressible.

ispartition(p::Polarity)

Check $P_{in} + P_{out} + P_{latent} = \mathrm{Id}$ with pairwise-zero products: every channel gets exactly one polarity. Guaranteed by the NamedTuple representation, so this is a total function returning true; kept as a named predicate because the corresponding check on a graph-wide polarity is not trivial.

isunidirectional(factor) -> Bool

Whether factor admits exactly one polarity, i.e. has a fixed input/output split and is therefore an explicit rather than an implicit factor.

Non-learnable structural nodes are usually unidirectional: a data source only emits, a loss only absorbs.

select(p::Polarity, ::Type{P}) where {P<:AbstractChannelPolarity}

The tuple of channel names carrying polarity P. This is the diagonal selection matrix $P_{in}$ / $P_{out}$ / $P_{latent}$ as a tuple of Symbols.

supported_polarities(factor) -> Tuple{Vararg{Polarity}}

The polarities factor can answer, enumerated.

supports_polarity is the predicate form; this is the listable form, which a scheduler needs in order to plan messages without guessing. The default is empty, matching supports_polarity's conservative false.

A factor with exactly one supported polarity is unidirectional: it can be evaluated in one direction only, like an ordinary Lux layer. See isunidirectional.

supports_polarity(factor, p::Polarity) -> Bool

Whether factor can answer the input/output split p.

This predicate replaces Lux's acyclicity restriction. A Lux Chain is well-formed iff the wiring is a DAG; a Lenticulum graph is well-formed iff every scheduled message uses a polarity its factor supports. The default is conservative: a factor supports nothing until it says otherwise.

DiracBelief(value)

A point mass. What a clamped (observed) channel carries, and what a categorical cup produces (see Copiers Cups and Caps.md). Inversions against a Dirac prior typically trivialise — which is exactly AutoBayes' Example 4: supervised learning is the case where the cup has collapsed the posterior.

OpenModelResult(observed, latent)

The result of running an open model forward: a value in $Y$ together with the value in $\llbracket c \rrbracket$ that composition would otherwise have destroyed.

This is the exact analogue of an autodiff tape entry. Reverse-mode AD caches activations so the backward pass can use them; an open model caches $\llbracket c \rrbracket$ so the inversion can. Same trade of memory for tractability, under the same chain rule.

SampleBelief(samples, [weights])

A particle representation of $\pi \in \mathcal{P}X$. The default fallback whenever no conjugate structure is available, which is most of the time.

TrivialBelief()

The unique belief on the one-point space $1$. The prior argument of a factor with $X \cong 1$ — a prior distribution, in AutoBayes' terminology, has this as its input.

forward(model, x, ps, st) -> (OpenModelResult, st)

Sample the kernel $c : X \rightsquigarrow \llbracket c \rrbracket \times Y$ at x.

isexact(x) -> Bool

Whether a pushforward or an inversion is exact rather than approximate. Defaults to false because approximation is the normal case and silence should not imply exactness.

ispure(model) -> Bool

$\llbracket c \rrbracket \cong 1$: an ordinary kernel $X \rightsquigarrow Y$.

Purity is not preserved by composition — that is precisely why the latent space exists. Composing two pure models yields a model whose latent space is the intermediate space.

latentspace(model)
observedspace(model)
unobservedspace(model)

The three spaces $\llbracket c \rrbracket$, $Y$, $X$ of AutoBayes Definition 1.

logdensity(model, x, a, y, ps, st) -> (Real, st)

$\log p_c(a, y \mid x)$. Needed whenever the energy is a negative log-likelihood, which is the default choice in every example of AutoBayes' Appendix A.

pushforward(model, π::AbstractBelief, ps, st) -> (AbstractBelief, st)

The pushforward prior $c_*\pi$.

AmortisedInversion(net)

$c'$ realised by a learned network — a VAE encoder. net is an AbstractLuxLayer, so the inversion carries its own parameters, independent of the forward kernel's.

That a factor has two independently parametrised halves is the structural reason a factor cannot be a Lux layer.

BayesianLens(model, inversion)

The pair $(c, c')$ of AutoBayes Definition 9.

Nothing constrains inversion to be exact. Exact posteriors, amortised encoders, mean-field families, root-finding solvers and diffusion denoisers are all legal, and their relative quality is precisely what the free energy measures.

ComposedLens(first, second)

$(d, d') \diamond (c, c')$ of AutoBayes Definition 12. Priors propagate forward by pushforward; corrections propagate backward by sampling the inversions in reverse order.

ExactInversion()

$c^\dagger$ of AutoBayes Definition 10: Bayes' law applied to the model's own kernel. Available only when the model declares a tractable posterior.

ProximalInversion(prox)

$c'$ realised by a proximal operator on an energy — the diffusion family (RED-Diff, ProxDM). See lib/VariationalDiffusion.jl and ImplicitREDDiff.md.

SolverInversion(solver)

$c'$ realised by root-finding on a residual: Implicit Learners.md's algebraic and equilibrium families. The backward pass is the implicit function theorem, which needs the Jacobian of the vector energy — see Scalar and Multivariate Energy.md §6.

A solver that stopped early is simply an inexact inversion, and the loss records the cost. That is a far better failure mode than a divergent unroll.

TensorLens(parts::Tuple)

$(c,c') \otimes (d,d')$ of AutoBayes Definition 15.

TrivialInversion()

The inversion of a prior $\pi : 1 \nrightarrow X$: there is nothing to infer.

AutoBayes Remark 24 uses exactly this to turn a prior into a statistical game with $l^\pi = -\log p_\pi$ and $H^\pi \equiv 0$, whose composite with c has the true variational free energy as its loss. The prior is a factor, not part of the model.

compose(c::AbstractBayesianLens, d::AbstractBayesianLens)

Build $d \diamond c$. Argument order follows the wiring (c then d), not the mathematical notation $d \diamond c$.

invert(lens, π::AbstractBelief, y, ps, st) -> (AbstractBelief, st)

Apply $c'_\pi$ to the observation y, returning a belief over $X \times \llbracket c \rrbracket$.

Note the return type reconstructs the latent space too, not just $X$. That is what the next factor upstream will consume, and dropping it breaks the chain rule.

EuclideanEnergySpace(n)

$E = \mathbb{R}^n$. The space a vector residual $r_\theta$ lands in; n must match the number of unobserved coordinates for the implicit function theorem to apply.

GradedEnergy(parts::NamedTuple)

An element of a GradedEnergySpace: the per-factor breakdown of a composite energy, before it is collapsed to a number.

Per-factor loss attribution is not a logging feature bolted on afterwards — it is what the composite energy is.

GradedEnergySpace(parts::NamedTuple)

The direct sum $E_G = \bigoplus_{f} E_f$ graded by factor name.

This is what a composite factor's energy space is: composition does not add, it files the summands away under their factors' names — exactly as an open model files intermediate values into $\llbracket c \rrbracket$ rather than integrating them out.

GradedScalarisation(parts::NamedTuple)

$\sigma_{dc}(e_c, e_d) = \sigma_c(e_c) + \sigma_d(e_d)$, the scalarisation induced on a direct sum. Additivity across summands is automatic; linearity is inherited from the parts.

IdentityScalarisation()

$\sigma = \mathrm{id}$ on ScalarEnergySpace. Linear. Recovers AutoBayes' Definition 22 verbatim.

ScalarEnergySpace()

$E = \mathbb{R}$, $K = [0,\infty)$. Recovers AutoBayes exactly when paired with IdentityScalarisation.

SquaredNorm(M = nothing)

$\sigma(e) = \tfrac12 \|e\|_M^2$. Convex, not linear.

The natural scalarisation for a residual factor: $r_\theta(x) \approx 0$ iff $\sigma(r_\theta(x)) \approx 0$. Its differential is $\mathrm{d}\sigma(e) = Me$, so the scalar gradient is $J^\top M r$ and the Gauss–Newton metric is $J^\top M J$ — both of which need the Jacobian of the vector energy, which is exactly what is lost by scalarising early.

WeightedSum(λ)

$\sigma_\lambda(e) = \langle \lambda, e\rangle$ for λ in the dual cone. Linear.

This is where β-VAE weights, KL annealing schedules, curriculum weights and the precisions $\rho$ of ImplicitREDDiff.md live. Because the energy is kept as a vector, λ can be changed after the graph is composed — with a scalar energy each λ is a different graph.

energy(factor, x, a, y, ps, st) -> (𝐥, st)

The vector energy $\mathbf{l}^c(x, a, y)$, an element of the cone of energyspace(factor).

Pointwise: it is evaluated at samples drawn from the inversion, so it needs no normalisation. That is why energies compose by direct sum with no expectation involved.

energyspace(factor) -> AbstractEnergySpace

The space $E_c$ of factor's vector energy. Defaults to ScalarEnergySpace().

entropy(factor, π, y, ps, st) -> (𝐇, st)

The vector entropy $\mathbf{H}^c(\pi, y)$, an element of the same space.

Note the argument type: unlike energy, this eats a distribution π, not a sample. That type difference is exactly why entropies chain (averaged under the downstream inversion, evaluated at the pushforward prior) while energies merely add.

Usually the entropy is scalar but must be told which coordinate of $E_c$ it regularises; returning h * u for a fixed direction u is the idiomatic way to say so.

islinear(σ::AbstractScalarisation) -> Bool

Whether $\sigma$ commutes with expectation.

This trait is load-bearing, not decorative. With a linear σ the composite scalar loss may be accumulated eagerly factor by factor, because $\sigma(\mathbb{E}[\cdot]) = \mathbb{E}[\sigma(\cdot)]$. With a convex σ it must be deferred until the vector loss is assembled, otherwise you silently compute $\mathbb{E}[\sigma(\mathbf{F})]$ when you asked for $\sigma(\mathbb{E}[\mathbf{F}])$. The two differ by jensen_gap.

jensen_gap(σ, samples) -> Real

$\mathbb{E}[\sigma(e)] - \sigma(\mathbb{E}[e])$ over a collection of energy samples: the exact discrepancy between the scalar chain rule (AutoBayes Theorem 23) and the multivariate one.

Zero for a linear σ; for SquaredNorm it equals $\tfrac12 \operatorname{tr} \operatorname{Cov}(e)$, the posterior variance of the upstream factor's loss. Report it — it is the same kind of object as the mutual information that measures the laxness of the tensor in AutoBayes' Remark 26.

⊕(a::AbstractEnergySpace, b::AbstractEnergySpace)
oplus(a, b)

Direct sum of energy spaces. Named summands are merged; unnamed ones are wrapped under :left / :right.

scalar_energy(factor, x, a, y, ps, st) -> (Real, st)

AutoBayes' $l^c$: the scalarisation of energy. Provided so that a factor written against the paper's interface needs no vector machinery.

scalarisation(factor) -> AbstractScalarisation

The $\sigma_c$ used to turn factor's vector loss into a number. Defaults to the identity, which makes the factor behave exactly as in AutoBayes.

scalarise(σ::AbstractScalarisation, e) -> Real

Apply $\sigma$ to a (possibly graded) energy.

ComposedFactor(first, second; coupling = DiagonalCoupling())

$d \diamond c$ of AutoBayes Definitions 22 and 28.

The composite's parameter space is the product $\Phi \times \Theta$, realised as the NamedTuple (first = ..., second = ...) — literally the same data structure Lux uses for Chain.

coupling records which of the gradient terms dropped by Definition 29 this edge restores.

TensorFactor(parts::NamedTuple)

$c \otimes d$ of AutoBayes Definition 25. Both energies and entropies add across summands, because in parallel there is no upstream/downstream.

Subject to the mean-field laxness of TensorLens.

chain_rule_defect(σ, 𝐅c_samples)

How far AutoBayes' Theorem 23 and the multivariate chain rule disagree across one edge:

\[F^{dc} - \sigma_{dc}(\mathbf{F}^{dc}) = \mathbb{E}[\sigma_c(\mathbf{F}^c)] - \sigma_c(\mathbb{E}[\mathbf{F}^c]) \ \ge 0\]

Zero for linear σ. For SquaredNorm it is $\tfrac12\operatorname{tr}\operatorname{Cov}(\mathbf{F}^c)$: how much the downstream posterior disagrees with itself about what the upstream factor should be doing.

compose_energy(𝐥c, 𝐥d)

The composite vector energy of AutoBayes Definition 22, adapted:

\[\mathbf{l}^{dc}(x,a,y,b,z) = (\mathbf{l}^c(x,a,y),\ \mathbf{l}^d(y,b,z))\]

The paper writes + here; we write a pair. Applying a GradedScalarisation to the pair recovers the paper's + exactly, and does so strictly — the direct sum is where no information is lost. Laxness enters only through compose_entropy's expectation.

compose_entropy(𝐇c_samples, 𝐇d)

The composite vector entropy of AutoBayes Definition 22, adapted:

\[\mathbf{H}^{dc}(\pi,z) = \Bigl(\mathbb{E}_{(y,b)\sim d'_{c_*\pi}(z)}[\mathbf{H}^c(\pi,y)],\ \mathbf{H}^d(c_*\pi,z)\Bigr)\]

𝐇c_samples are evaluations of $\mathbf{H}^c(\pi, y)$ at samples (y,b) drawn from the downstream inversion at the pushforward prior. Getting those two adjectives right is the whole content of the definition, and the commonest place to go wrong.

compose_free_energy(𝐅c_samples, 𝐅d)

The multivariate chain rule:

\[\mathbf{F}^{dc}(\pi,z) = \Bigl(\mathbb{E}_{(y,b)\sim d'_{c_*\pi}(z)}[\mathbf{F}^c(\pi,y)],\ \mathbf{F}^d(c_*\pi,z)\Bigr)\]

Compare AutoBayes Theorem 23, which is this composed with a scalarisation. The two agree strictly iff σ is linear; otherwise they differ by jensen_gap, and the multivariate one is the smaller (Jensen).

free_energy(factor, π::AbstractBelief, y, ps, st) -> (𝐅, st)

The vector loss

\[\mathbf{F}^c(\pi, y) = \mathbb{E}_{(x,a)\sim c'_\pi(y)}[\mathbf{l}^c(x,a,y)] - \mathbf{H}^c(\pi, y)\]

an element of energyspace(factor) — not a number. Collapse it with scalar_free_energy when you want a loss.

isfrozen(factor) -> Bool

Whether factor's parameters exist but are pinned for this run.

The distinction from islearnable matters for the backward pass: a frozen factor still propagates cotangents through to its inputs (so upstream factors learn), whereas a factor that is simply not learnable has no parameter wire at all. Confusing the two silently detaches a subgraph.

islearnable(factor) -> Bool

Whether factor contributes trainable parameters.

Defaults to LuxCore.parameterlength(factor) > 0, but is a trait, not a computation: a factor with parameters that are deliberately held fixed (a pretrained encoder, a physical constant, a data clamp) should override it to false. A graph uses this to decide which nodes need an optimiser attached and which parameter cotangents may be discarded unread.

See also isfrozen.

scalar_free_energy(factor, π, y, ps, st) -> (Real, st)

$F^c = \sigma_c \circ \mathbf{F}^c$: AutoBayes' Definition 20 loss.

setup(rng, factor) -> (ps, st)

As LuxCore.setup. Provided so that factors and Lux layers are set up identically.

LenticulumCore

Core abstractions for Lenticulum.jl: factors as parameterized statistical games.

LuxCore.jl is to Lux.jl as LenticulumCore.jl is to Lenticulum.jl, and the parameter/state interface is deliberately identical. The difference is what a node is:

A factor carries four pieces (AutoBayes Definition 20): a forward kernel c, an inversion c', an energy and an entropy. Lenticulum keeps the energy and entropy vector-valued, with a separate scalarisation, so that Jacobians — and hence the Gauss–Newton and Fisher metrics, and the implicit function theorem — survive composition. See Scalar and Multivariate Energy.md.

Concept notes live in vault/; per-file implementation notes sit next to each source file.