implementation

Open-model results retain latent values; belief types include probability representations and improper callable likelihood potentials.

Sources: code: open_model.jl

Theory (CT-ML wiki): Lax Functor · Bayesian Inversion · Bayesian Lens · Open Model · Reverse Derivative Category · Lens · Statistical Game

Implements: Open Models and Latent Channels (AutoBayes Def. 1) and the belief types that flow along the graph.

The one idea

The latent space exists so that composition can file away the intermediate value instead of integrating it out:

No integral. OpenModelResult(observed, latent) is the runtime witness of this: a forward pass returns both, and the latent field is exactly an autodiff tape entry. Same trade of memory for tractability, under the same chain rule.

Interface

functionpapernote
forward(m, x, ps, st)sample returns OpenModelResult
logdensity(m, x, a, y, ps, st)needed whenever the energy is a NLL
pushforward(m, π, ps, st)expensive — see below
latentspace / observedspace / unobservedspace / /
ispure(m)not preserved by composition

Beliefs: DiracBelief (a clamp / a cup), SampleBelief (particles), TrivialBelief (the unit space).

Implementation difficulties

1. pushforward is one of the two things that make this hard

The paper’s own closing discussion says it: computing is marginalisation, “similarly expensive to computing exact inversions”, and the recommended remedy is belief propagation or variational message passing. Mycelium supplies the graph and message-passing machinery.

The consequence for this file is that pushforward must be allowed to lie. It returns an AbstractBelief with no promise of exactness, and isexact (defaulting to false) is how a caller finds out. An implementation that returns a moment-matched Gaussian is legitimate; one that returns it while claiming isexact() == true is a bug.

2. Conjugate families are not preserved by pushforward

Also from the paper’s closing discussion: conjugate models make everything cheap, but pushing forward does not generally preserve the family, so you need moment-matching projections back into it — and encoding “which family this wire carries” into the game data “requires annotating the games with predicates, which can be done compositionally, but laxly”.

Nothing in this file does that yet. When it lands it will want a family(belief) trait and a project(belief, family) operation, and the laxness of the annotation will need the same treatment as everything else here: track it, report it, do not hide it.

3. ispure defaults to false and composition must not override it naively

Purity () is a genuine optimisation: a pure model needs no latent bookkeeping. It is also destroyed by composition — composing two pure models gives a model whose latent space is the intermediate space. Any future ispure(::ComposedModel) must return false unconditionally, not all(ispure, parts). Writing it the natural way would be wrong, which is why it is called out here.

4. Belief representations are the real open question

GaussianBelief now supplies canonical parameters, categorical beliefs store normalised log-probabilities, and samples provide a finite approximation. AbstractPotentialBelief separates likelihoods known only through a log-density from normalised probability representations. There is no shared exponential-family or natural-gradient interface for all these types; adding one still needs a declared family and projection contract.

Weight normalization

Categorical probabilities and mixture weights are normalized in log space after checking that the input is finite, nonnegative and has positive mass. This avoids overflow when several finite weights are near the floating-point maximum, preserves zero support, and keeps log probabilities finite for positive weights across the floating-point range. The construction tests cover Float32 and Float64 large and subnormal weights.

5. logdensity on an unnormalised measure

Open Models and Latent Channels notes that cyclic models require unnormalised measures, so logdensity may be a log-potential rather than a log-density, differing by an unknown constant. For energy purposes this is fine (the constant is irrelevant to gradients), but for anything that compares densities across models it is not. Currently unmarked. A isnormalised(m) trait will be needed before any model-comparison feature.

Related: Open Models and Latent Channels, Composition of Open Models, lens, Inversions and Bayesian Lenses

Likelihood potentials (LP-016)

AbstractPotentialBelief <: AbstractBelief represents an improper, unnormalised likelihood known through Mycelium.belief_logdensity(b, x), up to an additive constant independent of . LogPotentialBelief(logφ) wraps the callable logφ(x). It is intended as a factor→variable message, with no sampling, mean, covariance, or entropy contract.

The abstract type leaves room for likelihoods with gradients, a parametric form, or batched evaluation. Those types provide their own log-density and can add exact product rules. Mycelium sums log-potentials and reweights explicit particles; it refuses arbitrary parametric products. Core construction tests check the representation; Mycelium and AdversarialFactors tests check pooling and target exclusion against closed forms.

LP-011 positive-weight reverse derivatives (2026-10-27)

The optional ChainRules extension supplies the exact normalised log-weight pullback dy ./ weights - sum(dy)/sum(weights) for strictly positive weights. This avoids the tested Zygote/Julia broadcast failure while constructing categorical score messages. Zero-weight changing-support derivatives throw. Primal zero-weight beliefs remain legal. Evidence: the categorical score estimator tests in Mycelium’s test/gradient_couplings.jl.

LP-018 normalisation pullback (2026-10-29)

The normalised log-weight pullback still has the same formula, but its shared denominator is evaluated as (sum(dy)/maximum(weights))/sum(weights/maximum(weights)). This avoids an overflowing sum of finite weights and preserves cancellation for uniform cotangents. Evidence: the top-level test/lp018_interop.jl, “LP-018.9 finite particle weights across interop and AD”, checks the dimensionless scaled gradient at [1e308,1e308] against zero.

The weight-normalisation pullback (2026-10-29, LP-019 item 7)

The ChainRulesCore rule for _normalised_logweights computed with . The second term overflows for tiny weights: at [1e-308, 1e-308] the gradient of sum(logp) was [-Inf, -Inf], at [1e-320, 1e-320] NaN, though it is exactly zero. It is now with normalised by first, so the cancelling numerator is formed before any division. Tested in the top-level test/lp019_interop.jl (the extension does not load in LenticulumCore’s own test environment).

After review (2026-10-30): where underflows (, e.g. [1e-200, 1e200]) the numerator-first form loses ; there the rule uses , which is safe because is not small.

Empty sample beliefs (2026-11-04, LP-023)

SampleBelief refuses no samples and all-zero weights, which downstream operations handled inconsistently (a division by a zero total, or an empty reduction).

Sample moments (2026-11-12, P18)

belief_mean and belief_cov of a SampleBelief are defined in LenticulumCore (in gaussian_belief.jl, after the generic functions): the weighted mean and the weighted covariance , the weights divided by the largest and then by their sum (uniform without weights). Samples are the columns of a matrix or the elements of a vector. They were defined in AdversarialFactors (ratio.jl), which was type piracy: the moments of a sample belief existed only with AdversarialFactors loaded. AdversarialFactors’ weighted_mean and weighted_cov keep their names and call these. Tested in test/sample_moments.jl against the sums written out and against Statistics’ mean and uncorrected cov (40 random instances, matrix and vector samples, weights scaled by 1e300, scalar and integer samples, one sample).

Not done: Lenticulum’s moment_match(::SampleBelief) computes the same moments by its own code (and refuses a singular covariance); it was left as it is.