open-problem

Status: resolved (2026-10-26). Related: lib/Lenticulum.jl/src/gaussian.jl (GaussianFactor, GaussianPrior), Scalar and Multivariate Energy §8 (“Normalised or not”).

Problem. In GaussianFactor the noise covariance is a fixed hyperparameter, and GaussianPrior’s is not learnable either: learning them needs a positive-definite parametrisation, which the code defers. It matters beyond convenience. The pointwise energy leaves out the normaliser , a constant only while is fixed; once is learned, only the free energy, which keeps it as the complexity summand, is a valid objective.

Options.

  1. A Cholesky factor with a log-diagonal, , as parameters.
  2. A precision parametrisation, , which is what the scalarisation uses.
  3. Diagonal only, through log-variances.

Leaning. Not chosen. Whichever it is, training must use the free energy, and a test should show what goes wrong without it: the pointwise energy alone decreases as grows, so it drives (variance inflation); the normaliser in the free energy penalises that. At an exact fit () the likelihood is unbounded as , so even the free energy needs a prior on or a floor. (Direction corrected 2026-10-25; it said .)

Decision and implementation — 2026-10-26

The maintainer chose precision Cholesky coordinates, , with a log diagonal; diagonal mode stores log-precisions. GaussianFactor and GaussianPrior opt in with learn_noise=true / :full or :diagonal, initialised from their supplied covariance. Fixed noise remains the default; the prior mean stays fixed.

GaussianFactor.energy stays the residual, and scalar_energy is still quadratic fit without a normaliser. The full Gaussian normaliser belongs in free_energy and local_free_energy, as in L-ADR-0003 Pointwise energy and free energy are separate levels. GaussianPrior keeps its existing normalised pointwise energy. Optional noise-prior penalties belong only in free energy. The earlier options and leaning above record the original open question, not the current API.

A user-supplied WishartPrecisionPrior(df, scale) (full precision) or LogNormalVariancePrior(logmean, logstd) (diagonal variances) adds its normalised negative log density to complexity. MAP is in the prior’s declared density coordinates; no Jacobian for trainable Cholesky/log coordinates is added. Without a prior, a configurable covariance eigenvalue floor (default 1e-6) applies. The hard floor has zero derivative in saturated directions and can trap them. The prior replaces the floor. See noise and gaussian for formulas and limitations.

The LP-002 parameter-AD APIs and Mycelium’s parameter_step consume the new nested ps.noise tree. Optional ForwardDiff and ChainRulesCore extensions differentiate the spectral floor, including repeated eigenspaces on a smooth branch. A noise prior is charged once per local energy: shared-parameter dataset objectives must count it once per dataset, rather than accidentally once per sample.

Verification results — 2026-10-26

lib/Lenticulum.jl/test/learnable_noise.jl contributes 115 passing checks, with ForwardDiff and Zygote compared to central finite differences and densities compared to Distributions.jl. It checks:

  • MLE: eight balanced linear-Gaussian observations with known have residual second moment . Averaging parameter_step gradient-descent candidates from the same parameters is a batch step on the mean sample free energy. After 220 steps, full learning recovers and diagonal learning recovers , with atol=2e-6 under both backends. The affine parameters remain at their known values.
  • Variance inflation: pointwise fit decreases along increasingly inflated noise covariances; noise-only gradient descent grows scalar variance beyond 100 after 1000 steps. The normalised free energy grows along the inflation sequence instead.
  • Exact fit: full and diagonal noise reach the configured covariance floor 0.05. With one zero-residual observation and a Wishart prior, the precision converges to (atol=2e-6). The diagonal log-normal case converges to (atol=1e-9).
  • AD and inference: new terms agree with finite differences, including rotated eigenspaces with partial spectral clipping and repeated eigenvalues away from the kink. Learned covariance enters forward/backward messages, all four Dirac/Gaussian endpoint combinations, and GaussianPrior graph parameter updates.

The package runs used julia --project=<package> -e 'using Pkg; Pkg.test()' (root project . for Lenticulum). Wall times include package setup and compilation; jobs ran concurrently, so these are verification timings, not performance benchmarks.

suitepassing assertionswall time
Lenticulum (including LP-005)894232.41 s
ImplicitLayers (test dependant of Lenticulum)1195505.06 s
LenticulumCore31653.75 s
Mycelium49412.52 s

julia --project=docs docs/make.jl passed in 86.85 s, producing 26 API HTML pages and building the vault. The standalone docs/site/build.sh passed in 11.18 s with 248 HTML pages and no KaTeX errors; after recording these results it was rebuilt successfully in 7.78 s. Both used QUARTZ_DIR=/tmp/lp005-quartz, a disposable copy with its checkout/clean restore hook removed, and a temporary npm cache. No checkout/clean/reset/stash/commit command was run. Generated bibliography and release-metadata files retained their pre-existing contents.

Remaining scope: device execution, higher-order AD and other backends are not established. A hard spectral floor has a kink and can trap saturated directions; the prior alternatives avoid that floor. This resolves LP-005 for the tested CPU first-derivative API, not posterior inference over noise parameters or exposed parameter channels (see L-SPEC-003 Exposed Parameters and Energy Channels).