The discriminator, read as what it is: a density-ratio estimator. And the reason this file matters to Lenticulum has nothing to do with generating images.
Sources: code:
ratio.jlTheory (CT-ML wiki): Bayesian Inversion · Bayesian Lens · Variational Free Energy · Lens
Graph free-energy edge directions
RatioFactor declares Mycelium.uses_cavity == true: its local free energy reads the
incoming point or particles, forms the factor belief , and averages
the negative log ratio under that belief.
Graph free-energy evaluation rejects an attached Emitting() edge with ArgumentError;
use Bidirectional() if that channel must also send messages. Missing messages on permitted
edges retain their existing local behaviour. See free_energy and
LP-014 Emitting edges drop the cavity message from the free energy.
1. The identity
Train a classifier to separate (label 1) from (label 0) with equal class priors. At the optimum
The logit of the optimal discriminator is the log density ratio. That is Mohamed &
Lakshminarayanan §3, it is why GANs work, and logratio / discriminator are the two readings
of the same number — the test suite checks they are logistic transforms of one another against
a closed-form Gaussian pair.
2. Why the energy is
. So the factor’s energy is the integrand of a KL divergence, and minimising it over whatever produced pushes toward . That is the generator’s objective, expressed as a factor energy rather than as a training loop.
3. AmortisedInversion is literally true here
lens.md describes AmortisedInversion(net) as ” realised by a learned network — a VAE
encoder… That a factor has two independently parametrised halves is the structural reason
a factor cannot be a Lux layer.”
This is the first factor in the project where that description is exact. The discriminator is trained separately from whatever produced the samples it scores; the inversion genuinely is a network with its own parameters. Every other factor’s “inversion” is arithmetic on the forward model.
4. The payoff: a route around the combine gap
RatioFactor emits a LogPotentialBelief with log-potential .
Mycelium.combine pools it with a particle cavity by
reweight is the explicit posterior operation, also used by invert. The likelihood
message consumes neither the cavity nor target-channel inputs. Two ratio messages pool by
adding log-potentials; pooling their product with a particle prior applies each ratio once.
test/conformance.jl checks both factor insertion orders on a one-variable graph.
Weights are normalised in log space with a max-shift. Mycelium warns on severe ESS collapse;
callers of reweight can inspect effective_sample_size themselves. Applying the same
ratio twice squares it, as multiplying the same evidence twice should. This does not
violate associativity or commutativity of products. The previous note confused those laws
with idempotence and incorrectly used that as a reason to keep ratios out of combine.
The Gaussian oracle in test/runtests.jl draws 20 000 samples from
, reweights by the exact ratio against
, and checks the resulting mean and variance.
5. The energy is estimated, and nothing accounts for that
Every other factor in this project computes its energy. This one fits it: comes from a network trained on a finite sample, so the energy carries an estimation error of unknown sign and size.
Bayesian Lens.md licenses inexact inversions — “the quality of the choice is what the
free energy measures.” This is a different animal: an inexact energy, and the free energy
has no term for it because the free energy is the thing being estimated. You cannot measure
the error of your ruler with the ruler.
Recorded as a genuine gap in the framework’s accounting, not merely in this file. A candidate fix is the discriminator’s own validation loss as a confidence term, which nothing computes.
6. Implementation difficulties
Particle matrices use one column per sample, as in Mycelium’s belief algebra. Reweighting,
moments, ESS and generator pushforward share that convention with vector-of-vectors input;
the tests compare both representations. Zero prior weights stay zero under importance
reweighting: their log weights are -Inf, not a positive epsilon floor.
6.1 Likelihood messages and posterior inversion
The former particle message included its cavity (LP-016). The outgoing message now carries
only a log-potential; likelihood_message and factor_message do not read the cavity or
any target-channel input. invert reweights its prior and preserves Dirac clamps.
The callable captures the supplied parameters and network state. Message construction does not evaluate the network. Each density evaluation starts with that captured state and drops the returned state; use inference-mode state for deterministic scoring. Captured mutable parameters and state must not be mutated while a message is in use. Callable identity is only a conservative convergence check, not a numerical comparison of potentials.
6.2 Parametric products need more structure
invert supports SampleBelief, Dirac clamps, and absent/trivial priors (the latter retain
an uninformative posterior placeholder). Parametric beliefs throw. Mycelium likewise refuses
Gaussian, categorical, and mixture products with an arbitrary log-potential rather than
silently sampling or projecting. Even categorical labels need an explicit interpretation as
points; this implementation does not assume one.
AbstractPotentialBelief leaves room for richer likelihood types carrying gradients,
parametric structure, or batched evaluation and their own exact product rules.
6.3 No training, so logratio is only as good as what you hand it
This package scores; it does not fit. An untrained net gives a meaningless ratio and
everything downstream is confidently wrong, with no diagnostic. The Gaussian oracle in the
tests exists precisely because a perfectly trained discriminator is the only one whose
output can be checked.
Related: generator, AdversarialFactors, Implicit Generative Models, GANs as Two Factors, messages
Graph energy and game quantities (LP-016)
messages_into in Mycelium supplies incoming variable→factor messages , not variable
marginals. For the unary ratio factor the factor belief is , so
local_free_energy returns . It reweights the incoming particles
before taking that expectation. This is the Bethe energy contribution only: particle
clouds have no Lebesgue density, and no differential entropy is subtracted (LP-006).
It is not a complete continuous Bethe free energy or generally .
An absent/trivial cavity retains the zero contribution; unsupported cavity types throw.
For a Dirac prior the point remains fixed. The game methods retain energy -logratio and
the zero point-entropy convention. Particle game methods still throw ArgumentError:
no reference measure for their entropy has been chosen.
test/conformance.jl checks , , hence .
On equally weighted particles the posterior weights are
and the energy contribution is .
The outgoing potential, target exclusion, posterior inversion, and graph marginal with two
ratio factors are tested separately. The three former LP-016 broken assertions now pass.
Exact-law audit (2026-10-27, LP-009)
Non-Dirac game priors remain explicitly unsupported. An arbitrary discriminator and proper
Gaussian prior do not supply an analytic normaliser or entropy of the reweighted density.
SampleBelief supplies an empirical law, not a Lebesgue density; substituting finite-weight
Shannon entropy would change the reference measure (LP-006). Even the test’s analytic
Gaussian ratio is an arbitrary Lux layer to the public API, with no certified quadratic
log-potential contract. No Gaussian law is inferred from evaluations. Existing Dirac games
and extrinsic particle graph contributions remain checked in test/conformance.jl; no new
approximation is presented as exact.
LP-018 overflowing finite particle weights (2026-10-29)
Both ESS and moment/energy weights divide by the largest finite particle weight before
summing. Equal weights [1e308,1e308] therefore give mean 2, covariance 1 and ESS 2
on particles {1,3}. This also prevents a false disjoint-support error in ratio energy.
Log-weight normalisation already subtracts the maximum and needed no change.
Scaling uses non-mutating division so tuple weights remain supported as well.
Evidence: test/lp018.jl, “LP-018.9 finite overflowing particle weights”, including an
independent constant log-ratio energy of -2.