Status: mitigated (2026-11-04). An
outputchannel can now be declared, and with it the factor is a normalised conditional density.free_energy_gradientwarns once when the degenerate parameterisation meets data (Daniel chose a warning over a refusal). Found from Daniel’s question how a switching factor is kept from “accepting more points than are on the relation”; measured inlib/Lenticulum.jl/test/lp022.jl. Related: Switching Constraint Factors, LP-020 The fixed-message parameter gradient is not the free-energy gradient, constraint, switching, regime.
Sources: original repository analysis; code:
lib/Lenticulum.jl/src/constraint.jl(_coeffs,_output_coeff),lib/Lenticulum.jl/src/switching.jl,lib/Lenticulum.jl/src/regime.jl; tests:lib/Lenticulum.jl/test/lp022.jlTheory (CT-ML wiki): Variational Free Energy
Problem. A constraint factor (LinearConstraintFactor, SwitchingConstraintFactor,
RegimeFactor) is a Gaussian in its residual :
That density is normalised over , not over any of the channels. Data enter a Mycelium graph as point messages, and the free energy is of the product of factors at the data (latent variables integrated out). No partition function over the observed variables normalises it. So a factor is a density of the data only if its parameterisation makes it one. With every coefficient learnable nothing does, and the free energy rewards relations that accept every point:
- , . Every point is on the relation, , and , the
smallest value it can take. In
lib/Lenticulum.jl/test/lp022.jl(200 points of , ) the least-squares relation has , and has . At the least-squares fit the gradient in is , not 0, so gradient descent shrinks the relation. - , (learnable noise,
RegimeFactor(...; learn_noise = true)): , so changes by and has no lower bound. Measured in the same test: halving the scale lowers by to .
Neither is a bug in the messages or the free energy, which are exact for the factor as defined.
The factor is a relation, not a model of its channels. The fits in the repository were well-posed
only because the coefficient of the data channel was held fixed by hand
(lib/Lenticulum.jl/examples/structure_edits.jl: “the sum node’s coefficients are not updated”;
lib/Lenticulum.jl/examples/regime_lines.jl held the coefficient of at 1). The diffusion models do not have the
problem, since their objective bounds the log-density of a normalised model.
When it is fine. If the data channel’s coefficient is fixed and invertible, the change of
variables is fixed, and is the conditional density of given the other channels up to the
constant . Mixtures then cannot add mass: each component is normalised, and the
weights are a softmax. GaussianFactor is this case by construction, with and . A
rank-deficient relation (fewer equations than observed coordinates) is flat along . That is
fine when those coordinates are conditioned on, but then is not a density of all the data, and two
graphs can be compared by only when they normalise the same observed coordinates the same way.
Mitigation (2026-11-04)
output = :y on the three constraint factors (with dims[:y] == m) fixes the coefficient of y at
output_coeff (the identity, or an invertible matrix; for RegimeFactor the same in every
regime). The coefficient is dropped from the parameters: ps.A omits it, and passing it is an
ArgumentError. Every method reads the coefficients through _coeffs(f, ps), which merges the fixed
one in, so messages, free energies, cluster forms and partial_sums see the full relation.
lib/Lenticulum.jl/test/lp022.jl checks:
- the free energy equals ;
- gradient descent from 0 reaches least squares to ;
- a fixed coefficient of 2 gives the density up to ;
- a switching factor with an output in a cluster gives by the closed form, and its message to the outcome matches outside a cluster;
- the refusals.
Open
- A warning, not a refusal (added 2026-11-04, Daniel’s choice).
free_energy_gradientwarns once per session when a node whose factorlearns_unnormalised(the three constraint factors withoutoutput) is learnable, has no frozenA, and has a channel on a variable observed by aDataFactor. Inside a cluster node it names the clustered factor (c/rel). Data that arrive otherwise (pinned variables, point messages from other factors) are not detected, and a factor whose coefficients are fixed by other means (a parameter step that skipsA, aslib/Lenticulum.jl/examples/structure_edits.jldoes) still warns.lib/Lenticulum.jl/test/lp022.jltests the detection (_unnormalised_nodes), since the warning itself fires once per session. - Several data channels. One output channel per factor. A factor on two data channels whose joint density is meant needs both coefficients fixed, or a model written as a chain of conditionals.
partial_sumskeeps a sum factor’s output and refuses to move the output channel into a partial sum. This is not tested.- The same issue in other families (2026-11-04): unnormalised
ToricFactors on observed states (now warned too;lib/AlgebraicFactors.jl/test/lp022.jl) andDEQFactors trained by their residual energy with the state observed (not guarded). Every family is gone through in Why Learned Relations Do Not Collapse. - A second fix (2026-11-05):
OverlayFactor(:rel => f; normalise = :y)normalises a relation in its data channel, , with every coefficient learnable; is refused and maximum likelihood is least squares with its variance (lib/Lenticulum.jl/test/overlay.jl).