open-problem

Status: mitigated (2026-11-04). An output channel can now be declared, and with it the factor is a normalised conditional density. free_energy_gradient warns once when the degenerate parameterisation meets data (Daniel chose a warning over a refusal). Found from Daniel’s question how a switching factor is kept from “accepting more points than are on the relation”; measured in lib/Lenticulum.jl/test/lp022.jl. Related: Switching Constraint Factors, LP-020 The fixed-message parameter gradient is not the free-energy gradient, constraint, switching, regime.

Sources: original repository analysis; code: lib/Lenticulum.jl/src/constraint.jl (_coeffs, _output_coeff), lib/Lenticulum.jl/src/switching.jl, lib/Lenticulum.jl/src/regime.jl; tests: lib/Lenticulum.jl/test/lp022.jl

Theory (CT-ML wiki): Variational Free Energy

Problem. A constraint factor (LinearConstraintFactor, SwitchingConstraintFactor, RegimeFactor) is a Gaussian in its residual :

That density is normalised over , not over any of the channels. Data enter a Mycelium graph as point messages, and the free energy is of the product of factors at the data (latent variables integrated out). No partition function over the observed variables normalises it. So a factor is a density of the data only if its parameterisation makes it one. With every coefficient learnable nothing does, and the free energy rewards relations that accept every point:

  1. , . Every point is on the relation, , and , the smallest value it can take. In lib/Lenticulum.jl/test/lp022.jl (200 points of , ) the least-squares relation has , and has . At the least-squares fit the gradient in is , not 0, so gradient descent shrinks the relation.
  2. , (learnable noise, RegimeFactor(...; learn_noise = true)): , so changes by and has no lower bound. Measured in the same test: halving the scale lowers by to .

Neither is a bug in the messages or the free energy, which are exact for the factor as defined. The factor is a relation, not a model of its channels. The fits in the repository were well-posed only because the coefficient of the data channel was held fixed by hand (lib/Lenticulum.jl/examples/structure_edits.jl: “the sum node’s coefficients are not updated”; lib/Lenticulum.jl/examples/regime_lines.jl held the coefficient of at 1). The diffusion models do not have the problem, since their objective bounds the log-density of a normalised model.

When it is fine. If the data channel’s coefficient is fixed and invertible, the change of variables is fixed, and is the conditional density of given the other channels up to the constant . Mixtures then cannot add mass: each component is normalised, and the weights are a softmax. GaussianFactor is this case by construction, with and . A rank-deficient relation (fewer equations than observed coordinates) is flat along . That is fine when those coordinates are conditioned on, but then is not a density of all the data, and two graphs can be compared by only when they normalise the same observed coordinates the same way.

Mitigation (2026-11-04)

output = :y on the three constraint factors (with dims[:y] == m) fixes the coefficient of y at output_coeff (the identity, or an invertible matrix; for RegimeFactor the same in every regime). The coefficient is dropped from the parameters: ps.A omits it, and passing it is an ArgumentError. Every method reads the coefficients through _coeffs(f, ps), which merges the fixed one in, so messages, free energies, cluster forms and partial_sums see the full relation. lib/Lenticulum.jl/test/lp022.jl checks:

  • the free energy equals ;
  • gradient descent from 0 reaches least squares to ;
  • a fixed coefficient of 2 gives the density up to ;
  • a switching factor with an output in a cluster gives by the closed form, and its message to the outcome matches outside a cluster;
  • the refusals.

Open

  • A warning, not a refusal (added 2026-11-04, Daniel’s choice). free_energy_gradient warns once per session when a node whose factor learns_unnormalised (the three constraint factors without output) is learnable, has no frozen A, and has a channel on a variable observed by a DataFactor. Inside a cluster node it names the clustered factor (c/rel). Data that arrive otherwise (pinned variables, point messages from other factors) are not detected, and a factor whose coefficients are fixed by other means (a parameter step that skips A, as lib/Lenticulum.jl/examples/structure_edits.jl does) still warns. lib/Lenticulum.jl/test/lp022.jl tests the detection (_unnormalised_nodes), since the warning itself fires once per session.
  • Several data channels. One output channel per factor. A factor on two data channels whose joint density is meant needs both coefficients fixed, or a model written as a chain of conditionals.
  • partial_sums keeps a sum factor’s output and refuses to move the output channel into a partial sum. This is not tested.
  • The same issue in other families (2026-11-04): unnormalised ToricFactors on observed states (now warned too; lib/AlgebraicFactors.jl/test/lp022.jl) and DEQFactors trained by their residual energy with the state observed (not guarded). Every family is gone through in Why Learned Relations Do Not Collapse.
  • A second fix (2026-11-05): OverlayFactor(:rel => f; normalise = :y) normalises a relation in its data channel, , with every coefficient learnable; is refused and maximum likelihood is least squares with its variance (lib/Lenticulum.jl/test/overlay.jl).