implementation derivation

ToricFactor enumerates a finite integer-statistic exponential family. Sum-product sites and joint factor entropies are exact finite sums. Tree BP and its Bethe free energy are compared with independent enumeration in ../test/runtests.jl.

Sources: derivation from Toric Models as Factors and L-SPEC-004 Algebraic Factors Toric and Grassmannian §1.1–§1.5; code: toric.jl, ../test/runtests.jl, lib/Mycelium.jl/src/free_energy.jl.

Theory (CT-ML wiki): Statistical Game · Bayesian Inversion · Variational Free Energy

Representation and ordering

For alphabet sizes (s₁, …, sₙ), columns follow CartesianIndices: variable 1 varies fastest. In a 2×3 table the columns are (1,1), (2,1), (1,2), (2,2), (1,3), (2,3). Channels carry their alphabet size as their space. Names must be distinct Symbols, alphabets positive integers, and the product must fit in Int. The constructor copies the design matrix and label vectors. A zero-row matrix represents a uniform potential.

independence returns the matrix for singleton generators. hierarchical returns one indicator block per supplied generator, retaining redundant rows. Generators may name channels or give their indices. Within a block, the first listed variable varies fastest; reversing a generator permutes its rows. An empty generator gives a constant row and an empty generator collection gives no rows. These constructors do not split a graph into separate factors.

Default labels are 1:sᵢ. Explicit labels are a tuple of vectors, unique within each channel. Diracs and pointwise energy tuples use isequal to match actual labels; there is no integer-as-index fallback. An unlabelled categorical input is positional. A labelled one must have the same complete support, and can be reordered. Outputs use the factor’s canonical label order, with labels = nothing for default alphabets. Mycelium’s generic categorical combine still requires identical label order, so factors and priors sharing a graph variable must use that same output support and order.

Natural parameters are a Float64 zero vector initially, stored as ps.η; the state is empty and passes through unchanged. Redundant parameters are retained. Normalised models agree iff A' * (η - η′) is constant. No gauge reduction, integer kernel or augmented matrix is needed for these finite sums: they work even when the constant vector is absent from the row space of A.

Messages and conformance

For target t, the message is the normalised version of

Missing keys, nothing, missing, TrivialBelief() and Latent() have unit weight. Mycelium supplies only informative channels in inputs; the other channels are latent. All assignments with one Unobserved() and any combination of Observed()/Latent() elsewhere are declared. The implementation excludes the target input and prior, even if supplied. Uniform categorical inputs differ from unit weights only by a constant, so their normalised messages agree. The factor’s normalising constant also cancels in a message.

Computation uses shifted log weights and log-sum-exp, including -Inf for impossible states. Common logits are removed before incoming evidence is added; normalised energies use shifted log probabilities directly, avoiding cancellation against a large log partition. Zero joint support raises ArgumentError. The full state table is enumerated; the cost includes computing A'η and visiting every state and channel. It is exponential in arity. This is CPU enumeration, without GPU or AD guarantees.

The factor implements Mycelium’s direct-message interface. It does not implement assemble/invert: the core lens contract asks for a joint posterior including latent channels, whereas a categorical site on one target has marginalised those channels. The conformance checker’s explicit direct_message = true path requires independently computed site and pooled-posterior fixtures. It checks those values, excludes both the target input and prior, and shares all parameter, channel, energy, game and graph checks with the default path. Existing factors still take the unchanged lens checks. There is no forward generative model or correlated joint latent posterior API.

Factor and game free energy

Given variable-to-factor messages, form

local_free_energy returns this scalar and the unchanged state. It charges the energy of this factor alone, without the negative log of the incoming messages. The energy is -η'aₓ, plus the global log partition when normalised = true. The entropy is the joint factor entropy, including correlations, with zero probabilities contributing zero.

Mycelium adds sum((degree(v)-1)H(bᵥ)) separately. Thus the factor terms subtract each factor entropy once, and the variable terms correct the repeated variable entropies. The test graph has five variables, three toric factors and two priors, optionally an observation; every marginal and the total Bethe energy are checked against the joint enumeration. Prior edges are bidirectional so their local factor beliefs receive the cavity messages too. ToricFactor declares Mycelium.uses_cavity == true because its joint belief uses those incoming messages. Mycelium now rejects Emitting() edges on any factor with that trait before evaluating its local free energy; use Bidirectional() when the factor also sends messages. This closes LP-014 Emitting edges drop the cavity message from the free energy with option 2. The regression “LP-014: a prior on an Emitting edge” checks that the prior graph’s marginals still match enumeration, free energy throws ArgumentError, and the Bidirectional version matches .

Game-level quantities require normalised = true. π can be a NamedTuple of channel beliefs, and y a disjoint NamedTuple of beliefs or point labels. A single prior belief is assigned to the sole channel omitted from y; nothing means no evidence. Other unspecified channels are latent. Overlapping or unknown channel names throw rather than silently overwrite evidence. free_energy uses the same joint factor belief as above; entropy returns its positive Shannon entropy. With no evidence, free energy is zero.

The original factor testsets in ../test/runtests.jl check constructor failures, hand-written design columns, pointwise energies, all evidence kinds for every target at arities two and three, local/game quantities, tree inference and the common interface. The references compute ordinary products and finite sums independently of the log-domain implementation.

Parameter gradient at a fixed belief (LP-020, 2026-11-02)

The graph parameter gradient differentiates each factor at its belief held fixed, which at a belief-propagation fixed point is the derivative of the Bethe free energy (the envelope theorem; free_energy, “Fixed beliefs, not fixed messages”). Mycelium.factor_belief returns the joint table above as a ToricBelief, and Mycelium.belief_free_energy re-scores it at the current :

Before, the derivative also moved through its dependence on , with the messages held fixed; with categorical messages that is a different number. ../test/lp020.jl checks, on the five-variable tree above with two interaction rows added to each design (so is not a pure gauge), normalised or not and with clamped or not: belief_free_energy equals local_free_energy at the frozen belief to ; the graph gradient matches central differences of the converged to with reference differences, ForwardDiff and Zygote; and the old fixed-message derivative differs from it by more than in norm.

fit_mle, positive inmodel and supplied binomial_residuals are implemented in fit.jl; fit records the gauge, convergence/existence distinction and numerical limitations. No LP solver, Markov basis, manifold or polynomial inference is implemented. Label matching and finite summation are exact; large tables remain expensive.

Learned on observed states (2026-11-04, LP-022)

Unnormalised (the default), the potential is not a distribution of its channels: on observed states has no lower bound, and the free-energy gradient lacks the partition’s term . test/lp022.jl: with two binary variables observed and , for ; normalised, . Mycelium.learns_unnormalised(f) = !f.normalised, so free_energy_gradient warns once for an unnormalised toric factor on DataFactor variables. See Why Learned Relations Do Not Collapse.