overview

Section G of the Feature Backlog: control as inference, owned scalarisations in games, open games (asked for 2026-11-16). Open rows only; a row that is done moves to Backlog Done 2026-11 with its id. Status, priority, ids and the column needs are explained in Feature Backlog, which also holds the dependency graph.

Sources: original to this repository; the sources of each row are in its source column.

Daniel asked how far the library is from reinforcement learning “fully realized”: compositional multiplayer open games, explicit best responses, Nash equilibria. The rows below are the answer as a list, from a reading of the code and of GANs as Two Factors §4–§5 and The Two-Part Diagram §3 and §6; nothing was run or designed for them. They come in three tiers. G1–G4 are one agent and one free energy (the cooperative row of The Two-Part Diagram): new factors and one message rule on the engine as it is. G5–G8 build on P16 (owned scalarisations): once a parameter block descends its own scalarisation there is no one , and three things that rely on it change. G9 and G10 are a layer beside the statistical game, in which the backward pass carries a best response; it would be a package of its own with its own solve entry, so that infer! and its promises stay as they are. Whether a Polarity (or F8) can carry ownership, and whether a cluster node can serve as the solver of a stage game, were not checked.

idfeaturestatusneedsprioritysourcewhy
G1Reward and policy factors for control as inference: a reward (optimality) factor on state and action, a policy factor, and a constructor that unrolls a horizon with tied dynamics. The backward message on a state is then the soft value function (arXiv:1805.00909). Oracles: LQR against the Riccati recursion on a linear-Gaussian chain (The Linear Gaussian Chain), a finite MDP against soft value iterationproposed—lowDaniel, 2026-11-16; The Two-Part Diagram §3, §5expected to need no change to the engine, not tried: one agent and one free energy is what the Bethe machinery computes. Goes better after I9 (nested plates, stateful tied factors), which would make the horizon a plate
G2A posterior that keeps the model’s dynamics: in G1’s graph only the policy is free in the variational family, the transition factors are held at the model’sproposedG1lowDaniel, 2026-11-16with stochastic dynamics, plain inference given “optimal” also infers favourable transitions, so the agent plans as if it controlled the noise (risk-seeking); see arXiv:1805.00909 on this. With deterministic dynamics the two agree
G3A max-product message rule (an AbstractMessageRule beside BeliefPropagation, MeanField, ExpectationPropagationRule, MixtureReduction), for TableFactor first: the hard Bellman backup, and MAP assignments (Viterbi) in general. Oracle: value iteration and enumeration on small tablesproposed—lowDaniel, 2026-11-16; RQ-008 Overlays, Mixtures and Tropical Limitsno rule in lib/ maximises: every message is a sum-product one (searched 2026-11-16). It is the limit of G1, and of G8 for games
G4The Bellman residual as a loss for a learned value function: with the target held fixed (a stop-gradient wire) and a target copy kept as factor stateproposedL3lowDaniel, 2026-11-16temporal-difference learning descends a semi-gradient, which is not the gradient of any one objective; without L3 the graph differentiates the target too. Goes better after U1 (minibatches, for replay). Actor–critic beyond this is the general bilevel row of The Two-Part Diagram §6 and needs L9 or P16
G5The gradient of an owner’s own scalarisation through the beliefs: with P16, a block’s gradient is the total derivative of its payoff through the message fixed point (the graph adjoint or the coupled drivers), not the fixed-belief oneproposedP16lowDaniel, 2026-11-16; lib/MyceliumLearning.jl/src/gradient.jl (the envelope gradient’s docstring: the argument “is about the summed free energy”)the default gradient holds every belief fixed, which is right because beliefs and parameters descend the same . A payoff that is not has a nonzero derivative through the beliefs; dropping it gives a wrong gradient silently
G6Update rules for games that read the joint Jacobian of the players’ gradient field: extragradient, optimistic steps, symplectic gradient adjustment; with P16’s stability check as their diagnosticproposedP16lowGANs as Two Factors §4 “What simultaneous gradient play computes”simultaneous gradient steps spiral outwards on the bilinear game for every step size (derived there). An OptimiserFactor is one node with one block’s gradient; these rules couple blocks. Goes better after L7 (train!)
G7What the readers of the one do on a graph with owners: ConvergenceReport, the tree exactness check (P13), fit_em, and the structure scores (BIC, S6) either refuse such a graph or report per player (own scalarisation, stationarity in the own block)proposedP16lowGANs as Two Factors §4 (last paragraph)with owners there is no single number that decreases and “on a tree the free energy is ” has no counterpart; these functions would report it anyway
G8Entropy-regularised games: ownership extended from parameter keys to decision variables, so that each owner’s belief over its action minimises its own free energy (expected payoff minus its own entropy, at a temperature ). The fixed points are quantal-response equilibria. Oracle: the logit equilibrium of small matrix games by a direct fixed-point solve. A first version by sampling (Daniel’s route, 2026-11-16): GibbsSampler with a conditional per owner, each decision variable drawn from of its own component with the others pinned (logit-response dynamics); oracle for it: a potential game, whose chain has the Gibbs distribution of the potential as its stationary lawproposedP16lowDaniel, 2026-11-16; lib/Mycelium.jl/src/sampling.mdthe step from P16 that reuses the most: vector energies, beliefs and message updates stay, and only who answers to which component changes. Nash equilibria are its limit (G3). Whether the per-owner updates converge is the question of G6 again, for beliefs. The sampler returns to a statistical game (one joint, one free energy) only for potential games; from the literature, not run here: in a general game the per-owner conditionals belong to no joint and the chain’s stationary law is not the quantal-response equilibrium (a product of strategies), and in a zero-sum game the chain is not reversible. Today _conditional! draws from the marginal under all adjacent factors, and the variables not sampled are integrated out under the one : both would change
G9Open games as a type beside the statistical game, in a package of its own: a forward play (a Bayesian lens, reusing BayesianLens and pushforward), a backward coutility, a best-response relation on strategy profiles, sequential and parallel composition, and an equilibrium predicate (arXiv:1603.04641; Bayesian: arXiv:1910.03656). Oracles: the pure and mixed equilibria of finite games by enumeration, and composition against the composite’s equilibria on small gamesproposed—lowDaniel, 2026-11-16; GANs as Two Factors §5–§6a best response is a relation on profiles, not a message, and an equilibrium is a fixed point of it, not a minimum of a functional; lens.jl and statistical_game.jl are built on invert with a prior, so this is a sibling, not a subtype. Computing equilibria is a solver question apart from checking them. G8 at small should agree with it where both apply. With Dirac or sample beliefs a graph is already the parametrised lens underneath an open game (play forwards, energies backwards, ps as the strategy profile; Open Game: a parametrised lens plus a best-response relation), so the play half may need no new type and the new part is the relation: the owners of P16 and what each answers to. Read from the definitions, not tried
G10Markov games: a backward sweep over G1’s unrolled horizon in which each step solves a stage game (G9, or G8 at a temperature) and passes one value per player backwards. Oracle: Shapley’s value iteration for zero-sum stage gamesproposedG1, G9lowDaniel, 2026-11-16Bellman’s backup with an owner per action; the multiplayer case of G1 and G3