Section G of the Feature Backlog: control as inference, owned scalarisations in games, open games (asked for 2026-11-16). Open rows only; a row that is done moves to Backlog Done 2026-11 with its id. Status, priority, ids and the column needs are explained in Feature Backlog, which also holds the dependency graph.
Sources: original to this repository; the sources of each row are in its source column.
Daniel asked how far the library is from reinforcement learning “fully realized”: compositional
multiplayer open games, explicit best responses, Nash equilibria. The rows below are the answer as
a list, from a reading of the code and of GANs as Two Factors §4–§5 and The Two-Part Diagram
§3 and §6; nothing was run or designed for them. They come in three tiers. G1–G4 are one agent and
one free energy (the cooperative row of The Two-Part Diagram): new factors and one message
rule on the engine as it is. G5–G8 build on P16 (owned scalarisations): once a parameter block
descends its own scalarisation there is no one , and three things that rely on it change. G9
and G10 are a layer beside the statistical game, in which the backward pass carries a best
response; it would be a package of its own with its own solve entry, so that infer! and its
promises stay as they are. Whether a Polarity (or F8) can carry ownership, and whether a cluster
node can serve as the solver of a stage game, were not checked.
| id | feature | status | needs | priority | source | why |
|---|---|---|---|---|---|---|
| G1 | Reward and policy factors for control as inference: a reward (optimality) factor on state and action, a policy factor, and a constructor that unrolls a horizon with tied dynamics. The backward message on a state is then the soft value function (arXiv:1805.00909). Oracles: LQR against the Riccati recursion on a linear-Gaussian chain (The Linear Gaussian Chain), a finite MDP against soft value iteration | proposed | — | low | Daniel, 2026-11-16; The Two-Part Diagram §3, §5 | expected to need no change to the engine, not tried: one agent and one free energy is what the Bethe machinery computes. Goes better after I9 (nested plates, stateful tied factors), which would make the horizon a plate |
| G2 | A posterior that keeps the model’s dynamics: in G1’s graph only the policy is free in the variational family, the transition factors are held at the model’s | proposed | G1 | low | Daniel, 2026-11-16 | with stochastic dynamics, plain inference given “optimal” also infers favourable transitions, so the agent plans as if it controlled the noise (risk-seeking); see arXiv:1805.00909 on this. With deterministic dynamics the two agree |
| G3 | A max-product message rule (an AbstractMessageRule beside BeliefPropagation, MeanField, ExpectationPropagationRule, MixtureReduction), for TableFactor first: the hard Bellman backup, and MAP assignments (Viterbi) in general. Oracle: value iteration and enumeration on small tables | proposed | — | low | Daniel, 2026-11-16; RQ-008 Overlays, Mixtures and Tropical Limits | no rule in lib/ maximises: every message is a sum-product one (searched 2026-11-16). It is the limit of G1, and of G8 for games |
| G4 | The Bellman residual as a loss for a learned value function: with the target held fixed (a stop-gradient wire) and a target copy kept as factor state | proposed | L3 | low | Daniel, 2026-11-16 | temporal-difference learning descends a semi-gradient, which is not the gradient of any one objective; without L3 the graph differentiates the target too. Goes better after U1 (minibatches, for replay). Actor–critic beyond this is the general bilevel row of The Two-Part Diagram §6 and needs L9 or P16 |
| G5 | The gradient of an owner’s own scalarisation through the beliefs: with P16, a block’s gradient is the total derivative of its payoff through the message fixed point (the graph adjoint or the coupled drivers), not the fixed-belief one | proposed | P16 | low | Daniel, 2026-11-16; lib/MyceliumLearning.jl/src/gradient.jl (the envelope gradient’s docstring: the argument “is about the summed free energy”) | the default gradient holds every belief fixed, which is right because beliefs and parameters descend the same . A payoff that is not has a nonzero derivative through the beliefs; dropping it gives a wrong gradient silently |
| G6 | Update rules for games that read the joint Jacobian of the players’ gradient field: extragradient, optimistic steps, symplectic gradient adjustment; with P16’s stability check as their diagnostic | proposed | P16 | low | GANs as Two Factors §4 “What simultaneous gradient play computes” | simultaneous gradient steps spiral outwards on the bilinear game for every step size (derived there). An OptimiserFactor is one node with one block’s gradient; these rules couple blocks. Goes better after L7 (train!) |
| G7 | What the readers of the one do on a graph with owners: ConvergenceReport, the tree exactness check (P13), fit_em, and the structure scores (BIC, S6) either refuse such a graph or report per player (own scalarisation, stationarity in the own block) | proposed | P16 | low | GANs as Two Factors §4 (last paragraph) | with owners there is no single number that decreases and “on a tree the free energy is ” has no counterpart; these functions would report it anyway |
| G8 | Entropy-regularised games: ownership extended from parameter keys to decision variables, so that each owner’s belief over its action minimises its own free energy (expected payoff minus its own entropy, at a temperature ). The fixed points are quantal-response equilibria. Oracle: the logit equilibrium of small matrix games by a direct fixed-point solve. A first version by sampling (Daniel’s route, 2026-11-16): GibbsSampler with a conditional per owner, each decision variable drawn from of its own component with the others pinned (logit-response dynamics); oracle for it: a potential game, whose chain has the Gibbs distribution of the potential as its stationary law | proposed | P16 | low | Daniel, 2026-11-16; lib/Mycelium.jl/src/sampling.md | the step from P16 that reuses the most: vector energies, beliefs and message updates stay, and only who answers to which component changes. Nash equilibria are its limit (G3). Whether the per-owner updates converge is the question of G6 again, for beliefs. The sampler returns to a statistical game (one joint, one free energy) only for potential games; from the literature, not run here: in a general game the per-owner conditionals belong to no joint and the chain’s stationary law is not the quantal-response equilibrium (a product of strategies), and in a zero-sum game the chain is not reversible. Today _conditional! draws from the marginal under all adjacent factors, and the variables not sampled are integrated out under the one : both would change |
| G9 | Open games as a type beside the statistical game, in a package of its own: a forward play (a Bayesian lens, reusing BayesianLens and pushforward), a backward coutility, a best-response relation on strategy profiles, sequential and parallel composition, and an equilibrium predicate (arXiv:1603.04641; Bayesian: arXiv:1910.03656). Oracles: the pure and mixed equilibria of finite games by enumeration, and composition against the composite’s equilibria on small games | proposed | — | low | Daniel, 2026-11-16; GANs as Two Factors §5–§6 | a best response is a relation on profiles, not a message, and an equilibrium is a fixed point of it, not a minimum of a functional; lens.jl and statistical_game.jl are built on invert with a prior, so this is a sibling, not a subtype. Computing equilibria is a solver question apart from checking them. G8 at small should agree with it where both apply. With Dirac or sample beliefs a graph is already the parametrised lens underneath an open game (play forwards, energies backwards, ps as the strategy profile; Open Game: a parametrised lens plus a best-response relation), so the play half may need no new type and the new part is the relation: the owners of P16 and what each answers to. Read from the definitions, not tried |
| G10 | Markov games: a backward sweep over G1’s unrolled horizon in which each step solves a stage game (G9, or G8 at a temperature) and passes one value per player backwards. Oracle: Shapley’s value iteration for zero-sum stage games | proposed | G1, G9 | low | Daniel, 2026-11-16 | Bellman’s backup with an owner per action; the multiplayer case of G1 and G3 |