Compositional Information Geometry of Probabilistic Programs — Jacek Karwowski, Frank Nielsen & Sam Staton (2026). arXiv:2610.17886 (v1, PDF).
Assigns to every program of a first-order probabilistic language a degenerate Riemannian metric on its inputs, pulled back from its outputs: a contravariant lax monoidal functor that gives the Fisher metric for statistical models and the ordinary pullback for deterministic programs. The metric depends on the program’s presentation; a three-level inequality relates it to the Wasserstein–Fisher–Rao metric of the realised distributions. Weighted programs, continuous random variables (diffeological quasi-Borel spaces) and natural gradient are treated too.
Sources: the paper, arXiv:2610.17886v1, checked against the arXiv listing. Index: Papers.
Key definitions and results
- §3: the language L (flip with log-odds, let, if); Definitions 2–8: trace presentations, the copy-discard category FinTrace; Remark 4: not Markov
- Definitions 10–13, Theorem 1: jet traces; the semantics is a strong monoidal functor
- Definitions 15–16, Propositions 8–9: Fisher metric and expected tangent pullback; the chain rule g^{G∘F} = g^F + F♯ g
- Definitions 17–20, Theorem 2: the functor 𝒜 into AffCone, lax symmetric monoidal; the score-velocity metric
- Corollary 2, Example 1: invariant under trace isomorphism, not extensional
- Theorem 3: the universal space of programs with a fixed trace space as a hypersurface in a product of cones
- §6: weighted programs (observe, score), normalisation (Remark 13)
- Definition 33, Theorem 4: diffeological quasi-Borel spaces, complete, cocomplete, cartesian closed
- Theorems 9–10, Corollary 8: the trace data-processing inequality and G_WFR ⪯ G_out ⪯ G_F
- Theorem 12, Definition 56: experiments and natural gradient for degenerate metrics
- §9: loops, higher-order functions, a Solomonoff–Jeffreys prior, trans-dimensional RMHMC
Concept notes
Information Geometry of Probabilistic Programs, Fisher Information Metric, Natural Gradient, Quasi-Borel Space, Diffeological Space