paper

Compositional Information Geometry of Probabilistic Programs — Jacek Karwowski, Frank Nielsen & Sam Staton (2026). arXiv:2610.17886 (v1, PDF).

Assigns to every program of a first-order probabilistic language a degenerate Riemannian metric on its inputs, pulled back from its outputs: a contravariant lax monoidal functor that gives the Fisher metric for statistical models and the ordinary pullback for deterministic programs. The metric depends on the program’s presentation; a three-level inequality relates it to the Wasserstein–Fisher–Rao metric of the realised distributions. Weighted programs, continuous random variables (diffeological quasi-Borel spaces) and natural gradient are treated too.

Sources: the paper, arXiv:2610.17886v1, checked against the arXiv listing. Index: Papers.

Key definitions and results

  • §3: the language L (flip with log-odds, let, if); Definitions 2–8: trace presentations, the copy-discard category FinTrace; Remark 4: not Markov
  • Definitions 10–13, Theorem 1: jet traces; the semantics is a strong monoidal functor
  • Definitions 15–16, Propositions 8–9: Fisher metric and expected tangent pullback; the chain rule g^{G∘F} = g^F + F♯ g
  • Definitions 17–20, Theorem 2: the functor 𝒜 into AffCone, lax symmetric monoidal; the score-velocity metric
  • Corollary 2, Example 1: invariant under trace isomorphism, not extensional
  • Theorem 3: the universal space of programs with a fixed trace space as a hypersurface in a product of cones
  • §6: weighted programs (observe, score), normalisation (Remark 13)
  • Definition 33, Theorem 4: diffeological quasi-Borel spaces, complete, cocomplete, cartesian closed
  • Theorems 9–10, Corollary 8: the trace data-processing inequality and G_WFR ⪯ G_out ⪯ G_F
  • Theorem 12, Definition 56: experiments and natural gradient for degenerate metrics
  • §9: loops, higher-order functions, a Solomonoff–Jeffreys prior, trans-dimensional RMHMC

Concept notes

Information Geometry of Probabilistic Programs, Fisher Information Metric, Natural Gradient, Quasi-Borel Space, Diffeological Space