An Elementary Introduction to Information Geometry — Frank Nielsen (2018). arXiv:1808.08271 (v2, PDF); Entropy 22(10) (2020) 1100, doi:10.3390/e22101100.
A survey of information geometry: statistical manifolds with a metric and a pair of dual connections, dually flat (Bregman) geometry of exponential and mixture families, divergences and their induced geometries, the Fisher information metric and its invariance, and applications to natural gradient, hypothesis testing and clustering.
Sources: the paper, arXiv:1808.08271v2, checked against the arXiv listing. Index: Papers.
Key definitions and results
- §2: Riemannian manifolds, affine connections, the Levi-Civita connection
- §3.2–3.7: conjugate connections, statistical manifolds, dually flat manifolds
- §3.9, eqs. 79–80: score and Fisher information matrix
- §3.10, Theorem 8: uniqueness of the Fisher information metric under Markov embeddings (Čencov)
- §4.1.2, eqs. 172–178, Property 5: natural gradient as approximate Riemannian gradient, and its invariance
- §4.1.3: natural gradient and Bregman mirror descent in dually flat spaces