Computational Information Geometry Wonderland

The seminal paper of Rao written before he joined Cambridge for his PhD is available online at:

Breakthroughs in Statistics
page 235 is a reprint of:
Rao, Calyampudi (1945). "Information and the accuracy attainable in the estimation of statistical parameters". Bulletin of the Calcutta Mathematical Society 37: 81?89.

There we find three essential results:

• Cramer-Rao bound

• Population space and riemannian geometry using the Fisher information metric as the quadratic differential form.

• Test of significance (and classification)

Information geometry has since then spreaded, with the work of Chentsov on alpha-connections and its investigation by Amari. Historically, the space of distributions, was called the "population space".

The Rao distance for 1D normals is also given.

Frank.

Rao's Distance Measure, by Colin Atkinson and Ann F. S. Mitchell, 1981 Indian Statistical Institute.

Rao's distance is the Riemannian geodesic distance induced by the Fisher information matrix as the tensor. Computing Rao's distance for given parametric distributions (let d be the number of parameters), thus involved to compute explicitly the geodesic. The distance is the length of that geodesic, the sum of its infinitesimal elements along the shortest path curve. It is thus quite complicated to compute in practice the exact Rao's distance as we need to solve the differential equation of the geodesic stated by the Euler-Lagrange equations. In this paper, three different approaches are proposed;

• Classic Euler-Lagrange equations (d second order differential equations)
• Hamilton's equations (2d first order differential equations)
• Hamilton-Jacobi equations (nonlinear partial differential equation)

For uniparameter distribution, the geodesic length (Rao's distance) becomes easy, and the list of such distances are given for Poisson, binomial, exponential, chi-squared (gamma). For multi-parameter distributions, it becomes more difficult. The two classical examples are the normal distributions (which Rao geometry is hyperbolic geometry) and the multinomials (which Rao geometry is the spherical geometry). For same-mean multivariate normals, the Rao distance is given (an unpublished result due to Jensen in 1976). One problem we typically face with this distance computation is to know whether it admits a closed-form equation or not.

Frank.