Computational Information Geometry Wonderland

Let us give some examples of information manifolds:

• Statistical manifolds (parametric distributions),
• Neural manifolds (Boltzmann machines with fixed topology, i.e., number of nodes),
• ARMA(p,q) time-series manifolds (e-flat=-1-flat)

Strictly speaking, geometrizing information-theoretic problems does not provide a more powerful framework in theory. This is because synthetical and analytical geometries are equivalent. Informally, that means that we can do geometry by algebraic equations.

However, geometrizing problems help grab intuition on the problem at hand. Geometry also yields novel notions to mathematical theories. For example, let us cite the two curvature notions in statistics: exponential and mixture curvatures emanating from conjugate connections. So although synthetical geometry provides the same power as analytical geometry, the third-order asymptotic theory of statistics has been obtained so far only from synthetical information geometry.

Dual differential geometries are also useful to tackle information-theoretic problems such as

• Multiterminal problems met in information theory,
• Linear programming problems (e.g., continuous Karmarkar inner method walking along the m-geodesic),
• Clustering (negative entropy and dual Legendre log-normalizer conjugate for soft/hard clustering).

Frank.