Computational Information Geometry Wonderland

The paper I would like to quickly present today is

Divergence function, duality, and convex analysis (Neural Computation 2004)

The strong points of the paper is to unify the three most famous families of divergences: Bregman, Csiszar alpha-div, and Burbea-Rao from the view point of differential geometry. Bregman divergences yield the usual dually flat space with natural/expectation biorthogonal coordinate systems

Everything is built from the fundamental convex inequality: Jensen's remainder or Jensen's difference of a strictly convex function. Metrization and connections are derived for these three parameterized families.

Overall, this lengthy paper is a must to get clear ideas about the links of families of divergences with characteristics of the differential geometries.

The full paper is at: neco.mitpress.org/cgi/reprint/16/1/159.pdf

You'll learn about the wording "biduality" embodying both referential and representational duality. The article will also consider infinite-dimensional functional space (vs vector-space)

Jensen's inequality is used in many different settings. For example, in the computations of the expectation too: