Computational Information Geometry Wonderland

Csiszar C_f f-divergences preserve information monotonicity, Bregman divergences B_F are canonical divergences of dually flat spaces. These two families intersect only at the Kullback-Leibler divergence.

Consider now using a parameter representation function, say k (strictly monotone). And define B_{F,k}(p||q)=B_F(k(p)||k(q)) then you can obtain alpha and beta-divergences using such an extended Bregman divergence. Furthermore, add an external divergence representation function so that C_{h,f}(p||q)=h(C_f(p||q)). Then you get Renyi, Sharma Mittal and Bhattacharyya divergences using external representations of f-divergences.
Convexity and monotonicity are two puzzling ingredients for function rewriting.

I came across the book: Pardo L: Statistical Inference Based on Divergence Measures. Chapman&Hall, London, 2006.
It is very nice to have a monography focusing on statistical inference wrt. f-divergences.

Frank.

Today, I'd like to present a COLT'99 important paper:

Additive models, boosting, and inference for generalized divergences

The paper considers Bregman divergences with additive models to build up incremental learning algorithms. More precisely, Lafferty brings to the reader attention the 1991 Csiszar's paper 1D parameterized family of divergences and shows that one member of it closely approximates the loss function of Adaboost.

The paper is worth reading, especially if you feel like learning more on the beauty of Legendre transformations and Bregman divergences. Mathematical beauty almost, since most of these transforms cannot be explicitly computed. Well, that is all for today. Short post, I apologize but I'm behind schedule. -:)

By the way, Bregman divergences are marvelous tools to by pass integral computations of Kullback-Leibler divergences, defined as: