Computational Information Geometry Wonderland

The Burbea-Rao centroids allows one to compute the Bhattacharyya distance of exponential families in closed forms. Moreover, by skewing the Burbea-Rao divergence, we obtain Bregman divergences in limit cases (and Kullback-Leibler divergence on exponential families). Today, at ICPR, we will present a simple algorithm to compute those Burbea-Rao centroids using the concave-convex procedure (CCCP).

All details:

• The Arxiv report: The Burbea-Rao and Bhattacharyya centroids
• The presentation slides BRcentroid-ICPR2010-Pres.pdf (20 minutes)

Frank.

All good things come in threes. So here is the third installement of the information-theoretic quizz! Suggestions welcome.
Frank.

After 3 years of work and a very careful reviewing (thanks to the reviewers), we are pleased to announce that the seminal paper "On Bregman Voronoi diagrams" is available online at the journal "Discrete and Computational Geometry" .

The title is Bregman Voronoi Diagrams (we removed applications and shortened significantly the length. Interested Readers are welcome to look at the arXiv report)

In the paper The dual Voronoi diagrams with respect to representational Bregman divergences (ISVD 2009), International Symposium on Voronoi Diagrams, we show that Beta divergences is a (representational) Bregman divergence with

• Beta=1 -> Kullback-Leibler
• Beta=0 -> Itakura-Saito
• Beta=2 -> squared Euclidean distance

In the paper, we derive formula for the beta left and right-sided centroids. The program RepresentationalBetaBregman.java shows this equivalence (up to numerical errors).

Shows that beta divergences can be obtained from representational Bregman divergences
beta-div=0.0060607537896566754  equals Bregman rep. div=0.006060753789656717
beta-div=0.03596836005227946    equals Bregman rep. div=0.03596836005227946
beta-div=0.03786639961675385    equals Bregman rep. div=0.03786639961675385
beta-div=0.015356733711556145   equals Bregman rep. div=0.015356733711556173
beta-div=0.16665973512136045    equals Bregman rep. div=0.16665973512136045
beta-div=0.006143185064308276   equals Bregman rep. div=0.006143185064308207
beta-div=0.012777128199946086   equals Bregman rep. div=0.012777128199946072
beta-div=2.42453303134018E-4    equals Bregman rep. div=2.4245330313402494E-4
beta-div=0.07962156613977964    equals Bregman rep. div=0.07962156613977961
beta-div=2.6549301732092453E-4  equals Bregman rep. div=2.6549301732092974E-4
Press any key to continue...

Frank.

In this note alphadivergencemeans.pdf, we summarize the following work

• Shun-ichi Amari, Integration of Stochastic Models by Minimizing \alpha-Divergence, Neural Computation (NECO), (19)10:2780-2796, October 2007.

• F. Nielsen and R. Nock, The dual Voronoi diagrams with respect to representational Bregman divergences, International Symposium on Voronoi Diagrams (ISVD), June 2009.

• F. Nielsen and R. Nock, Sided and Symmetrized Bregman Centroids, IEEE transactions on information theory (2009), vol. 55, no. 6, pp. 2882-2904

I've read the following paper:

Empirical evaluation of dissimilarity measures for color and texture

The paper compares 9 kinds of divergences for several applications of computer vision such as classification supervised/unsupervised segmentation, and, image annotation and retrieval. I am not going to cite verbatim their concluding remarks, but for short (1) EMD is performing very good for partial matches, and (2) there is no winner for all tasks, which confort my point of view: divergences should be tailored and learnt from data-sets on the fly.
Also adaptive binning in histogram seems to be a key for improved performance.