Monthly Archives: April 2010

We study the centroid with respect to the class of information-theoretic distortion measures called Burbea-Rao divergences. Burbea-Rao divergences generalize the Jensen-Shannon divergence by measuring the non-negative Jensen difference induced by a strictly convex and differentiable function expressing a measure of entropy. We first show that a symmetrization of Bregman divergences called Jensen-Bregman distances yields a natural definition of Burbea-Rao divergences. We then define skew Burbea-Rao divergences, and prove that skew Burbea-Rao divergences amount to compute Bregman divergences in asymptotical cases. We prove that Burbea-Rao centroids are always unique, and we design a generic iterative algorithm for efficiently estimating those centroids with guaranteed convergence. In statistics, the Bhattacharyya distance is widely used to measure the degree of overlap of probability distributions. This distance notion is all the more useful as it provides both upper and lower bounds on Bayes misclassification error, and turns out to be equal at the infinitesimal level to Fisher information. We show that Bhattacharyya distances on members of the same exponential family amount to calculate a Burbea-Rao divergence. We thus get as a byproduct an efficient algorithm for computing the Bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families, improving over former specialized methods that were mostly limited to univariate or “diagonal” multivariate Gaussians.

Here is the arxiv report

Well, the title of the blog is computational information geometry. It includes the word “computational”. Of course, one aspects is to bring computational geometry vistas to information geometry by fostering algorithmic techniques. Another aspect, is to ponder what can be done with computations in the field of information geometry. The basic elements being numbers, let us realize that computational reals are of measure zero in the set of reals.
A real is said computationable if we can enumerate all its bits one by one (up to infinity). Emire Borel in the 1920’s already felt strange with this notion (that came in time for Turing later): consider all YES/NO questions (say, in French -:) ) and let each bit of the Borel real B answer the corresponding question.

Being able to compute such a real would yield insights to all questions, including all yes/no questions of science, technology in the future, etc. We expect B by contradiction thus to be uncomputable. Otherwise, imagine having this oracle (Is there advanced life in other planets, will human have wings and fly someday, etc.)

Two major dawbacks of course is the notion of infinity in digital expansion and the notion of time in languages. Actually, to construct a non computable real, one uses Cantor diagonal arguments…

Anyway, in Computational science, one has from time to time to think of the basic notion of computable numbers. There are the basic atoms of any computational science.
Frank.