Computational Information Geometry Wonderland

Unifying Jeffreys symmetric KL with Jensen-Shannon divergences is interesting not only from the viewpoint of understanding but also from the standpoint of applications. In information retrieval, we often require symmetric divergences. It is not so much a problem for f-divergences since they can always be symmetrized. In fact KL and JS are both f-divergences. However JS requires to compute a distance to the mean average distribution. For parametric continuous density like Gaussians, this may rises a problem: the mixture of two Gaussians is not a Gaussian. So computation is hard for those non-discrete densities. Here, we consider another smooth family of symmetric divergences unifying sKL with JS based on Jensen's inequality (a mother for statistical distances -:) )

The preprint is here.

Frank.