@FrnkNlsn
]]>
For example, the video on
The Burbea-Rao and Bhattacharyya centroids
is available from the past events tab.
There is a much more to listen to.
Also, there is the international mailing list on information geometry.
For f-divergences, convexity in both arguments ensure that it is the case, and setting the gradient to zero does the job.
But when it is not convex, we may still hope (and prove) that the centroid is unique.
This was the case for the Burbea-Rao centroid (or Jensen centroid induced by the Jensen convexity gap of a function):
The Burbea-Rao and Bhattacharyya centroids using a fixed-point equation and property of interness of quasi-arithmetic means.
However, in general the centroid defined as a minimizer may not be unique if the average distortion has many local minima.
Such simple functions may indeed yield exponentially many maxima as shown by Auer and his colleagues:
Exponentially many local minima for single neurons
They proved that a single neuron with the logistic transfer function and square loss to optimze can yield exponential many extrema for learning the weight vector.
@inproceedings{ManyLocalMinima-1995,
author = {Peter Auer and Mark Herbster and Manfred K. Warmuth},
title = {Exponentially many local minima for single neurons},
booktitle = {Neural Information Processing Society (NIPS)},
year = {1995},
pages = {316-322}
}
Nowadays, with Big Data hunt for Better Data, model selection and sufficiency is crucial.
The question remains on how much information is there in the data.
How crazy or realistic is this idea? Well, I suggest you to look at that video for convincing you that it is not so SciFI:
$k$-MLE: A fast algorithm for learning statistical mixture models
The two major questions:
Many interesting trade-offs to unravel!
Frank.
]]>
dvips -T 128mm,96mm -Ic slides.dvi
ps2pdf slides.ps
Frank.
]]>
In
Shape Retrieval Using Hierarchical Total
Bregman Soft Clustering
similarly it is proven that
for total Bregman divergences (tBD), there exists a distribution which belongs to the lifted exponential family of statistical distributions
This leads to a new clustering technique namely, the total Bregman soft clustering algorithm.
See paper.
Frank.
sci.ans
Frank.
More details in the memo:
NoteLegendreTransformation.pdf
++x-x,
Frank.
Digital cameras are quickly merging with smart phones, and visual computing applications [1] that support computational photography and augmented reality applications are flourishing at a fast pace. By 2013, the annual worldwide IP traffic is predicted to be a zettabyte: 90% of consumer IP traffic and 60% of mobile IP traffic will be video.
How do we extract and use rich information from those massive data sets? As visual data abound, computer vision and computer graphics are increasingly relying on machine learning and information-theoretic methods. Computational Information Geometry is a novel paradigm to perform high-fidelity data analysis using the language and thinking of geometry.
Geometry allows us to map the data in space for efficient processing and retrieval of intrinsic information. Geometry is in essence coordinate-free and allows one to extract the very information from data.
Geometrization of statistics has provided novel algorithms for manipulating statistical mixture models such as Gaussian mixture models [2] that are commonly used in image processing: An image pixel at position (x,y) with color attributes (red, green, blue) is embedded into a 5D space so that a 2D color image is interpreted as a 5D spatial point cloud. We then seek for a compact generative statistical representation of the image point set. Such statistical methods are useful for explaning human cognitive and learning skills [3], and analyzing emerging phenomena of complex systems using hierarchical Bayesian models.
Geometry is well alive and continue to play a crucial role in natural sciences. For example, the propagation of seismic waves from an epicenter follows Fermat's principle of shortest paths (minimum arrival time). Since the Earth is made of anisotropic media such as the peridotite, shortest paths are not line segments: The geometry is not Euclidean. Seismic wave propagation is currently best modeled using Finsler geometry that extends Riemmanian geometry by taking into account the anisotropic direction of materials. In [4], we recently show how to aggregate and cluster information in such Finslerian spaces. Finsler geometry is also considered for advanced medical imaging of DT-MRI data-sets.
Last but not least, the theory of portfolio allocation has been traditionally carried out using the mean-variance method of Markowitz. Considering universal statistical distributions (exponential families) allows one to bypass the Gaussian assumption, and to derive the exact expression of the risk premium (a Bregman divergence) and certainty equivalent [5]. Moreover, we design an on-line learning algorithm with guaranteed lower bounds on its cumulated certainty equivalents [5]. It is interesting to note that portfolio theory has also been considered to explain robustness trade-offs of cells in biology [6] (bioeconomics).
REFERENCES:
[6] Hiroaki Kitano: Violations of robustness trade-offs Mol Syst Biol. 2010; 6: 384. 10.1038/msb.2010.40
]]>
@article{1751-8121-45-3-032003,
author={Frank Nielsen and Richard Nock},
title={A closed-form expression for the Sharma-Mittal entropy of exponential families},
journal={Journal of Physics A: Mathematical and Theoretical},
volume={45},
number={3},
pages={032003},
url={http://stacks.iop.org/1751-8121/45/i=3/a=032003},
year={2012}
}
]]>
"Abstract A Jensen-Bregman divergence is a distortion measure defined by a Jensen convexity gap induced by a strictly convex functional generator. Jensen-Bregman divergences unify the squared Euclidean and Mahalanobis distances with the celebrated information-theoretic Jensen-Shannon divergence, and can further be skewed to include Bregman divergences in limit cases. We study the geometric properties and combinatorial complexities of both the Voronoi diagrams and the centroidal Voronoi diagrams induced by such as class of divergences. We show that Jensen-Bregman divergences occur in two contexts: (1) when symmetrizing Bregman divergences, and (2) when computing the Bhattacharyya distances of statistical distributions. Since the Bhattacharyya distance of popular parametric exponential family distributions in statistics can be computed equivalently as Jensen-Bregman divergences, these skew Jensen-Bregman Voronoi diagrams allow one to define a novel family of statistical Voronoi diagrams.
Keywords Jensen?s inequality ? Bregman divergences ? Jensen-Shannon divergence ? Jensen-von Neumann divergence ? Bhattacharyya distance ? information geometry
]]>
Here is a toy reconstruction (that I made by hand in a few minutes)
Best, Frank.
I am looking for Finslerian quasi-metric distances with applications in computer vision/medical imaging. Any recommendation?
Comments welcome!
Frank.
Details in the paper:
Nielsen, F., Nock, R., 2005. ClickRemoval: Interactive pinpoint image object removal, ACM multimedia, Hilton, Singapore, pp. 315?318.
Here it is:
Frank.