Computational Information Geometry Wonderland

Ok, is that obvious that there is a Heaven world of random variables, were us Human beings can observe samples. I let you decide (do you think also that in that World, things are iid., but here on Earth everything seems to be dependent and different, think of the butterfly effect... are two atoms exactly alike?).

Anyway, to come back to pragmatic discussion. Suppose that we are given many samples, then in 1933, it was proven that the empirical CDF (cumulative distribution function) was converging to the underlying true distribution at an exponential rate. This bears the name of Glivenko-Cantelli theorem. So the inverse probability problem might be potentially tackled. Of course, Fisher key idea was to consider class of CDFs by parameterizing them (say, Gaussian, etc.) instead of looking for the non-parametric case.

Here are then bibtex entries for those articles (I had to brush up my Italian to create those entries to my bibtex file... -:) )

@article{Glivenko-1933,
author={Valery Ivanovich Glivenko},
title={Sulla determinazione empirica delle leggi di probabilita},
journal={Giornale dell'Istituto Italiano degli Attuari},
number={4},
pages={92-99},
year=1933
}
@article{Cantelli-1933,
author={Francesco Paolo Cantelli},
title={Sulla determinazione empirica delle leggi di probabilita},
journal={Giornale dell'Istituto Italiano degli Attuari},
number={4},
pages={221-424},
year=1933
}

Today, I would like to introduce the paper:

Imprecise probability models for inference in exponential families (2005)

The paper starts from the definition of exponential families, introduce the conjugate priors for Bayesian estimation, and then goes on imprecise models. Imprecision can either be manipulated on the conjugate prior hyperparameters, or by using lower/upper envelopes on density functions.

The beauty of this theory is to derive predictive distributions, and distinguish the initial prior from the loop prior-posterior updating. The paper depicts graphically the updating scheme for the normal distributions.

Overall, this field of imprecise probabilities (or robust probabilities?) seems quite challenging, and yet yielding applications.

There is a page on Wikipedia devoted to imprecise probabilities