## Tags : bregman

Entries in this Tags : 1logs Showing : 1 - 1 / 1

## Jul 06, 2010

### A misbelief (?) on the concavity of entropies

Post @ 18:30:24 | bregman, tsallis

Shannon entropy is well-known to be concave. The entropy and cross-entropy (inaccuracy) yield the notion of relative entropy. Bregman divergences can be interpreted as relative entropies for a convex generator. The negative (and hence concave) generator can thus be interpreted as an entropy measure. Bregman divergences include both extensive (eg., Shannon) and non-extensive (eg., quadratic) entropies.

However, entropies need NOT BE always concave.

For example, Tsallis entropies (Havrda-Charvat) generalizing Shannon entropy are CONVEX for q<0

This proves also that Tsallis relative entropy is not a Bregman divergence for q<0.