Computational Information Geometry Wonderland

Finite mixture models occur ubiquituously in many signal processing applications. The standard base solution is to use the Expectation-Maximization algorithm (EM) that is proven to converge monotonically by improving the incomplete likelihood. One problem is proper initialization, another problem is the stopping criterion to force EM to stop. What if we rather maximizes the complete likelihood. Then for components of exponential families (like Gaussian mixture models), we can design a simple algorithm that performs a dual additive Bregman clustering for updating expectation parameters followed by a cross-entropy minimization for updating weights, and reiterate finitely until it reaches a local optimum: This is the essence of k-MLE. MLE stands for maximum likelihood estimation.

$k$-MLE: A fast algorithm for learning statistical mixture models

The two major questions:

• How good can you learn a mixture model?,
• and how fast can you do it?

Many interesting trade-offs to unravel!

Frank.