To get the best, tame the beast: constrained ML estimation for mixture models

This paper presents a review about the usage of eigenvalues restrictions for constrained parameter estimation in mixtures of elliptical distributions according to the likelihood approach. The restrictions serve a twofold purpose: to avoid convergence to degenerate solutions and to reduce the onset of non- interesting (spurious) local maximizers, related to complex likelihood surfaces. The likelihood function may present local spurious maxima when a fitted component has a very small variance or generalized variance (i.e., the determinant of the covariance matrix), compared to the others (Day, 1969). Such a component usually corresponds to a cluster containing few data points either relatively close together or almost lying in a lower-dimensional subspace, in the case of multivariate data. These solutions are of little practical use or real world interpretation (McLachlan and Peel, 2000). To get the best from the EM algorithm widely used for model estimation, a constrained estimation of the covariance matrices can help in driving the maximum likelihood approach toward sensible solutions and far from the ”wild”, non-useful ones. We begin presenting the strongest constraints, i.e. equality among spherical covariance matrices, and the well-known homoscedastic assumption. Afterwards, some authors reinterpreted the safe, above mentioned methods, looking for milder conditions, like the lightest assumption of equality of covariance determinants (McLachlan and Peel, 2000, Section 3.9.1). Gaussian parsimonious mixture models have been proposed by Banfield and Raftery (1993), to get intermediate component covariance matrices lying between homoscedasticity and heteroscedasticity. A broad family of contributions in the literature arise from Hathaway’s seminal paper and deals with setting constraints on the ratio between the maximum and the minimum eigenvalue (among many others, Ingrassia, 2004 and García-Escudero et al. 2008). The same approach, simultaneously with impartial trimming, can be adopted in the context of robust statistical methods. Unfortunately these proposals are not equivariant, and this remark motivated Gallegos and Ritter (2009) and Rocci et al. (2017) to introduce affine equivariant constraints. The methods herein described have been extended to other mixtures of elliptical models. In our survey, for each position recalled so far, we discuss the algorithms needed for their exact or approximate implementation through the EM and recall theoretical results on the obtained estimator (whenever available). When “taming the beast”, the constrained maximization of the likelihood provides stability to the obtained solutions.