Tempered Expectation-Maximization algorithm for discrete latent variable models

The Latent Class (LC) model is one of the most well-known latent variable models; it is very popular for the analysis of categorical response variables, and it is typically used to cluster subjects, by assuming the existence of individual-specific latent variables having a discrete distribution. A Hidden (or Latent) Markov (HM) model represents a generalization of the LC model to the case of longitudinal data. It assumes the existence of a discrete latent process generally following a first-order Markov chain, corresponding to subpopulations, usually referred to as latent states. As typically happens for discrete latent variable models, despite maximum likelihood estimation of both LC and HM model parameters can be rather simply performed using the Expectation-Maximization (EM) algorithm, a well-known drawback of this estimation method is related to the multimodality of the log-likelihood function. The consequence is that the estimation algorithm could converge to one of the local maxima, not corresponding to the global optimum. In order to face the multimodality problem described above, we propose a Tempered EM (T-EM) algorithm, which is able to explore the parameter space adequately. It consists in rescaling the objective function depending on a parameter known as the temperature, which controls global and local maxima prominence. High temperatures allow us to explore wide regions of the parameter space, avoiding the maximization algorithm being trapped in non-global maxima; low temperatures, instead, guarantee a sharp optimization in a local region of the parameter space. By properly tuning the sequence of temperature values, the target function is gradually attracted towards the global maximum, escaping local sub-optimal solutions. We rely on an accurate Monte Carlo simulation study to compare the proposal with the standard EM algorithm, evaluating both the ability to hit the global maximum and the computational time of the proposed algorithm. We also show the results for both LC and HM models, using the proposal on discrete and continuous cross-sectional and longitudinal data in connection with some applications of interest. We conclude that the proposal outperforms the standard EM algorithm, significantly improving the chance to reach the global maximum in the overwhelming majority of considered cases. The advantage is relevant even considering the overall computing time.