We propose a variable selection method for multivariate hidden Markov models with continuous responses that are partially or completely missing at a given time occasion. Through this procedure, we achieve a dimensionality reduction by selecting the subset of the most informative responses for clustering individuals and simultaneously choosing the optimal number of these clusters corresponding to latent states. The approach is based on comparing different model specifications in terms of the subset of responses assumed to be dependent on the latent states, and it relies on a greedy search algorithm based on the Bayesian information criterion seen as an approximation of the Bayes factor. A suitable expectation-maximization algorithm is employed to obtain maximum likelihood estimates of the model parameters under the missing-at-random assumption. The proposal is illustrated via Monte Carlo simulation and an application where development indicators collected over eighteen years are selected, and countries are clustered into groups to evaluate their growth over time.

Pennoni, F., Bartolucci, F., Pandolfi, S. (2024). Variable selection for hidden Markov models with continuous variables and missing data. JOURNAL OF CLASSIFICATION, 1-22 [10.1007/s00357-023-09457-9].

Variable selection for hidden Markov models with continuous variables and missing data

Pennoni, F
;
2024

Abstract

We propose a variable selection method for multivariate hidden Markov models with continuous responses that are partially or completely missing at a given time occasion. Through this procedure, we achieve a dimensionality reduction by selecting the subset of the most informative responses for clustering individuals and simultaneously choosing the optimal number of these clusters corresponding to latent states. The approach is based on comparing different model specifications in terms of the subset of responses assumed to be dependent on the latent states, and it relies on a greedy search algorithm based on the Bayesian information criterion seen as an approximation of the Bayes factor. A suitable expectation-maximization algorithm is employed to obtain maximum likelihood estimates of the model parameters under the missing-at-random assumption. The proposal is illustrated via Monte Carlo simulation and an application where development indicators collected over eighteen years are selected, and countries are clustered into groups to evaluate their growth over time.
Articolo in rivista - Articolo scientifico
Expectation-maximization algorithm; Greedy search algorithm; Missing-at-random assumption; Model-based variables selection; Sustainable development;
English
23-gen-2024
2024
1
22
open
Pennoni, F., Bartolucci, F., Pandolfi, S. (2024). Variable selection for hidden Markov models with continuous variables and missing data. JOURNAL OF CLASSIFICATION, 1-22 [10.1007/s00357-023-09457-9].
File in questo prodotto:
File Dimensione Formato  
Pennoni-2024-J Classificat-VoR.pdf

accesso aperto

Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Licenza: Creative Commons
Dimensione 1.73 MB
Formato Adobe PDF
1.73 MB Adobe PDF Visualizza/Apri
Pennoni-2024-J Classificat.pdf

accesso aperto

Descrizione: Supplementary Material
Tipologia di allegato: Other attachments
Licenza: Creative Commons
Dimensione 1.6 MB
Formato Adobe PDF
1.6 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/457218
Citazioni
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
Social impact