eXplainable AI for Word Embeddings: A Survey

In recent years, word embeddings have become integral to natural language processing (NLP), offering sophisticated machine understanding and manipulation of human language. Yet, the complexity of these models often obscures their inner workings, posing significant challenges in scenarios requiring transparency and explainability. This survey conducts a comprehensive review of eXplainable artificial intelligence (XAI) strategies focused on enhancing the interpretability of word embeddings. By classifying the existing body of work into six broad categories based on their methodological approaches—a classification that, to our knowledge, does not exist in the literature—we provide a structured overview of current techniques and their characteristics. Additionally, we uncover a noteworthy oversight: a predominant emphasis on interpreting model outputs at the expense of exploring the models’ internal mechanics. This finding underscores the necessity of shifting research efforts toward not only clarifying the results these models produce but also demystifying the models themselves. Such a shift is crucial for uncovering and addressing biases inherent in word embeddings, thus ensuring the development of fair and trustworthy AI systems. Through this analysis, we identify key research questions for future studies and advocate for a holistic approach to transparency in word embeddings, encouraging the research community to explore both the outcomes and the underlying algorithms of these models.