Neural Semantic Decoding through Distributional Semantics: Comparing Linear Models and Neural Networks

Decoding of linguistics information from electrical brain states aims at understanding how language is represented in the brain. In this work we combined electrophysiology and distributional semantics: the former provides large datasets in which semantic content is encoded in the neural signal; the latter provides a quantitative theory of meaning. We used data from the Kiloword megastudy and we modeled meaning with fastText, studying the extent to which semantic information could be recovered from the neural data. We modeled the relationship between the two initially with linear models, and then with convolutional neural networks, by using a Leave-Two-Out procedure to assess how much our models were able to distinguish a target word from a random word selected as control. Our models show that semantic information can be decoded from the neural signal, that it is distributed according to the time window of the N400 ERP component, and that neural networks are more powerful in information extraction as compared to linear models. Our work eventually brings support to distributional semantics, showing that linguistic information encoded in semantic vectors can be recovered from brain signals.