Predicting false memories with convolutional neural networks: the effect of visual similarity in a DRM paradigm with pictorial stimuli

The Deese-Roediger-McDermott task (DRM) is one of the most widely used methods to account for false memories’ formation in the verbal domain. In this task, participants are typically presented with a list of words to memorise (encoding). In a second phase (recognition), they are asked to recognise whether a given word was part of the memorised lists or a novel one. During this latter phase, people tend to erroneously recognise as part of the memorised list even those novel words that are semantically related to the ones presented in the encoding phase. Although the role of semantic similarity in the generation of false memories has well been established, little is known about the role of other types of similarity on memory distortions. Here, we focused on vision-based similarity between images and directly tested its effect on the generation of false memories by predicting, through Convolutional Neural Networks (CNN), participants’ performance in a visual variant of the DRM task. Participants were asked to memorise lists of photographs selected from homogeneous categories in ImageNet. For the encoding phase, we selected only images visually close to the centroid of the image space of the category. The novel images of the recognition phase were selected at various degrees of similarity to the centroid of the image space. Results showed that, in the recognition phase, participants tended to identify the novel images that were visually closer to the targets, as part of the memorised list. Our evidence indicates that the similarity measured by CNN is mirrored in human behaviour. In conclusion, the present study confirms the critical role of the similarity between stimuli in generating false memories. Such similarity is not limited to the semantic similarity between words but also extends to the visual similarity between pictorial stimuli.