Do you see what I hear? Audio-visual semantic interactions

Music has been investigated in a variety of contexts. Previous studies reported evidence of the influence of music on the perception and emotional evaluation of faces. Moreover, it has been demonstrated how an art context promotes evaluation processes that influence the emotional experience of an artwork. The aim of the present study was to investigate how emotionally significant music can affect the perception of emotions elicited by artworks. It was hypothesized that listening to a musical piece with an incongruent emotional value compared to the emotion induced by a painting would negatively affect the emotional categorization of the visual stimulus. On the basis of a validation process, for both visual and auditory stimuli, the considered emotional categories were: Happiness, Relax, Fear, and Sadness. Each category comprised 48 paintings. They were paired with either a congruent or an incongruent background music. Participants were 20 students, who were asked to categorize the artworks on the basis of the emotion expressed. A 2-way rmANOVA was performed on the artworks’ categorization judgements. Background music could be congruent, incongruent, or null (silence). The analysis revealed the significance of the background music factor. Post-hoc tests showed significant differences between congruent and incongruent conditions, and between silence and incongruent conditions. In the incongruent condition, the ability to correctly categorize the emotions expressed by the paintings was significantly worse, regardless of the emotional category. These data further suggest that background music can influence emotions aroused by other sensory channels. Further research should explain when does the integration occur.