Perceptual Attributes of Natural Dynamic Audiovisual Scenes

Abstract

This work analyzed the perceptual attributes of natural dynamic audiovisual scenes in two consecutive experiments. First, we presented 30 naive participants with 19 natural scenes depicting urban environments reproduced with an immersive audiovisual display utilizing surrounding visual projections and spatial audio reproduction. The aim was to assess the perceptual dimensionality of natural scenes, and to identify significant perceptual attributes by means of a similarity categorization task and an interview. A two-dimensional perceptual map of the stimulus scenes and perceptual attributes was formed, and the exploratory results show the amount of movement and perceived noisiness of the scene to be the most important perceptual attributes in naturalistically reproduced real-world urban environments. We found the scene gist properties openness and expansion to remain as important factors in scenes with no salient auditory or visual events. Our second experiment was organized with 23 naive participants to assess the modality contributions in three salient perceptual attributes through pairwise unimodal and bimodal scene discrimination tasks with short (< 500 ms) natural scene exposures. The chosen attributes were movement, noisiness and openness. Both visual and auditory information were found to affect scene discrimination in all the attributes, and bimodal discrimination was found superior to either of the unimodal accuracies in most cases. We propose that the study of natural scene perception should move forward to understand better the processes behind multimodal scene processing in real-world environments. The stimulus scenes are available as a public database of spherical video recordings and A-format audio recordings.

Publication
In 7th Forum Acusticum