When sound modulates vision: VR applications for art and entertainment

SE Bialkova, M.S. Van Gisbergen

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

21 Citations (Scopus)


The interplay between sound and vision is a key determinant of human perception. With the development of Virtual Reality (VR) technologies and their commercial applications, there is emergent need to better understand how audio-visual signals manipulated in virtual environments influence perception and human behaviour. The current study addresses this challenge in simulated VR environments mirroring real life scenarios. In particular, we investigated the parameters that might enhance perception, and thus VR experiences when sound and vision are manipulated. A VR museum was created mimicking a real art gallery featuring Japanese paintings. Participants were exposed to the gallery via Samsung Gear VR, head mounted display, and could freely walk in. To half of the participants newly composed music clips were played, during the VR gallery visit. The other participants were exposed to the same environment, but no music was played (control condition). The results showed that music played altered the way people are engaged in, perceive and experience the VR art gallery. Opposite to our expectation, the VR experience was liked more when no music was played. The naturalness and presence were perceived to be relatively high, and did not differ significantly depending on whether music was played or not. Regression modelling further explored the relationship between the parameters hypothesised to influence the VR experiences. The findings are summarised in a theoretical model. The study outcomes could be implemented to successfully develop efficient VR applications for art and entertainment.

Original languageEnglish
Title of host publication2017 IEEE 3rd Workshop on Everyday Virtual Reality, WEVR 2017
ISBN (Electronic)9781538638811
Publication statusPublished - 2017
EventIEEE 2017 Virtual Reality - Los Angeles, United States
Duration: 18 Mar 201722 Mar 2017

Publication series

Name2017 IEEE 3rd Workshop on Everyday Virtual Reality, WEVR 2017


ConferenceIEEE 2017 Virtual Reality
Country/TerritoryUnited States
CityLos Angeles


  • sound
  • vision
  • VR applications
  • entertainment
  • art
  • music
  • Sound-vision interplay


Dive into the research topics of 'When sound modulates vision: VR applications for art and entertainment'. Together they form a unique fingerprint.

Cite this