Virtual reality technology incorporating six degrees-of-freedom introduces new challenges for the evaluation of audio quality. Here, a real-time “online” evaluation platform is proposed, allowing multiple stimulus comparison of binaural renderers within the virtual environment, to perceptually evaluate audio quality. To evaluate the sensitivity of the platform, tests were conducted using the online platform with audiovisual content, and two traditional platforms with pre-rendered “off-line” audiovisual content. Conditions employed had known relative levels of impairments. A comparison of the results across platforms indicates that only the proposed online platform produced results representative of the known impaired audio conditions. Off-line platforms were found to be not sufficient in detecting the tested impairments for audio as part of a multi-modal virtual reality environment.