Forecasting of viewers’ discomfort

Introduction

Nowadays, numerous movies are produced in stereoscopic format. Despite the development of stereoscopic movie production, stereoscopic artifacts causing discomfort right up to headaches continue to appear even in high-budget movies.

In this study, the influence of geometric, color and temporal artifacts was examined.

Distortions

Experiments

In a series of experiments, participants were asked to evaluate the level of discomfort while watching a specially prepared stereoscopic video. Over 300 people took part.

Experiment comparison

University #subjects #subjects per video Duration #videos #videos per test #scores Year 3D tech. Stimuli
University of Surrey 30 30 25 40 40 1200 2010 Autostereo. Synthetic S3D seq., encoding with different QP
Catholic University of Korea 20 20 18 36 36 720 2011 Passive Captured S3D seq., different parallax and motion
Telecom Innovation Labs 24 24 50 64 64 1536 2011 Active Open S3D DB, different parallax
Philips Research Labs 24 24 24 7 7 168 2011 Autostereo. 3D movie (converted), different parallax
Beijing Institute of Ophthal. 30 30 30 1 1 30 2012 Active+passive 3D movie (captured)
LUNAM University 29 29 28 110 110 3190 2012 Active Open S3D DB (NAMA3DS1-COSPAD1),
different degradations
Yonsei University 28 28 29 110 110 3080 2013 Passive Open S3D DB,10 degradation types
Acreo Institute 48 28 29 110 110 5280 2013 Passive Open S3D DB,10 degradation types
Tampere University of Tech. 10 10 45 40 40 400 2013 Passive Captured S3D seq., encoding with different QP
Roma Tre University 854 43-255 90 - 1 854 2014 Passive Screening after watching S3D movies in cinemas
Yonsei University 56 56 10.5 30.1 31 1736 2014 Autostereo. Open S3D DB, different SI and TI
University College Dublin 40 4 5.5 25.11 33 1320 2015 Active Open S3D DB, packet losses
University of British Columbia 88 88 48 208 208 18304 2015 Passive Open S3D DB, different SI/TI and compression degradations
University North 146 18 8 184 26 146 2016 Active Open S3D DB, 22 degradation types
(packet losses, encoding, resizing,
disparity, brightness, geometry etc.)
University of Coimbra 35 2-6 7-1 184 26 146 2016 Active Open S3D DB, 22 degradation types
(packet losses, encoding, resizing,
disparity, brightness, geometry etc.)
Lomonosov MSU 302 370 40 60 60 22200 2017 Passive Scenes from 3D movies (captured), 4 types of S3D distortions,
5 intensities

Experiment process

Experiment process

Experiment process

Experiment process


Discomfort level sorted by mean discomfort (red for high)

Experiment results

Here “C” is the color mismatch between stereoscopic views, “R” is the rotation mismatch, “T” is the temporal shift and “S” is the scale mismatch. The distortion intensity is indicated by numbers from 0 to 4, where 0 corresponds to the absence of distortion (the original scene was demonstrated).

To solve the problem of assessing the discomfort level, we used machine learning algorithms on the processed experimental data. More than 30,000 configurations were tested on the obtained stratified cross-validation dataset.

The best result was shown by a linear regression model with the Huber loss function and L2–regularization.

Results

The proposed models were applied to evaluate 60 stereoscopic movies.

The predicted level of discomfort that viewers may experience and the predicted percentage of viewers who will probably feel the discomfort while watching the analyzed movies are illustrated below:

Level of discomfort

Percentage of viewers

05 May 2020
See Also
Call for HEVC codecs 2019
Fourteen modern video codec comparison
Parallax range estimation in S3D video
Parallax determines the depth of S3D movies. The range of parallaxes should be both comfortable and entertaining for spectators.
Geometric distortions analysis and correction
Production of low-budget movies is prone to errors. Our method automatically corrects rotation and scale mismatch.
Automatic detection of artifacts in converted S3D videos
Our set of algorithms detects edge sharpness mismatch, cardboard effect, and crosstalk noticeability.
Temporal shift estimation for stereoscopic videos
How to take into account geometric distortions in the estimation of the temporal shift?
Neural network-based algorithm for classification of stereoscopic video by the production method
What method was used to create the 3D scene?
Site structure