Swapped views detection in S3D movies
- Author: Sergey Lavrushkin
- Supervisor: Dmitry Vatolin
Channel mismatch is one of the most painful-for-viewer problems in stereo 3D movies. This problem is very hard to detect and simply to eliminate: just swap the views.
Our technique provides an opportunity for automatic detection of scenes with channel mismatch.
Details of the algorithm
Our algorithm uses five baselines.
Firstly, it is checking on three basic heuristics (i.e certain assumptions have to work for the scene):
- Objects with less depth are often at the bottom of the frame, and objects with a greater depth are often at the top of the frame.
- Usually, the objects that are most interesting to the viewer are “falling out”, that is, they have the smallest depth among other objects and are closer to the center of the screen.
- Usually, the areas in front of the screen plane occupy a third of the screen space, and the areas behind the screen plane - two thirds.
Secondly, two occlusion-analysis based algorithms are performing.
In the end, the weighted sum of all baselines results in one factor indicating whether the scene has a channel mismatch
Here listed some results of channel mismatch detection in S3D movies. The analysis showed the swapped views occurred even in the high-budget movies (e.g. Avatar).
How often channel mismatch occurs
|Movie name||Release year||Budget, $M||Scenes with CM||Total CM duration, sec||Film duration, sec||CM percentage|
|The Child’s Eye||2010||$4.5||15||57.5||5823||0.9875%|
|The Nutcracker in 3D||2010||$90||9||28,9||6480||0,447%|
|3D Sex and Zen: Extreme Ecstasy||2011||$2.5||9||23,1||6775||0,341%|
|Spy Kids 3D — Game Over||2003||$39||5||10,3||5063||0,203%|
Comparison with other algorithms
Due to the fact that there were no other channel mismatch detection algorithms, a comparison was conducted to our previous version of this algorithm.
|Algorithm||AUC ROC||Time for a scene, sec|
Instead of conclusion
Channel mismatch perceptibility
A subjective study of perceptibility of channel mismatch was conducted.
We composed a test sequence of 56 scenes with swapped views. In addition, the sequence included scenes preceding and following each scene. 59 people took part in the experiment, each of them rated perceptibility of swapped views for every scene from 1 to 5.
Two graphs show dependencies between channel mismatch perceptibility, movie budget, and release date.
Due to a large number of clear films, the trends are improving, but the “outliers” (even in 2D-3D conversion) are still encountered.
The swapped views scenes more often occur in low budget films.
- Codecs Comparison & Optimization
- Video Filters
Semiautomatic Visual-Attention Modeling