Neural network-based algorithm for classification of stereoscopic video by the production method

Introduction

Creating an efficient and correct algorithm for quality assessment of stereoscopic videos requires knowledge of the method used for its making. Stereoscopic videos, made through different production methods, contain various types of artifacts, so it’s necessary to analyze them with different quality assessment algorithms.

Artifacts of stereoscopic shooting

Artifacts of stereoscopic shooting


Artifacts of 2D-to-3D conversion

Artifacts of 2D-to-3D conversion


Common artifacts

Common artifacts

Proposed method

The algorithm classifies stereoscopic video scenes.

Features which can be used for attribution to one of the three main classes are calculated for an input video. Then, using the information obtained from these features, the classifier predicts the production method of the input.

Due to the fact that flat and low-contrast stereoscopic video is uninformative and difficult to classify, it is excluded from analysis.

Algorithm scheme

Algorithm scheme

Experiments

To find the best parameters, various configurations of features based on VQMT3D were tested:

Conversion/shooting classification results

Conversion/shooting


CGI/conversion classification results

Graphics/conversion


CGI/shooting classification results

Graphics/shooting

Results

Within the framework of the VQMT3D project, the proposed algorithm was used to analyze 105 full-length stereoscopic movies.

Converted scene statistics for shot movies

Film Budget, $K/min Number of scenes found
47 Ronin 1470 9
A Very Harold & Kumar 3D Christmas 211 19
Bait 301 11
Battle of the Year 181 24
Dark Country 45 2
Dredd 520 48
Drive Angry 480 2
Final Destination 5 434 4
Flying Swords of Dragon Gate 286 14
Fright Night 283 6
Ghosts of the Abyss 213 14
Great Canyon Adventure Unknown 4
Hansel & Gretel: Witch Hunters 568 5
Hugo 1349 9
Jack the Giant Slayer 1710 2
Journey 2: The Mysterious Island 840 3
Katy Perry: Part of Me 127 36
Life of Pi 944 6
One Direction: This Is Us 93 20
Oz the Great and Powerful 1653 2
Pina 40 2
Piranha 3DD 60 23
Pirates of the Caribbean: On Stranger Tides 1838 19
Prometheus 1048 5
Resident Evil: Afterlife 618 3
Resident Evil: Retribution 677 3
Sanctum 277 1
Saw 3D: The Final Chapter 222 48
Sea Rex 3D: Journey to a Prehistoric World 121 12
Silent Hill: Revelation 3D 210 3
Stalingrad 229 1
Step Up Revolution 333 7
Texas Chainsaw 3D 108 15
The Amazing Spider-Man 1691 2
The Darkest Hour 337 14
The Great Gatsby 739 1
The Hobbit: An Unexpected Journey 1914 2
The Legend of Hercules 707 16
Transformers: Dark of the Moon 1266 11
TRON: Legacy 1360 17
Underworld: Awakening 786 11


Shooted scene statistics for converted movies

Film Budget, $K/min Number of scenes found
Gulliver’s Travels 1317 1
The Green Hornet 1008 40
Spy Kids: All the Time in the World in 4D 306 38
Thor 1304 3
Priest 689 1
Piranha 3D 272 3
05 May 2020
See Also
Call for HEVC codecs 2019
Fourteen modern video codec comparison
Parallax range estimation in S3D video
Parallax determines the depth of S3D movies. The range of parallaxes should be both comfortable and entertaining for spectators.
Geometric distortions analysis and correction
Production of low-budget movies is prone to errors. Our method automatically corrects rotation and scale mismatch.
Automatic detection of artifacts in converted S3D videos
Our set of algorithms detects edge sharpness mismatch, cardboard effect, and crosstalk noticeability.
Temporal shift estimation for stereoscopic videos
How to take into account geometric distortions in the estimation of the temporal shift?
Perspective distortions estimation
How to detect a mismatch in the vertical position of the cameras?
Site structure