VQMT3D Project: Report 9 on 3D-video Quality Analysis

In cooperation with IITP RAS

About the Report

The ninth report presents results from an analysis of 105 S3D movies that examined temporal shifts between views and introduces a new metric for detecting stuck-to-background objects in converted S3D scenes. The report also introduces a classifier that determines which technique was used during production of a scene (capture, conversion or CGI).

Download report

Examined problems:

Over 100 films analysed, including:

  • Creature from the Black Lagoon (1954)
  • World War Z (2013)
  • Dredd (2012)
  • Thor (2011)
  • The Hobbit: An Unexpected Journey (2012)
  • Life of Pi (2012)
  • Oz the Great and Powerful (2013)
  • The Legend of Hercules (2014)
  • Clash of the Titans (2010)
  • Men in Black 3 (2012)
  • Dial M for Murder (1954)
  • Pirates of the Caribbean: On Stranger Tides (2011)
  • The Last Airbender (2010)
  • TRON: Legacy (2010)
  • Jack the Giant Slayer (2013)
  • The Great Gatsby (2013)
  • Pacific Rim (2013)
  • The Avengers (2012)

The report comprises three main parts:

  1. Firstly, we present our approach to analyzing stereoscopic video for temporal shift. We provide the results of our analysis for 105 S3D films and present overall charts for these films.
  2. Then we present our new metric for detecting foreground objects stuck to the background of a scene. The report also presents visualizations of these scenes.
  3. Finally, we introduce our algorithm which classifies shots in S3D films into three categories: natively captured, 2D-3D converted and CGI. The section contains a lot of examples with converted shots in natively captured films and natively captured shots in converted S3D films.

Example of a "stuck-to-background" object

Professional stereographers have contributed to the analysis by providing valuable comments and general feedback.

Contributed Stereographers

(in alphabetical order; for main part of the report)

Overall evaluation results

In this section we present selected overall comparison charts.

Please, read more about the charts and particular films in the report.

Number of scenes detected with major “stuck-to-background artifacts”

Temporal-shift distribution in analyzed 105 films

Publications

If you want to make a reference to this project, please refer to one of the following publications:

  1. Mikhail Erofeev, Dmitriy Vatolin, Alexander Voronov, Alexey Fedorov,
    “Toward an Objective Stereo-Video Quality Metric: Depth Perception of Textured Areas,”
    International Conference on 3D Imaging,
    pp. 1–6, 2012. doi:10.1109/IC3D.2012.6615120
  2. Dmitriy Akimov, Alexey Shestov, Alexander Voronov, Dmitriy Vatolin,
    “Automatic Left-Right Channel Swap Detection,”
    International Conference on 3D Imaging,
    pp. 1–6, 2012. doi:10.1109/IC3D.2012.6615126
  3. Alexander Voronov, Alexey Borisov, Dmitriy Vatolin,
    “System for automatic detection of distorted scenes in stereo video,”
    International Workshop on Video Processing and Quality Metrics for Consumer Electronic (VPQM-2012),
    pp. 138–143, 2012.
  4. Alexander Voronov, Dmitriy Vatolin, Denis Sumin, Vyacheslav Napadovsky, Alexey Borisov,
    “Towards Automatic Stereo-video Quality Assessment and Detection of Color and Sharpness Mismatch,”
    International Conference on 3D Imaging,
    pp. 1–6, 2012. doi:10.1109/IC3D.2012.6615121
  5. Alexander Voronov, Dmitriy Vatolin, Denis Sumin, Vyacheslav Napadovsky, Alexey Borisov,
    “Methodology for stereoscopic motion-picture quality assessment,”
    Proc. SPIE 8648, Stereoscopic Displays and Applications XXIV,
    vol. 8648, pp. 864810-1–864810-14, 2013. doi:10.1117/12.2008485
  6. Alexander Bokov, Dmitriy Vatolin, Anton Zachesov, Alexander Belous, Mikhail Erofeev,
    “Automatic detection of artifacts in converted S3D video,”
    Proc. SPIE 9011, Stereoscopic Displays and Applications XXV (March 6, 2014),
    vol. 901112, pp. 1–14, 2014. doi:10.1117/12.2054330
  7. Stanislav Dolganov, Mikhail Erofeev, Dmitriy Vatolin, Yury Gitman,
    “Detection of stuck-to-background objects in converted S3D movies,”
    2015 International Conference on 3D Imaging, IC3D 2015,
    pp. 1–6, 2015. doi:10.1109/IC3D.2015.7391839
  8. Yury Gitman, Can Bal, Mikhail Erofeev, Ankit Jain, Sergey Matyunin, Kyoung-Rok Lee, Alexander Voronov, Jason Juang, Dmitriy Vatolin, Truong Nguyen,
    “Delivering Enhanced 3D Video,”
    Intel Technology Journal,
    vol. 19, pp. 162–200, 2015.
  9. Dmitriy Vatolin, Alexander Bokov, Mikhail Erofeev, Vyacheslav Napadovsky,
    “Trends in S3D-Movie Quality Evaluated on 105 Films Using 10 Metrics,”
    Proceedings of Stereoscopic Displays and Applications XXVII,
    pp. SDA-439.1–SDA-439.10, 2016. doi:10.2352/ISSN.2470-1173.2016.5.SDA-439
  10. Alexander Bokov, Sergey Lavrushkin, Mikhail Erofeev, Dmitriy Vatolin, Alexey Fedorov,
    “Toward fully automatic channel-mismatch detection and discomfort prediction for S3D video,”
    2016 International Conference on 3D Imaging (IC3D),
    pp. 1–7, 2016. doi:10.1109/IC3D.2016.7823462
  11. Dmitriy Vatolin, Sergey Lavrushkin,
    “Investigating and predicting the perceptibility protect of channel mismatch in stereoscopic video,”
    Moscow University Computational Mathematics and Cybernetics,
    vol. 40, pp. 185–191, 2016. doi:10.3103/s0278641916040075
  12. Anastasia Antsiferova, Dmitriy Vatolin,
    “The influence of 3D video artifacts on discomfort of 302 viewers,”
    2017 International Conference on 3D Immersion (IC3D),
    pp. 1–8, 2017. doi:10.1109/IC3D.2017.8251897
  13. Dmitriy Vatolin, Alexander Bokov,
    “Sharpness Mismatch and 6 Other Stereoscopic Artifacts Measured on 10 Chinese S3D Movies,”
    Proceedings of Stereoscopic Displays and Applications XXVIII,
    pp. 137–144, 2017. doi:10.2352/ISSN.2470-1173.2017.5.SDA-340
  14. Sergey Lavrushkin, Vitaliy Lyudvichenko, Dmitriy Vatolin,
    “Local Method of Color-Difference Correction Between Stereoscopic-Video Views,”
    Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
    pp. 1–4, 2018. doi:10.1109/3DTV.2018.8478453
  15. Aidar Khatiullin, Mikhail Erofeev, Dmitriy Vatolin,
    “Fast Occlusion Filling Method For Multiview Video Generation,”
    Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
    pp. 1–4, 2018. doi:10.1109/3DTV.2018.8478562
  16. Sergey Lavrushkin, Dmitriy Vatolin,
    “Channel-Mismatch Detection Algorithm for Stereoscopic Video Using Convolutional Neural Network,”
    Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
    pp. 1–4, 2018. doi:10.1109/3DTV.2018.8478542
  17. Alexander Ploshkin, Dmitriy Vatolin,
    “Accurate Method of Temporal Shift Estimation for 3D Video,”
    Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
    pp. 1–4, 2018. doi:10.1109/3DTV.2018.8478431

Reports overview

Stereo-analysis project description

Report 1 (S3D shooting quality analysis of 5 movies) Download
(Additional info for bloggers and press)
Pages: 246
Figures: 295
Report 2 (S3D shooting quality analysis of 5 movies) Download
(Additional info for bloggers and press)
Pages: 342
Figures: 442
Report 3 (2D-3D conversion quality analysis of 5 movies) Download
(Additional info for bloggers and press)
Pages: 305
Figures: 336
Report 4 (S3D shooting quality analysis of 5 movies) Download Pages: 301
Figures: 402
Report 5 (2D-3D conversion quality analysis of 5 movies) Download
(Additional info for bloggers and press)
Pages: 384
Figures: 404
Report 6 (Stereo Window analysis of 10 movies) Download
(Additional info for bloggers and press)
Pages: 415
Figures: 455
Report 7 (Stereo Window analysis of 10 movies) Download
(Additional info for bloggers and press)
Pages: 333
Figures: 348
Report 8 (Rotate Analysis, Temporal Shift, Channels Swap, Zoom Mismatch in 25 movies) Download
(Additional info for bloggers and press)
Pages: 366
Figures: 361
Report 9 (Temporal Shift, Stuck-to-Background Objects, 2D to S3D conversion in Captured Films) Download
(Additional info for bloggers and press)
Pages: 467
Figures: 529

Report 10 (Overall analysis of 105 movies)

Approximate release date: 2020

Report 11 (Overall analysis of 10 selected Chinese movies)

Approximate release date: 2021

Download

Full report (467 pages) is absolutely free but download is available only for industry professionals (due to publication of real mistakes in the post-production process).

To download the report, please fill-in the request form.
You will get the download link for all reports (Reports #1–9) via e-mail.

All materials in this evaluation are used for research purposes and in accordance with Fair Use conditions. This evaluation is not published widely, but it is available for free under a subscription for stereoscopy and cinema technology professionals.

Contacts

For questions and proposition please contact us 3dmovietest@graphics.cs.msu.ru

24 Jan 2017
See Also
Call for HEVC codecs 2019
Fourteen modern video codec comparison
Detection of stereo window violation
How to find objects that are present only in one view?
Depth continuity estimation in S3D video
How smooth is the depth transition between scenes?
Detection of 3D movie scenes shot on converged axes
Another cause of headaches when watching 3D movies.
Parallax range estimation in S3D video
The parallax range should be both comfortable and entertaining for spectators.
Geometric distortions analysis and correction
Automatic correction of vertical disparity, rotation mismatch and scale mismatch.
Site structure