MSU Graphics & Media Lab Report #9 on stereo films quality assessment

VQMT3D project continues publishing its updates. This time 9th report includes a detailed analysis of 105 stereo movies re: time shifts between camera views. The most interesting part of the report regards 2 new automatic quality analysis tools for stereoscopic video, which were used for a large-scale analysis of 105 stereo movies.

Image #1: Blu-ray boxes with discs that were used for 105 films analysis.

The metric for time shift detection alongside with the detailed description was presented by the authors in the previous report, where its results were demonstrated on the analysis of a considerably smaller number of movies. The new report demonstrates the outcomes of working on a large-scale analysis of 105 movies. Time shift was detected in the majority of the movies analyzed, with the maximum of 2 frames and the median of 1 frame exactly. Time shift was evident in most contemporary blockbusters. For example, Pirates of the Caribbean: On Stranger Tides had two scenes of this kind. Interestingly, one of the detected scenes was clearly produced by the means of 2D-3D conversion, which serves as proof of sloppy editing.

These convincing results give the authors evidence that the development of computer vision technologies may help us completely get rid of the problem of time shifts throughout production stage.

Image #2: Example of a temporal shift

The most interesting part of the report is dedicated to our new metric for converted movie analysis. More and more movies are being produced in this way, which calls for creation of analytics tools that are able to detect artefacts unseen in natural environment. Our new instrument allows to detect a common problem of low-quality conversion, when 3D movie-makers ignore small objects (usually in the background) and do not assign them a distinct depth. As a consequence, those objects look stuck to background.

This problem is hard to detect automatically, because the algorithm should estimate distance from each object to the camera and compare its estimate with the depth map used in the conversion process (which in turn should be constructed by a separate stereo-matching algorithm). Thus, a number of fundamental problems of computer vision are involved. Therefore, the proposed approach is based on a simplifying assumption that moving objects should have different depth from the background.

Using the abovementioned approach the authors managed to detect numerous stuck-to-background objects with some notable artifacts found even in blockbuster movies (28 out of 105 analyzed movies had budgets over $100 million). The report showcases the examples of problems and offers detailed case studies with comments from industry professionals.

Image #3: Example of a stuck-to-background object

Last but not least, the report provides a method of scene classification by production type - shooting, conversion, computer graphics. 2D-to-3D conversion technique is more and more commonly used not only for creating full-length 3D movies, but also for converting individual scenes in natively captured movies. In the majority of cases this method is applied when one of the original views was somehow damaged. Another case is when converted movies are enriched with captured scenes to cut down expenses. Usually those are simple scenes (panoramic views) shot by the second unit team. In both cases such scenes are made as an afterthought which causes quality deterioration and makes them more interesting to analyse. The algorithm developed by the authors allows to quickly detect converted scenes in captured movies (and vice versa) and improve movie analysis quality by using different algorithms for captured and converted scenes.

The report offers a vast number of examples of converted scenes in natively captured movies (and the other way round) and includes S3D industry professionals’ comments. All the analyzed movies are included in overall bar charts with scenes sorted by production type, which allows to evaluate the general relationship between the technologies in use today.

Image #4: Example of a scene converted from 2D that was detected in natively captured film Bait

You can download the ninth report here: download report. Note, that the full report is absolutely free but download is available only for industry professionals (due to publication of real mistakes in the post-production process).

Acknowledgements

We would like to thank the following people for providing valuable comments, opinions and feedback, which helped to improve this report (in alphabetical order):

A key motive of our reports is to improve the stereo quality and therefore to reduce eyestrain and headache. The Graphics & Media Lab hopes that a high 3D video quality would contribute to a continually increasing number of people, leaving the modern theaters without annoyance and with good impressions from the image quality and storytelling.

Currently we are looking for collaboration on upcoming reports. If you would like to participate by commenting on the draft versions of the reports, please feel free to contact us: 3dmovietest@graphics.cs.msu.ru. If, for some reason, you cannot make public comments for inclusion in the reports, we would really appreciate any other kind of feedback.

Reports overview

Stereo-analysis project description

Report 1 (S3D shooting quality analysis of 5 movies) Download
(Additional info for bloggers and press)
Pages: 246
Figures: 295
Report 2 (S3D shooting quality analysis of 5 movies) Download
(Additional info for bloggers and press)
Pages: 342
Figures: 442
Report 3 (2D-3D conversion quality analysis of 5 movies) Download
(Additional info for bloggers and press)
Pages: 305
Figures: 336
Report 4 (S3D shooting quality analysis of 5 movies) Download Pages: 301
Figures: 402
Report 5 (2D-3D conversion quality analysis of 5 movies) Download
(Additional info for bloggers and press)
Pages: 384
Figures: 404
Report 6 (Stereo Window analysis of 10 movies) Download
(Additional info for bloggers and press)
Pages: 415
Figures: 455
Report 7 (Stereo Window analysis of 10 movies) Download
(Additional info for bloggers and press)
Pages: 333
Figures: 348
Report 8 (Rotate Analysis, Temporal Shift, Channels Swap, Zoom Mismatch in 25 movies) Download
(Additional info for bloggers and press)
Pages: 366
Figures: 361
Report 9 (Temporal Shift, Stuck-to-Background Objects, 2D to S3D conversion in Captured Films) Download
(Additional info for bloggers and press)
Pages: 467
Figures: 529
Report 10 (Overall analysis of 105 movies) Download Pages: 211
Figures: 270

Report 11 (Overall analysis of 10 selected Chinese movies)

Approximate release date: 2021

Contacts

For questions and proposition please contact us 3dmovietest@graphics.cs.msu.ru

24 Jan 2017
See Also
MSU Deinterlacer Benchmark 2020
MSU 3D-video Quality Analysis. Report 10
Detection of stereo window violation
How to find objects that are present only in one view?
Depth continuity estimation in S3D video
How smooth is the depth transition between scenes?
Detection of 3D movie scenes shot on converged axes
Another cause of headaches when watching 3D movies.
Parallax range estimation in S3D video
The parallax range should be both comfortable and entertaining for spectators.
Site structure