Detection of object boundary inconsistencies between 2D-3D conversion results and depth maps

Introduction

The creation of S3D movies by converting 2D captured footage requires the high-quality depth maps. In case of inaccurate depth maps’ usage during the conversion different types of artifacts can occur. Such artifacts can make the viewing experience significantly worse even if it appears only in the background.

Our method extracts information about motion in the scene and detects conversion-related discrepancies between motion strength and depth.

An example of a scene containing a defect found by the proposed method: the foreground object was merged with the background.

Example

Proposed method

Algorithm scheme

Algorithm scheme

The disparity map estimation step may be skipped if the original depth map is available.

Experiments

The algorithm was tested on 39 converted stereoscopic feature movies. As a result, 125 scenes with this artifact were found.

The number of scenes with notable artifacts found during the analysis of feature movies are illustrated below.

Number of scenes

The examples of detected scenes are listed below. In these scenes, foreground objects merge with the background.

Example

05 May 2020
See Also
Call for HEVC codecs 2019
Fourteen modern video codec comparison
Parallax range estimation in S3D video
Parallax determines the depth of S3D movies. The range of parallaxes should be both comfortable and entertaining for spectators.
Geometric distortions analysis and correction
Production of low-budget movies is prone to errors. Our method automatically corrects rotation and scale mismatch.
Automatic detection of artifacts in converted S3D videos
Our set of algorithms detects edge sharpness mismatch, cardboard effect, and crosstalk noticeability.
Temporal shift estimation for stereoscopic videos
How to take into account geometric distortions in the estimation of the temporal shift?
Neural network-based algorithm for classification of stereoscopic video by the production method
What method was used to create the 3D scene?
Site structure