Automatic detection of artifacts in converted S3D videos

Introduction

Our set of algorithms provides an objective quality assessment for stereoscopic videos converted from a 2D source. In particular, the algorithms are able to detect three types of unwanted effects: edge sharpness mismatch, cardboard effect, and crosstalk noticeability.

Edge sharpness mismatch

A distorted region detected with edge sharpness mismatch metric

Due to the nature of 2D-to-3D conversion, the sharpness of edges may differ between the left-eye and right-eye views. This artifact is typically caused by the following:

Such an effect is rarely natural. Therefore, the edge sharpness mismatch may cause difficulties with scene interpretation and discomfort for the viewer.

Cardboard effect

The cardboard effect is a term referring to an unnatural flattening of objects in perceived visual images. Although this artifact may appear in stereoscopic videos captured by a stereoscopic camera system, it appears more frequently in converted S3D videos.

Crosstalk

Crosstalk is a very common problem across modern stereo display devices. Both viewer’s eyes see a combination of left and right views. This effect is caused solely by display devices, but it is possible to reduce its noticeability during film production.

Crosstalk visibility depends on the technology used for views separation:

Proposed method

For each type of the artifacts we have developed a detection algorithm.

Edge sharpness mismatch algorithm

This complex task can be simplified assuming that background remains the same near edges with sharpness mismatch.

Cardboard effect

In a 2D video the true depth map is not available and the converted video’s disparity cannot be verified. Thus, our goal is to detect completely flat objects and scenes.

Flat scenes are detected in the same way. Scenes that were rotated (with disparity changing linearly) are also considered flat.

Crosstalk noticeability

In highly textured regions the crosstalk can be masked by the human visual system. That is why we use a structural similarity function to compute the noticeability of differences. We blend the left and right views in 85% to 15% proportion.

Experiments

The following images demonstrate the work of our algorithm. The regions with distortions are enlarged, and a thumbnail of the full view is placed in the corner.

Image order: left view, right view, edge sharpness mismatch map. Red color indicates the severity of mismatch.

Image order: left view, right view, edge sharpness mismatch map. Red color indicates the severity of mismatch.

A foreground object is detected as flat and has the same depth as background.

Another example of a flat foreground object.

Visualization of crosstalk noticeability map.

Potential crosstalk noticeability analyzed in 105 movies. The metric shows what movies can be viewed on displays susceptible to crosstalk.

Results

Publications

A. Bokov et al., "Automatic detection of artifacts in converted S3D video," in Stereoscopic Displays and Applications XXV, Vol. 9011, 2014.

06 May 2020
See Also
Call for HEVC codecs 2019
Fourteen modern video codec comparison
Parallax range estimation in S3D video
Parallax determines the depth of S3D movies. The range of parallaxes should be both comfortable and entertaining for spectators.
Geometric distortions analysis and correction
Production of low-budget movies is prone to errors. Our method automatically corrects rotation and scale mismatch.
Temporal shift estimation for stereoscopic videos
How to take into account geometric distortions in the estimation of the temporal shift?
Neural network-based algorithm for classification of stereoscopic video by the production method
What method was used to create the 3D scene?
Perspective distortions estimation
How to detect a mismatch in the vertical position of the cameras?
Site structure