Automatic color mismatch estimation in S3D videos using confidence maps

Introduction

Usually, the cause of discomfort while watching stereoscopic video is the mismatch between the left and right views. One of the most common types of distortion is color mismatch between stereoscopic views.

We propose a new confidence-map-based algorithm that automatically detects the frames containing color distortions between two views of a stereoscopic video.

Preview

Proposed method

Usage of confidence maps allows to detect the unreliable disparity values and therefore decreases the amount of false positives produced by our approach.

The proposed algorithm is a modified baseline model that consists of the following stages:

  1. Global color correction of the right view
  2. Matching of the right and left views
  3. Compensation of the left view to the right based on the results of the comparison
  4. Cutting off part of the minimum and maximum difference values between the right and compensated left views, filtering views to restore the discarded values
  5. Calculation the normalized sum of squared difference values

A significant difference from the previous version of the algorithm is the use of confidence maps, that significantly reduces the number of algorithm’s false positives. Additionally, the value filtering algorithm in step 4 of the basic algorithm was modified, and mean square error (MSE) function was replaced with the sum of absolute differences (SAD).

Example

a — left view
b — right view
c — compensated left view
d — visualization of the result of the basic algorithm
e — visualization of the result using confidence
f — visualization of the confidence map

Experiments

Comparisons were based on samples of scenes from 105 stereoscopic movies. Only scenes, where the values of the baseline algorithm exceeded the preset threshold, were selected. The sample size was 957 frames. Then, all selected frames were manually classified as true positive and false positive.

Comparison of the basic algorithm with various modifications

Comparison

During testing, the proposed modifications showed better results compared to the basic version of the algorithm.

05 May 2020
See Also
Call for HEVC codecs 2019
Fourteen modern video codec comparison
Parallax range estimation in S3D video
Parallax determines the depth of S3D movies. The range of parallaxes should be both comfortable and entertaining for spectators.
Geometric distortions analysis and correction
Production of low-budget movies is prone to errors. Our method automatically corrects rotation and scale mismatch.
Automatic detection of artifacts in converted S3D videos
Our set of algorithms detects edge sharpness mismatch, cardboard effect, and crosstalk noticeability.
Temporal shift estimation for stereoscopic videos
How to take into account geometric distortions in the estimation of the temporal shift?
Neural network-based algorithm for classification of stereoscopic video by the production method
What method was used to create the 3D scene?
Site structure