Automatic sharpness mismatch detection and compensation in stereo

Introduction

Enlarged fragment of left and right views with visible sharpness mismatch from “Jack the Giant Slayer”

A lot of S3D movies contain artifacts despite large budgets and modern post-processing methods. It is caused by insufficiency in automatization of post-processing and high rate of human errors.

Differences in sharpness between views are usually caused by incorrectly calibrated cameras. While watching movies with sharpness mismatch the spectator may lose sense of 3D, which is the primary goal of stereo, or even get a headache.

Proposed method

The algorithm estimates the amount of local sharpness mismatch for each stereo pair, trying to fix it if possible.

The steps of the algorithm:

Experiments

The “Driving Angry” movie: left — input stereo pair, right — corrected stereo pair

We compared our algorithm with a commercial plug-in for Nuke — Ocula 3.0 FocusMatcher on a set of 8 stereo sequences.

Pros over Ocula 3.0:

Results of the proposed algorithm

Results of comparison with Ocula 3.0 (FocusMatching)

Pictures below illustrate the differences between proposed algorithm and FocusMatcher on a single stereo pair. It is notable that FocusMarcher corrupts the background of semi-transparent objects (red boxes), while proposed algorithm doesn’t have such an effect.

Input views from “Driving Angry”

Result of proposed algorithm (corrected sharpness mismatch, no new artifacts introduced)

Ocula 3.0's result (red boxes contain produced artifacts)

Results

27 May 2019
See Also
Call for HEVC codecs 2019
Fourteen modern video codec comparison
Parallax range estimation in S3D video
Parallax determines the depth of S3D movies. The range of parallaxes should be both comfortable and entertaining for spectators.
Geometric distortions analysis and correction
Production of low-budget movies is prone to errors. Our method automatically corrects rotation and scale mismatch.
Automatic detection of artifacts in converted S3D videos
Our set of algorithms detects edge sharpness mismatch, cardboard effect, and crosstalk noticeability.
Temporal shift estimation for stereoscopic videos
How to take into account geometric distortions in the estimation of the temporal shift?
Neural network-based algorithm for classification of stereoscopic video by the production method
What method was used to create the 3D scene?
Site structure