MSU Video Quality Measurement tools: Metric Performance

MSU Graphics & Media Lab (Video Group)

Table of contents

Metrics GPU acceleration

Now SSIM-based algorithms show best subjective quality correlation among other video quality algorithms. To increase performance of SSIM-based metrics, these algorithms was implemented on graphics hardware. For implementations CUDA technology was used. SSIM, 3-SSIM, and MS-SSIM metrics implemented for now. Our CPU Implementation of the metrics above are using Intel IPP. We don’t know precision and operations order of these algorithm in the IPP, so metric values for GPU implementations can slightly differ from the CPU. These implementations can be found in the metric list in the GUI or via -metr ssim_cuda, 3ssim_cuda, msssim_cuda parameters via console line interface.
Data for this graph was obtained in the following way:

  1. Speed was measured 3 times for each metric, PC configuration and resolution via console interface of VQMT PRO version
  2. Result was calculated as median value between 3 values from the first step

Speedup results provided in the graphs below as fps graph and speedup graphs:


SSIM metric speedup graph

SSIM metric fps graph

3-SSIM metric speedup graph

3-SSIM metric fps graph

MS-SSIM metric speedup graph

MS-SSIM metric fps graph

Metric speed performance

Measurements of large files can take a very long time. We are always trying to maximize speed of metric implementations using features like multi-threading, SSE\MMX optimizations, high-performance libraries, GPU acceleration. In example, using console interface it is able compute four most popular metrics (PSNR, SSIM, 3-SSIM, MS-SSIM) in almost the same time as the slowest of them.
Data for this graph was obtained in the following way:

  1. Speed was measured 3 times for each metric, PC configuration and resolution via console interface of VQMT PRO version
  2. Time for multi-metric measurement was obtained via multiple “-metr” parameters in the console interface
  3. Result was calculated as median value between 3 values from the first step

Here we provide metric performance graph for different resolutions for two PC configurations:


Speed graph for PC Configuration: Intel Core i7 920 @ 2.67 GHz, 12GB RAM, NVIDIA GTX 580
High Resolution Video - 720p & 1080p

Speed graph for PC Configuration: Intel Core i7 920 @ 2.67 GHz, 12GB RAM, NVIDIA GTX 580
Low Resolution Video - CIF & SD

Speed graph for PC Configuration: Intel Core Quad Q6600 @ 2.4 GHz, 4GB RAM, NVIDIA GTX 285
High Resolution Video - 720p & 1080p

Speed graph for PC Configuration: Intel Core Quad Q6600 @ 2.4 GHz, 4GB RAM, NVIDIA GTX 285
Low Resolution Video - CIF & SD

Metric speed performance with correlation

We are also providing Speed/Correlation plot, which is allows user to understand difference between metrics.
Information about subjective quality you can find here.
Data provided for 1080p resolution and following configuration: Intel Core i7 920 @ 2.67 GHz, 12GB RAM, NVIDIA GTX 580.


Metrics speed/correlation plot for 1080p resolution on Intel Core i7 with NVIDIA GTX 580

Contacts

E-mail: video-measure@graphics.cs.msu.ru

cs.msu.ru>

21 Feb 2019
See Also
Automatic local color correction in S3D video
Stereo video may contain a huge color discrepancy. Most of the problems are hard to eliminate because of possible different distortions in each area in the frame.
MSU Video Quality Measurement Tool: Why upgrade?
MSU Video Quality Measurement Tool (PSNR, MSE, VQM, SSIM)
MSU Video Quality Measurement Tool (VQMT) is a program for objective video quality assessment.
MSU VQMT Subjective quality correlation
Comparison provided for the most popular metrics: PSNR, SSIM, 3-SSIM, MS-SSIM and new stSSIM
MSU Video Quality Measurement Tool: SDK
MSU Perceptual Video Quality: Subjective video quality methods information
Site structure