Switch Mode Based Deep Fractional Interpolation in Video Coding
Abstract
Fractional interpolation is a significant technology in motion compensation of video coding. It generates subpixel level reference samples in inter prediction to facilitate temporal redundancy removal between video frames. Recently, some methods explore to introduce the deep learning technique for fractional interpolation and have obtained better compression results. However, existing deep learning based methods still treat fractional interpolation as a traditional interpolation problem but fail to adjust it to the motion compensation scenario. In this paper, we design a switch mode based deep fractional interpolation method to introduce integer pixels of different positions to the interpolation of sub-pixel position samples. By switching between integer pixels of different positions, our method can infer the sub-pixels with smaller variations and achieve better fractional interpolation results. Consequently the motion compensation performance can be further improved. Experimental results have also verified the efficiency of the switch mode based deep fractional interpolation. Compared with High Efficiency Video Coding, our method achieves 2.8% bit saving on average and up to 6.2% bit saving under low-delay P configuration.
Framework
Figure. 1. Framework of the proposed variation learning network. The network first extracts the shared feature map from the integer-position sample. Then the variations that identify the differences between different sub-pixel position samples and integer-position samples are inferred separately from the shared feature map by group convolution. Sub-pixel position samples can be later derived by adding the inferred variations to corresponding integer-position samples.
Results
Table 1: BD-rate reduction of the proposed method compared to HEVC.
Citation
@inproceedings{SwitchMode, author={S. Xia and W. Yang and Y. Hu and W. H. Cheng and J. Liu}, booktitle={IEEE International Symposium on Circuits and Systems (ISCAS)}, title={Switch Mode Based Deep Fractional Interpolation in Video Coding}, year={2019}, }