The goal of this work is to improve the quality of 3D videos using depth information in video plus depth format, which transmit with low frame rate. The depth information is utilized in three ways to improve the accuracy of motion estimation. One usage is that depth frame is added in block matching criterion to estimate initial motion. Another usage is to detect object boundary region combining depth value variation and local depth distribution adjustment technique. Finally, motion vector field is refined adaptively by segmenting macroblocks into object regions. Any existing block-based motion-compensated frame interpolation methods may utilize our method to refine motion vector field by using depth frames.

Example of interpolated frames (a) original (b) Full Search(FS) (C) FS with proposed method (d) original (e) Multi-level Successive Eliminate Algorithm(MSEA) (f) MSEA with proposed method.

References

[1] Y. Lee, Z. Lee, T. Q. Nguyen, "Frame Rate Up Conversion Of 3D Video by Motion and Depth Fusion", IVMSP, Jun. 2013.[link]
[2] X. Yang, J. Liu, Y. Lee, and T. Q. Nguyen, "Depth-Assisted Frame Rate Up-Conversion for Stereo-scopic Video", SPL, 2014.[link]
[3] S.H. Chan, D.T. V˜o, T.Q. Nguyen, "Subpixel Motion Estimation without Interpolation”, ICASSP, 2010.
[4] Y.-L. Lee and T.Q. Nguyen, "High Frame Rate Motion Compensated Frame Interpolation in High-Definition video Processing” , ICASSP, 2010.