• 
• 
• 
• 
• 
• 

Multiview Video Coding

We are working on developing advanced video compression algorithms for multiview video, i.e., video sequences recorded simultaneously from multiple cameras. We are also actively participating in the multiview video coding standardization activity in MPEG. Target applications for this work include 3D display and free viewpoint video.

Background & Objective:  The need for multiview video coding is driven by two recent technological developments: new 3D display technologies and the growing use of multi-camera arrays. A variety of companies are starting to produce 3D displays that do not require glasses and can be viewed by multiple people simultaneously. The immersive experience provided by these 3D displays are compelling and have the potential to create a growing market for 3D video and hence for multiview video compression. Furthermore, even with 2D displays, multi-camera arrays are increasingly being used to capture a scene from many angles. The resulting multiview data sets allow the viewer to observe a scene from any viewpoint and serve as another application of multiview video compression.

Technical Discussion:  In July 2008, MPEG officially approved an amendment of the ITU-T Rec. H.264 & ISO/IEC 14996-10 Advanced Video Coding (AVC) standard on Multiview Video Coding. This new standard enables an efficient compressed representation of stereo and multiview video by exploiting correlation among neighboring camera views to support 3D and free-viewpoint video applications. MERL has contributed several technologies to this standard including methods for reference picture management, spatial random access and signaling of scene information.

Future Direction:  Continue to work on improving the coding efficiency of multiview video compression while actively being involved in related standardization activities such as MPEG EE(Exploration Experiment) in 3D Video.

Publications:
Vetro, A.; Yea, S; Smolic, A., "Towards a 3D Video Format for Auto-Stereoscopic Displays", SPIE Conference on Applications of Digital Image Processing XXXI, Vol. 7073, September 2008 (Proceedings of SPIE, TR2008-057)

Yea, S.; Vetro, A., "View Synthesis Prediction for Rate-Overhead Reduction in FTV", 3DTV-Conference, ISBN: 978-1-4244-1670-5, pp. 145-148, May 2008 (IEEE Xplore, TR2008-016)

Zwicker, M.; Vetro, A.; Yea, S.; Matusik, W.; Pfister, H.; Durand, F., "Resampling, Antialiasing, and Compression in Multiview 3-D Displays", IEEE Signal Processing Magazine, ISSN: 1053-5888, Vol. 24, Issue 6, pp. 88-96, November, 2007 (IEEE Xplore, TR2007-084)

Zwicker, M.; Yea, S.; Vetro, A.; Forlines, C.; Matusik, W.; Pfister, H., "Display Pre-filtering for Multi-view Video Compression", International Conference on Multimedia (ACM Multimedia), ISBN: 978-1-59593-702-5, Session: Systems 4 - Coding Support - pp. 1046-1053, September 2007 (ACM Press, TR2007-073)

Yea, S.; Vetro, A., "RD-Optimized View Synthesis Prediction for Multiview Video Coding", IEEE International Conference on Image Processing (ICIP), ISSN: 1522-4880, Vol. 1, pp I-209 - I-212, September 2007 (IEEE Xplore, TR2007-026)

Ince, S.; Martinian, E.; Yea, S.; Vetro, A., "Depth Estimation for View Synthesis in Multiview Video Coding", 3DTV Conference (3DTV-CON), May 2007 (3DTV-CON, TR2007-025)

Martinian, E.; Behrens, A.; Xin, J.; Vetro, A.; Sun, H., "Extensions of H.264/AVC for Multiview Video Compression", IEEE International Conference on Image Processing (ICIP), ISSN: 1522-4880, pp. 2981-2984, October 2006 (IEEE Xplore, TR2006-048)

Martinian, E.; Behrens, A.; Xin, J.; Vetro, A., "View Synthesis for Multiview Video Compression", Picture Coding Symposium (PCS), April 2006 (PCS 2006, TR2006-035)

Vetro, A.; Matusik, W.; Pfister, H.; Xin, J., "Coding Approaches for End-to-End 3D TV Systems", Picture Coding Symposium (PCS), December 2004 (TR2004-137)

Technology Areas:
Multimedia
Digital Video

Modification Date:  October 2, 2008