TR2006-106

Natural Video Matting using Camera Arrays


    •  Joshi, N., Matusik, W., Avidan, S., "Natural Video Matting using Camera Arrays", ACM SIGGRAPH, July 2006, vol. 25, pp. 779-786.
      BibTeX TR2006-106 PDF
      • @inproceedings{Joshi2006jul,
      • author = {Joshi, N. and Matusik, W. and Avidan, S.},
      • title = {Natural Video Matting using Camera Arrays},
      • booktitle = {ACM SIGGRAPH},
      • year = 2006,
      • volume = 25,
      • number = 3,
      • pages = {779--786},
      • month = jul,
      • issn = {0730-0301},
      • url = {https://www.merl.com/publications/TR2006-106}
      • }
  • Research Area:

    Computer Vision

Abstract:

We present an algorithm and a system for high-quality natural video matting using a camera array. The system uses high frequencies present in natural scenes to compute mattes by creating a synthetic aperture image that is focused on the foreground object, which reduces the variance of pixels reprojected from the foreground while increasing the variance of pixels reprojected from the background. We modify the standard matting equation to work directly with variance measurements and show how these statistics can be used to construct a trimap that is later upgraded to an alpha matte. The entire process is completely automatic, including an automatic method for focusing the synthetic aperture image on the foreground object and an automatic method to compute the trimap and the alpha matte. The proposed algorithm is very efficient and has a per-pixel running time that is linear in the number of cameras. Our current system runs at several frames per second, and we believe that it is the first system capable of computing high-quality alpha mattes at near real-time rates without the use of active illumination or special backgrounds.

 

  • Related News & Events

    •  NEWS    ACM SIGGRAPH 2006: 7 publications by Paul Beardsley, Ron Perry, Darren Leigh, Ramesh Raskar and Jeroen van Baar
      Date: July 30, 2006
      Where: ACM SIGGRAPH
      Brief
      • The papers "Cartoon Dioramas in Motion" by Raskar, R., Ziegler, R. and Willwacher, T., "Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering Using Multi-flash Imaging" by Raskar, R., Feris, R., Yu, J. and Turk, M., "RFIG Lamps: Interacting with a Self-Describing World via Photosensing Wireless Tags and Projectors" by Raskar, R., Beardsley, P., van Baar, J., Wang, Y., Dietz, P., Lee, J., Leigh, D. and Willwacher, T., "Designing with Distance Fields" by Frisken, S.F. and Perry, R.N., "An Improved Representation for Stroke-based Fonts" by Jakubiak, E.J., Perry, R.N. and Frisken, S.F., "iLamps: Geometrically Aware and Self-configuring Projectors" by Raskar, R., van Baar, J., Beardsley, P., Willwacher, T., Rao, S. and Forlines, C. and "Natural Video Matting using Camera Arrays" by Joshi, N., Matusik, W. and Avidan, S. were presented at ACM SIGGRAPH.
    •