TR2008-064

Improved Seam Carving for Video Retargeting


    •  Rubinstein, M.; Shamir, A.; Avidan, A., "Improved Seam Carving for Video Retargeting", ACM Transactions on Graphics (TOG), Vol. 27, No. 3, August 2008.
      BibTeX Download PDF
      • @article{Rubinstein2008aug,
      • author = {Rubinstein, M. and Shamir, A. and Avidan, A.},
      • title = {Improved Seam Carving for Video Retargeting},
      • journal = {ACM Transactions on Graphics (TOG)},
      • year = 2008,
      • volume = 27,
      • number = 3,
      • month = aug,
      • url = {http://www.merl.com/publications/TR2008-064}
      • }
  • MERL Contact:
  • Research Areas:

    Computational Photography, Computer Vision


TR Image
Improved seam carving for video sequences combines the frames of the video to form a 3D cube and finds 2D monotonic and connected manifold seams using graph cuts. The intersection of the manifolds with each frame defines the seams on the frame. The manifolds are found using a new forward-energy criterion that reduces both spatial and temporal artifacts considerably.

Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes. In the new formulation, a seam is given by a minimal cut in the graph and we show how to construct a graph such that the resulting cut is a valid seam. That is, the cut is monotonic and connected. In addition, we present a novel energy criterion that improves the visual quality of the retargeted images and videos. The original seam carving operator is focused on removing seams with the least amount of energy, ignoring energy that is introduced into the images and video by applying the operator. To counter this, the new criterion is looking forward in time - removing seams that introduce the least amount of energy into the retargeted result. We show how to encode the improved criterion into graph cuts (for images and video) as well as dynamic programming (for images). We apply our technique to images and videos and present results of various applications.