Morphable 3D models from video


Nonrigid 3D structure-from-motion and 2D optical flow can both be formulated as tensor factorization problems. The two problems can be made equivalent through a noisy affine transform, yielding a combined nonrigid structure-from-intensities problem that we solve via structured matrix decompositions. Often the preconditions for this factorization are violated by image noise and deficiencies of the data vis-a-vis the sample complexity of the problem. Both issues are remediated with careful use of rank constraints, norm constraints, and integration over uncertainty in the intensity values, yielding novel solutions for SVD under uncertainty, factorization under uncertainty, nonrigid factorization, and subspace optical flow. The resulting integrated algorithm can track and 3D-reconstruct nonrigid surfaces that have very little texture, for example the smooth parts of the face. Working with low-resolution low-texture "found video," these methods produce good tracking and 3D reconstruction results where prior algorithms fail. NB: Winner, IEEE CVPR 2001 Best Paper Award.


  • Related News & Events

    •  NEWS    CVPR 2001: 4 publications by Paul Beardsley, Matthew Brand, Ramesh Raskar and Michael Jones
      Date: December 9, 2001
      Where: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
      MERL Contacts: Michael J. Jones; Matthew Brand
      • The papers "Morphable 3D Models from Video" by Brand, M.E., "Flexible Flow for 3D Nonrigid Tracking and Shape Recovery" by Brand, M.E. and Bhotika, R., "A Self-Correcting Projector" by Raskar, R. and Beardsley, P.A. and "Rapid Object Detection Using a Boosted Cascade of Simple Features" by Viola, P. and Jones, M. were presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).