TR2007-116

Multi-Scale Capture of Facial Geometry and Motion


    •  Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., Gross, M., "Multi-scale Capture of Facial Geometry and Motion", ACM Transactions on Graphics (TOG), Vol. 26, No. 3, July 2007.
      BibTeX TR2007-116 PDF
      • @article{Bickel2007jul,
      • author = {Bickel, B. and Botsch, M. and Angst, R. and Matusik, W. and Otaduy, M. and Pfister, H. and Gross, M.},
      • title = {Multi-scale Capture of Facial Geometry and Motion},
      • journal = {ACM Transactions on Graphics (TOG)},
      • year = 2007,
      • volume = 26,
      • number = 3,
      • month = jul,
      • issn = {0730-0301},
      • url = {https://www.merl.com/publications/TR2007-116}
      • }
  • Research Area:

    Computer Vision

Abstract:

We present a novel multi-scale representation and acquisition method for the animation of high-resolution facial geometry and wrinkles. We first acquire a static scan of the face including reflectance data at the highest possible quality. We then augment a traditional marker-based facial motion-capture system by two synchronized video cameras to track expression wrinkles. The resulting model consists of high-resolution geometry, motion-capture data, and expression wrinkles in 2D parametric form. This combination represents the facial shape and its salient feature at multiple scales. During motion synthesis the motion-capture data deforms the high-resolution geometry using a linear shell-based mesh-deformation method. The wrinkle geometry is added to the facial base mesh using nonlinear energy optimization. We present the results of our approach for performance replay as well as for wrinkle editing.

 

  • Related News & Events

    •  NEWS    ACM Transactions on Graphics (TOG): 6 publications by Ashok Veeraraghavan, Jay Thornton, Amit Agrawal, John Barnwell and others
      Date: July 15, 2007
      Where: ACM Transactions on Graphics (TOG)
      Brief
      • The articles "Prakash: Lighting Aware Motion Capture using Photosensing Markers and Multiplexed Illuminators" by Raskar, R., Nii, H., deDecker, B., Hashimoto, Y., Summet, J., Moore, D., Zhao, Y., Westhues, J., Dietz, P., Barnwell, J., Nayar, S., Inami, M., Bekaert, P., Noland, M., Branzoi, V. and Bruns, E., "Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing" by Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A. and Tumblin, J., "Seam Carving for Content-Aware Image Resizing" by Avidan, S. and Shamir, A., "Factored Time-Lapse Video" by Sunkavalli, K., Matusik, W., Pfister, H. and Rusinkiewicz, S., "Post-production Facial Performance Relighting using Reflectance Transfer" by Peers, P., Tamura, N., Matusik, W. and Debevec, P. and "Multi-scale Capture of Facial Geometry and Motion" by Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H. and Gross, M. were published in ACM Transactions on Graphics (TOG).
    •