A Conditional Random Field for Automatic Photo Editing

    •  Brand, M.; Pletscher, P., "A Conditional Random Field for Automatic Photo Editing", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), ISSN: 1063-6919, June 2008, pp. 1-7.
      BibTeX Download PDF
      • @inproceedings{Brand2008jun,
      • author = {Brand, M. and Pletscher, P.},
      • title = {A Conditional Random Field for Automatic Photo Editing},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2008,
      • pages = {1--7},
      • month = jun,
      • issn = {1063-6919},
      • url = {}
      • }
  • MERL Contact:
  • Research Area:


TR Image
We visually represent the distribution over labellings with a false-color image that indicates, at each pixel site, the label having maximum marginal likelihood (henceforth, maximum-of-marginals). Class likelihoods in turn inform local texture filtering or replacement, here yielding a resynthesized image with red-eye corrected and blemishes removed, but birthmarks preserved. The entire process is automatic and trained discriminatively from before-and-after images.

We introduce a method for fully automatic touch-up of face images by making inferences about the structure of the scene and undesirable textures in the image. A distribution over image segmentations and labelings is computed via a conditional random field; this distribution controls the application of various local image transforms to regions in the image. Parameters governing both the labeling and transforms are jointly optimized w.r.t. a training set of before-and-after example images. One major advantage of our formulation is the ability to marginalize over all possible labeling and thus exploit all the information in the distribution; this yield better results than MAP inference. We demonstrate with a system that is trained to correct red-eye, reduce specularities, and remove acne and other blemishes from faces, showing results with test images scavenged from acne-themed internet message boards.