Computational Photography

Combining smart optics, sensors and processing to enable novel imaging applications.

With advances in sensors and computational power, cameras coupled with computation offer us new possibilities which were not possible with traditional film based cameras. Computational photography is emerging as a new field combining computer vision, graphics, optics and imaging to overcome the limitations of current cameras.

A key aspect is to design novel sensors for specific applications that go beyond the traditional capturing of the scene as a regular grid of pixel intensities. Such sensors, for example, include coding and modulation strategies along dimensions of space, time, angle and/or wavelength. The imaging group at MERL has been at the forefront of developing fundamental technologies for computational photography. The flutter shutter (coded exposure) camera and mask-based heterodyne light field camera are few of the key inventions of our group.

  • Researchers

  • News & Events

    •  NEWS   MERL presents 3 papers at ASRU 2017, John Hershey serves as general chair
      Date: December 16, 2017 - December 20, 2017
      Where: Okinawa, Japan
      MERL Contacts: Chiori Hori; Takaaki Hori; Jonathan Le Roux
      Research Areas: Multimedia, Speech & Audio
      Brief
      • MERL presented three papers at the 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), which was held in Okinawa, Japan from December 16-20, 2017. ASRU is the premier speech workshop, bringing together researchers from academia and industry in an intimate and collegial setting. More than 270 people attended the event this year, a record number. MERL's Speech and Audio Team was a key part of the organization of the workshop, with John Hershey serving as General Chair, Chiori Hori as Sponsorship Chair, and Jonathan Le Roux as Demonstration Chair. Two of the papers by MERL were selected among the 10 finalists for the best paper award. Mitsubishi Electric and MERL were also Platinum sponsors of the conference, with MERL awarding the MERL Best Student Paper Award.
    •  
    •  EVENT   MERL leads organization of dialog technology challenges and associated workshop
      Date: Sunday, December 10, 2017
      MERL Contacts: Bret Harsham; Chiori Hori; Takaaki Hori
      Location: Hyatt Regency, Long Beach, CA
      Research Areas: Multimedia, Speech & Audio
      Brief
      • MERL researcher Chiori Hori led the organization of the 6th edition of the Dialog System Technology Challenges (DSTC6). This year's edition of DSTC is split into three tracks: End-to-End Goal Oriented Dialog Learning, End-to-End Conversation Modeling, and Dialogue Breakdown Detection. A total of 23 teams from all over the world competed in the various tracks, and will meet at the Hyatt Regency in Long Beach, CA, USA on December 10 to present their results at a dedicated workshop colocated with NIPS 2017.

        MERL's Speech and Audio Team and Mitsubishi Electric Corporation jointly submitted a set of systems to the End-to-End Conversation Modeling Track, obtaining the best rank among 19 submissions in terms of objective metrics.
    •  

    See All News & Events for Computational Photography
  • Recent Publications

    •  Liu, M.-Y., Tuzel, O., "Coupled Generative Adversarial Nets", arXiv, June 2016.
      BibTeX Download PDFRead TR2016-070
      • @techreport{MERL_TR2016-070,
      • author = {Liu, M.-Y. and Tuzel, O.},
      • title = {Coupled Generative Adversarial Nets},
      • institution = {MERL - Mitsubishi Electric Research Laboratories},
      • address = {Cambridge, MA 02139},
      • number = {TR2016-070},
      • month = jun,
      • year = 2016,
      • url = {http://www.merl.com/publications/TR2016-070/}
      • }
    •  Kadambi, A.; Boufounos, P.T., "Coded Aperture Compressive 3-D LIDAR", IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), DOI: 10.1109/ICASSP.2015.7178153, April 2015, pp. 1166-1170.
      BibTeX Download PDFRead TR2015-028
      • @inproceedings{Kadambi2015apr,
      • author = {Kadambi, A. and Boufounos, P.T.},
      • title = {Coded Aperture Compressive 3-D LIDAR},
      • booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
      • year = 2015,
      • pages = {1166--1170},
      • month = apr,
      • publisher = {IEEE},
      • doi = {10.1109/ICASSP.2015.7178153},
      • url = {http://www.merl.com/publications/TR2015-028}
      • }
    •  Ataer-Cansizoglu, E.; Taguchi, Y.; Ramalingam, S.; Miki, Y., "Calibration of Non-overlapping Cameras Using an External SLAM System", International Conference on 3D Vision (3DV), DOI: 10*1109/eDV.2014.106, December 2014, pp. 509-516.
      BibTeX Download PDFRead TR2014-106
      • @inproceedings{Cansizoglu2014dec,
      • author = {Ataer-Cansizoglu, E. and Taguchi, Y. and Ramalingam, S. and Miki, Y.},
      • title = {Calibration of Non-overlapping Cameras Using an External SLAM System},
      • booktitle = {International Conference on 3D Vision (3DV)},
      • year = 2014,
      • pages = {509--516},
      • month = dec,
      • publisher = {IEEE},
      • doi = {10*1109/eDV.2014.106},
      • url = {http://www.merl.com/publications/TR2014-106}
      • }
    •  Taguchi, Y., "Rainbow Flash Camera: Depth Edge Extraction Using Complementary Colors", International Journal of Computer Vision, DOI: 10.1007/s11263-014-0726-4, ISSN: 1573-1405, ISBN: 0920-5691, Vol. 110, No. 2, pp. 156-171, November 2014.
      BibTeX Download PDFRead TR2014-059
      • @article{Taguchi2014may,
      • author = {Taguchi, Y.},
      • title = {Rainbow Flash Camera: Depth Edge Extraction Using Complementary Colors},
      • journal = {International Journal of Computer Vision},
      • year = 2014,
      • volume = 110,
      • number = 2,
      • pages = {156--171},
      • month = nov,
      • doi = {10.1007/s11263-014-0726-4},
      • issn = {1573-1405},
      • isbn = {0920-5691},
      • url = {http://www.merl.com/publications/TR2014-059}
      • }
    •  Agrawal, A.; Ramalingam, S., "Single Image Calibration of Multi-Axial Imaging Systems", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI: 10.1109/CVPR.2013.184, ISSN: 1063-6919, June 2013, pp. 1399-1406.
      BibTeX Download PDFRead TR2013-041
      • @inproceedings{Agrawal2013jun1,
      • author = {Agrawal, A. and Ramalingam, S.},
      • title = {Single Image Calibration of Multi-Axial Imaging Systems},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2013,
      • pages = {1399--1406},
      • month = jun,
      • doi = {10.1109/CVPR.2013.184},
      • issn = {1063-6919},
      • url = {http://www.merl.com/publications/TR2013-041}
      • }
    •  Liu, M.-Y.; Tuzel, O.; Taguchi, Y., "Joint Geodesic Upsampling of Depth Images", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI: 10.1109/CVPR.2013.29, ISSN: 1063-6919, June 2013, pp. 169-176.
      BibTeX Download PDFRead TR2013-042
      • @inproceedings{Liu2013jun,
      • author = {Liu, M.-Y. and Tuzel, O. and Taguchi, Y.},
      • title = {Joint Geodesic Upsampling of Depth Images},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2013,
      • pages = {169--176},
      • month = jun,
      • doi = {10.1109/CVPR.2013.29},
      • issn = {1063-6919},
      • url = {http://www.merl.com/publications/TR2013-042}
      • }
    •  Agrawal, A.; Ramalingam, S.; Taguchi, Y.; Chari, V., "A Theory of Multi-Layer Flat Refractive Geometry", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2012, pp. 3346-3353.
      BibTeX Download PDFRead TR2012-047
      • @inproceedings{Agrawal2012jun,
      • author = {Agrawal, A. and Ramalingam, S. and Taguchi, Y. and Chari, V.},
      • title = {A Theory of Multi-Layer Flat Refractive Geometry},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2012,
      • pages = {3346--3353},
      • month = jun,
      • url = {http://www.merl.com/publications/TR2012-047}
      • }
    •  Wu, D.; O'Toole, M.; Velten, A.; Agrawal, A.; Raskar, R., "Decomposing Global Light Transport using Time of Flight Imaging", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI: 10.1109/CVPR.2012.6247697, June 2012, pp. 366-373.
      BibTeX Download PDFRead TR2012-042
      • @inproceedings{Wu2012jun,
      • author = {Wu, D. and O'Toole, M. and Velten, A. and Agrawal, A. and Raskar, R.},
      • title = {Decomposing Global Light Transport using Time of Flight Imaging},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2012,
      • pages = {366--373},
      • month = jun,
      • doi = {10.1109/CVPR.2012.6247697},
      • url = {http://www.merl.com/publications/TR2012-042}
      • }
    See All Publications for Computational Photography
  • Free Downloads