Computational Photography

Combining smart optics, sensors and processing to enable novel imaging applications.

With advances in sensors and computational power, cameras coupled with computation offer us new possibilities which were not possible with traditional film based cameras. Computational photography is emerging as a new field combining computer vision, graphics, optics and imaging to overcome the limitations of current cameras.

A key aspect is to design novel sensors for specific applications that go beyond the traditional capturing of the scene as a regular grid of pixel intensities. Such sensors, for example, include coding and modulation strategies along dimensions of space, time, angle and/or wavelength. The imaging group at MERL has been at the forefront of developing fundamental technologies for computational photography. The flutter shutter (coded exposure) camera and mask-based heterodyne light field camera are few of the key inventions of our group.

  • Researchers

  • News & Events

    •  NEWS   Andrew Knyazev (MERL) presents at WPI SIAM Industry Speaker Series about his career path
      Date: April 19, 2018
      Where: Room 202 Stratton Hall Worcester Polytechnic Institute
      MERL Contacts: Joseph Katz; Andrew Knyazev
      Research Areas: Algorithms, Advanced Control Systems, Computational Photography, Computational Sensing, Decision Optimization, Digital Video, Machine Learning, Optical Communications & Devices, Predictive Modeling, Wireless Communications & Signal Processing
      Brief
      • Andrew Knyazev, Distinguished Research Scientist of MERL, has accepted an invitation to speak at the Worcester Polytechnic Institute (WPI) chapter of the Society for Industrial and Applied Mathematics (SIAM) located in Worcester, MA, at a series of industry speakers about different career paths for applied mathematicians.

        Andrew Knyazev studied at the Department of Computational Mathematics and Cybernetics of the Moscow State University in 1976-1981. He obtained PhD Degree in Numerical Mathematics at the Russian Academy of Sciences (RAS) in 1985. Knyazev worked at the Kurchatov Institute in 1981-1983 and at the Institute of Numerical Mathematics RAS in 1983-1992, where he collaborated with Academician Bakhvalov (Erdos number 3 via Kantorovich) on numerical methods for homogenization. In 1993-1994, Knyazev held a visiting position at the Courant Institute of Mathematical Sciences of New York University. From 1994 and until retirement in 2014, he was a Professor of Mathematics at the University of Colorado Denver (CU Denver), supported by many grants from the National Science Foundation and the United States Department of Energy. He was awarded the title of CU Denver Professor Emeritus and named the SIAM Fellow in 2016. During his 30 years in the academy, Knyazev supervised 7 PhD students. He is best known for his Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) eigenvalue solver. In 2012, Knyazev starts his industrial research career joining Mitsubishi Electric Research Laboratories (MERL) in Cambridge, MA, where he invents and develops algorithms for control, machine learning, data sciences, computer vision, coding, communications, material sciences, and signal processing, having 11 US patent applications filed (6 issued, 5 pending) and over 20 papers published.
    •  
    •  NEWS   Andrew Knyazev (MERL) invited to 2018 MathWorks Research Summit
      Date: June 2, 2018 - June 4, 2018
      Where: Newton, Massachusetts (USA)
      MERL Contact: Andrew Knyazev
      Research Areas: Algorithms, Advanced Control Systems, Computational Photography, Dynamical Systems, Machine Learning, Predictive Modeling
      Brief
      • Dr. Andrew Knyazev of MERL has accepted an invitation to participate at the 2018 MathWorks Research Summit. The objective of the Research Summit is to provide a forum for leading researchers in academia and industry to explore the latest research and technology results and directions in computation and its use in technology, engineering, and science. The event aims to foster discussion among scientists, engineers, and research faculty about challenges and research opportunities for the respective communities with a particular interest in exploring cross-disciplinary research avenues.
    •  

    See All News & Events for Computational Photography
  • Recent Publications

    •  Liu, M.-Y., Tuzel, O., "Coupled Generative Adversarial Nets," Tech. Rep. TR2016-070, arXiv, June 2016.
      BibTeX Download PDFRead TR2016-070
      • @techreport{MERL_TR2016-070,
      • author = {Liu, M.-Y. and Tuzel, O.},
      • title = {Coupled Generative Adversarial Nets},
      • institution = {MERL - Mitsubishi Electric Research Laboratories},
      • address = {Cambridge, MA 02139},
      • number = {TR2016-070},
      • month = jun,
      • year = 2016,
      • url = {http://www.merl.com/publications/TR2016-070/}
      • }
    •  Kadambi, A.; Boufounos, P.T., "Coded Aperture Compressive 3-D LIDAR", IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), DOI: 10.1109/ICASSP.2015.7178153, April 2015, pp. 1166-1170.
      BibTeX Download PDFRead TR2015-028
      • @inproceedings{Kadambi2015apr,
      • author = {Kadambi, A. and Boufounos, P.T.},
      • title = {Coded Aperture Compressive 3-D LIDAR},
      • booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
      • year = 2015,
      • pages = {1166--1170},
      • month = apr,
      • publisher = {IEEE},
      • doi = {10.1109/ICASSP.2015.7178153},
      • url = {http://www.merl.com/publications/TR2015-028}
      • }
    •  Ataer-Cansizoglu, E.; Taguchi, Y.; Ramalingam, S.; Miki, Y., "Calibration of Non-overlapping Cameras Using an External SLAM System", International Conference on 3D Vision (3DV), DOI: 10*1109/eDV.2014.106, December 2014, pp. 509-516.
      BibTeX Download PDFRead TR2014-106
      • @inproceedings{Cansizoglu2014dec,
      • author = {Ataer-Cansizoglu, E. and Taguchi, Y. and Ramalingam, S. and Miki, Y.},
      • title = {Calibration of Non-overlapping Cameras Using an External SLAM System},
      • booktitle = {International Conference on 3D Vision (3DV)},
      • year = 2014,
      • pages = {509--516},
      • month = dec,
      • publisher = {IEEE},
      • doi = {10*1109/eDV.2014.106},
      • url = {http://www.merl.com/publications/TR2014-106}
      • }
    •  Taguchi, Y., "Rainbow Flash Camera: Depth Edge Extraction Using Complementary Colors", International Journal of Computer Vision, DOI: 10.1007/s11263-014-0726-4, ISSN: 1573-1405, ISBN: 0920-5691, Vol. 110, No. 2, pp. 156-171, November 2014.
      BibTeX Download PDFRead TR2014-059
      • @article{Taguchi2014may,
      • author = {Taguchi, Y.},
      • title = {Rainbow Flash Camera: Depth Edge Extraction Using Complementary Colors},
      • journal = {International Journal of Computer Vision},
      • year = 2014,
      • volume = 110,
      • number = 2,
      • pages = {156--171},
      • month = nov,
      • doi = {10.1007/s11263-014-0726-4},
      • issn = {1573-1405},
      • isbn = {0920-5691},
      • url = {http://www.merl.com/publications/TR2014-059}
      • }
    •  Agrawal, A.; Ramalingam, S., "Single Image Calibration of Multi-Axial Imaging Systems", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI: 10.1109/CVPR.2013.184, ISSN: 1063-6919, June 2013, pp. 1399-1406.
      BibTeX Download PDFRead TR2013-041
      • @inproceedings{Agrawal2013jun1,
      • author = {Agrawal, A. and Ramalingam, S.},
      • title = {Single Image Calibration of Multi-Axial Imaging Systems},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2013,
      • pages = {1399--1406},
      • month = jun,
      • doi = {10.1109/CVPR.2013.184},
      • issn = {1063-6919},
      • url = {http://www.merl.com/publications/TR2013-041}
      • }
    •  Liu, M.-Y.; Tuzel, O.; Taguchi, Y., "Joint Geodesic Upsampling of Depth Images", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI: 10.1109/CVPR.2013.29, ISSN: 1063-6919, June 2013, pp. 169-176.
      BibTeX Download PDFRead TR2013-042
      • @inproceedings{Liu2013jun,
      • author = {Liu, M.-Y. and Tuzel, O. and Taguchi, Y.},
      • title = {Joint Geodesic Upsampling of Depth Images},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2013,
      • pages = {169--176},
      • month = jun,
      • doi = {10.1109/CVPR.2013.29},
      • issn = {1063-6919},
      • url = {http://www.merl.com/publications/TR2013-042}
      • }
    •  Agrawal, A.; Ramalingam, S.; Taguchi, Y.; Chari, V., "A Theory of Multi-Layer Flat Refractive Geometry", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2012, pp. 3346-3353.
      BibTeX Download PDFRead TR2012-047
      • @inproceedings{Agrawal2012jun,
      • author = {Agrawal, A. and Ramalingam, S. and Taguchi, Y. and Chari, V.},
      • title = {A Theory of Multi-Layer Flat Refractive Geometry},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2012,
      • pages = {3346--3353},
      • month = jun,
      • url = {http://www.merl.com/publications/TR2012-047}
      • }
    •  Wu, D.; O'Toole, M.; Velten, A.; Agrawal, A.; Raskar, R., "Decomposing Global Light Transport using Time of Flight Imaging", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI: 10.1109/CVPR.2012.6247697, June 2012, pp. 366-373.
      BibTeX Download PDFRead TR2012-042
      • @inproceedings{Wu2012jun,
      • author = {Wu, D. and O'Toole, M. and Velten, A. and Agrawal, A. and Raskar, R.},
      • title = {Decomposing Global Light Transport using Time of Flight Imaging},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2012,
      • pages = {366--373},
      • month = jun,
      • doi = {10.1109/CVPR.2012.6247697},
      • url = {http://www.merl.com/publications/TR2012-042}
      • }
    See All Publications for Computational Photography
  • Free Downloads