Computer Vision

Researchers

Awards

AWARD Best Paper - Honorable Mention Award at WACV 2021
Date: January 6, 2021
Awarded to: Rushil Anirudh, Suhas Lohit, Pavan Turaga
MERL Contact: Suhas Lohit
Research Areas: Computational Sensing, Computer Vision, Machine Learning
Brief
- A team of researchers from Mitsubishi Electric Research Laboratories (MERL), Lawrence Livermore National Laboratory (LLNL) and Arizona State University (ASU) received the Best Paper Honorable Mention Award at WACV 2021 for their paper "Generative Patch Priors for Practical Compressive Image Recovery".
  
  The paper proposes a novel model of natural images as a composition of small patches which are obtained from a deep generative network. This is unlike prior approaches where the networks attempt to model image-level distributions and are unable to generalize outside training distributions. The key idea in this paper is that learning patch-level statistics is far easier. As the authors demonstrate, this model can then be used to efficiently solve challenging inverse problems in imaging such as compressive image recovery and inpainting even from very few measurements for diverse natural scenes.
AWARD MERL Researchers win Best Paper Award at ICCV 2019 Workshop on Statistical Deep Learning in Computer Vision
Date: October 27, 2019
Awarded to: Abhinav Kumar, Tim K. Marks, Wenxuan Mou, Chen Feng, Xiaoming Liu
MERL Contact: Tim K. Marks
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
Brief
- MERL researcher Tim Marks, former MERL interns Abhinav Kumar and Wenxuan Mou, and MERL consultants Professor Chen Feng (NYU) and Professor Xiaoming Liu (MSU) received the Best Oral Paper Award at the IEEE/CVF International Conference on Computer Vision (ICCV) 2019 Workshop on Statistical Deep Learning in Computer Vision (SDL-CV) held in Seoul, Korea. Their paper, entitled "UGLLI Face Alignment: Estimating Uncertainty with Gaussian Log-Likelihood Loss," describes a method which, given an image of a face, estimates not only the locations of facial landmarks but also the uncertainty of each landmark location estimate.
AWARD CVPR 2011 Longuet-Higgins Prize
Date: June 25, 2011
Awarded to: Paul A. Viola and Michael J. Jones
Awarded for: "Rapid Object Detection using a Boosted Cascade of Simple Features"
Awarded by: Conference on Computer Vision and Pattern Recognition (CVPR)
MERL Contact: Michael J. Jones
Research Area: Machine Learning
Brief
- Paper from 10 years ago with the largest impact on the field: "Rapid Object Detection using a Boosted Cascade of Simple Features", originally published at Conference on Computer Vision and Pattern Recognition (CVPR 2001).

See All Awards for MERL

News & Events

NEWS MERL launches Postdoctoral Research Fellow program
Date: September 21, 2022
MERL Contacts: Philip V. Orlik; Anthony Vetro
Research Areas: Applied Physics, Artificial Intelligence, Communications, Computational Sensing, Computer Vision, Control, Data Analytics, Dynamical Systems, Electric Systems, Electronic and Photonic Devices, Machine Learning, Multi-Physical Modeling, Optimization, Robotics, Signal Processing, Speech & Audio
Brief
- Mitsubishi Electric Research Laboratories (MERL) invites qualified postdoctoral candidates to apply for the position of Postdoctoral Research Fellow. This position provides early career scientists the opportunity to work at a unique, academically-oriented industrial research laboratory. Successful candidates will be expected to define and pursue their own original research agenda, explore connections to established laboratory initiatives, and publish high impact articles in leading venues. Please refer to our web page for further details.
TALK [MERL Seminar Series 2022] Prof. Chuang Gan presents talk titled Learning to Perceive Physical Scenes from Multi-Sensory Data
Date & Time: Tuesday, September 6, 2022; 12:00 PM EDT
Speaker: Chuang Gan, UMass Amherst & MIT-IBM Watson AI Lab
MERL Host: Jonathan Le Roux
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & Audio
Abstract
- Human sensory perception of the physical world is rich and multimodal and can flexibly integrate input from all five sensory modalities -- vision, touch, smell, hearing, and taste. However, in AI, attention has primarily focused on visual perception. In this talk, I will introduce my efforts in connecting vision with sound, which will allow machine perception systems to see objects and infer physics from multi-sensory data. In the first part of my talk, I will introduce a. self-supervised approach that could learn to parse images and separate the sound sources by watching and listening to unlabeled videos without requiring additional manual supervision. In the second part of my talk, I will show we may further infer the underlying causal structure in 3D environments through visual and auditory observations. This enables agents to seek the sound source of repeating environmental sound (e.g., alarm) or identify what object has fallen, and where, from an intermittent impact sound.

See All News & Events for Computer Vision

Research Highlights

Internships

CV1703: Software development in ROS for robotic manipulation

MERL is offering an internship position for non-research software development for robotic manipulation. The scope of the internship is to develop robust ROS packages by refactoring existing experimental code. The position is open to prospective candidates with very strong programming skills in ROS (Robot Operating System) using C++ primarily and Python respectively. The selected intern will have a software engineering role rather than research oriented. The position is open to both senior undergraduate students and master students. Flexible start and end dates.
CV1862: 6D pose estimation for machine vision

The Computer Vision group at MERL, in collaboration with Mitsubishi Electric Automotive America, is seeking a highly skilled graduate student for an internship position for Online Pose Estimation of vehicles using machine vision. The research work is expected to be deployed in real-world applications such as vehicle range and pose detection. Candidates should have advanced knowledge of full-6D pose estimation from stereo imaging using both Machine Learning and geometric approaches, point cloud processing using ML and computational geometry. Exceptional programming skills are required, including ROS, C++ and Python. The expected start date is Sep/Oct 2022 and the duration of the internship is 6 months.

See All Internships for Computer Vision

Recent Publications

Rambhatla, S., Jones, M.J., Chellappa, R., "An Empirical Analysis of Boosting Deep Networks", International Joint Conference on Neural Networks (IJCNN), July 2022.
BibTeX TR2022-075 PDF Presentation
- @inproceedings{Rambhatla2022jul,
- author = {Rambhatla, Sai and Jones, Michael J. and Chellappa, Rama},
- title = {An Empirical Analysis of Boosting Deep Networks},
- booktitle = {International Joint Conference on Neural Networks (IJCNN)},
- year = 2022,
- month = jul,
- url = {https://www.merl.com/publications/TR2022-075}
- }
Miraldo, P., Iglesias, J.P., "A Unified Model for Line Projections in Catadioptric Cameras with Rotationally Symmetric Mirrors", IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
BibTeX TR2022-084 PDF
- @inproceedings{Miraldo2022jun,
- author = {Miraldo, Pedro and Iglesias, Jose Pedro},
- title = {A Unified Model for Line Projections in Catadioptric Cameras with Rotationally Symmetric Mirrors},
- booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2022,
- month = jun,
- url = {https://www.merl.com/publications/TR2022-084}
- }
Sullivan, A., Wang, J., Li, X., Chen, S., Abbot, L., "PointMotionNet: Point-Wise Motion Learning for Large-Scale LiDAR Point Clouds Sequences", CVPR Workshop on Autonomous Driving, June 2022.
BibTeX TR2022-083 PDF
- @inproceedings{Sullivan2022jun,
- author = {Sullivan, Alan and Wang, Jun and Li, Xiaolong and Chen, Siheng and Abbot, Lynn},
- title = {PointMotionNet: Point-Wise Motion Learning for Large-Scale LiDAR Point Clouds Sequences},
- booktitle = {CVPR Workshop on Autonomous Driving},
- year = 2022,
- month = jun,
- url = {https://www.merl.com/publications/TR2022-083}
- }
Chatterjee, M., Ahuja, N., Cherian, A., "Quantifying Predictive Uncertainty for Stochastic Video Synthesis from Audio", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
BibTeX TR2022-082 PDF
- @inproceedings{Chatterjee2022jun,
- author = {Chatterjee, Moitreya and Ahuja, Narendra and Cherian, Anoop},
- title = {Quantifying Predictive Uncertainty for Stochastic Video Synthesis from Audio},
- booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2022,
- month = jun,
- url = {https://www.merl.com/publications/TR2022-082}
- }
Zhu, X., Jain, S., Tomizuka, M., van Baar, J., "Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints", IEEE International Conference on Robotics and Automation (ICRA), DOI: 10.1109/ICRA46639.2022.9812092, May 2022, pp. 4833-4839.
BibTeX TR2022-055 PDF
- @inproceedings{Zhu2022may2,
- author = {Zhu, Xinghao and Jain, Siddarth and Tomizuka, Masayoshi and van Baar, Jeroen},
- title = {Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints},
- booktitle = {2022 IEEE International Conference on Robotics and Automation (ICRA)},
- year = 2022,
- pages = {4833--4839},
- month = may,
- publisher = {IEEE},
- doi = {10.1109/ICRA46639.2022.9812092},
- isbn = {978-1-7281-9681-7},
- url = {https://www.merl.com/publications/TR2022-055}
- }
Shah, A.P., Geng, S., Gao, P., Cherian, A., Hori, T., Marks, T.K., Le Roux, J., Hori, C., "Audio-Visual Scene-Aware Dialog and Reasoning Using Audio-Visual Transformers with Joint Student-Teacher Learning", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), April 2022, pp. 7732-7736.
BibTeX TR2022-019 PDF
- @inproceedings{Shah2022apr,
- author = {Shah, Ankit Parag and Geng, Shijie and Gao, Peng and Cherian, Anoop and Hori, Takaaki and Marks, Tim K. and Le Roux, Jonathan and Hori, Chiori},
- title = {Audio-Visual Scene-Aware Dialog and Reasoning Using Audio-Visual Transformers with Joint Student-Teacher Learning},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2022,
- pages = {7732--7736},
- month = apr,
- publisher = {IEEE},
- issn = {1520-6149},
- isbn = {978-1-6654-0540-9},
- url = {https://www.merl.com/publications/TR2022-019}
- }
Hori, C., Shah, A.P., Geng, S., Gao, P., Cherian, A., Hori, T., Le Roux, J., Marks, T.K., "Overview of Audio Visual Scene-Aware Dialog with Reasoning Track for Natural Language Generation in DSTC10", The 10th Dialog System Technology Challenge Workshop at AAAI, February 2022.
BibTeX TR2022-016 PDF
- @inproceedings{Hori2022feb,
- author = {Hori, Chiori and Shah, Ankit Parag and Geng, Shijie and Gao, Peng and Cherian, Anoop and Hori, Takaaki and Le Roux, Jonathan and Marks, Tim K.},
- title = {Overview of Audio Visual Scene-Aware Dialog with Reasoning Track for Natural Language Generation in DSTC10},
- booktitle = {The 10th Dialog System Technology Challenge Workshop at AAAI},
- year = 2022,
- month = feb,
- url = {https://www.merl.com/publications/TR2022-016}
- }
Shah, A.P., Hori, T., Le Roux, J., Hori, C., DSTC10-AVSD Submission System with Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning, February 2022.
BibTeX TR2022-025 PDF
- @book{Shah2022feb,
- author = {Shah, Ankit Parag and Hori, Takaaki and Le Roux, Jonathan and Hori, Chiori},
- title = {DSTC10-AVSD Submission System with Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning},
- year = 2022,
- month = feb,
- url = {https://www.merl.com/publications/TR2022-025}
- }

See All Publications for Computer Vision

Quick Links

Tim K.Marks

AnoopCherian

Michael J.Jones

ChioriHori

Matthew E.Brand

HassanMansour

JonathanLe Roux

AnthonyVetro

RaduCorcodel

Devesh K.Jha

SuhasLohit

Petros T.Boufounos

DehongLiu

Daniel N.Nikovski

DiegoRomeres

YeWang

SiddarthJain

ArvindRaghunathan

GordonWichern

William S.Yerazunis

JoseAmaya

ToshiakiKoike-Akino

PedroMiraldo

Philip V.Orlik

Kuan-ChuanPeng

HuifangSun

YebinWang

MoitreyaChatterjee

Robust Machine Learning

Video Anomaly Detection

MERL Shopping Dataset

Point-Plane SLAM

CV1703: Software development in ROS for robotic manipulation

CV1862: 6D pose estimation for machine vision

[MERL Seminar Series Spring 2022] Self-Supervised Scene Representation Learning

[MERL Seminar Series Spring 2022] Learning Speech Representations with Multimodal Self-Supervision

HealthCam: A system for non-contact monitoring of vital signs

[MERL Seminar Series 2021] Learning to See by Moving: Self-supervising 3D scene representations for perception, control, and visual reasoning

[MERL Seminar Series 2021] Look and Listen: From Semantic to Spatial Audio-Visual Perception

Towards Human-Level Learning of Complex Physical Puzzles

Scene-Aware Interaction Technology

3D Object Discovery and Modeling Using Single RGB-D Images Containing Multiple Object Instances

Joint 3D Reconstruction of a Static Scene and Moving Objects

Direct Multichannel Tracking

FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds

FasTFit: A fast T-spline fitting algorithm

CASENet: Deep Category-Aware Semantic Edge Detection

Object Detection and Tracking in RGB-D SLAM via Hierarchical Feature Grouping

Pinpoint SLAM: A Hybrid of 2D and 3D Simultaneous Localization and Mapping for RGB-D Sensors

Action Detection Using A Deep Recurrent Neural Network

Dose optimization for particle beam therapy

3D Reconstruction

MERL Research on Autonomous Vehicles

Saffron - Digital Type System

Obstacle Detection

Semantic Scene Labeling

Robot Bin Picking

Sapphire - High Accuracy NC Milling Simulation

Deep Hierarchical Parsing for Semantic Segmentation

Global Local Face Upsampling Network

Gaussian Conditional Random Field Network for Semantic Segmentation

Fast Graspability Evaluation on Single Depth Maps for Bin Picking with General Grippers

Point-Plane SLAM for Hand-Held 3D Sensors

Tracking an RGB-D Camera Using Points and Planes

Fast Plane Extraction in Organized Point Clouds Using Agglomerative Hierarchical Clustering

Calibration of Non-Overlapping Cameras Using an External SLAM System

Voting-Based Pose Estimation for Robotic Assembly Using a 3D Sensor

Fast Object Localization and Pose Estimation in Heavy Clutter for Robotic Bin Picking

Learning to rank 3D features

SOurce-free Cross-modal KnowledgE Transfer

Instance Segmentation GAN

Audio Visual Scene-Graph Segmentor

Generating Visual Dynamics from Sound and Context

Adversarially-Contrastive Optimal Transport

MotionNet

Contact-Implicit Trajectory Optimization

FoldingNet++

Landmarks’ Location, Uncertainty, and Visibility Likelihood

Gradient-based Nikaido-Isoda

Tim K.
Marks

Anoop
Cherian

Michael J.
Jones

Chiori
Hori

Matthew E.
Brand

Hassan
Mansour

Jonathan
Le Roux

Anthony
Vetro

Radu
Corcodel

Devesh K.
Jha

Suhas
Lohit

Petros T.
Boufounos

Dehong
Liu

Daniel N.
Nikovski

Diego
Romeres

Ye
Wang

Siddarth
Jain

Arvind
Raghunathan

Gordon
Wichern

William S.
Yerazunis

Jose
Amaya

Toshiaki
Koike-Akino

Pedro
Miraldo

Philip V.
Orlik

Kuan-Chuan
Peng

Huifang
Sun

Yebin
Wang

Moitreya
Chatterjee