Computer Vision
Extracting meaning and building representations of visual objects and events in the world.
Our main research themes cover the areas of deep learning and artificial intelligence for object and action detection, classification and scene understanding, robotic vision and object manipulation, 3D processing and computational geometry, as well as simulation of physical systems to enhance machine learning systems.
Quick Links
-
Researchers
Jeroen
van Baar
Tim
Marks
Michael
Jones
Anoop
Cherian
Alan
Sullivan
Matthew
Brand
Chiori
Hori
Siheng
Chen
Ronald
Perry
Hassan
Mansour
Takaaki
Hori
Jay
Thornton
Petros
Boufounos
Radu
Corcodel
Devesh
Jha
Daniel
Nikovski
Arvind
Raghunathan
Diego
Romeres
Anthony
Vetro
Ye
Wang
Dehong
Liu
Gordon
Wichern
William
Yerazunis
Bret
Harsham
Toshiaki
Koike-Akino
Suhas
Lohit
Philip
Orlik
Huifang
Sun
Yebin
Wang
Siddarth
Jain
Kuan-Chuan
Peng
-
Awards
-
AWARD MERL Researchers win Best Paper Award at ICCV 2019 Workshop on Statistical Deep Learning in Computer Vision Date: October 27, 2019
Awarded to: Abhinav Kumar, Tim K. Marks, Wenxuan Mou, Chen Feng, Xiaoming Liu
MERL Contact: Tim Marks
Research Areas: Artificial Intelligence, Computer Vision, Machine LearningBrief- MERL researcher Tim Marks, former MERL interns Abhinav Kumar and Wenxuan Mou, and MERL consultants Professor Chen Feng (NYU) and Professor Xiaoming Liu (MSU) received the Best Oral Paper Award at the IEEE/CVF International Conference on Computer Vision (ICCV) 2019 Workshop on Statistical Deep Learning in Computer Vision (SDL-CV) held in Seoul, Korea. Their paper, entitled "UGLLI Face Alignment: Estimating Uncertainty with Gaussian Log-Likelihood Loss," describes a method which, given an image of a face, estimates not only the locations of facial landmarks but also the uncertainty of each landmark location estimate.
-
AWARD R&D100 award for Deep Learning-based Water Detector Date: November 16, 2018
Awarded to: Ziming Zhang, Alan Sullivan, Hideaki Maehara, Kenji Taira, Kazuo Sugimoto
MERL Contact: Alan Sullivan
Research Areas: Artificial Intelligence, Computer Vision, Machine LearningBrief- Researchers and developers from MERL, Mitsubishi Electric and Mitsubishi Electric Engineering (MEE) have been recognized with an R&D100 award for the development of a deep learning-based water detector. Automatic detection of water levels in rivers and streams is critical for early warning of flash flooding. Existing systems require a height gauge be placed in the river or stream, something that is costly and sometimes impossible. The new deep learning-based water detector uses only images from a video camera along with 3D measurements of the river valley to determine water levels and warn of potential flooding. The system is robust to lighting and weather conditions working well during the night as well as during fog or rain. Deep learning is a relatively new technique that uses neural networks and AI that are trained from real data to perform human-level recognition tasks. This work is powered by Mitsubishi Electric's Maisart AI technology.
-
AWARD CVPR 2011 Longuet-Higgins Prize Date: June 25, 2011
Awarded to: Paul A. Viola and Michael J. Jones
Awarded for: "Rapid Object Detection using a Boosted Cascade of Simple Features"
Awarded by: Conference on Computer Vision and Pattern Recognition (CVPR)
MERL Contact: Michael Jones
Research Area: Machine LearningBrief- Paper from 10 years ago with the largest impact on the field: "Rapid Object Detection using a Boosted Cascade of Simple Features", originally published at Conference on Computer Vision and Pattern Recognition (CVPR 2001).
See All Awards for MERL -
-
News & Events
-
NEWS Anoop Cherian gave an invited talk at the Multi-modal Video Analysis Workshop, ECCV 2020 Date: August 23, 2020
Where: European Conference on Computer Vision (ECCV), online, 2020
MERL Contact: Anoop Cherian
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & AudioBrief- MERL Principal Research Scientist Anoop Cherian gave an invited talk titled "Sound2Sight: Audio-Conditioned Visual Imagination" at the Multi-modal Video Analysis workshop held in conjunction with the European Conference on Computer Vision (ECCV), 2020. The talk was based on a recent ECCV paper that describes a new multimodal reasoning task called Sound2Sight and a generative adversarial machine learning algorithm for producing plausible video sequences conditioned on sound and visual context.
-
NEWS MERL's Scene-Aware Interaction Technology Featured in Mitsubishi Electric Corporation Press Release Date: July 22, 2020
Where: Tokyo, Japan
MERL Contacts: Siheng Chen; Anoop Cherian; Bret Harsham; Chiori Hori; Takaaki Hori; Jonathan Le Roux; Tim Marks; Alan Sullivan; Anthony Vetro
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & AudioBrief- Mitsubishi Electric Corporation announced that the company has developed what it believes to be the world’s first technology capable of highly natural and intuitive interaction with humans based on a scene-aware capability to translate multimodal sensing information into natural language.
The novel technology, Scene-Aware Interaction, incorporates Mitsubishi Electric’s proprietary Maisart® compact AI technology to analyze multimodal sensing information for highly natural and intuitive interaction with humans through context-dependent generation of natural language. The technology recognizes contextual objects and events based on multimodal sensing information, such as images and video captured with cameras, audio information recorded with microphones, and localization information measured with LiDAR.
Scene-Aware Interaction for car navigation, one target application, will provide drivers with intuitive route guidance. The technology is also expected to have applicability to human-machine interfaces for in-vehicle infotainment, interaction with service robots in building and factory automation systems, systems that monitor the health and well-being of people, surveillance systems that interpret complex scenes for humans and encourage social distancing, support for touchless operation of equipment in public areas, and much more. The technology is based on recent research by MERL's Speech & Audio and Computer Vision groups.
Demonstration Video:
Link:
Mitsubishi Electric Corporation Press Release
- Mitsubishi Electric Corporation announced that the company has developed what it believes to be the world’s first technology capable of highly natural and intuitive interaction with humans based on a scene-aware capability to translate multimodal sensing information into natural language.
See All News & Events for Computer Vision -
-
Research Highlights
-
Internships
-
SP1419: Simulation of Multimodal Sensors
MERL is seeking a motivated intern to assist in generating simulated multimodal data for machine learning applications. The project involves integrating several existing software components to generate optical and radar data in a variety of sensing scenarios, and executing the simulations under a variety of conditions. The ideal candidate should have experience with C++, Python, and scripting methods. Some knowledge or experience with Blender, computer graphics, and computer vision would be preferred, but is not required. Project duration is flexible in the range of 1-2 months. Intern has the choice of part-time or full-time occupation and may start immediately.
See All Internships for Computer Vision -
-
Openings
See All Openings at MERL -
Recent Publications
- , "Sound2Sight: Generating Visual Dynamics from Sound and Context", European Conference on Computer Vision (ECCV), August 2020.BibTeX TR2020-121 PDF
- @inproceedings{Cherian2020aug,
- author = {Cherian, Anoop and Chatterjee, Moitreya and Ahuja, Narendra},
- title = {Sound2Sight: Generating Visual Dynamics from Sound and Context},
- booktitle = {European Conference on Computer Vision (ECCV)},
- year = 2020,
- month = aug,
- url = {https://www.merl.com/publications/TR2020-121}
- }
- , "Representation Learning via Adversarially-Contrastive Optimal Transport", International Conference on Machine Learning (ICML), July 2020.BibTeX TR2020-093 PDF
- @inproceedings{Cherian2020jul,
- author = {Cherian, Anoop and Aeron, Shuchin},
- title = {Representation Learning via Adversarially-Contrastive Optimal Transport},
- booktitle = {International Conference on Machine Learning (ICML)},
- year = 2020,
- month = jul,
- url = {https://www.merl.com/publications/TR2020-093}
- }
- , "Collaborative Motion Prediction via Neural Motion Message Passing", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.BibTeX TR2020-072 PDF
- @inproceedings{Hu2020jun,
- author = {Hu, Yue and Chen, Siheng and Zhang, Ya and Gu, Xiao},
- title = {Collaborative Motion Prediction via Neural Motion Message Passing},
- booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2020,
- month = jun,
- url = {https://www.merl.com/publications/TR2020-072}
- }
- , "Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.BibTeX TR2020-073 PDF
- @inproceedings{Li2020jun,
- author = {Li, Maosen and Chen, Sihen and Zhao, Yangheng and Zhang, Ya and Wang, Yanfeng and Tia, Qi},
- title = {Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction},
- booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2020,
- month = jun,
- url = {https://www.merl.com/publications/TR2020-073}
- }
- , "LUVLi Face Alignment: Estimating Landmarks’ Location, Uncertainty, and Visibility Likelihood", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.BibTeX TR2020-067 PDF Video Data
- @inproceedings{Kumar2020jun,
- author = {Kumar, Abhinav and Marks, Tim and Mou, Wenxuan and Wang, Ye and Cherian, Anoop and Jones, Michael J. and Liu, Xiaoming and Koike-Akino, Toshiaki and Feng, Chen},
- title = {LUVLi Face Alignment: Estimating Landmarks’ Location, Uncertainty, and Visibility Likelihood},
- booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2020,
- month = jun,
- url = {https://www.merl.com/publications/TR2020-067}
- }
- , "MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird’s Eye View Maps", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.BibTeX TR2020-068 PDF Software
- @inproceedings{Wu2020jun,
- author = {Wu, Pengxiang and Chen, Siheng},
- title = {MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird’s Eye View Maps},
- booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2020,
- month = jun,
- url = {https://www.merl.com/publications/TR2020-068}
- }
- , "3D Point Cloud Processing and Learning for Autonomous Driving", IEEE Signal Processing Magazine, May 2020.BibTeX TR2020-066 PDF
- @article{Chen2020may2,
- author = {Chen, Siheng and Liu, Baoan and Feng, Chen and Vallespi-Gonzalez, Carlos and Wellington, Carl},
- title = {3D Point Cloud Processing and Learning for Autonomous Driving},
- journal = {IEEE Signal Processing Magazine},
- year = 2020,
- month = may,
- url = {https://www.merl.com/publications/TR2020-066}
- }
- , "Tuning-Free Contact-Implicit Trajectory Optimization", IEEE International Conference on Robotics and Automation (ICRA), May 2020.BibTeX TR2020-065 PDF Video Software
- @inproceedings{Onol2020may,
- author = {Onol, Aykut O. and Corcodel, Radu and Long, Philip and Padir, Taskin},
- title = {Tuning-Free Contact-Implicit Trajectory Optimization},
- booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
- year = 2020,
- month = may,
- url = {https://www.merl.com/publications/TR2020-065}
- }
- , "Sound2Sight: Generating Visual Dynamics from Sound and Context", European Conference on Computer Vision (ECCV), August 2020.
-
Videos
-
Calibration of Non-Overlapping Cameras Using an External SLAM System
-
Gaussian Conditional Random Field Network for Semantic Segmentation
-
Fast Plane Extraction in Organized Point Clouds Using Agglomerative Hierarchical Clustering
-
Fast Graspability Evaluation on Single Depth Maps for Bin Picking with General Grippers
-
Point-Plane SLAM for Hand-Held 3D Sensors
-
FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds
-
CASENet: Deep Category-Aware Semantic Edge Detection
-
Pinpoint SLAM: A Hybrid of 2D and 3D Simultaneous Localization and Mapping for RGB-D Sensors
-
MERL Research on Autonomous Vehicles
-
Tracking an RGB-D Camera Using Points and Planes
-
Voting-Based Pose Estimation for Robotic Assembly Using a 3D Sensor
-
3D Reconstruction
-
Learning to rank 3D features
-
Fast Object Localization and Pose Estimation in Heavy Clutter for Robotic Bin Picking
-
Global Local Face Upsampling Network
-
3D Object Discovery and Modeling Using Single RGB-D Images Containing Multiple Object Instances
-
Joint 3D Reconstruction of a Static Scene and Moving Objects
-
Direct Multichannel Tracking
-
Saffron - Digital Type System
-
Obstacle Detection
-
Semantic Scene Labeling
-
Robot Bin Picking
-
Object Detection and Tracking in RGB-D SLAM via Hierarchical Feature Grouping
-
Action Detection Using A Deep Recurrent Neural Network
-
Dose optimization for particle beam therapy
-
Sapphire - High Accuracy NC Milling Simulation
-
FasTFit: A fast T-spline fitting algorithm
-
Deep Hierarchical Parsing for Semantic Segmentation
-
-
Software Downloads
