- Date & Time: Wednesday, March 30, 2022; 11:00 AM EDT
Speaker: Vincent Sitzmann, MIT
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
Abstract 
Given only a single picture, people are capable of inferring a mental representation that encodes rich information about the underlying 3D scene. We acquire this skill not through massive labeled datasets of 3D scenes, but through self-supervised observation and interaction. Building machines that can infer similarly rich neural scene representations is critical if they are to one day parallel people’s ability to understand, navigate, and interact with their surroundings. This poses a unique set of challenges that sets neural scene representations apart from conventional representations of 3D scenes: Rendering and processing operations need to be differentiable, and the type of information they encode is unknown a priori, requiring them to be extraordinarily flexible. At the same time, training them without ground-truth 3D supervision is an underdetermined problem, highlighting the need for structure and inductive biases without which models converge to spurious explanations.
I will demonstrate how we can equip neural networks with inductive biases that enables them to learn 3D geometry, appearance, and even semantic information, self-supervised only from posed images. I will show how this approach unlocks the learning of priors, enabling 3D reconstruction from only a single posed 2D image, and how we may extend these representations to other modalities such as sound. I will then discuss recent work on learning the neural rendering operator to make rendering and training fast, and how this speed-up enables us to learn object-centric neural scene representations, learning to decompose 3D scenes into objects, given only images. Finally, I will talk about a recent application of self-supervised scene representation learning in robotic manipulation, where it enables us to learn to manipulate classes of objects in unseen poses from only a handful of human demonstrations.
-
- Date: March 1, 2022
MERL Contacts: Anoop Cherian; Chiori Hori; Jonathan Le Roux; Tim K. Marks; Anthony Vetro
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & Audio
Brief - MERL's research on scene-aware interaction was recently featured in an IEEE Spectrum article. The article, titled "At Last, A Self-Driving Car That Can Explain Itself" and authored by MERL Senior Principal Research Scientist Chiori Hori and MERL Director Anthony Vetro, gives an overview of MERL's efforts towards developing a system that can analyze multimodal sensing information for highly natural and intuitive interaction with humans through context-dependent generation of natural language. The technology recognizes contextual objects and events based on multimodal sensing information, such as images and video captured with cameras, audio information recorded with microphones, and localization information measured with LiDAR.
Scene-Aware Interaction for car navigation, one target application that the article focuses on, will provide drivers with intuitive route guidance. Scene-Aware Interaction technology is expected to have wide applicability, including human-machine interfaces for in-vehicle infotainment, interaction with service robots in building and factory automation systems, systems that monitor the health and well-being of people, surveillance systems that interpret complex scenes for humans and encourage social distancing, support for touchless operation of equipment in public areas, and much more. MERL's Scene-Aware Interaction Technology had previously been featured in a Mitsubishi Electric Corporation Press Release.
IEEE Spectrum is the flagship magazine and website of the IEEE, the world’s largest professional organization devoted to engineering and the applied sciences. IEEE Spectrum has a circulation of over 400,000 engineers worldwide, making it one of the leading science and engineering magazines.
-
- Date & Time: Tuesday, November 2, 2021; 1:00 PM EST
Speaker: Dr. Hsiao-Yu (Fish) Tung, MIT BCS
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Robotics
Abstract 
Current state-of-the-art CNNs can localize and name objects in internet photos, yet, they miss the basic knowledge that a two-year-old toddler has possessed: objects persist over time despite changes in the observer’s viewpoint or during cross-object occlusions; objects have 3D extent; solid objects do not pass through each other. In this talk, I will introduce neural architectures that learn to parse video streams of a static scene into world-centric 3D feature maps by disentangling camera motion from scene appearance. I will show the proposed architectures learn object permanence, can imagine RGB views from novel viewpoints in truly novel scenes, can conduct basic spatial reasoning and planning, can infer affordability in sentences, and can learn geometry-aware 3D concepts that allow pose-aware object recognition to happen with weak/sparse labels. Our experiments suggest that the proposed architectures are essential for the models to generalize across objects and locations, and it overcomes many limitations of 2D CNNs. I will show how we can use the proposed 3D representations to build machine perception and physical understanding more close to humans.
-
- Date: November 16, 2020
MERL Contacts: Devesh K. Jha; Daniel N. Nikovski; Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, Robotics
Brief - MERL researchers, in collaboration with researchers from MELCO and the Department of Brain and Cognitive Science at MIT, have released simulation software Circular Maze Environment (CME). This system could be used as a new benchmark for evaluating different control and robot learning algorithms. The control objective in this system is to tip and the tilt the maze so as to drive one (or multiple) marble(s) to the innermost ring of the circular maze. Although the system is very intuitive for humans to control, it is very challenging for artificial intelligence agents to learn efficiently. It poses several challenges for both model-based as well as model-free methods, due to its non-smooth dynamics, long planning horizon, and non-linear dynamics. The released Python package provides the simulation environment for the circular maze, where movement of multiple marbles could be simulated simultaneously. The package also provides a trajectory optimization algorithm to design a model-based controller in simulation.
-
- Date: July 22, 2020
Where: Tokyo, Japan
MERL Contacts: Anoop Cherian; Chiori Hori; Jonathan Le Roux; Tim K. Marks; Anthony Vetro
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & Audio
Brief - Mitsubishi Electric Corporation announced that the company has developed what it believes to be the world’s first technology capable of highly natural and intuitive interaction with humans based on a scene-aware capability to translate multimodal sensing information into natural language.
The novel technology, Scene-Aware Interaction, incorporates Mitsubishi Electric’s proprietary Maisart® compact AI technology to analyze multimodal sensing information for highly natural and intuitive interaction with humans through context-dependent generation of natural language. The technology recognizes contextual objects and events based on multimodal sensing information, such as images and video captured with cameras, audio information recorded with microphones, and localization information measured with LiDAR.
Scene-Aware Interaction for car navigation, one target application, will provide drivers with intuitive route guidance. The technology is also expected to have applicability to human-machine interfaces for in-vehicle infotainment, interaction with service robots in building and factory automation systems, systems that monitor the health and well-being of people, surveillance systems that interpret complex scenes for humans and encourage social distancing, support for touchless operation of equipment in public areas, and much more. The technology is based on recent research by MERL's Speech & Audio and Computer Vision groups.
-
- Date: November 16, 2018
Awarded to: Ziming Zhang, Alan Sullivan, Hideaki Maehara, Kenji Taira, Kazuo Sugimoto
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
Brief - Researchers and developers from MERL, Mitsubishi Electric and Mitsubishi Electric Engineering (MEE) have been recognized with an R&D100 award for the development of a deep learning-based water detector. Automatic detection of water levels in rivers and streams is critical for early warning of flash flooding. Existing systems require a height gauge be placed in the river or stream, something that is costly and sometimes impossible. The new deep learning-based water detector uses only images from a video camera along with 3D measurements of the river valley to determine water levels and warn of potential flooding. The system is robust to lighting and weather conditions working well during the night as well as during fog or rain. Deep learning is a relatively new technique that uses neural networks and AI that are trained from real data to perform human-level recognition tasks. This work is powered by Mitsubishi Electric's Maisart AI technology.
-
- Date: October 15, 2018 - October 19, 2018
Where: CEATEC'18, Makuhari Messe, Tokyo
MERL Contacts: Devesh K. Jha; Daniel N. Nikovski; Diego Romeres; William S. Yerazunis
Research Areas: Artificial Intelligence, Computer Vision, Data Analytics, Robotics
Brief - MERL's work on robot learning algorithms was demonstrated at CEATEC'18, Japan's largest IT and electronics exhibition and conference held annually at Makuhari Messe near Tokyo. A team of researchers from the Data Analytics Group at MERL and the Artificial Intelligence Department of the Information Technology Center (ITC) of MELCO presented an interactive demonstration of a model-based artificial intelligence algorithm that learns how to control equipment autonomously. The algorithm developed at MERL constructs models of mechanical equipment through repeated trial and error, and then learns control policies based on these models. The demonstration used a circular maze, where the objective is to drive a ball to the center of the maze by tipping and tilting the maze, a task that is difficult even for humans; approximately half of the CEATEC'18 visitors who tried to steer the ball by means of a joystick could not bring it to the center of the maze within one minute. In contrast, MERL's algorithm successfully learned how to drive the ball to the goal within ten seconds without the need for human programming. The demo was at the entrance of MELCO's booth at CEATEC'18, inviting visitors to learn more about MELCO's many other AI technologies on display, and was seen by an estimated more than 50,000 visitors over the five days of the expo.
-
- Date: February 14, 2018
Where: Tokyo, Japan
MERL Contacts: Devesh K. Jha; Daniel N. Nikovski; Diego Romeres; William S. Yerazunis
Research Areas: Optimization, Computer Vision
Brief - New technology for model-based AI learning for equipment control was demonstrated by MERL researchers at a recent press release event in Tokyo. The AI learning method constructs predictive models of the equipment through repeated trial and error, and then learns control rules based on these models. The new technology is expected to significantly reduce the cost and time needed to develop control programs in the future. Please see the link below for the full text of the Mitsubishi Electric press release.
-
- Date & Time: Tuesday, March 28, 2017; 1:30 - 5:30PM
Location: Google (355 Main St., 5th Floor, Cambridge MA)
MERL Contacts: Daniel N. Nikovski; Anthony Vetro; Jinyun Zhang Brief - How will AI and robotics reshape the economy and create new opportunities (and challenges) across industries? Who are the hottest companies that will compete with the likes of Google, Amazon, and Uber to create the future? And what are New England innovators doing to strengthen the local cluster and help lead the national discussion?
MERL will be participating in Xconomy's third annual conference on AI and robotics in Boston to address these questions. MERL President & CEO, Dick Waters, will be on a panel discussing the status and future of self-driving vehicles. Lab members will also be on hand demonstrate and discuss recent advances AI and robotics technology.
The agenda and registration for the event can be found online: https://xconomyforum85.eventbrite.com.
-
- Date & Time: Tuesday, January 17, 2017; 6:00 pm
Location: 201 Broadway, Cambridge, MA
Speaker: Tim Marks, Esra Cansizoglu and Carl Vondrick, MERL and MIT
Research Area: Computer Vision
Brief - MERL was pleased to host the Boston Imaging and Vision Meetup held on January 17. The meetup is an informal gathering of people interested in the field of computer imaging and vision. According to the group's website "the meetup provides an opportunity for the image processing/computer vision community to network, socialize and learn". The event held at MERL featured three speakers, Tim Marks and Esra Cansizoglu from MERL, as well as Carl Vondrick, an MIT CS graduate student in the group of Prof. Antonio Torralba. Roughly 70 people attended to eat pizza, hear the speakers and network.
-
- Date: June 13, 2013
Where: CIRP Conference on Modeling of Machining Operations (CIRP CMMO)
Research Area: Computer Vision
Brief - The paper "Cutter Workpiece Engagement Calculations for Five-axis Milling using Composite Adaptively Sampled Distance Fields" by Erdim, H. and Sullivan, A. was presented at the CIRP Conference on Modeling of Machining Operations (CIRP CMMO).
-
- Date: February 12, 2013
Brief - Mitsubishi Electric Corporation (TOKYO: 6503) announced today it has developed ultra-high resolution 3D shape representation technology for numerically controlled (NC) machine tools. The technology displays detailed shapes of machined surfaces down to a resolution of 1 micrometer (um) in a 3D machining simulation, which machine operators can use to evaluate surface textures without trial cutting through a high quality machining process.
-
- Date: October 29, 2012
Where: CIRP Conference on Process Machine Interactions (PMI)
Research Area: Computer Vision
Brief - The paper "High Accuracy Computation of Geometric Properties of Cutter Workpiece Intersection using Distance Fields for NC Milling" by Erdim, H. and Sullivan, A. was presented at the CIRP Conference on Process Machine Interactions (PMI).
-
- Date: June 1, 2012
Where: Computer-Aided Design
Research Area: Computer Vision
Brief - The article "High Accuracy NC Milling Simulation Using Composite Adaptively Sampled Distance Fields" by Sullivan, A., Erdim, H., Perry, R.N. and Frisken, S.F. was published in Computer-Aided Design.
-
- Date: August 3, 2009
Where: SPIE Conference on Applications of Digital Image Processing
MERL Contact: Anthony Vetro
Research Area: Digital Video
Brief - The paper "Intermediate View Generation for Perceived Depth Adjustment of Stereo Video" by Arican, Z., Yea, S., Sullivan, A., Vetro, A. and Tescher, A.G. was presented at the SPIE Conference on Applications of Digital Image Processing.
-