- Date & Time: Tuesday, April 12, 2022; 11:00 AM EDT
Speaker: Sebastien Gros, NTNU
Research Areas: Control, Dynamical Systems, Optimization
Abstract 
Reinforcement Learning (RL), similarly to many AI-based techniques, is currently receiving a very high attention. RL is most commonly supported by classic Machine Learning techniques, i.e. typically Deep Neural Networks (DNNs). While there are good motivations for using DNNs in RL, there are also significant drawbacks. The lack of “explainability” of the resulting control policies, and the difficulty to provide guarantees on their closed-loop behavior (safety, stability) makes DNN-based policies problematic in many applications. In this talk, we will discuss an alternative approach to support RL, via formal optimal control tools based on Model Predictive Control (MPC). This approach alleviates the issues detailed above, but also presents some challenges. In this talk, we will discuss why MPC is a valid tool to support RL, and how MPC can be combined with RL (RLMPC). We will then discuss some recent results regarding this combination, the known challenges, and the kind of control applications where we believe that RLMPC will be a valuable approach.
-
- Date & Time: Tuesday, April 5, 2022; 11:00 AM EDT
Speaker: Albert Benveniste, Benoît Caillaud, and Mathias Malandain, Inria
MERL Host: Scott A. Bortoff
Research Areas: Dynamical Systems, Multi-Physical Modeling
Abstract 
Since its 3.3 release, Modelica offers the possibility to specify models of dynamical systems with multiple modes having different DAE-based dynamics. However, the handling of such models by the current Modelica tools is not satisfactory, with mathematically sound models yielding exceptions at runtime. In our introduction, will briefly explain why and when the approximate structural analysis implemented in current Modelica tools leads to such errors. Then we will present our multimode Pryce Sigma-method for index reduction, in which the mode-dependent Sigma-matrix is represented in a dual form, by attaching, to every valuation of the sigma_ij entry of the Sigma matrix, the predicate characterizing the set of modes in which sigma_ij takes this value. We will illustrate this multimode analysis on example, by using our IsamDAE tool. In a second part, we will complement this multimode DAE structural analysis by a new structural analysis of mode changes (and, more generally, transient modes holding for zero time). Also, mode changes often give raise to impulsive behaviors: we will present a compile-time analysis identifying such behaviors. Our structural analysis of mode changes deeply relies on nonstandard analysis, which is a mathematical framework in which infinitesimals and infinities are first class citizens.
-
- Date & Time: Wednesday, March 30, 2022; 11:00 AM EDT
Speaker: Vincent Sitzmann, MIT
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
Abstract 
Given only a single picture, people are capable of inferring a mental representation that encodes rich information about the underlying 3D scene. We acquire this skill not through massive labeled datasets of 3D scenes, but through self-supervised observation and interaction. Building machines that can infer similarly rich neural scene representations is critical if they are to one day parallel people’s ability to understand, navigate, and interact with their surroundings. This poses a unique set of challenges that sets neural scene representations apart from conventional representations of 3D scenes: Rendering and processing operations need to be differentiable, and the type of information they encode is unknown a priori, requiring them to be extraordinarily flexible. At the same time, training them without ground-truth 3D supervision is an underdetermined problem, highlighting the need for structure and inductive biases without which models converge to spurious explanations.
I will demonstrate how we can equip neural networks with inductive biases that enables them to learn 3D geometry, appearance, and even semantic information, self-supervised only from posed images. I will show how this approach unlocks the learning of priors, enabling 3D reconstruction from only a single posed 2D image, and how we may extend these representations to other modalities such as sound. I will then discuss recent work on learning the neural rendering operator to make rendering and training fast, and how this speed-up enables us to learn object-centric neural scene representations, learning to decompose 3D scenes into objects, given only images. Finally, I will talk about a recent application of self-supervised scene representation learning in robotic manipulation, where it enables us to learn to manipulate classes of objects in unseen poses from only a handful of human demonstrations.
-
- Date: June 19, 2022
Research Areas: Communications, Electronic and Photonic Devices, Machine Learning
Brief - MERL Researcher Rui Ma will give an invited talk titled "All Digital Transmitter with GaN Switching Mode Power Amplifiers"at a technical workshop during International Microwave Symposium (IMS)2022. This IMS workshop (WSN) invites members from academia and industry to discuss the latest development activities in the area of digital-intensive power amplifiers and transmitters for RF communications.
In addition, Dr. Rui Ma is chairing a Technical Session(We2C) on "AI/ML on RF and mmWave Applications" at IMS2022.
IMS is the flagship annual conference of IEEE Microwave Theory and Technology Society(MTT-S).
Learn more here:
Sessions
Workshops
-
- Date & Time: Tuesday, March 15, 2022; 1:00 PM EDT
Speaker: Arjuna Madanayake, Florida International University
Research Areas: Applied Physics, Electronic and Photonic Devices, Multi-Physical Modeling
Abstract 
Analog computers are making a comeback. In fact, they are taking the world by storm. After decades of “analog computing winter” that followed the invention of the digital computing paradigm in the 1940s, classical physics-based analog computers are being reconsidered for improving the computational throughput of demanding applications. The research is driven by exponential growth in transistor densities and bandwidths in the integrated circuits world, which in turn, has led to new possibilities for the creative circuit designer. Fast analog chips not only furnish communication/radar front-ends, but can also be used to accelerate mathematical operations. Most analog computer today focus on AI and machine learning. E.g., analog in-memory computing plays an exciting role in AI acceleration because linear algebra operations can be mapped efficiently to compute in memory. However, many scientific computing tasks are built on linear and non-linear partial differential equations (PDEs) that require recursive numerical PDE solution across spatial and temporal dimensions. The adoption of analog parallel processors that are built around speed vs power efficiency vs precision trade-offs available from circuitry for PDE solution require new research in computer architecture. We report on recent progress on CMOS based analog computers for solving computational electromagnetics and non-linear pressure wave equations. Our first analog computing chip was measured to be more than 400x faster than a top-of-the-line NVIDIA GPU while consuming 1000x less power for elementary computational electromagnetics computations using finite-difference time-domain scheme.
-
- Date: March 15, 2022
Awarded to: Yukimasa Nagai, Jianlin Guo, Philip Orlik, Takenori Sumi, Benjamin A. Rolfe and Hiroshi Mineno
MERL Contacts: Jianlin Guo; Philip V. Orlik
Research Areas: Communications, Machine Learning
Brief - MELCO/MERL research paper “Sub-1 GHz Frequency Band Wireless Coexistence for the Internet of Things” has won the 37th Telecommunications Advancement Foundation Award (Telecom System Technology Award) in Japan. This award started in 1984, and is given to research papers and works related to information and telecommunications that have made significant contributions and achievements to the advancement, development, and standardization of information and telecommunications from technical and engineering perspectives. The award recognizes both the IEEE 802.19.3 standardization efforts and the technological advancements using reinforcement learning and robust access methodologies for wireless communication system. This year, there were 43 entries with 5 winning awards and 3 winning encouragement awards. This is the first time MELCO/MERL has received this award. Our paper has been published by IEEE Access in 2021 and authors are Yukimasa Nagai, Jianlin Guo, Philip Orlik, Takenori Sumi, Benjamin A. Rolfe and Hiroshi Mineno.
-
- Date: March 10, 2022
Where: Department of Mathematics, KTH Royal Institute of Technology
MERL Contact: Arvind Raghunathan
Research Area: Optimization
Brief - Arvind Raghunathan will present the Optimization and System Theorem seminar in the Department of Mathematics at the KTH Royal Institute of Technology. The title of the talk is Recursive McCormick Linearizations of Multilinear Programs: Minimum Size Formulations.
-
- Date: March 1, 2022
Where: Online/Zoom
MERL Contact: Devesh K. Jha
Research Areas: Artificial Intelligence, Machine Learning, Robotics
Brief - Devesh Jha, a Principal Research Scientist in MERL's Data Analytics group, gave an invited talk at the Mechanical and Aerospace Engineering Department, NYU. The title of the talk was "Robotic Manipulation in the Wild: Planning, Learning and Control through Contacts". The talk presented some of the recent work done at MERL for robotic manipulation in unstructured environments in the presence of significant uncertainty.
-
- Date: March 1, 2022
MERL Contacts: Anoop Cherian; Chiori Hori; Jonathan Le Roux; Tim K. Marks; Anthony Vetro
Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & Audio
Brief - MERL's research on scene-aware interaction was recently featured in an IEEE Spectrum article. The article, titled "At Last, A Self-Driving Car That Can Explain Itself" and authored by MERL Senior Principal Research Scientist Chiori Hori and MERL Director Anthony Vetro, gives an overview of MERL's efforts towards developing a system that can analyze multimodal sensing information for highly natural and intuitive interaction with humans through context-dependent generation of natural language. The technology recognizes contextual objects and events based on multimodal sensing information, such as images and video captured with cameras, audio information recorded with microphones, and localization information measured with LiDAR.
Scene-Aware Interaction for car navigation, one target application that the article focuses on, will provide drivers with intuitive route guidance. Scene-Aware Interaction technology is expected to have wide applicability, including human-machine interfaces for in-vehicle infotainment, interaction with service robots in building and factory automation systems, systems that monitor the health and well-being of people, surveillance systems that interpret complex scenes for humans and encourage social distancing, support for touchless operation of equipment in public areas, and much more. MERL's Scene-Aware Interaction Technology had previously been featured in a Mitsubishi Electric Corporation Press Release.
IEEE Spectrum is the flagship magazine and website of the IEEE, the world’s largest professional organization devoted to engineering and the applied sciences. IEEE Spectrum has a circulation of over 400,000 engineers worldwide, making it one of the leading science and engineering magazines.
-
- Date & Time: Tuesday, March 1, 2022; 1:00 PM EST
Speaker: David Harwath, The University of Texas at Austin
MERL Host: Chiori Hori
Research Areas: Artificial Intelligence, Machine Learning, Speech & Audio
Abstract 
Humans learn spoken language and visual perception at an early age by being immersed in the world around them. Why can't computers do the same? In this talk, I will describe our ongoing work to develop methodologies for grounding continuous speech signals at the raw waveform level to natural image scenes. I will first present self-supervised models capable of discovering discrete, hierarchical structure (words and sub-word units) in the speech signal. Instead of conventional annotations, these models learn from correspondences between speech sounds and visual patterns such as objects and textures. Next, I will demonstrate how these discrete units can be used as a drop-in replacement for text transcriptions in an image captioning system, enabling us to directly synthesize spoken descriptions of images without the need for text as an intermediate representation. Finally, I will describe our latest work on Transformer-based models of visually-grounded speech. These models significantly outperform the prior state of the art on semantic speech-to-image retrieval tasks, and also learn representations that are useful for a multitude of other speech processing tasks.
-
- Date: February 24, 2022
Research Areas: Control, Optimization
Brief - Rien Quirynen has accepted an invitation to serve on the editorial board of Journal of Optimal Control Applications and Methods (OCAM) as an Associate Editor.
OCAM provides a forum for papers on the full range of optimal control and related control design methods. The aim is to encourage new developments in optimal control theory and design methodologies that may lead to advances in real control applications.
-
- Date & Time: Tuesday, February 15, 2022; 1:00 PM EST
Speaker: Katie Bouman, California Institute of Technology
MERL Host: Joshua Rapp
Research Area: Computational Sensing
Abstract 
As imaging requirements become more demanding, we must rely on increasingly sparse and/or noisy measurements that fail to paint a complete picture. Computational imaging pipelines, which replace optics with computation, have enabled image formation in situations that are impossible for conventional optical imaging. For instance, the first black hole image, published in 2019, was only made possible through the development of computational imaging pipelines that worked alongside an Earth-sized distributed telescope. However, remaining scientific questions motivate us to improve this computational telescope to see black hole phenomena still invisible to us and to meaningfully interpret the collected data. This talk will discuss how we are leveraging and building upon recent advances in machine learning in order to achieve more efficient uncertainty quantification of reconstructed images as well as to develop techniques that allow us to extract the evolving structure of our own Milky Way's black hole over the course of a night, perhaps even in three dimensions.
-
- Date & Time: Tuesday, February 8, 2022; 1:00 PM EST
Speaker: Raphaël Pestourie, MIT
MERL Host: Matthew Brand
Research Areas: Applied Physics, Electronic and Photonic Devices, Optimization
Abstract 
Thin large-area structures with aperiodic subwavelength patterns can unleash the full power of Maxwell’s equations for focusing light and a variety of other wave transformation or optical applications. Because of their irregularity and large scale, capturing the full scattering through these devices is one of the most challenging tasks for computational design: enter extreme optics! This talk will present ways to harness the full computational power of modern large-scale optimization in order to design optical devices with thousands or millions of free parameters. We exploit various methods of domain-decomposition approximations, supercomputer-scale topology optimization, laptop-scale “surrogate” models based on Chebyshev interpolation and/or new scientific machine learning models, and other techniques to attack challenging problems: achromatic lenses that simultaneously handle many wavelengths and angles, “deep” images, hyperspectral imaging, and more.
-
- Date: July 5, 2022 - July 7, 2022
MERL Contact: Mouhacine Benosman
Research Areas: Control, Data Analytics, Dynamical Systems
Brief - The Benelux meeting is an annual conference gathering of the scientific community of Belgium, the Netherlands, and Luxemburg around systems and control. It is especially intended for PhD researchers and a number of activities are dedicated to them, including plenary talks and a mini-course.
Dr. Benosman has been invited to give the mini-course of the 2022 edition of the conference. This course, entitled 'A hybrid approach to control: classical control theory meets machine learning theory', will be centered around the topic of safe and robust machine learning-based control.
-
- Date: February 3, 2022
MERL Contact: Radu Corcodel
Research Areas: Robotics, Human-Computer Interaction
Brief - Radu Corcodel, a Principal Research Scientist in MERL's computer vision group, has been invited to join the IEEE-RAS Standing Committee for Standards & Human-Robot Interaction Terminology. This committee defines standard terms relevant to human-robot interaction in service, social, education, industrial, and research robotic applications. It establishes and defines a common terminology for practitioners and users of human-robot interaction (HRI) technologies. It is also intended to address issues common within the field of HRI, particularly surrounding the use of inconsistent and/or conflicting terms and definitions.
The invitation is a recognition of Radu's excellent record of robotics research and a significant opportunity for him to contribute to new standards in robotics terminology.
-
- Date: May 31, 2022
MERL Contact: Arvind Raghunathan
Research Area: Optimization
Brief - Arvind Raghunathan from MERL's Data Analytics group has been invited to serve on the The Howard Rosenbrock Prize committee. Instituted in 2015, Optimization and Engineering journal's Howard Rosenbrock Prize is awarded annually to honor the authors of the best paper published in the journal in the previous year.
-
- Date: January 24, 2022
Where: The TWIML AI Podcast
MERL Contact: Jonathan Le Roux
Research Areas: Artificial Intelligence, Machine Learning, Speech & Audio
Brief - MERL Speech & Audio Senior Team Leader Jonathan Le Roux was featured in an extended interview on the popular TWIML AI Podcast, presenting MERL's work towards solving the "cocktail party problem". Humans have the extraordinary ability to focus on particular sounds of interest within a complex acoustic scene, such as a cocktail party. MERL's Speech & Audio Team has been at the forefront of the field's effort to develop algorithms giving machines similar abilities. Jonathan talked with host Sam Charrington about the group's decade-long journey on this topic, from early pioneering work using deep learning for speech enhancement and speech separation, to recent works on weakly-supervised separation, hierarchical sound separation, as well as the separation of real-world soundtracks into speech, music, and sound effects (aka the "cocktail fork problem").
The TWIML AI podcast, formerly known as This Week in Machine Learning & AI, was created in 2016 and is followed by more than 10,000 subscribers on Youtube and Twitter. Jonathan's interview marks the 555th episode of the podcast.
-
- Date: December 20, 2021
Awarded to: Joshua Rapp
MERL Contact: Joshua Rapp
Research Areas: Computational Sensing, Signal Processing
Brief - Joshua Rapp has won the 2021 Best PhD Dissertation Award from the IEEE Signal Processing Society.
The award recognizes a PhD thesis completed on a signal processing subject within the past three years for its relevant work in signal processing while stimulating further research in the field.
Dr. Rapp completed his PhD at Boston University in 2020 with a thesis entitled "Probabilistic Modeling for Single-Photon Lidar." The dissertation tackles challenges of the acquisition and processing of 3D depth maps reconstructed from time-of-flight data captured one photon at a time.
The award will be presented at the 2022 IEEE International Conference on Image Processing (ICIP) in France.
-
- Date: December 14, 2021
Research Area: Control
Brief - MERL researcher Uroš Kalabić has been appointed to serve as an associate editor of the IEEE Transactions on Control Systems Technology.
The Transactions on Control Systems Technology bridge the gap between the theory and practice of control engineering. They feature publications on engineering needed to implement practical control systems.
-
- Date & Time: Tuesday, December 14, 2021; 1:00 PM EST
Speaker: Prof. Chris Fletcher, University of Waterloo
MERL Host: Ankush Chakrabarty
Research Areas: Dynamical Systems, Machine Learning, Multi-Physical Modeling
Abstract 
Decision-making and adaptation to climate change requires quantitative projections of the physical climate system and an accurate understanding of the uncertainty in those projections. Earth system models (ESMs), which solve the Navier-Stokes equations on the sphere, are the only tool that climate scientists have to make projections forward into climate states that have not been observed in the historical data record. Yet, ESMs are incredibly complex and expensive codes and contain many poorly constrained physical parameters—for processes such as clouds and convection—that must be calibrated against observations. In this talk, I will describe research from my group that uses ensembles of ESM simulations to train statistical models that learn the behavior and sensitivities of the ESM. Once trained and validated the statistical models are essentially free to run, which allows climate modelling centers to make more efficient use of precious compute cycles. The aim is to improve the quality of future climate projections, by producing better calibrated ESMs, and to improve the quantification of the uncertainties, by better sampling the equifinality of climate states.
-
- Date & Time: December 9, 2021; 7pm EST
Where: virtual
MERL Contact: Toshiaki Koike-Akino
Research Areas: Communications, Machine Learning, Signal Processing
Brief - Toshiaki Koike-Akino (Signal Processing group, Network Intelligence Team) is giving an invited talk titled, `Evolution of Machine Learning for Photonic Research' for the Boston Photonic Chapter of the IEEE Photonic Society on December 9. The talk covers recent MERL research on machine learning for nonlinearity compensation and nanophotonic device design.
-
- Date & Time: Tuesday, December 7, 2021; 1:00 PM EST
Speaker: Prof. Eric Severson, University of Wisconsin-Madison
MERL Host: Bingnan Wang
Research Area: Electric Systems
Abstract 
Electric motors pump our water, heat and cool our homes and offices, drive critical medical and surgical equipment, and, increasingly, operate our transportation systems. Approximately 99% of the world’s electric energy is produced by a rotating generator and 45% of that energy is consumed by an electric motor. The efficiency of this technology is vital in enabling our energy sustainability and reducing our carbon footprint. The reliability and lifetime of this technology have severe, and sometimes life-altering, consequences. Today’s motor technology largely relies upon mechanical bearings to support the motor’s shaft. These bearings are the first components to fail, create frictional losses, and rely on lubricants that create contamination challenges and require periodic maintenance. In short, bearings are the Achilles' heel of modern electric motors.
This seminar will explore the use of actively controlled magnetic forces to levitate the motor shaft, eliminating mechanical bearings and the problems associated with them. The working principles of traditional magnetic levitation technology (active magnetic bearings) will be reviewed and used to explain why this technology has not been successfully applied to the most high-impact motor applications. Research into “bearingless” motors offers a new levitation approach by manipulating the inherent magnetic force capability of all electric motors. While traditional motors are carefully designed to prevent shaft forces, the bearingless motor concept controls these forces to make the motor simultaneously function as an active magnetic bearing. The seminar will showcase the potential of bearingless technology to revolutionize motor systems of critical importance for energy and sustainability—from industrial compressors and blowers, such as those found in HVAC systems and wastewater aeration equipment, to power grid flywheel energy storage devices and electric turbochargers in fuel-efficient vehicles.
-
- Date & Time: Thursday, December 9, 2021; 1:00pm - 5:30pm EST
Location: Virtual Event
Speaker: Prof. Melanie Zeilinger, ETH
Research Areas: Applied Physics, Artificial Intelligence, Communications, Computational Sensing, Computer Vision, Control, Data Analytics, Dynamical Systems, Electric Systems, Electronic and Photonic Devices, Machine Learning, Multi-Physical Modeling, Optimization, Robotics, Signal Processing, Speech & Audio, Digital Video, Human-Computer Interaction, Information Security
Brief - MERL is excited to announce the second keynote speaker for our Virtual Open House 2021:
Prof. Melanie Zeilinger from ETH .
Our virtual open house will take place on December 9, 2021, 1:00pm - 5:30pm (EST).
Join us to learn more about who we are, what we do, and discuss our internship and employment opportunities. Prof. Zeilinger's talk is scheduled for 3:15pm - 3:45pm (EST).
Registration: https://mailchi.mp/merl/merlvoh2021
Keynote Title: Control Meets Learning - On Performance, Safety and User Interaction
Abstract: With increasing sensing and communication capabilities, physical systems today are becoming one of the largest generators of data, making learning a central component of autonomous control systems. While this paradigm shift offers tremendous opportunities to address new levels of system complexity, variability and user interaction, it also raises fundamental questions of learning in a closed-loop dynamical control system. In this talk, I will present some of our recent results showing how even safety-critical systems can leverage the potential of data. I will first briefly present concepts for using learning for automatic controller design and for a new safety framework that can equip any learning-based controller with safety guarantees. The second part will then discuss how expert and user information can be utilized to optimize system performance, where I will particularly highlight an approach developed together with MERL for personalizing the motion planning in autonomous driving to the individual driving style of a passenger.
-
- Date: January 1, 2022
Awarded to: Petros T. Boufounos
MERL Contact: Petros T. Boufounos
Research Areas: Computational Sensing, Signal Processing
Brief - MERL’s Petros Boufounos has been elevated to IEEE Fellow, effective January 2022, for “contributions to compressed sensing.”
IEEE Fellow is the highest grade of membership of the IEEE. It honors members with an outstanding record of technical achievements, contributing importantly to the advancement or application of engineering, science and technology, and bringing significant value to society. Each year, following a rigorous evaluation procedure, the IEEE Fellow Committee recommends a select group of recipients for elevation to IEEE Fellow. Less than 0.1% of voting members are selected annually for this member grade elevation.
-
- Date & Time: Thursday, December 9, 2021; 1:00pm - 5:30pm EST
Location: Virtual Event
Speaker: Prof. Ashok Veeraraghavan, Rice University
Research Areas: Applied Physics, Artificial Intelligence, Communications, Computational Sensing, Computer Vision, Control, Data Analytics, Dynamical Systems, Electric Systems, Electronic and Photonic Devices, Machine Learning, Multi-Physical Modeling, Optimization, Robotics, Signal Processing, Speech & Audio, Digital Video, Human-Computer Interaction, Information Security
Brief - MERL is excited to announce the first keynote speaker for our Virtual Open House 2021:
Prof. Ashok Veeraraghavan from Rice University.
Our virtual open house will take place on December 9, 2021, 1:00pm - 5:30pm (EST).
Join us to learn more about who we are, what we do, and discuss our internship and employment opportunities. Prof. Veeraraghavan's talk is scheduled for 1:15pm - 1:45pm (EST).
Registration: https://mailchi.mp/merl/merlvoh2021
Keynote Title: Computational Imaging: Beyond the limits imposed by lenses.
Abstract: The lens has long been a central element of cameras, since its early use in the mid-nineteenth century by Niepce, Talbot, and Daguerre. The role of the lens, from the Daguerrotype to modern digital cameras, is to refract light to achieve a one-to-one mapping between a point in the scene and a point on the sensor. This effect enables the sensor to compute a particular two-dimensional (2D) integral of the incident 4D light-field. We propose a radical departure from this practice and the many limitations it imposes. In the talk we focus on two inter-related research projects that attempt to go beyond lens-based imaging.
First, we discuss our lab’s recent efforts to build flat, extremely thin imaging devices by replacing the lens in a conventional camera with an amplitude mask and computational reconstruction algorithms. These lensless cameras, called FlatCams can be less than a millimeter in thickness and enable applications where size, weight, thickness or cost are the driving factors. Second, we discuss high-resolution, long-distance imaging using Fourier Ptychography, where the need for a large aperture aberration corrected lens is replaced by a camera array and associated phase retrieval algorithms resulting again in order of magnitude reductions in size, weight and cost. Finally, I will spend a few minutes discussing how the wholistic computational imaging approach can be used to create ultra-high-resolution wavefront sensors.
-