News & Events

1,453 News items, Awards, Events and Talks related to MERL and its staff.

  •  NEWS    MERL researchers win ASME Energy Systems Technical Committee Best Paper Award at 2022 American Control Conference
    Date: June 8, 2022
    Where: 2022 American Control Conference
    MERL Contacts: Ankush Chakrabarty; Christopher R. Laughman
    Research Areas: Control, Machine Learning, Multi-Physical Modeling, Optimization
    • Researchers from EPFL (Wenjie Xu, Colin Jones) and EMPA (Bratislav Svetozarevic), in collaboration with MERL researchers Ankush Chakrabarty and Chris Laughman, recently won the ASME Energy Systems Technical Committee Best Paper Award at the 2022 American Control Conference for their work on "VABO: Violation-Aware Bayesian Optimization for Closed-Loop Performance Optimization with Unmodeled Constraints" out of 19 nominations and 3 finalists. The paper describes a data-driven framework for optimizing the performance of constrained control systems by systematically re-evaluating how cautiously/aggressively one should explore the search space to avoid sustained, large-magnitude constraint violations while tolerating small violations, and demonstrates these methods on a physics-based model of a vapor compression cycle.
  •  NEWS    MERL researchers presented 9 papers at the American Control Conference (ACC)
    Date: June 8, 2022 - June 10, 2022
    Where: Atlanta, GA
    MERL Contacts: Karl Berntorp; Scott A. Bortoff; Ankush Chakrabarty; Stefano Di Cairano; Christopher R. Laughman; Marcel Menner; Rien Quirynen; Abraham P. Vinod; Avishai Weiss
    Research Areas: Control, Machine Learning, Optimization
    • At the American Control Conference in Atlanta, GA, MERL presented 9 papers on subjects including autonomous-vehicle decision making and motion planning, realtime Bayesian inference and learning, reference governors for hybrid systems, Bayesian optimization, and nonlinear control.
  •  NEWS    MERL work on 3D Printing in Orbit featured in IEEE Spectrum
    Date: June 3, 2022
    Where: IEEE Spectrum
    MERL Contacts: Avishai Weiss; William S. Yerazunis
    Research Areas: Applied Physics, Communications, Robotics
    • MERL's research on on-orbit manufacturing was recently featured in an IEEE Spectrum article. The article, titled How Satellites Will 3D Print Their Own Antennas in Space gives an overview of MERL's efforts towards developing a system that construct spacecraft parts in their natural environment-- that is, in space. The technology, called OOM for On-Orbit Manufacturing, provides a way to manufacture not just antenna dishes, but general freeform sturctures on orbit and in a vacuum, using an solar-hardened resin based approach. This technology includes both a special high performance liquid resin, as well as a 3D freeform printer capable of building objects far larger than the as-launched satellite.

      An important aspect of the special resin is that all components have extremely low vapor pressures and do not boil away even in a vacuum. When exposed to solar ultraviolet, the resin hardens by polymerization crosslinking, forming a tough, rigid solid in a few seconds of exposure. No separate UV source is needed, making the entire process very energy efficient. Additionally, the crosslinking resin is heat resistant, and is unaffected to at least 400 degrees C. The 3D printer needed to print the resin is unlike common liquid-resin SLA printers- there is no vat of liquid resin, instead a shielded nozzle delivers the liquid resin directly to where the resin is needed. The result is the ability to construct large and very large structures, not just parabolic dishes, but also solar panel supports and structural trusswork, while in orbit. The system could even construct parts that were unanticipated during mission design and launch.

      MERL's On-Orbit Manufacturing Technology had previously been featured in a Mitsubishi Electric Corporation Press Release and was recently on display at a recent press exhibition in Tokyo, Japan.

      IEEE Spectrum is the flagship magazine and website of the IEEE, the world’s largest professional organization devoted to engineering and the applied sciences. IEEE Spectrum has a circulation of over 400,000 engineers worldwide, making it one of the leading science and engineering magazines.
  •  NEWS    MERL researchers presented 5 papers and an invited workshop talk at ICRA 2022
    Date: May 23, 2022 - May 27, 2022
    Where: International Conference on Robotics and Automation (ICRA)
    MERL Contacts: Ankush Chakrabarty; Stefano Di Cairano; Siddarth Jain; Devesh K. Jha; Pedro Miraldo; Daniel N. Nikovski; Rien Quirynen; Arvind Raghunathan; Diego Romeres; Abraham P. Vinod; Yebin Wang
    Research Areas: Artificial Intelligence, Machine Learning, Robotics
    • MERL researchers presented 5 papers at the IEEE International Conference on Robotics and Automation (ICRA) that was held in Philadelphia from May 23-27, 2022. The papers covered a broad range of topics from manipulation, tactile sensing, planning and multi-agent control. The invited talk was presented in the "Workshop on Collaborative Robots and Work of the Future" which covered some of the work done by MERL researchers on collaborative robotic assembly. The workshop was co-organized by MERL, Mitsubishi Electric Automation's North America Development Center (NADC), and MIT.
  •  NEWS    MERL presenting 8 papers at ICASSP 2022
    Date: May 22, 2022 - May 27, 2022
    Where: Singapore
    MERL Contacts: Anoop Cherian; Chiori Hori; Toshiaki Koike-Akino; Jonathan Le Roux; Tim K. Marks; Philip V. Orlik; Kuan-Chuan Peng; Pu (Perry) Wang; Gordon Wichern
    Research Areas: Artificial Intelligence, Computer Vision, Signal Processing, Speech & Audio
    • MERL researchers are presenting 8 papers at the IEEE International Conference on Acoustics, Speech & Signal Processing (ICASSP), which is being held in Singapore from May 22-27, 2022. A week of virtual presentations also took place earlier this month.

      Topics to be presented include recent advances in speech recognition, audio processing, scene understanding, computational sensing, and classification.

      ICASSP is the flagship conference of the IEEE Signal Processing Society, and the world's largest and most comprehensive technical conference focused on the research advances and latest technological development in signal and information processing. The event attracts more than 2000 participants each year.
  •  NEWS    MERL Scientists Presenting 5 Papers at IEEE International Conference on Communications (ICC) 2022
    Date: May 16, 2022 - May 20, 2022
    Where: Seoul, Korea
    MERL Contacts: Jianlin Guo; Toshiaki Koike-Akino; Philip V. Orlik; Kieran Parsons; Pu (Perry) Wang; Ye Wang
    Research Areas: Artificial Intelligence, Communications, Computational Sensing, Computer Vision, Machine Learning, Signal Processing
    • MERL Connectivity & Information Processing Team scientists remotely presented 5 papers at the IEEE International Conference on Communications (ICC) 2022, held in Seoul Korea on May 16-20, 2022. Topics presented include recent advancements in communications technologies, deep learning methods, and quantum machine learning (QML). Presentation videos are also found on our YouTube channel. In addition, K. J. Kim organized "Industrial Private 5G-and-beyond Wireless Networks Workshop" at the conference.

      IEEE ICC is one of two IEEE Communications Society’s flagship conferences (ICC and Globecom). Each year, close to 2,000 attendees from over 70 countries attend IEEE ICC to take advantage of a program which consists of exciting keynote session, robust technical paper sessions, innovative tutorials and workshops, and engaging industry sessions. This 5-day event is known for bringing together audiences from both industry and academia to learn about the latest research and innovations in communications and networking technology, share ideas and best practices, and collaborate on future projects.
  •  NEWS    MERL's On-Orbit 3D Printing Technology Featured in Mitsubishi Electric Corporation Press Release
    Date: May 17, 2022
    Where: Tokyo, Japan
    MERL Contacts: Avishai Weiss; William S. Yerazunis
    Research Areas: Applied Physics, Communications
    • Mitsubishi Electric Corporation announced that the company has developed an on-orbit additive-manufacturing technology that uses photosensitive resin and solar ultraviolet light for the freeform printing of satellite antennas in the vacuum of outer space.

      The novel technology makes use of a newly developed liquid resin that was custom formulated for stability in vacuum. The resin enables structures to be fabricated in space using a low-power process that utilizes the sun’s ultraviolet rays for photopolymerization. The technology specifically addresses the challenge of equipping small, inexpensive spacecraft buses with large structures, such as high-gain antenna reflectors, and enables on-orbit fabrication of structures that greatly exceed the dimensions of launch vehicle fairings. Resin-based on-orbit manufacturing is expected to enable spacecraft structures to be made thinner and lighter than conventional designs, which must survive the stresses of launch and orbital insertion, thereby reducing both total satellite weight and launch costs.

      Mitsubishi Electric’s resin-based on-orbit manufacturing enables small satellites to have large satellite capability, which reduces launch costs and allows for satellite technology to be used more than ever in applications such as communication and Earth observation. The technology is based on recent research by MERL's Control for Autonomy and Data Analytics groups.


      Mitsubishi Electric Corporation Press Release
      SatMagazine: UV In The Sky With Resin: A novel, on-orbit manufacturing technique
  •  NEWS    Arvind Raghunathan's publication is Featured Article in the current issue of the INFORMS Journal on Computing
    Date: April 1, 2022
    Where: INFORMS Journal on Computing (
    MERL Contact: Arvind Raghunathan
    Research Areas: Artificial Intelligence, Machine Learning, Optimization
    • Arvind Raghunathan co-authored a publication titled "JANOS: An Integrated Predictive and Prescriptive Modeling Framework" which has been chosen as a Featured Article in the current issue of the INFORMS Journal on Computing. The article was co-authored with Prof. David Bergman, a collaborator of MERL and Teng Huang, a former MERL intern, among others.

      The paper describes a new software tool, JANOS, that integrates predictive modeling and discrete optimization to assist decision making. Specifically, the proposed solver takes as input user-specified pretrained predictive models and formulates optimization models directly over those predictive models by embedding them within an optimization model through linear transformations.
  •  NEWS    Toshiaki Koike-Akino gave an invited lecture to USPTO on advanced photonics
    Date: May 4, 2022
    MERL Contact: Toshiaki Koike-Akino
    Research Areas: Artificial Intelligence, Communications, Electronic and Photonic Devices, Machine Learning, Optimization, Signal Processing
    • Toshiaki Koike-Akino gave an invited lecture on advanced photonic devices at the United States Patent and Trademark Office (USPTO) Technology Fair on May 4, 2022. Topics of the lecture included the recent progress of applied artificial intelligence (AI) technologies for optical systems, nano-photonic devices, and quantum technology. During the 2-hour interactive online presentation, he lectured to more than 200 patent examiner participants.

      USPTO Tech Fair Organizer mentioned:
      "Thank you very much for representing Advanced Photonic Devices at this year’s Technology Center 2800 Virtual Tech Fair held May 4th, 2022. Tech Fair is an important part of the United States Patent and Trademark Office’s Patent Examiner Technical Training Program (PETTP). Having a scientifically well-trained examiner workforce and ensuring the quality, consistency, and reliability of issued patents are top priorities at the USPTO. The PETTP is designed to achieve those priorities by giving examiners direct access to technical experts who are willing to share their knowledge about prior art and industry standards for both emerging and established technologies. Experts like yourself help to maintain our high quality of patent examination by keeping examiners updated on technologies and innovations pertinent to their field of examination.
      We very much appreciate your efforts, time, and contributions."
  •  TALK    [MERL Seminar Series 2022] Prof. Michael Posa presents talk titled Hybrid robotics and implicit learning
    Date & Time: Tuesday, May 3, 2022; 1:00 PM
    Speaker: Michael Posa, University of Pennsylvania
    MERL Host: Devesh K. Jha
    Research Areas: Control, Optimization, Robotics
    • Machine learning has shown incredible promise in robotics, with some notable recent demonstrations in manipulation and sim2real transfer. These results, however, require either an accurate a priori model (for simulation) or a large amount of data. In contrast, my lab is focused on enabling robots to enter novel environments and then, with minimal time to gather information, accomplish complex tasks. In this talk, I will argue that the hybrid or contact-driven nature of real-world robotics, where a robot must safely and quickly interact with objects, drives this high data requirement. In particular, the inductive biases inherent in standard learning methods fundamentally clash with the non-differentiable physics of contact-rich robotics. Focusing on model learning, or system identification, I will show both empirical and theoretical results which demonstrate that contact stiffness leads to poor training and generalization, leading to some healthy skepticism of simulation experiments trained on artificially soft environments. Fortunately, implicit learning formulations, which embed convex optimization problems, can dramatically reshape the optimization landscape for these stiff problems. By carefully reasoning about the roles of stiffness and discontinuity, and integrating non-smooth structures, we demonstrate dramatically improved learning performance. Within this family of approaches, ContactNets accurately identifies the geometry and dynamics of a six-sided cube bouncing, sliding, and rolling across a surface from only a handful of sample trajectories. Similarly, a piecewise-affine hybrid system with thousands of modes can be identified purely from state transitions. Time permitting, I'll discuss how these learned models can be deployed for control via recent results in real-time, multi-contact MPC.
  •  NEWS    Radu Corcodel to present invited seminar at NYU on Robot Vision
    Date: May 4, 2022
    MERL Contact: Radu Corcodel
    Research Areas: Computer Vision, Robotics
    • Radu Corcodel, a Principal Research Scientist in MERL's Computer Vision Group, will present an overview of the Robot Perception research published by MERL for advanced manipulation. The talk will mainly cover topics pertaining to robotic manipulation in unstructured environments such as machine vision, tactile sensing and autonomous grasping. The seminar will also cover specific perception problems in non-prehensile interactions such as Contact-Implicit Trajectory Optimization and Tactile Classification, and is intended for a broader audience.
  •  NEWS    MERL's Yanting Ma completed the 2022 Boston Marathon
    Date: April 18, 2022
    Where: Boston, MA
    MERL Contact: Yanting Ma
    • Researcher, Yanting Ma, qualified for and completed the Boston Marathon on Monday April 18th. We would like to congratulate her on achieving this personal goal in an impressive time of 3:18.48!!
  •  TALK    [MERL Seminar Series 2022] Prof. Sebastien Gros presents talk titled RLMPC: An Ideal Combination of Formal Optimal Control and Reinforcement Learning?
    Date & Time: Tuesday, April 12, 2022; 11:00 AM EDT
    Speaker: Sebastien Gros, NTNU
    MERL Host: Rien Quirynen
    Research Areas: Control, Dynamical Systems, Optimization
    • Reinforcement Learning (RL), similarly to many AI-based techniques, is currently receiving a very high attention. RL is most commonly supported by classic Machine Learning techniques, i.e. typically Deep Neural Networks (DNNs). While there are good motivations for using DNNs in RL, there are also significant drawbacks. The lack of “explainability” of the resulting control policies, and the difficulty to provide guarantees on their closed-loop behavior (safety, stability) makes DNN-based policies problematic in many applications. In this talk, we will discuss an alternative approach to support RL, via formal optimal control tools based on Model Predictive Control (MPC). This approach alleviates the issues detailed above, but also presents some challenges. In this talk, we will discuss why MPC is a valid tool to support RL, and how MPC can be combined with RL (RLMPC). We will then discuss some recent results regarding this combination, the known challenges, and the kind of control applications where we believe that RLMPC will be a valuable approach.
  •  TALK    [MERL Seminar Series 2022] Albert Benveniste, Benoît Caillaud, and Mathias Malandain present talk titled Exact Structural Analysis of Multimode Modelica Models
    Date & Time: Tuesday, April 5, 2022; 11:00 AM EDT
    Speaker: Albert Benveniste, Benoît Caillaud, and Mathias Malandain, Inria
    MERL Host: Scott A. Bortoff
    Research Areas: Dynamical Systems, Multi-Physical Modeling
    • Since its 3.3 release, Modelica offers the possibility to specify models of dynamical systems with multiple modes having different DAE-based dynamics. However, the handling of such models by the current Modelica tools is not satisfactory, with mathematically sound models yielding exceptions at runtime. In our introduction, will briefly explain why and when the approximate structural analysis implemented in current Modelica tools leads to such errors. Then we will present our multimode Pryce Sigma-method for index reduction, in which the mode-dependent Sigma-matrix is represented in a dual form, by attaching, to every valuation of the sigma_ij entry of the Sigma matrix, the predicate characterizing the set of modes in which sigma_ij takes this value. We will illustrate this multimode analysis on example, by using our IsamDAE tool. In a second part, we will complement this multimode DAE structural analysis by a new structural analysis of mode changes (and, more generally, transient modes holding for zero time). Also, mode changes often give raise to impulsive behaviors: we will present a compile-time analysis identifying such behaviors. Our structural analysis of mode changes deeply relies on nonstandard analysis, which is a mathematical framework in which infinitesimals and infinities are first class citizens.
  •  TALK    [MERL Seminar Series 2022] Prof. Vincent Sitzmann presents talk titled Self-Supervised Scene Representation Learning
    Date & Time: Wednesday, March 30, 2022; 11:00 AM EDT
    Speaker: Vincent Sitzmann, MIT
    Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
    • Given only a single picture, people are capable of inferring a mental representation that encodes rich information about the underlying 3D scene. We acquire this skill not through massive labeled datasets of 3D scenes, but through self-supervised observation and interaction. Building machines that can infer similarly rich neural scene representations is critical if they are to one day parallel people’s ability to understand, navigate, and interact with their surroundings. This poses a unique set of challenges that sets neural scene representations apart from conventional representations of 3D scenes: Rendering and processing operations need to be differentiable, and the type of information they encode is unknown a priori, requiring them to be extraordinarily flexible. At the same time, training them without ground-truth 3D supervision is an underdetermined problem, highlighting the need for structure and inductive biases without which models converge to spurious explanations.

      I will demonstrate how we can equip neural networks with inductive biases that enables them to learn 3D geometry, appearance, and even semantic information, self-supervised only from posed images. I will show how this approach unlocks the learning of priors, enabling 3D reconstruction from only a single posed 2D image, and how we may extend these representations to other modalities such as sound. I will then discuss recent work on learning the neural rendering operator to make rendering and training fast, and how this speed-up enables us to learn object-centric neural scene representations, learning to decompose 3D scenes into objects, given only images. Finally, I will talk about a recent application of self-supervised scene representation learning in robotic manipulation, where it enables us to learn to manipulate classes of objects in unseen poses from only a handful of human demonstrations.
  •  NEWS    Rui Ma gives an Invited Talk on Digital Intensive PA/Transmitter for RF Communications Workshop at IMS2022
    Date: June 19, 2022
    Research Areas: Communications, Electronic and Photonic Devices, Machine Learning
    • MERL Researcher Rui Ma will give an invited talk titled "All Digital Transmitter with GaN Switching Mode Power Amplifiers"at a technical workshop during International Microwave Symposium (IMS)2022. This IMS workshop (WSN) invites members from academia and industry to discuss the latest development activities in the area of digital-intensive power amplifiers and transmitters for RF communications.

      In addition, Dr. Rui Ma is chairing a Technical Session(We2C) on "AI/ML on RF and mmWave Applications" at IMS2022.

      IMS is the flagship annual conference of IEEE Microwave Theory and Technology Society(MTT-S).

      Learn more here:
  •  TALK    [MERL Seminar Series 2022] Analog CMOS Computing Chips for Fast and Energy-Efficient Solution of PDE Systems
    Date & Time: Tuesday, March 15, 2022; 1:00 PM EDT
    Speaker: Arjuna Madanayake, Florida International University
    Research Areas: Applied Physics, Electronic and Photonic Devices, Multi-Physical Modeling
    • Analog computers are making a comeback. In fact, they are taking the world by storm. After decades of “analog computing winter” that followed the invention of the digital computing paradigm in the 1940s, classical physics-based analog computers are being reconsidered for improving the computational throughput of demanding applications. The research is driven by exponential growth in transistor densities and bandwidths in the integrated circuits world, which in turn, has led to new possibilities for the creative circuit designer. Fast analog chips not only furnish communication/radar front-ends, but can also be used to accelerate mathematical operations. Most analog computer today focus on AI and machine learning. E.g., analog in-memory computing plays an exciting role in AI acceleration because linear algebra operations can be mapped efficiently to compute in memory. However, many scientific computing tasks are built on linear and non-linear partial differential equations (PDEs) that require recursive numerical PDE solution across spatial and temporal dimensions. The adoption of analog parallel processors that are built around speed vs power efficiency vs precision trade-offs available from circuitry for PDE solution require new research in computer architecture. We report on recent progress on CMOS based analog computers for solving computational electromagnetics and non-linear pressure wave equations. Our first analog computing chip was measured to be more than 400x faster than a top-of-the-line NVIDIA GPU while consuming 1000x less power for elementary computational electromagnetics computations using finite-difference time-domain scheme.
  •  AWARD    Japan Telecommunications Advancement Foundation Award
    Date: March 15, 2022
    Awarded to: Yukimasa Nagai, Jianlin Guo, Philip Orlik, Takenori Sumi, Benjamin A. Rolfe and Hiroshi Mineno
    MERL Contacts: Jianlin Guo; Philip V. Orlik
    Research Areas: Communications, Machine Learning
    • MELCO/MERL research paper “Sub-1 GHz Frequency Band Wireless Coexistence for the Internet of Things” has won the 37th Telecommunications Advancement Foundation Award (Telecom System Technology Award) in Japan. This award started in 1984, and is given to research papers and works related to information and telecommunications that have made significant contributions and achievements to the advancement, development, and standardization of information and telecommunications from technical and engineering perspectives. The award recognizes both the IEEE 802.19.3 standardization efforts and the technological advancements using reinforcement learning and robust access methodologies for wireless communication system. This year, there were 43 entries with 5 winning awards and 3 winning encouragement awards. This is the first time MELCO/MERL has received this award. Our paper has been published by IEEE Access in 2021 and authors are Yukimasa Nagai, Jianlin Guo, Philip Orlik, Takenori Sumi, Benjamin A. Rolfe and Hiroshi Mineno.
  •  NEWS    Arvind Raghunathan delivers Optimization and Systems Theory seminar at KTH Royal Institute of Technology
    Date: March 10, 2022
    Where: Department of Mathematics, KTH Royal Institute of Technology
    MERL Contact: Arvind Raghunathan
    Research Area: Optimization
    • Arvind Raghunathan will present the Optimization and System Theorem seminar in the Department of Mathematics at the KTH Royal Institute of Technology. The title of the talk is Recursive McCormick Linearizations of Multilinear Programs: Minimum Size Formulations.
  •  NEWS    Devesh Jha delivers invited talk at Mechanical and Aerospace Engineering Department, NYU
    Date: March 1, 2022
    Where: Online/Zoom
    MERL Contact: Devesh K. Jha
    Research Areas: Artificial Intelligence, Machine Learning, Robotics
    • Devesh Jha, a Principal Research Scientist in MERL's Data Analytics group, gave an invited talk at the Mechanical and Aerospace Engineering Department, NYU. The title of the talk was "Robotic Manipulation in the Wild: Planning, Learning and Control through Contacts". The talk presented some of the recent work done at MERL for robotic manipulation in unstructured environments in the presence of significant uncertainty.
  •  NEWS    MERL work on scene-aware interaction featured in IEEE Spectrum
    Date: March 1, 2022
    MERL Contacts: Anoop Cherian; Chiori Hori; Jonathan Le Roux; Tim K. Marks; Anthony Vetro
    Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & Audio
    • MERL's research on scene-aware interaction was recently featured in an IEEE Spectrum article. The article, titled "At Last, A Self-Driving Car That Can Explain Itself" and authored by MERL Senior Principal Research Scientist Chiori Hori and MERL Director Anthony Vetro, gives an overview of MERL's efforts towards developing a system that can analyze multimodal sensing information for highly natural and intuitive interaction with humans through context-dependent generation of natural language. The technology recognizes contextual objects and events based on multimodal sensing information, such as images and video captured with cameras, audio information recorded with microphones, and localization information measured with LiDAR.

      Scene-Aware Interaction for car navigation, one target application that the article focuses on, will provide drivers with intuitive route guidance. Scene-Aware Interaction technology is expected to have wide applicability, including human-machine interfaces for in-vehicle infotainment, interaction with service robots in building and factory automation systems, systems that monitor the health and well-being of people, surveillance systems that interpret complex scenes for humans and encourage social distancing, support for touchless operation of equipment in public areas, and much more. MERL's Scene-Aware Interaction Technology had previously been featured in a Mitsubishi Electric Corporation Press Release.

      IEEE Spectrum is the flagship magazine and website of the IEEE, the world’s largest professional organization devoted to engineering and the applied sciences. IEEE Spectrum has a circulation of over 400,000 engineers worldwide, making it one of the leading science and engineering magazines.
  •  TALK    [MERL Seminar Series 2022] Learning Speech Representations with Multimodal Self-Supervision
    Date & Time: Tuesday, March 1, 2022; 1:00 PM EST
    Speaker: David Harwath, The University of Texas at Austin
    MERL Host: Chiori Hori
    Research Areas: Artificial Intelligence, Machine Learning, Speech & Audio
    • Humans learn spoken language and visual perception at an early age by being immersed in the world around them. Why can't computers do the same? In this talk, I will describe our ongoing work to develop methodologies for grounding continuous speech signals at the raw waveform level to natural image scenes. I will first present self-supervised models capable of discovering discrete, hierarchical structure (words and sub-word units) in the speech signal. Instead of conventional annotations, these models learn from correspondences between speech sounds and visual patterns such as objects and textures. Next, I will demonstrate how these discrete units can be used as a drop-in replacement for text transcriptions in an image captioning system, enabling us to directly synthesize spoken descriptions of images without the need for text as an intermediate representation. Finally, I will describe our latest work on Transformer-based models of visually-grounded speech. These models significantly outperform the prior state of the art on semantic speech-to-image retrieval tasks, and also learn representations that are useful for a multitude of other speech processing tasks.
  •  NEWS    Rien Quirynen appointed as an Associate Editor for Journal of Optimal Control Applications and Methods (OCAM)
    Date: February 24, 2022
    MERL Contact: Rien Quirynen
    Research Areas: Control, Optimization
    • Rien Quirynen has accepted an invitation to serve on the editorial board of Journal of Optimal Control Applications and Methods (OCAM) as an Associate Editor.

      OCAM provides a forum for papers on the full range of optimal control and related control design methods. The aim is to encourage new developments in optimal control theory and design methodologies that may lead to advances in real control applications.
  •  TALK    [MERL Seminar Series 2022] Beyond the First Portrait of a Black Hole
    Date & Time: Tuesday, February 15, 2022; 1:00 PM EST
    Speaker: Katie Bouman, California Institute of Technology
    MERL Host: Joshua Rapp
    Research Area: Computational Sensing
    • As imaging requirements become more demanding, we must rely on increasingly sparse and/or noisy measurements that fail to paint a complete picture. Computational imaging pipelines, which replace optics with computation, have enabled image formation in situations that are impossible for conventional optical imaging. For instance, the first black hole image, published in 2019, was only made possible through the development of computational imaging pipelines that worked alongside an Earth-sized distributed telescope. However, remaining scientific questions motivate us to improve this computational telescope to see black hole phenomena still invisible to us and to meaningfully interpret the collected data. This talk will discuss how we are leveraging and building upon recent advances in machine learning in order to achieve more efficient uncertainty quantification of reconstructed images as well as to develop techniques that allow us to extract the evolving structure of our own Milky Way's black hole over the course of a night, perhaps even in three dimensions.
  •  TALK    [MERL Seminar Series 2022] Extreme optics design as a large-scale optimization problem
    Date & Time: Tuesday, February 8, 2022; 1:00 PM EST
    Speaker: Raphaël Pestourie, MIT
    MERL Host: Matthew Brand
    Research Areas: Applied Physics, Electronic and Photonic Devices, Optimization
    • Thin large-area structures with aperiodic subwavelength patterns can unleash the full power of Maxwell’s equations for focusing light and a variety of other wave transformation or optical applications. Because of their irregularity and large scale, capturing the full scattering through these devices is one of the most challenging tasks for computational design: enter extreme optics! This talk will present ways to harness the full computational power of modern large-scale optimization in order to design optical devices with thousands or millions of free parameters. We exploit various methods of domain-decomposition approximations, supercomputer-scale topology optimization, laptop-scale “surrogate” models based on Chebyshev interpolation and/or new scientific machine learning models, and other techniques to attack challenging problems: achromatic lenses that simultaneously handle many wavelengths and angles, “deep” images, hyperspectral imaging, and more.