Machine Learning
Data-driven approaches to design intelligent algorithms.
MERL has a long history of research activity in machine learning, including the development of various boosting algorithms and contributing to the theory and practice of highly scalable collaborative filtering. Our recent work has focused on deep learning and reinforcement learning, with application to a wide range of applications including automotive, robotics, factory automation, transportation, as well as building and home systems.
Quick Links
-
Researchers
Toshiaki
Koike-Akino
Jonathan
Le Roux
Ye
Wang
Ankush
Chakrabarty
Anoop
Cherian
Philip V.
Orlik
Gordon
Wichern
Tim K.
Marks
Daniel N.
Nikovski
Michael J.
Jones
Stefano
Di Cairano
Devesh K.
Jha
Kieran
Parsons
Karl
Berntorp
Chiori
Hori
Diego
Romeres
Christopher R.
Laughman
Yebin
Wang
Pu
(Perry)
WangMouhacine
Benosman
Bingnan
Wang
Hassan
Mansour
Matthew
Brand
Suhas
Lohit
Arvind
Raghunathan
Rien
Quirynen
Jing
Zhang
Jianlin
Guo
Siddarth
Jain
Petros T.
Boufounos
Scott A.
Bortoff
Radu
Corcodel
Kuan-Chuan
Peng
Abraham P.
Vinod
William S.
Yerazunis
Saviz
Mowlavi
Hongtao
Qiao
Hongbo
Sun
Vedang M.
Deshpande
Chungwei
Lin
Dehong
Liu
Wataru
Tsujita
Marcus
Greiff
Koon Hoo
Teo
Anthony
Vetro
Jinyun
Zhang
Jose
Amaya
Francois
Germain
Abraham
Goldsmith
Yanting
Ma
Avishai
Weiss
Jing
Liu
Zexu
Pan
-
Awards
-
AWARD Joint University of Padua-MERL team wins Challenge 'AI Olympics With RealAIGym' Date: August 25, 2023
Awarded to: Alberto Dalla Libera, Niccolo' Turcato, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
The International Joint Conference on Artificial Intelligence (IJCAI) is a premier gathering for AI researchers and organizes several competitions. This year the competition CC7 "AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World?" consisted of two stages: simulation and real-robot experiments on two under-actuated robotic systems. The two robotics systems were treated as separate tracks and one final winner was selected for each track based on specific performance criteria in the control tasks.
The UniPD-MERL team competed and won in both tracks. The team's system made strong use of a Model-based Reinforcement Learning algorithm called (MC-PILCO) that we recently published in the journal IEEE Transaction on Robotics.
- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
-
AWARD MERL Intern and Researchers Win ICASSP 2023 Best Student Paper Award Date: June 9, 2023
Awarded to: Darius Petermann, Gordon Wichern, Aswin Subramanian, Jonathan Le Roux
MERL Contacts: Jonathan Le Roux; Gordon Wichern
Research Areas: Artificial Intelligence, Machine Learning, Speech & AudioBrief- Former MERL intern Darius Petermann (Ph.D. Candidate at Indiana University) has received a Best Student Paper Award at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023) for the paper "Hyperbolic Audio Source Separation", co-authored with MERL researchers Gordon Wichern and Jonathan Le Roux, and former MERL researcher Aswin Subramanian. The paper presents work performed during Darius's internship at MERL in the summer 2022. The paper introduces a framework for audio source separation using embeddings on a hyperbolic manifold that compactly represent the hierarchical relationship between sound sources and time-frequency features. Additionally, the code associated with the paper is publicly available at https://github.com/merlresearch/hyper-unmix.
ICASSP is the flagship conference of the IEEE Signal Processing Society (SPS). ICASSP 2023 was held in the Greek island of Rhodes from June 04 to June 10, 2023, and it was the largest ICASSP in history, with more than 4000 participants, over 6128 submitted papers and 2709 accepted papers. Darius’s paper was first recognized as one of the Top 3% of all papers accepted at the conference, before receiving one of only 5 Best Student Paper Awards during the closing ceremony.
- Former MERL intern Darius Petermann (Ph.D. Candidate at Indiana University) has received a Best Student Paper Award at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023) for the paper "Hyperbolic Audio Source Separation", co-authored with MERL researchers Gordon Wichern and Jonathan Le Roux, and former MERL researcher Aswin Subramanian. The paper presents work performed during Darius's internship at MERL in the summer 2022. The paper introduces a framework for audio source separation using embeddings on a hyperbolic manifold that compactly represent the hierarchical relationship between sound sources and time-frequency features. Additionally, the code associated with the paper is publicly available at https://github.com/merlresearch/hyper-unmix.
-
AWARD MERL’s Paper on Wi-Fi Sensing Earns Top 3% Paper Recognition at ICASSP 2023, Selected as a Best Student Paper Award Finalist Date: June 9, 2023
Awarded to: Cristian J. Vaca-Rubio, Pu Wang, Toshiaki Koike-Akino, Ye Wang, Petros Boufounos and Petar Popovski
MERL Contacts: Petros T. Boufounos; Toshiaki Koike-Akino; Pu (Perry) Wang; Ye Wang
Research Areas: Artificial Intelligence, Communications, Computational Sensing, Dynamical Systems, Machine Learning, Signal ProcessingBrief- A MERL Paper on Wi-Fi sensing was recognized as a Top 3% Paper among all 2709 accepted papers at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023). Co-authored by Cristian Vaca-Rubio and Petar Popovski from Aalborg University, Denmark, and MERL researchers Pu Wang, Toshiaki Koike-Akino, Ye Wang, and Petros Boufounos, the paper "MmWave Wi-Fi Trajectory Estimation with Continous-Time Neural Dynamic Learning" was also a Best Student Paper Award finalist.
Performed during Cristian’s stay at MERL first as a visiting Marie Skłodowska-Curie Fellow and then as a full-time intern in 2022, this work capitalizes on standards-compliant Wi-Fi signals to perform indoor localization and sensing. The paper uses a neural dynamic learning framework to address technical issues such as low sampling rate and irregular sampling intervals.
ICASSP, a flagship conference of the IEEE Signal Processing Society (SPS), was hosted on the Greek island of Rhodes from June 04 to June 10, 2023. ICASSP 2023 marked the largest ICASSP in history, boasting over 4000 participants and 6128 submitted papers, out of which 2709 were accepted.
- A MERL Paper on Wi-Fi sensing was recognized as a Top 3% Paper among all 2709 accepted papers at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023). Co-authored by Cristian Vaca-Rubio and Petar Popovski from Aalborg University, Denmark, and MERL researchers Pu Wang, Toshiaki Koike-Akino, Ye Wang, and Petros Boufounos, the paper "MmWave Wi-Fi Trajectory Estimation with Continous-Time Neural Dynamic Learning" was also a Best Student Paper Award finalist.
See All Awards for Machine Learning -
-
News & Events
-
NEWS MERL presents 9 papers at 2023 IFAC World Congress Date: July 9, 2023 - July 14, 2023
MERL Contacts: Karl Berntorp; Scott A. Bortoff; Ankush Chakrabarty; Stefano Di Cairano; Christopher R. Laughman; Diego Romeres; Abraham P. Vinod
Research Areas: Control, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Optimization, RoboticsBrief- MERL researchers presented 9 papers and organized 2 invited/workshop sessions at the 2023 IFAC World Congress held in Yokohama, JP.
MERL's contributions covered topics including decision-making for autonomous vehicles, statistical and learning-based estimation for GNSS and energy systems, impedance control for delta robots, learning for system identification of rigid body dynamics and time-varying systems, and meta-learning for deep state-space modeling using data from similar systems. The invited session (MERL co-organizer: Ankush Chakrabarty) was on the topic of “Estimation and observer design: theory and applications” and the workshop (MERL co-organizer: Karl Berntorp) was on “Gaussian Process Learning for Systems and Control”.
- MERL researchers presented 9 papers and organized 2 invited/workshop sessions at the 2023 IFAC World Congress held in Yokohama, JP.
-
NEWS MERL researchers present 3 papers on Dexterous Manipulation at RSS 23. Date: July 11, 2023
Where: Daegu, Korea
MERL Contacts: Siddarth Jain; Devesh K. Jha; Arvind Raghunathan
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- MERL researchers presented 3 papers at the 19th edition of Robotics:Science and Systems Conference in Daegu, Korea. RSS is the flagship conference of the RSS foundation and is run as a single track conference presenting a limited number of high-quality papers. This year the main conference had a total of 112 papers presented. MERL researchers presented 2 papers in the main conference on planning and perception for dexterous manipulation. Another paper was presented in a workshop of learning for dexterous manipulation. More details can be found here https://roboticsconference.org.
See All News & Events for Machine Learning -
-
Research Highlights
-
Internships
-
CI1950: Quantum Machine Learning
MERL is seeking an intern to work on research for quantum machine learning (QML). The ideal candidate is an experienced PhD student or post-graduate researcher having an excellent background in quantum computing, deep learning, and signal processing. Proficient programming skills with PyTorch and PennyLane will be additional assets to this position.
-
MS1851: Dynamic Modeling and Control for Grid-Interactive Buildings
MERL is looking for a highly motivated and qualified candidate to work on modeling for smart sustainable buildings. The ideal candidate will have a strong understanding of modeling renewable energy sources, grid-interactive buildings, occupant behavior, and dynamical systems with expertise demonstrated via, e.g., peer-reviewed publications. Hands-on programming experience with Modelica is preferred. The minimum duration of the internship is 12 weeks; start time is flexible. This internship is preferred to be onsite at MERL, but may be done remotely where you live if the COVID pandemic makes it necessary.
-
ST1750: THz (Terahertz) Sensing
The Signal Processing (SP) group at MERL is seeking a highly motivated intern to conduct fundamental research in THz (Terahertz) sensing. Expertise in statistical inference, unsupervised anomaly detection, and deep learning (spatial-temporal representation learning) is required. Previous hands-on experience in THz data analysis is a plus. Familiarity with python and deep learning libraries is a must. The intern will collaborate with a small group of MERL researchers to develop novel algorithms, design experiments with collaborators, and prepare results for patents and publication. The expected duration of the internship is 3 months with a flexible start date.
See All Internships for Machine Learning -
-
Openings
-
EA2051: Research Scientist - Electric Systems Automation
-
CA2053: Research Scientist - Control for Autonomy
See All Openings at MERL -
-
Recent Publications
- "EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation", 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2023.BibTeX TR2023-118 PDF
- @inproceedings{Huang2023oct,
- author = {Huang, Baichuan and Yu, Jingjin and Jain, Siddarth},
- title = {EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation},
- booktitle = {2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
- year = 2023,
- month = oct,
- url = {https://www.merl.com/publications/TR2023-118}
- }
, - "Semi-Supervised Machine Learning for Motor Eccentricity Fault Diagnosis", Asia Pacific Conference of the Prognostics and Health Management Society, September 2023.BibTeX TR2023-117 PDF
- @inproceedings{Wang2023sep,
- author = {Wang, Bingnan and Zhang, Shen and Inoue, Hiroshi and Kanemaru, Makoto},
- title = {Semi-Supervised Machine Learning for Motor Eccentricity Fault Diagnosis},
- booktitle = {Asia Pacific Conference of the Prognostics and Health Management Society},
- year = 2023,
- month = sep,
- url = {https://www.merl.com/publications/TR2023-117}
- }
, - "Unrolled IPPG: Video Heart Rate Esitmation via Unrolling Proximal Gradient Descent", IEEE International Conference on Image Processing (ICIP), September 2023.BibTeX TR2023-116 PDF
- @inproceedings{Shenoy2023sep,
- author = {Shenoy, Vineet and Marks, Tim K. and Mansour, Hassan and Lohit, Suhas},
- title = {Unrolled IPPG: Video Heart Rate Esitmation via Unrolling Proximal Gradient Descent},
- booktitle = {IEEE International Conference on Image Processing (ICIP)},
- year = 2023,
- month = sep,
- url = {https://www.merl.com/publications/TR2023-116}
- }
, - "A Virtual Testbed for Robust and Reproducible Calibration of Building Energy Simulation Models", 18th IBPSA International Conference and Exhibition Building Simulation, September 2023.BibTeX TR2023-114 PDF
- @inproceedings{Zhan2023sep,
- author = {Zhan, Sicheng and Chakrabarty, Ankush and Laughman, Christopher R. and Chong, Adrian},
- title = {A Virtual Testbed for Robust and Reproducible Calibration of Building Energy Simulation Models},
- booktitle = {18th IBPSA International Conference and Exhibition Building Simulation},
- year = 2023,
- month = sep,
- url = {https://www.merl.com/publications/TR2023-114}
- }
, - "Overview of the Tenth Dialog System Technology Challenge: DSTC10", IEE/ACM Transactions on Audio, Speech, and Language Processing, August 2023.BibTeX TR2023-109 PDF
- @article{Yoshino2023aug,
- author = {Yoshino, Koichiro and Chen, Yun-Nung and Crook, Paul and Kottur, Satwik and Li, Jinchao and Hedayatnia, Behnam and Moon, Seungwhan and Fe, Zhengcong and Li, Zekang and Zhang, Jinchao and Fen, Yang and Zhou, Jie and Kim, Seokhwan and Liu, Yang and Jin, Di and Papangelis, Alexandros and Gopalakrishnan, Karthik and Hakkani-Tur, Dilek and Damavandi, Babak and Geramifard, Alborz and
Hori, Chiori and Shah, Ankit and Zhang, Chen and Li, Haizhou and Sedoc, João and D’Haro, Luis F. and Banchs, Rafael and Rudnicky, Alexander}, - title = {Overview of the Tenth Dialog System Technology Challenge: DSTC10},
- journal = {IEE/ACM Transactions on Audio, Speech, and Language Processing},
- year = 2023,
- month = aug,
- url = {https://www.merl.com/publications/TR2023-109}
- }
, - "Trajectory Generation for Online Payload Estimation of Robot Manipulators: A Supervised Learning Based Approach", IEEE Conference on Automation and Science Engineering, August 2023.BibTeX TR2023-106 PDF
- @inproceedings{Duan2023aug,
- author = {Duan, Xiaoming and Wang, Yebin and Romeres, Diego and Koike-Akino, Toshiaki and Orlik, Philip V.},
- title = {Trajectory Generation for Online Payload Estimation of Robot Manipulators: A Supervised Learning Based Approach},
- booktitle = {IEEE Conference on Automation and Science Engineering},
- year = 2023,
- month = aug,
- url = {https://www.merl.com/publications/TR2023-106}
- }
, - "Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos", Interspeech, August 2023.BibTeX TR2023-104 PDF
- @inproceedings{Hori2023aug,
- author = {Hori, Chiori and Peng, Puyuang and Harwath, David and Liu, Xinyu and Ota, Kei and Jain, Siddarth and Corcodel, Radu and Jha, Devesh K. and Romeres, Diego and Le Roux, Jonathan},
- title = {Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos},
- booktitle = {Interspeech},
- year = 2023,
- month = aug,
- url = {https://www.merl.com/publications/TR2023-104}
- }
, - "Digital Twins for Vapor Compression Cycles: Challenges & Opportunities", International Congress of Refrigeration (ICR), August 2023.BibTeX TR2023-103 PDF
- @inproceedings{Laughman2023aug,
- author = {Laughman, Christopher R. and Deshpande, Vedang M. and Qiao, Hongtao and Bortoff, Scott A. and Chakrabarty, Ankush},
- title = {Digital Twins for Vapor Compression Cycles: Challenges & Opportunities},
- booktitle = {International Congress of Refrigeration (ICR)},
- year = 2023,
- month = aug,
- url = {https://www.merl.com/publications/TR2023-103}
- }
,
- "EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation", 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2023.
-
Videos
-
[MERL Seminar Series Fall 2023] A Process Systems Engineering Perspective on Carbon Capture: Key Challenges and Opportunities
-
Safe multi-agent motion planning under uncertainty using filtered reinforcement learning
-
Are Deep Neural Networks SMARTer than Second Graders?
-
[CVPR 2023] EVAL: Explainable Video Anomaly Localization
-
[MERL Seminar Series Spring 2023] Learning and Dynamical Systems
-
[MERL Seminar Series Spring 2023] Investigating Multi-Agent Reinforcement Learning for Grid-Interactive Smart Communities using CityLearn
-
[MERL Seminar Series Spring 2023] Pitfalls and Opportunities in Interpretable Machine Learning
-
[MERL Seminar Series Spring 2023] Neural Implicit Flow
-
[MERL Seminar Series Spring 2023] Towards Complex Language in Partially Observed Environments
-
[MERL Seminar Series Spring 2022] Hybrid robotics and implicit learning
-
Toshiaki Koike-Akino Gives Seminar Talk at IEEE Boston Photonics
-
[MERL Seminar Series Spring 2022] RLMPC: An Ideal Combination of Formal Optimal Control and Reinforcement Learning?
-
[MERL Seminar Series Spring 2022] Self-Supervised Scene Representation Learning
-
[MERL Seminar Series Spring 2022] Learning Speech Representations with Multimodal Self-Supervision
-
[MERL Seminar Series Spring 2022] Extreme optics design as a large-scale optimization problem
-
HealthCam: A system for non-contact monitoring of vital signs
-
[MERL Seminar Series 2021] Harnessing machine learning to build better Earth system models for climate projection
-
[MERL Seminar Series 2021] Deep probabilistic regression
-
[MERL Seminar Series 2021] Learning to See by Moving: Self-supervising 3D scene representations for perception, control, and visual reasoning
-
Control of Mechanical Systems via Feedback Linearization Based on Black-Box Gaussian Process Models
-
Application of Deep Learning for Nanophotonic Device Design (Invited)
-
Towards Human-Level Learning of Complex Physical Puzzles
-
Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry
-
Scene-Aware Interaction Technology
-
Action Detection Using A Deep Recurrent Neural Network
-
MERL Research on Autonomous Vehicles
-
Semantic Scene Labeling
-
Obstacle Detection
-
Deep Hierarchical Parsing for Semantic Segmentation
-
Global Local Face Upsampling Network
-
-
Downloads
-
DeepBornFNO
-
Simple Multimodal Algorithmic Reasoning Task Dataset
-
SOurce-free Cross-modal KnowledgE Transfer
-
Audio-Visual-Language Embodied Navigation in 3D Environments
-
Nonparametric Score Estimators
-
Instance Segmentation GAN
-
Audio Visual Scene-Graph Segmentor
-
Generalized One-class Discriminative Subspaces
-
Hierarchical Musical Instrument Separation
-
Generating Visual Dynamics from Sound and Context
-
Adversarially-Contrastive Optimal Transport
-
Online Feature Extractor Network
-
MotionNet
-
FoldingNet++
-
Quasi-Newton Trust Region Policy Optimization
-
Landmarks’ Location, Uncertainty, and Visibility Likelihood
-
Robust Iterative Data Estimation
-
Gradient-based Nikaido-Isoda
-
Circular Maze Environment
-
Discriminative Subspace Pooling
-
Kernel Correlation Network
-
Fast Resampling on Point Clouds via Graphs
-
FoldingNet
-
Deep Category-Aware Semantic Edge Detection
-
MERL Shopping Dataset
-
Partial Group Convolutional Neural Networks
-