Artificial Intelligence
Making machines smarter for improved safety, efficiency and comfort.
Our AI research encompasses advances in computer vision, speech and audio processing, as well as data analytics. Key research themes include improved perception based on machine learning techniques, learning control policies through model-based reinforcement learning, as well as cognition and reasoning based on learned semantic representations. We apply our work to a broad range of automotive and robotics applications, as well as building and home systems.
Quick Links
-
Researchers
Jonathan
Le Roux
Takaaki
Hori
Toshiaki
Koike-Akino
Ye
Wang
Michael
Jones
Tim
Marks
Daniel
Nikovski
Chiori
Hori
Anoop
Cherian
Devesh
Jha
Gordon
Wichern
Alan
Sullivan
Niko
Moritz
Keisuke
Kojima
Philip
Orlik
Kieran
Parsons
Jeroen
van Baar
Diego
Romeres
William
Yerazunis
Arvind
Raghunathan
Petros
Boufounos
Hassan
Mansour
Saleh
Nabi
Hongbo
Sun
Zhong-Qiu
Wang
Radu
Corcodel
Kyeong Jin
(K.J.)
Kim
Chungwei
Lin
Yebin
Wang
Mouhacine
Benosman
Matthew
Brand
Suhas
Lohit
Anthony
Vetro
Bingnan
Wang
Jinyun
Zhang
Jianlin
Guo
Siddarth
Jain
Kuan-Chuan
Peng
Abraham
P. Vinod
-
Awards
-
AWARD Best Poster Award and Best Video Award at the International Society for Music Information Retrieval Conference (ISMIR) 2020 Date: October 15, 2020
Awarded to: Ethan Manilow, Gordon Wichern, Jonathan Le Roux
MERL Contacts: Jonathan Le Roux; Gordon Wichern
Research Areas: Artificial Intelligence, Machine Learning, Speech & AudioBrief- Former MERL intern Ethan Manilow and MERL researchers Gordon Wichern and Jonathan Le Roux won Best Poster Award and Best Video Award at the 2020 International Society for Music Information Retrieval Conference (ISMIR 2020) for the paper "Hierarchical Musical Source Separation". The conference was held October 11-14 in a virtual format. The Best Poster Awards and Best Video Awards were awarded by popular vote among the conference attendees.
The paper proposes a new method for isolating individual sounds in an audio mixture that accounts for the hierarchical relationship between sound sources. Many sounds we are interested in analyzing are hierarchical in nature, e.g., during a music performance, a hi-hat note is one of many such hi-hat notes, which is one of several parts of a drumkit, itself one of many instruments in a band, which might be playing in a bar with other sounds occurring. Inspired by this, the paper re-frames the audio source separation problem as hierarchical, combining similar sounds together at certain levels while separating them at other levels, and shows on a musical instrument separation task that a hierarchical approach outperforms non-hierarchical models while also requiring less training data. The paper, poster, and video can be seen on the paper page on the ISMIR website.
- Former MERL intern Ethan Manilow and MERL researchers Gordon Wichern and Jonathan Le Roux won Best Poster Award and Best Video Award at the 2020 International Society for Music Information Retrieval Conference (ISMIR 2020) for the paper "Hierarchical Musical Source Separation". The conference was held October 11-14 in a virtual format. The Best Poster Awards and Best Video Awards were awarded by popular vote among the conference attendees.
-
AWARD Best Paper Award at the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) 2019 Date: December 18, 2019
Awarded to: Xuankai Chang, Wangyou Zhang, Yanmin Qian, Jonathan Le Roux, Shinji Watanabe
MERL Contact: Jonathan Le Roux
Research Areas: Artificial Intelligence, Machine Learning, Speech & AudioBrief- MERL researcher Jonathan Le Roux and co-authors Xuankai Chang, Shinji Watanabe (Johns Hopkins University), Wangyou Zhang, and Yanmin Qian (Shanghai Jiao Tong University) won the Best Paper Award at the 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU 2019), for the paper "MIMO-Speech: End-to-End Multi-Channel Multi-Speaker Speech Recognition". MIMO-Speech is a fully neural end-to-end framework that can transcribe the text of multiple speakers speaking simultaneously from multi-channel input. The system is comprised of a monaural masking network, a multi-source neural beamformer, and a multi-output speech recognition model, which are jointly optimized only via an automatic speech recognition (ASR) criterion. The award was received by lead author Xuankai Chang during the conference, which was held in Sentosa, Singapore from December 14-18, 2019.
-
AWARD MERL Researchers win Best Paper Award at ICCV 2019 Workshop on Statistical Deep Learning in Computer Vision Date: October 27, 2019
Awarded to: Abhinav Kumar, Tim K. Marks, Wenxuan Mou, Chen Feng, Xiaoming Liu
MERL Contact: Tim Marks
Research Areas: Artificial Intelligence, Computer Vision, Machine LearningBrief- MERL researcher Tim Marks, former MERL interns Abhinav Kumar and Wenxuan Mou, and MERL consultants Professor Chen Feng (NYU) and Professor Xiaoming Liu (MSU) received the Best Oral Paper Award at the IEEE/CVF International Conference on Computer Vision (ICCV) 2019 Workshop on Statistical Deep Learning in Computer Vision (SDL-CV) held in Seoul, Korea. Their paper, entitled "UGLLI Face Alignment: Estimating Uncertainty with Gaussian Log-Likelihood Loss," describes a method which, given an image of a face, estimates not only the locations of facial landmarks but also the uncertainty of each landmark location estimate.
See All Awards for Artificial Intelligence -
-
News & Events
-
NEWS MERL researchers Diego Romeres, Devesh Jha and Siddarth Jain co-organized a workshop on Robotic Manipulation at the RSS 2021 conference Date: July 13, 2021
Where: Robotics: Science and Systems
MERL Contacts: Siddarth Jain; Devesh Jha; Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- MERL researchers Diego Romeres, Devesh Jha, and Siddarth Jain together with research groups at MIT, NVIDIA, NIST, TUM, Google DeepMind, ETH Zurich, Google AI, and UMASS Lowell organized a workshop at the Robotics: Science and Systems 2021 conference. The workshop was on "Advancing Artificial Intelligence and Manipulation for Robotics: Understanding Gaps, Industry and Academic Perspectives, and Community Building". The workshop had a list of excellent speakers both from academia and industry. Recording of the talks and of the panel discussion can be found in the link below.
-
NEWS Arvind Raghunathan joins editorial board of Journal of Optimization Theory and Applications Date: June 21, 2021
MERL Contact: Arvind Raghunathan
Research Areas: Artificial Intelligence, OptimizationBrief- Arvind Raghunathan has accepted an invitation to serve on the editorial board of Journal of Optimization Theory and Applications (JOTA).
JOTA is devoted to the publication of carefully selected high quality regular papers, invited papers, survey papers, technical notes, book notices, and forums that cover mathematical optimization techniques, computational methodologies of optimization algorithms and their applications to science, engineering, and business. Typical theoretical areas include linear, nonlinear, discrete, stochastic, and dynamic optimization. Among the areas of application covered are mathematical economics, mathematical physics and biology, all areas of engineering, and novel areas, such as artificial intelligence and quantum computing optimization.
- Arvind Raghunathan has accepted an invitation to serve on the editorial board of Journal of Optimization Theory and Applications (JOTA).
See All News & Events for Artificial Intelligence -
-
Research Highlights
-
Internships
-
SP1517: AI-based spectrum management for 5G wireless networks and beyond
MERL is seeking a highly motivated, qualified intern to join a thirteen weeks internship program. The ideal candidate will be expected to carry out research on emerging 5G wireless networks and beyond for industrial applications. The candidate is expected to develop innovative spectrum-based traffic recognition and optimal scheduling for local spectrum access. Candidates should have strong knowledge about 5G networks, spectrum management, cognitive radio, and neural network. Proficient programming skills with MATLAB, C++, Python (Pytorch), experience with ns-3 simulator, and strong mathematical analysis will be additional assets to this position. Candidates in their junior or senior years of a Ph.D. program are encouraged to apply. This internship is preferred to be onsite at MERL, but may be done remotely where you live if the COVID pandemic makes it necessary.
-
SP1537: Machine Learning for Wireless Communications
MERL is seeking an intern to work on machine learning for wireless communication systems. The ideal candidate would be an experienced PhD student or post-graduate researcher working in wireless communications with a focus on machine learning methods. The candidate should have a detailed knowledge of wireless communications, with some experience in machine learning, MIMO, and/or channel equalization preferred. Strong programming skills in Python and machine learning frameworks are essential. The expected duration of the internship is 3-6 months with flexible start date and length. This internship is preferred to be onsite at MERL, but may be done remotely where you live if the COVID pandemic makes it necessary.
-
MD1593: Design Optimization for Electric Machines
MERL is seeking a motivated and qualified intern to conduct research on design optimization of electrical machines. The ideal candidate should have solid background and demonstrated research experience in mathematical optimization methods, especially in topology optimization, robust optimization, sensitivity analysis, and machine learning techniques. Hands-on experiences with the implementation of optimization algorithms, machine learning and deep learning methods are highly desirable. Knowledge and experience with electric machine principle, design and finite-element analysis is a strong plus. Senior Ph.D. students in related expertise are encouraged to apply. Start date for this internship is flexible and the duration is about 3-6 months. This internship is preferred to be onsite at MERL, but may be done remotely where you live if the COVID pandemic makes it necessary.
See All Internships for Artificial Intelligence -
-
Recent Publications
- , "Robust Machine Learning via Privacy/Rate-Distortion Theory", IEEE International Symposium on Information Theory (ISIT), July 2021.BibTeX TR2021-082 PDF Video Presentation
- @inproceedings{Wang2021jul,
- author = {Wang, Ye and Aeron, Shuchin and Rakin, Adnan S and Koike-Akino, Toshiaki and Moulin, Pierre},
- title = {Robust Machine Learning via Privacy/Rate-Distortion Theory},
- booktitle = {IEEE International Symposium on Information Theory (ISIT)},
- year = 2021,
- month = jul,
- url = {https://www.merl.com/publications/TR2021-082}
- }
- , "Generalized One-Class Learning Using Pairs of Complementary Classifiers", IEEE Transactions on Pattern Analysis and Machine Intelligence, June 2021.BibTeX TR2021-076 PDF
- @article{Cherian2021jun,
- author = {Cherian, Anoop and Wang, Jue},
- title = {Generalized One-Class Learning Using Pairs of Complementary Classifiers},
- journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
- year = 2021,
- month = jun,
- url = {https://www.merl.com/publications/TR2021-076}
- }
- , "Cross-domain Imitation from Observations", International Conference on Machine Learning (ICML), June 2021.BibTeX TR2021-074 PDF
- @inproceedings{Raychaudhuri2021jun,
- author = {Raychaudhuri, Dripta S. and van Baar, Jeroen and Paul, Sujoy and Roy-Chowdhury, Amit K.},
- title = {Cross-domain Imitation from Observations},
- booktitle = {International Conference on Machine Learning (ICML)},
- year = 2021,
- month = jun,
- url = {https://www.merl.com/publications/TR2021-074}
- }
- , "Capturing Multi-Resolution Context by Dilated Self-Attention", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), June 2021.BibTeX TR2021-036 PDF
- @inproceedings{Moritz2021jun,
- author = {Moritz, Niko and Hori, Takaaki and Le Roux, Jonathan},
- title = {Capturing Multi-Resolution Context by Dilated Self-Attention},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2021,
- month = jun,
- url = {https://www.merl.com/publications/TR2021-036}
- }
- , "Time-Varying Graph Signal Inpainting via Unrolling Networks", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), June 2021.BibTeX TR2021-070 PDF
- @inproceedings{Chen2021jun,
- author = {Chen, Siheng and Eldar, Yonina},
- title = {Time-Varying Graph Signal Inpainting via Unrolling Networks},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2021,
- month = jun,
- url = {https://www.merl.com/publications/TR2021-070}
- }
- , "Graph Signaling Denoising via Unrolling Networks", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), June 2021.BibTeX TR2021-071 PDF
- @inproceedings{Chen2021jun3,
- author = {Chen, Siheng and Eldar, Yonina},
- title = {Graph Signaling Denoising via Unrolling Networks},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2021,
- month = jun,
- url = {https://www.merl.com/publications/TR2021-071}
- }
- , "Transcription Is All You Need: Learning to Separate Musical Mixtures with Score as Supervision", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), June 2021.BibTeX TR2021-069 PDF
- @inproceedings{Hung2021jun,
- author = {Hung, Yun-Ning and Wichern, Gordon and Le Roux, Jonathan},
- title = {Transcription Is All You Need: Learning to Separate Musical Mixtures with Score as Supervision},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2021,
- month = jun,
- url = {https://www.merl.com/publications/TR2021-069}
- }
- , "Unsupervised Domain Adaptation For Speech Recognition via Uncertainty Driven Self-Training", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), June 2021.BibTeX TR2021-039 PDF
- @inproceedings{Khurana2021jun,
- author = {Khurana, Sameer and Moritz, Niko and Hori, Takaaki and Le Roux, Jonathan},
- title = {Unsupervised Domain Adaptation For Speech Recognition via Uncertainty Driven Self-Training},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2021,
- month = jun,
- url = {https://www.merl.com/publications/TR2021-039}
- }
- , "Robust Machine Learning via Privacy/Rate-Distortion Theory", IEEE International Symposium on Information Theory (ISIT), July 2021.
-
Videos
-
Software Downloads
-
Goal directed RL with Safety Constraints
-
Generating Visual Dynamics from Sound and Context
-
Adversarially-Contrastive Optimal Transport
-
Online Feature Extractor Network
-
MotionNet
-
FoldingNet++
-
Quasi-Newton Trust Region Policy Optimization
-
Landmarks’ Location, Uncertainty, and Visibility Likelihood
-
Robust Iterative Data Estimation
-
Gradient-based Nikaido-Isoda
-
Discriminative Subspace Pooling
-
