Robotics
Where hardware, software and machine intelligence come together.
Our research is interdisciplinary and focuses on sensing, planning, reasoning, and control of single and multi-agent systems, including both manipulation and mobile robots. We strive to develop algorithms and methods for factory automation, smart building and transportation applications using machine learning, computer vision, RF/optical sensing, wireless communications, control theory and signal processing. Key research themes include bin picking and object manipulation, sensing and mapping of indoor areas, coordinated control of robot swarms, as well as robot learning and simulation.
Quick Links
-
Researchers
Devesh K.
Jha
Diego
Romeres
Daniel N.
Nikovski
Arvind
Raghunathan
Stefano
Di Cairano
Siddarth
Jain
William S.
Yerazunis
Yebin
Wang
Karl
Berntorp
Radu
Corcodel
Mouhacine
Benosman
Toshiaki
Koike-Akino
Tim K.
Marks
Scott A.
Bortoff
Abraham P.
Vinod
Avishai
Weiss
Ye
Wang
Matthew
Brand
Chiori
Hori
Jonathan
Le Roux
Philip V.
Orlik
Bingnan
Wang
Anoop
Cherian
Abraham
Goldsmith
Jianlin
Guo
Hassan
Mansour
Koon Hoo
Teo
Anthony
Vetro
Pedro
Miraldo
James
Queeney
-
Awards
-
AWARD Honorable Mention Award at NeurIPS 23 Instruction Workshop Date: December 15, 2023
Awarded to: Lingfeng Sun, Devesh K. Jha, Chiori Hori, Siddharth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka and Diego Romeres
MERL Contacts: Radu Corcodel; Chiori Hori; Siddarth Jain; Devesh K. Jha; Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- MERL Researchers received an "Honorable Mention award" at the Workshop on Instruction Tuning and Instruction Following at the NeurIPS 2023 conference in New Orleans. The workshop was on the topic of instruction tuning and Instruction following for Large Language Models (LLMs). MERL researchers presented their work on interactive planning using LLMs for partially observable robotic tasks during the oral presentation session at the workshop.
-
AWARD Joint University of Padua-MERL team wins Challenge 'AI Olympics With RealAIGym' Date: August 25, 2023
Awarded to: Alberto Dalla Libera, Niccolo' Turcato, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
The International Joint Conference on Artificial Intelligence (IJCAI) is a premier gathering for AI researchers and organizes several competitions. This year the competition CC7 "AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World?" consisted of two stages: simulation and real-robot experiments on two under-actuated robotic systems. The two robotics systems were treated as separate tracks and one final winner was selected for each track based on specific performance criteria in the control tasks.
The UniPD-MERL team competed and won in both tracks. The team's system made strong use of a Model-based Reinforcement Learning algorithm called (MC-PILCO) that we recently published in the journal IEEE Transaction on Robotics.
- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
-
AWARD MERL Researchers Win Best Workshop Poster Award at the 2023 IEEE International Conference on Robotics and Automation (ICRA) Date: June 2, 2023
Awarded to: Yuki Shirai, Devesh Jha, Arvind Raghunathan and Dennis Hong
MERL Contacts: Devesh K. Jha; Arvind Raghunathan
Research Areas: Artificial Intelligence, Optimization, RoboticsBrief- MERL's paper titled: "Closed-Loop Tactile Controller for Tool Manipulation" Won the Best Poster Award in the workshop on "Embracing contacts : Making robots physically interact with our world". First author and MERL intern, Yuki Shirai, was presented with the award at a ceremony held at ICRA in London. MERL researchers Devesh Jha, Principal Research Scientist, and Arvind Raghunathan, Senior Principal Research Scientist and Senior Team Leader as well as Prof. Dennis Hong of University of California, Los Angeles are also coauthors.
The paper presents a technique to manipulate an object using a tool in a closed-loop fashion using vision-based tactile sensors. More information about the workshop and the various speakers can be found here https://sites.google.com/view/icra2023embracingcontacts/home.
- MERL's paper titled: "Closed-Loop Tactile Controller for Tool Manipulation" Won the Best Poster Award in the workshop on "Embracing contacts : Making robots physically interact with our world". First author and MERL intern, Yuki Shirai, was presented with the award at a ceremony held at ICRA in London. MERL researchers Devesh Jha, Principal Research Scientist, and Arvind Raghunathan, Senior Principal Research Scientist and Senior Team Leader as well as Prof. Dennis Hong of University of California, Los Angeles are also coauthors.
See All Awards for Robotics -
-
News & Events
-
NEWS Invited talk given by Diego Romeres at Bentley University Date: November 1, 2023
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- Principal Research Scientist and Team Leader Diego Romeres gave an invited talk with title 'Applications of Machine Learning to Robotics' in the Machine Learning graduate course at Bentley University. The presentation focused mainly on Reinforcement Learning research applied to robotics. The audience consisted mostly of Master’s in Business Analytics (MSBA) students and students in the MBA w/ Business Analytics Concentration program.
-
TALK [MERL Seminar Series 2023] Prof. Shaoshuai Mou presents talk titled Inverse Optimal Control for Autonomous Systems Date & Time: Tuesday, October 10, 2023; 1:00 PM
Speaker: Shaoshuai Mou, Purdue University
MERL Host: Yebin Wang
Research Areas: Control, Dynamical Systems, RoboticsAbstract- Inverse Optimal Control (IOC) aims to achieve an objective function corresponding to a certain task from an expert robot driven by optimal control, which has become a powerful tool in many applications in robotics. We will present our recent solutions to IOC based on incomplete observations of systems' trajectories, which enables an autonomous system to “sense-and-adapt", i.e., incrementally improving the learning of objective functions as new data arrives. This also leads to a distributed algorithm to solve IOC in multi-agent systems, in which each agent can only access part of the overall trajectory of an optimal control system and cannot solve IOC by itself. This is perhaps the first distributed method to IOC. Applications of IOC into human prediction will also be given.
See All News & Events for Robotics -
-
Internships
-
OR2103: Human Robot Collaboration in Assembly Tasks
MERL is looking for a self-motivated and qualified candidate to work on human-robot-interaction for manipulation and assembly collaborative scenarios. The ideal candidate is a PhD student and should have experience and records in one or multiple of the following areas. 1) Control, estimation and perception for Robotic manipulation 2) Task and Motion Planning 3) Learning from demonstration algorithms applied to robotic manipulation 4) Machine learning techniques for modeling and control as well as regression and classification problems. 5) Experience in working with robotic systems and familiarity with physics engine simulators like Mujoco, Isaac Gym, PyBullet. The successful candidate will be expected to develop, in collaboration with MERL employees, state of the art algorithms to solve complex manipulation tasks that involve human and robot collaborations. Proficiency in Python and ROS are required. The expectation is that the research will lead to one or more scientific publications. The expected duration s 3-4 months, with a flexible starting date.
-
CA2131: Collaborative Legged Robots
MERL is seeking a highly motivated and qualified intern to collaborate with the Control for Autonomy team in research on control and planning algorithms for legged robots for support activities of and collaboration with humans. The ideal candidate is expected to be working towards a PhD with strong emphasis in robotics control and planning and to have interest and background in as many as possible of: motion planning algorithms, control for legged robot locomotions, legged robots, perception and sensing with multiple sensors, SLAM, vision-based control. Good programming skills in Python or C/C++ are required. The expected start of of the internship is flexible, with duration of 3--6 months.
-
CI2075: Human-Machine Interface with Biosignal Processing
MERL is seeking an intern to work on research for human-machine interface with multi-modal bio-sensors. The ideal candidate is an experienced PhD student or post-graduate researcher having an excellent background in brain-machine interface (BMI), deep learning, mixed reality (XR), remote robot manipulation, bionics, and bio sensing. The expected duration of the internship is 3-6 months, with a flexible start date.
See All Internships for Robotics -
-
Recent Publications
- "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, December 2024.BibTeX TR2024-008 PDF
- @article{Shirai2024dec,
- author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind and Romeres, Diego},
- title = {Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming},
- journal = {Nonlinear Analysis: Hybrid Systems},
- year = 2024,
- month = dec,
- url = {https://www.merl.com/publications/TR2024-008}
- }
, - "Stochastic Learning Manipulation of Object Pose With Under-Actuated Impulse Generator Arrays", International Conference on Machine Learning and Applications (ICMLA), DOI: 10.1109/ICMLA58977.2023.00024, December 2023, pp. 112-119.BibTeX TR2023-151 PDF
- @inproceedings{Kong2023dec,
- author = {Kong, Chuizheng and Yerazunis, William S. and Nikovski, Daniel},
- title = {Stochastic Learning Manipulation of Object Pose With Under-Actuated Impulse Generator Arrays},
- booktitle = {International Conference on Machine Learning and Applications (ICMLA)},
- year = 2023,
- pages = {112--119},
- month = dec,
- doi = {10.1109/ICMLA58977.2023.00024},
- url = {https://www.merl.com/publications/TR2023-151}
- }
, - "Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks", Advances in Neural Information Processing Systems (NeurIPS) Workshop on Instruction Tuning and Instruction Following, December 2023.BibTeX TR2023-148 PDF
- @inproceedings{Sun2023dec,
- author = {Sun, Lingfeng and Jha, Devesh K. and Hori, Chiori and Jain, Siddarth and Corcodel, Radu and Zhu, Xinghao and Tomizuka, Masayoshi and Romeres, Diego},
- title = {Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks},
- booktitle = {Advances in Neural Information Processing Systems (NeurIPS) Workshop on Instruction Tuning and Instruction Following},
- year = 2023,
- month = dec,
- url = {https://www.merl.com/publications/TR2023-148}
- }
, - "Physics Informed Gaussian Process Regression Methods for Robot Inverse Dynamics Identification", Conferenza Italiana di Robotica e Macchine Intelligenti, October 2023.BibTeX TR2023-132 PDF
- @inproceedings{Giacomuzzo2023oct2,
- author = {Giacomuzzo, Giulio and Dalla Libera, Alberto and Romeres, Diego and Carli, Ruggero},
- title = {Physics Informed Gaussian Process Regression Methods for Robot Inverse Dynamics Identification},
- booktitle = {Conferenza Italiana di Robotica e Macchine Intelligenti},
- year = 2023,
- month = oct,
- url = {https://www.merl.com/publications/TR2023-132}
- }
, - "EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation", 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), DOI: 10.1109/IROS55552.2023.10341988, October 2023, pp. 2963-2970.BibTeX TR2023-118 PDF Video
- @inproceedings{Huang2023oct,
- author = {Huang, Baichuan and Yu, Jingjin and Jain, Siddarth},
- title = {EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation},
- booktitle = {2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
- year = 2023,
- pages = {2963--2970},
- month = oct,
- publisher = {IEEE},
- doi = {10.1109/IROS55552.2023.10341988},
- issn = {2153-0866},
- isbn = {978-1-6654-9190-7},
- url = {https://www.merl.com/publications/TR2023-118}
- }
, - "Constrained Dynamic Movement Primitives for Collision Avoidance in Novel Environments", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), DOI: 10.1109/IROS55552.2023.10341839, September 2023, pp. 3672-3679.BibTeX TR2023-121 PDF Video
- @inproceedings{Shaw2023sep,
- author = {Shaw, Seiji and Jha, Devesh K. and Raghunathan, Arvind and Corcodel, Radu and Romeres, Diego and Konidaris, George and Nikovski, Daniel},
- title = {Constrained Dynamic Movement Primitives for Collision Avoidance in Novel Environments},
- booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
- year = 2023,
- pages = {3672--3679},
- month = sep,
- publisher = {IEEE},
- doi = {10.1109/IROS55552.2023.10341839},
- issn = {2153-0866},
- isbn = {978-1-6654-9190-7},
- url = {https://www.merl.com/publications/TR2023-121}
- }
, - "Contact-Aware Covariance Control of Stochastic Contact-Rich Systems", IROS 2023 Workshop on Leveraging Models for Contact-Rich Manipulation, September 2023.BibTeX TR2023-120 PDF
- @inproceedings{Shirai2023sep,
- author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind},
- title = {Contact-Aware Covariance Control of Stochastic Contact-Rich Systems},
- booktitle = {IROS 2023 Workshop on Leveraging Models for Contact-Rich Manipulation},
- year = 2023,
- month = sep,
- url = {https://www.merl.com/publications/TR2023-120}
- }
, - "Estimation of Extrinsic Contact Patch for Stable Placement", IEEE IROS 2023 Workshop on Visuo-Tactile Perception, Learning, Control for Manipulation and HRI, September 2023.BibTeX TR2024-018 PDF
- @inproceedings{Ota2023sep2,
- author = {Ota, Kei and Jha, Devesh K. and Jatavallabhula, Krishna and Kanezaki, Asako and Tenenbaum, Joshua B.},
- title = {Estimation of Extrinsic Contact Patch for Stable Placement},
- booktitle = {IEEE IROS 2023 Workshop on Visuo-Tactile Perception, Learning, Control for Manipulation and HRI},
- year = 2023,
- month = sep,
- url = {https://www.merl.com/publications/TR2024-018}
- }
,
- "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, December 2024.
-
Videos
-
Software & Data Downloads