Robotics
Where hardware, software and machine intelligence come together.
Our research is interdisciplinary and focuses on sensing, planning, reasoning, and control of single and multi-agent systems, including both manipulation and mobile robots. We strive to develop algorithms and methods for factory automation, smart building and transportation applications using machine learning, computer vision, RF/optical sensing, wireless communications, control theory and signal processing. Key research themes include bin picking and object manipulation, sensing and mapping of indoor areas, coordinated control of robot swarms, as well as robot learning and simulation.
Quick Links
-
Researchers
Devesh K.
Jha
Diego
Romeres
Daniel N.
Nikovski
Arvind
Raghunathan
Stefano
Di Cairano
Siddarth
Jain
Yebin
Wang
William S.
Yerazunis
Karl
Berntorp
Mouhacine
Benosman
Toshiaki
Koike-Akino
Radu
Corcodel
Tim K.
Marks
Scott A.
Bortoff
Abraham P.
Vinod
Avishai
Weiss
Rien
Quirynen
Ye
Wang
Matthew
Brand
Marcus
Greiff
Jonathan
Le Roux
Philip V.
Orlik
Bingnan
Wang
Anoop
Cherian
Abraham
Goldsmith
Jianlin
Guo
Chiori
Hori
Hassan
Mansour
Koon Hoo
Teo
Anthony
Vetro
Pedro
Miraldo
-
Awards
-
AWARD Joint University of Padua-MERL team wins Challenge 'AI Olympics With RealAIGym' Date: August 25, 2023
Awarded to: Alberto Dalla Libera, Niccolo' Turcato, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
The International Joint Conference on Artificial Intelligence (IJCAI) is a premier gathering for AI researchers and organizes several competitions. This year the competition CC7 "AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World?" consisted of two stages: simulation and real-robot experiments on two under-actuated robotic systems. The two robotics systems were treated as separate tracks and one final winner was selected for each track based on specific performance criteria in the control tasks.
The UniPD-MERL team competed and won in both tracks. The team's system made strong use of a Model-based Reinforcement Learning algorithm called (MC-PILCO) that we recently published in the journal IEEE Transaction on Robotics.
- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
-
AWARD MERL Researchers Win Best Workshop Poster Award at the 2023 IEEE International Conference on Robotics and Automation (ICRA) Date: June 2, 2023
Awarded to: Yuki Shirai, Devesh Jha, Arvind Raghunathan and Dennis Hong
MERL Contacts: Devesh K. Jha; Arvind Raghunathan
Research Areas: Artificial Intelligence, Optimization, RoboticsBrief- MERL's paper titled: "Closed-Loop Tactile Controller for Tool Manipulation" Won the Best Poster Award in the workshop on "Embracing contacts : Making robots physically interact with our world". First author and MERL intern, Yuki Shirai, was presented with the award at a ceremony held at ICRA in London. MERL researchers Devesh Jha, Principal Research Scientist, and Arvind Raghunathan, Senior Principal Research Scientist and Senior Team Leader as well as Prof. Dennis Hong of University of California, Los Angeles are also coauthors.
The paper presents a technique to manipulate an object using a tool in a closed-loop fashion using vision-based tactile sensors. More information about the workshop and the various speakers can be found here https://sites.google.com/view/icra2023embracingcontacts/home.
- MERL's paper titled: "Closed-Loop Tactile Controller for Tool Manipulation" Won the Best Poster Award in the workshop on "Embracing contacts : Making robots physically interact with our world". First author and MERL intern, Yuki Shirai, was presented with the award at a ceremony held at ICRA in London. MERL researchers Devesh Jha, Principal Research Scientist, and Arvind Raghunathan, Senior Principal Research Scientist and Senior Team Leader as well as Prof. Dennis Hong of University of California, Los Angeles are also coauthors.
-
AWARD Marcus Greiff receives Outstanding Student Paper Award at CCTA 2022 Date: August 25, 2022
Awarded to: Marcus Greiff
MERL Contact: Marcus Greiff
Research Areas: Control, Dynamical Systems, RoboticsBrief- Marcus Greiff, a Visiting Research Scientist at MERL, was awarded one of three outstanding student paper awards at the IEEE CCTA 2022 conference for his paper titled "Quadrotor Control on SU(2)xR3 with SLAM Integration". The award was given for originality, clarity, and potential impact on practical applications of control. The work presents a complete UAV control system design, facilitating autonomous supermarket inventorying without the need for external motion capture systems. A video of the experiments is on YouTube, including both simulations and real-time examples.
See All Awards for Robotics -
-
News & Events
-
NEWS MERL presents 9 papers at 2023 IFAC World Congress Date: July 9, 2023 - July 14, 2023
MERL Contacts: Karl Berntorp; Scott A. Bortoff; Ankush Chakrabarty; Stefano Di Cairano; Christopher R. Laughman; Diego Romeres; Abraham P. Vinod
Research Areas: Control, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Optimization, RoboticsBrief- MERL researchers presented 9 papers and organized 2 invited/workshop sessions at the 2023 IFAC World Congress held in Yokohama, JP.
MERL's contributions covered topics including decision-making for autonomous vehicles, statistical and learning-based estimation for GNSS and energy systems, impedance control for delta robots, learning for system identification of rigid body dynamics and time-varying systems, and meta-learning for deep state-space modeling using data from similar systems. The invited session (MERL co-organizer: Ankush Chakrabarty) was on the topic of “Estimation and observer design: theory and applications” and the workshop (MERL co-organizer: Karl Berntorp) was on “Gaussian Process Learning for Systems and Control”.
- MERL researchers presented 9 papers and organized 2 invited/workshop sessions at the 2023 IFAC World Congress held in Yokohama, JP.
-
NEWS MERL researchers present 3 papers on Dexterous Manipulation at RSS 23. Date: July 11, 2023
Where: Daegu, Korea
MERL Contacts: Siddarth Jain; Devesh K. Jha; Arvind Raghunathan
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- MERL researchers presented 3 papers at the 19th edition of Robotics:Science and Systems Conference in Daegu, Korea. RSS is the flagship conference of the RSS foundation and is run as a single track conference presenting a limited number of high-quality papers. This year the main conference had a total of 112 papers presented. MERL researchers presented 2 papers in the main conference on planning and perception for dexterous manipulation. Another paper was presented in a workshop of learning for dexterous manipulation. More details can be found here https://roboticsconference.org.
See All News & Events for Robotics -
-
Internships
-
OR2041: Object Pose Estimation and Tracking for Robotic Manipulation
MERL is looking for a highly motivated and qualified intern to work on 6D pose estimation and tracking of multiple objects for robotic manipulation. The ideal candidate would be a Ph.D. student familiar with the state-of-the-art methods for 6D pose estimation, tracking of rigid objects and vision based scene understanding. The candidate should have a strong background in deep learning using unsupervised and self-supervised approaches and geometric scene understanding. The internship requires developing and implementing algorithms to estimate accurate 6D poses of multiple objects in a scene captured by an RGB-D camera with unknown positions. The method will be applied for robotic manipulation where the knowledge of accurate position and orientation of objects within the scene would allow the robot to interact with the objects. Experience working with a physics engine simulator like PyBullet, Issac Gym, or Mujoco is preferred. Proficiency in Python programming is necessary, and experience with ROS is a plus. The successful candidate will collaborate with MERL researcher, and publication of the relevant results is expected. Start date in the summer or early fall is preferable, and the expected duration of the internship is 3-4 months. Interested candidates are encouraged to apply with their recent CV and list of publications in related topics.
-
CA1940: Autonomous vehicle planning and contro in uncertain environments
MERL is seeking a highly motivated and qualified intern to collaborate with the Control for Autonomy team in research on planning and control for autonomous vehicles in uncertain surrounding environments. The research domain includes algorithms for path planning and control in environments that are uncertain and perceived by sensing and predicted according to models and data. The ideal candidate is expected to be working towards a PhD with strong emphasis in vehicle guidance and control, and to have interest and background in as many as possible of: vehicle dynamics modeling and control, sensor uncertainty modeling, data-driven prediction, predictive control for uncertain systems, motion planning. Good programming skills in MATLAB, Python are required, knowledge of C/C++, rapid prototyping systems, automatic code generation, vehicle simulation packages (CarSim, CarMaker) or ROS are a plus. The expected start of of the internship is in the late Spring/Early Summer 2022, for a duration of 3-6 months.
-
OR2042: Robotic Manipulation
MERL is looking for a highly motivated individual to work on robust control of robotic manipulation system using vision and tactile feedback. The research will develop complex system using optimization, contact sensing and control techniques that can be used for closed-loop control of manipulation systems. The ideal candidate should have experience in either one or multiple of the following topics: optimization for contact-rich systems, manipulation, grasping and tactile sensing. Senior PhD students in robotics and engineering with a focus on contact-rich manipulation are encouraged to apply. Prior experience working with physical robotic systems (and vision and tactile sensors) is required as results need to be implemented on a physical hardware. A successful internship will result in submission of results to peer-reviewed conference and journals. Good coding skills in Python and state-of-the-art optimization packages like IPOPT, SNOPT, etc. is required. The expected duration of internship is 3-4 months with start date in Aug/Sept 2023. This internship is preferred to be onsite at MERL.
See All Internships for Robotics -
-
Openings
See All Openings at MERL -
Recent Publications
- "EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation", 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2023.BibTeX TR2023-118 PDF
- @inproceedings{Huang2023oct,
- author = {Huang, Baichuan and Yu, Jingjin and Jain, Siddarth},
- title = {EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation},
- booktitle = {2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
- year = 2023,
- month = oct,
- url = {https://www.merl.com/publications/TR2023-118}
- }
, - "Athletic Intelligence Olympics challenge with Model-Based Reinforcement Learning", International Joint Conference on Artificial Intelligence, August 2023.BibTeX TR2023-111 PDF
- @inproceedings{DallaLibera2023aug,
- author = {Dalla Libera, Alberto and Turcato, Niccolò and Giacomuzzo, Giulio and Carli, Ruggero and Romeres, Diego},
- title = {Athletic Intelligence Olympics challenge with Model-Based Reinforcement Learning},
- booktitle = {International Joint Conference on Artificial Intelligence},
- year = 2023,
- month = aug,
- url = {https://www.merl.com/publications/TR2023-111}
- }
, - "Trajectory Generation for Online Payload Estimation of Robot Manipulators: A Supervised Learning Based Approach", IEEE Conference on Automation and Science Engineering, August 2023.BibTeX TR2023-106 PDF
- @inproceedings{Duan2023aug,
- author = {Duan, Xiaoming and Wang, Yebin and Romeres, Diego and Koike-Akino, Toshiaki and Orlik, Philip V.},
- title = {Trajectory Generation for Online Payload Estimation of Robot Manipulators: A Supervised Learning Based Approach},
- booktitle = {IEEE Conference on Automation and Science Engineering},
- year = 2023,
- month = aug,
- url = {https://www.merl.com/publications/TR2023-106}
- }
, - "Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos", Interspeech, August 2023.BibTeX TR2023-104 PDF
- @inproceedings{Hori2023aug,
- author = {Hori, Chiori and Peng, Puyuang and Harwath, David and Liu, Xinyu and Ota, Kei and Jain, Siddarth and Corcodel, Radu and Jha, Devesh K. and Romeres, Diego and Le Roux, Jonathan},
- title = {Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos},
- booktitle = {Interspeech},
- year = 2023,
- month = aug,
- url = {https://www.merl.com/publications/TR2023-104}
- }
, - "Motion Planning of Articulated Vehicles with Active Trailer Steering by Particle Filtering", Conference on Control Technology and Applications (CCTA), August 2023.BibTeX TR2023-101 PDF
- @inproceedings{Iqbal2023aug,
- author = {Iqbal, Hassan and Di Cairano, Stefano and Berntorp, Karl},
- title = {Motion Planning of Articulated Vehicles with Active Trailer Steering by Particle Filtering},
- booktitle = {Conference on Control Technology and Applications (CCTA)},
- year = 2023,
- month = aug,
- url = {https://www.merl.com/publications/TR2023-101}
- }
, - "Forward Dynamics Estimation from Data-Driven Inverse Dynamics Learning", World Congress of the International Federation of Automatic Control (IFAC), July 2023.BibTeX TR2023-092 PDF
- @inproceedings{DallaLibera2023jul2,
- author = {Dalla Libera, Alberto and Giacomuzzo, Giulio and Carli, Ruggero and Nikovski, Daniel and Romeres, Diego},
- title = {Forward Dynamics Estimation from Data-Driven Inverse Dynamics Learning},
- booktitle = {World Congress of the International Federation of Automatic Control (IFAC)},
- year = 2023,
- month = jul,
- url = {https://www.merl.com/publications/TR2023-092}
- }
, - "Bandit-based multi-agent search under noisy observations", World Congress of the International Federation of Automatic Control (IFAC), July 2023.BibTeX TR2023-085 PDF
- @inproceedings{Thaker2023jul,
- author = {Thaker, Parth and Di Cairano, Stefano and Vinod, Abraham P.},
- title = {Bandit-based multi-agent search under noisy observations},
- booktitle = {World Congress of the International Federation of Automatic Control (IFAC)},
- year = 2023,
- month = jul,
- url = {https://www.merl.com/publications/TR2023-085}
- }
, - "Decision Making for Automated Driving by Reachability of Parameterized Maneuvers", World Congress of the International Federation of Automatic Control (IFAC), July 2023.BibTeX TR2023-084 PDF
- @inproceedings{DiCairano2023jul,
- author = {Di Cairano, Stefano and Skibik, Terrence and Vinod, Abraham P. and Weiss, Avishai and Berntorp, Karl},
- title = {Decision Making for Automated Driving by Reachability of Parameterized Maneuvers},
- booktitle = {World Congress of the International Federation of Automatic Control (IFAC)},
- year = 2023,
- month = jul,
- url = {https://www.merl.com/publications/TR2023-084}
- }
,
- "EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation", 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2023.
-
Videos
-
Simultaneous Trajectory Optimization and Contact Selection for Multi-Modal Manipulation Planning
-
A Virtual Reality Teleoperation Interface for Industrial Robot Manipulators
-
Simultaneous Tactile Estimation and Control of Extrinsic Contact
-
Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization
-
Tactile-Filter: Interactive Tactile Perception for Part Mating
-
[MERL Seminar Series Spring 2023] Towards Complex Language in Partially Observed Environments
-
Tactile tool manipulation
-
Robot Locomotion by Automated Controller Tuning
-
Real-time Mixed-integer Programming for Vehicle Decision Making and Motion Planning
-
[MERL Seminar Series Spring 2022] Hybrid robotics and implicit learning
-
[MERL Seminar Series Spring 2022] Exact Structural Analysis of Multimode Modelica Models
-
[MERL Seminar Series Spring 2022] Self-Supervised Scene Representation Learning
-
[MERL Seminar Series 2021] Learning to See by Moving: Self-supervising 3D scene representations for perception, control, and visual reasoning
-
Robotic Research at MERL
-
Control of Mechanical Systems via Feedback Linearization Based on Black-Box Gaussian Process Models
-
Modelica-Based Modeling and Control of a Delta Robot
-
Towards Human-Level Learning of Complex Physical Puzzles
-
Assembly of Belt Drive Units
-
Examples of Robotic Manipulation
-
Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry
-
Cooperating Modular Goal Selection and Motion Planning for Autonomous Driving
-
Deep Reactive Planning in Dynamic Environments
-
Monte Carlo Probabilistic Inference for Learning Control
-
Experimental Validation of Reachability-based Decision Making for Autonomous Driving
-
-
Downloads