Tactile Pose Feedback for Closed-loop Manipulation Tasks

    •  Ota, K., Jain, S., Zhang, M., Jha, D.K., "Tactile Pose Feedback for Closed-loop Manipulation Tasks", RSS 23 Workshop on Learning Dexterous Manipulation, July 2023.
      BibTeX TR2023-082 PDF
      • @inproceedings{Ota2023jul,
      • author = {Ota, Kei and Jain, Siddarth and Zhang, Mengchao and Jha, Devesh K.},
      • title = {Tactile Pose Feedback for Closed-loop Manipulation Tasks},
      • booktitle = {RSS 23 Workshop on Learning Dexterous Manipulation},
      • year = 2023,
      • month = jul,
      • url = {}
      • }
  • MERL Contacts:
  • Research Area:



Current generation manipulation systems operate in an open-loop fashion resulting in poor performance in the presence of disturbances. Robust manipulation requires a robot to compensate for uncertainties and errors arising due to contact in- teraction during manipulation. Consequently, it is essential that a robot can estimate an object’s state and the relevant contact states so that manipulation can be controlled precisely. However, precise object state estimation is difficult due to occlusions and complex contact interactions during manipulation. This paper presents several different manipulation tasks where a robot may have to perform complex manipulation, which can introduce uncertainty leading to failure. To deal with this problem, we use in-hand pose estimation using vision-based tactile sensors to adjust our plan during manipulation. We present several different analyses for pose estimation using vision as well as tactile sensors to evaluate the importance of different modalities for these precision tasks. We demonstrate that using the proposed approach, we can perform the desired task successfully by incorporating feedback from the tactile pose estimation framework. See supplementary video at


  • Related News & Events

    •  NEWS    MERL researchers present 3 papers on Dexterous Manipulation at RSS 23.
      Date: July 11, 2023
      Where: Daegu, Korea
      MERL Contacts: Siddarth Jain; Devesh K. Jha; Arvind Raghunathan
      Research Areas: Artificial Intelligence, Machine Learning, Robotics
      • MERL researchers presented 3 papers at the 19th edition of Robotics:Science and Systems Conference in Daegu, Korea. RSS is the flagship conference of the RSS foundation and is run as a single track conference presenting a limited number of high-quality papers. This year the main conference had a total of 112 papers presented. MERL researchers presented 2 papers in the main conference on planning and perception for dexterous manipulation. Another paper was presented in a workshop of learning for dexterous manipulation. More details can be found here