Simultaneous Tactile Estimation and Control of Extrinsic Contact

    •  Kim, S., Jha, D.K., Romeres, D., Rodriguez, A., "Simultaneous Tactile Estimation and Control of Extrinsic Contact", IEEE International Conference on Robotics and Automation (ICRA), DOI: 10.1109/​ICRA48891.2023.10161158, May 2023, pp. 12563-12569.
      BibTeX TR2023-025 PDF Video
      • @inproceedings{Kim2023may,
      • author = {Kim, Sangwoon and Jha, Devesh K. and Romeres, Diego and Rodriguez, Alberto},
      • title = {Simultaneous Tactile Estimation and Control of Extrinsic Contact},
      • booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
      • year = 2023,
      • pages = {12563--12569},
      • month = may,
      • publisher = {IEEE},
      • doi = {10.1109/ICRA48891.2023.10161158},
      • url = {}
      • }
  • MERL Contacts:
  • Research Area:



We propose a method that simultaneously estimates and controls extrinsic contact with tactile feedback. The method enables challenging manipulation tasks that require controlling light forces and accurate motions in contact, such as balancing an unknown object on a thin rod standing upright. A factor graph-based framework fuses a sequence of tactile and kinematic measurements to estimate and control the interaction between gripper-object-environment, including the location and wrench at the extrinsic contact between the grasped object and the environment and the grasp wrench transferred from the gripper to the object. The same framework simultaneously plans the gripper motions that make it possible to estimate the state while satisfying regularizing control objectives to prevent slip, such as minimizing the grasp wrench and minimizing frictional force at the extrinsic contact. We show results with sub-millimeter contact localization error and good slip prevention even on slippery environments, for multiple contact formations (point, line, patch contact) and transitions between them. See supplementary video and results at


  • Related News & Events

  • Related Video