Abstract:

Existing industrial systems often rely on specialized end effectors that grasp objects in pre-defined poses. Designing systems that can solve high-precision tasks with simple grippers is an important goal, which high-accuracy in-hand pose estimation can ease. Image-based tactile sensors hold promise for this task, but high-accuracy tactile pose estimation from arbitrary grasps remains challenging for several reasons. First, many grasps are inherently ambiguous [1] without additional information from vision [2 ], extrinsic contacts [ 3 ], or multiple grasps [ 4]. Second, training pose estimation models from real data is expensive [ 5], whereas sim2real with RGB tactile images is difficult [6]. Motivated by these challenges, we present a solution for high-accuracy tactile pose estimation with the following contributions:

1) We use tactile depth images as an intermediate repre- sentation between binary masks [1 ] and RGB to regress discrete pose distributions.
2) We introduce a refinement network to improve the accuracy beyond the discrete pose resolution.
3) We introduce a suite of data augmentations that allow
Depth2Pose to sim2real with high fidelity.
4) We introduce a simple ambiguity detection method to identify grasps that can be localized accurately.
5) We demonstrate Depth2Pose on a connector assembly task, and show that for some connectors, we achieve high success rates with a simple force controller.