Abstract:

Establishing correspondences between keypoints of an object in different poses in multiple images captured by a camera is a key step in estimating its pose for the purpose of manipulation, material handling, and assembly in robotics. Full-range in-plane rotations in the image plane are particularly challenging, because the majority of keypoint detection, description, and matching methods are neither rotationally invariant nor covariant. In addition, the appearance of the object tends to undergo significant changes due to varying illumination and specular effects. We investigate the performance of four keypoint matching and two keypoint tracking methods on a representative dataset, and propose a novel memory-based method to improve matching under large in- plane rotations, where matching is done against a set of reference images whose relative poses are estimated in a self-supervised manner.