Abstract:

We propose a method for visual servocontrol of rigid bodies in the eye-to-hand setting, where the controlled body’s movement is observed by a stationary RGB-D camera. The method is based on establishing and maintaining correspondences between keypoints on the body’s surface across multiple images of the body in various configurations, and leverages recent advances in keypoint tracking and matching methods based on deep learning. The proposed method is pose-based, using the 3D positions of the keypoints extracted by the depth camera to estimate the relative pose of the body with respect to a reference pose. Furthermore, the method identifies the true configuration space of the controlled body by performing principal component analysis on the computed relative poses over a training sequence and decouples the control loop along each identified configuration variable.