Abstract:

We propose a method for estimating the joint configuration of articulated mechanisms without joint encoders and with unknown forward kinematics, based solely on RGB-D images of the mechanism captured by a stationary camera. The method collects a sequence of such images under a suitable excitation control policy, extracts the 3D locations of keypoints in these images, and determines which of these points must belong to the same link of the mechanism by means of testing their pairwise distances and clustering them using agglomerative clustering. By computing the rigid-body transforms of all bodies with respect to the keypoints’ positions in a reference image and analyzing each body’s transform expressed relative to all other bodies’ coordinate reference frames, the algorithm discovers which pairs of bodies must be connected by a single- degree-of-freedom joint and based on this, discovers the ordering of the bodies in the kinematic chain of the mechanism. The method can be used for pose-based visual servocontrol and other robotics tasks where inverse kinematics is needed, without providing forward kinematics or measurements of the end tool of the robot.