Abstract:

Although steady progress has been made on developing vision-based gesture recognition systems, state-of-the-art approaches are still limited to discriminate hand configurations with high amounts of finger occlusions, a common scenario in most fingerspelling alphabets. In this article, we propose a novel method for recognition of isolated fingerspelling gestures based on depth edge features. Our approach is based on a simple and inexpensive modification of the capture setup: a multi-flash camera is used with flashes strategically positioned to cast shadows along depth discontinuities in the scene, allowing efficient and accurate extraction of depth edges. We then use a shift and scale invariant shape descriptor for fingerspelling recognition, demonstrating great improvement over methods that rely on features acquired by traditional edge detection and segmentation algorithms.