Freehand Touch Gestures
Hand gestures can be 'natural' expressions in mapping input hand action to rendered effect. We can leverage many degrees of freedom movements to allow for significantly richer, potentially more efficient and effective interaction. Ultimately, we hope to enable users to more quickly learn input techniques and to more easily express their intentions.
Technical Discussion: Freehand gestural interaction on touch display surfaces has been the focus of significant research activity recently. While many interesting gestures have been proposed, their design has been mostly ad-hoc. The scope of our gesture interaction research includes:
*An extensible hand-posture engine: Offering a set of generic hand postures for application designers to work with, enabling the construction of expressive gestures.
*Speech and Bimanual Gesture input examples for Google Earth and games (PDF): Combining gesture input and speech recognition to even more closely mimic 'natural' behavior.
*ExpressiveTouch (PDF): A constructive design framework, embodying a set of principles for the systematic construction of multi-hand gestures on touch surfaces that can sense multiple points and areas, and can also accommodate conventional point-based input. As part of this framework, we developed the concepts of gesture "registration", gesture "relaxation" and gesture "reuse", allowing an initial touch action (e.g., a hand posture, a gesture, a dwell, a pen stroke or finger tabs) to initialise a continuous operation that is then carried out without the need to maintain the indicative and limited posture. These concepts are useful in designing continuous and compound multi-gesture phrasing.
*DTLens (movie) (PDF): Turning an interactive tabletop into a bi-focal display, enabling two-handed multi-lens multi-user geospatial visual exploration.
*Modal spaces (movie) (PDF): Designed as a workbench, a large touch surface is spatially multiplexed.
Contact: Clifton Forlines
Publications:
Tse, E.; Shen, C.; Greenberg, S.; Forlines, C., "Enabling Interaction with Single User Applications through Speech and Gestures on a Multi-User Tabletop", Advanced Visual Interfaces (AVI), May 2006 (AVI 2006, TR2005-130)
Tse, E.; Greenberg, S.; Shen, C.; Forlines, C., "Multimodal Multiplayer Tabletop Gaming", International Workshop on Pervasive Gaming Applications (PerGames), May 2006 (PerGames 2006, TR2006-009)
Wu, M.; Shen, C.; Ryall, K.; Forlines, C.; Balakrishnan, R., "Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces", IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TableTop), pp. 185-192, January 2006 (IEEE Xplore, TR2005-109)
Forlines, C.; Shen, C., "DTLens: Multi-user Tabletop Spatial Data Exploration", ACM Symposium on User Interface Software and Technology (UIST), ISBN: 1-59593-271-2, pp. 119-122, October 2005 (ACM Press, TR2005-096)
Everitt, K.; Shen, C.; Ryall, K.; Forlines, C., "Modal Spaces: Spatial Multiplexing to Mediate Direct-Touch Input on Large Displays", ACM Conference on Human Factors in Computing Systems (CHI) Short Paper, April 2005 (CHI 2005, TR2005-002)
Technology Area: Off the Desktop Interaction and Display
Modification Date: July 7, 2008