TR2005-130

Enabling Interaction with Single User Applications through Speech and Gestures on a Multi-User Tabletop


    •  Tse, E., Shen, C., Greenberg, S., Forlines, C., "Enabling Interaction with Single User Applications through Speech and Gestures on a Multi-User Tabletop", International Working Conference on Advanced Visual Interfaces (AVI), May 2006.
      BibTeX TR2005-130 PDF
      • @inproceedings{Tse2006may1,
      • author = {Tse, E. and Shen, C. and Greenberg, S. and Forlines, C.},
      • title = {Enabling Interaction with Single User Applications through Speech and Gestures on a Multi-User Tabletop},
      • booktitle = {International Working Conference on Advanced Visual Interfaces (AVI)},
      • year = 2006,
      • month = may,
      • url = {https://www.merl.com/publications/TR2005-130}
      • }
Abstract:

Co-located collaborators often work over physical tabletops with rich geospatial information. Previous research shows that people use gestures and speech as they interact with artefacts on the table and communicate with one another. With the advent of large multi-touch surfaces, developers are now applying this knowledge to create appropriate technical innovations in digital table design. Yet they are limited by the difficulty of building a truly useful collaborative application from the ground up. In this paper, we circumvent this difficulty by: (a) building a multimodal speech and gesture engine around the Diamond Touch multi-user surface, and (b) wrapping existing, widely-used off-the-shelf single-user interactive spatial applications with a multimodal interface created from this engine. Through case studies of two quite different geospatial systems -- Google Earth and Warcraft III -- we show the new functionalities, feasibility and limitations of leveraging such single-user applications within a multi user, multimodal tabletop. This research informs the design of future multimodal tabletop applications that can exploit single-user software conveniently available in the market. We also contribute (1) a set of technical and behavioural affordances of multimodal interaction on a tabletop, and (2) lessons learnt from the limitations of single user applications.

 

  • Related News & Events

    •  NEWS    AVI 2006: 4 publications by Derek Schwenke, Kent Wittenburg, Anthony Vetro, Kathy Ryall and Alan Esenther
      Date: May 23, 2006
      Where: International Working Conference on Advanced Visual Interfaces (AVI)
      MERL Contact: Anthony Vetro
      Brief
      • The papers "Enabling Interaction with Single User Applications through Speech and Gestures on a Multi-User Tabletop" by Tse, E., Shen, C., Greenberg, S. and Forlines, C., "Table-Centric Interactive Spaces for Real-Time Collaboration" by Wigdor, D., Shen, C., Forlines, C. and Balakrishnan, R., "The Prospects for Unrestricted Speech Input for TV Content Search" by Wittenburg, K., Lanning, T., Schwenke, D., Shubin, H. and Vetro, A. and "Fluid DTMouse: Better Mouse Support for Touch-Based Interactions" by Esenther, A. and Ryall, K. were presented at the International Working Conference on Advanced Visual Interfaces (AVI).
    •