Mouse Emulation and Legacy Support for Multi-User/Multi-Touch Surfaces
The DiamondTouch Mouse mouse emulation utility, one outcome of the Mouse Emulation and Legacy Application Support on Multi-User/Multi-Touch Surfaces effort (DTMouse), is provided with the DiamondTouch SDK to convert DiamondTouch table touch input to Windows system mouse input. DTMouse is a project to investigate issues related to providing backwards compatibility to (legacy) mouse-based software for users of a multi-user multi-touch DiamondTouch (DT) table. It is part of the broader DiamondTouch Applications project.
Background & Objective: Contemporary operating systems fundamentally only allow the use of one mouse at a time. Only one window has mouse focus, only one text area can be selected for copying, etc. By providing support for legacy software, DTMouse enables arbitrary applications to run on DiamondTouch. It was originally developed as part of a remote collaboration system. The first standalone version of DTMouse offered arbitrary mapping of touch events to mouse events, and included a novel "2-Finger Resizing" capability for resizing arbitrary windows. Subsequent versions included support for laptop fingerpad emulation, multiple displays, zoomed views, and launching the onscreen keyboard. The current version supports fluidly conducting precision interactions with arbitrary applications, as well as a multi-user annotation tool for simultaneously scribbling on screenshots and then saving the annotations as image files.
Technical Discussion: By default, the DiamondTouch Mouse utility lets any toucher act as the mouse. In the case of conflicts, the first to touch wins - it will ignore other touchers while the first toucher is interacting. Some of the challenges for realizing a mouse emulation system on a finger touch surface include: resolution, how to specify a particular pixel with a "fat" fingertip; obscured content, how to tell what is being touched if a finger is covering the content; movement vs. engagement, how to specify when the left mouse button is to be held down (just moving the mouse versus dragging with the mouse); timing and spacing, how two taps with a finger are not as temporally or spatially as close to each other as two clicks with a mouse; full mouse functionality, how to right-click, middle-click, and use the mouse wheel; integration, how to seamlessly incorporate this into existing operating systems user environments. This project also includes best techniques for automating existing software to support multi-touch (one user touching more than one place at a time), and multi-user (more than one user touching the surface at the same time) capabilities. These issues were addressed in the current software iteration through several mechanisms including support for touch modes, configuration settings, touch-friendly utilities, and O.S. integration. See the documentation for more details.
Technology Area: Off the Desktop Interaction and Display
Modification Date: December 10, 2007