Computer Vision for Computer Games
We developed vision-based interfaces for several computer games. These allow the player to move or gesture to affect the game, instead of pressing buttons. The characters in the game may imitate those motions, or respond accordingly. These interfaces allow for engaging, exciting games. This prototype led to a business arrangement with Nintendo where they combined Mitsubishi Electric's Artificial Retina chip with their GameBoy handheld game. 700,000 units have been sold in Japan; they went on sale in North America in June, 1998.
Background & Objective: Vision can be a powerful interface device for computers. There is the potential to sense body position, head orientation, direction of gaze, pointing commands, and gestures. Such unencumbered interaction can make computers easier to use. The application of vision to computer games poses special challenges. The response time must be very fast, while the total hardware cost must be very low. We sought to meet these challenges using special hardware, and fast and simple algorithms.
Technical Discussion: The low-cost, real-time control required for the computer game is very challenging. Fortunately, the computer game itself makes the computer vision task easier. The game restricts the possible visual interpretations. For example, if the game context requires that the player is running in place, the vision system may only need to ascertain how fast the player runs. This is a much easier vision problem to solve than a full 3D reconstruction of a player's unknown motion. Mitsubishi Electric's low-cost Artifical Retina chip is well-suited to this problem, since it is both a detector and a simple image processor. We also used a fast motion analysis technique, developed by colleagues at Mitsubishi Electric, to analyze gestures of the game players. Throw timing and running rate parameters of the game player can be measured from global averages of the optical flow, which the motion analysis method calculates very quickly.
Publications:
| Technical Reports: | |
| Computer vision for computer games | |
Technology Area: Computer Vision
Modification Date: June 13, 2008