Integrated Event Recognition
We are developing an approach to identification of human behaviors in a large spaces that are instrumented with video cameras and a variety of sensors. The approach is expected to make in impact in surveillance and monitoring applications.
Background & Objective: Typically in monitoring spatially extended and geometrically complex areas problems of coverage arise. In order to provide a video coverage of the space many video cameras need to be deployed. In such situations recognition of human activities inevitably begins with attempths to solve smaller irrelevant problems, such as multi-camera tracking, camera handover, object correspondence, multi-camera geometry calibration, etc. This requires massive computational resources and does not address directly the main problem of what is going on in the scene. We solve this problem by using a wide area sensor network to provide the sensor context for the sparse array of video cameras and improve search speeds by several orders of magnitude.
Technical Discussion: We developed algorithms that have the potential to sidestep the video coverage issues by providing a global context of the event. We use the wide area motion sensor network jointly with a sparse array of video cameras to specify and search for events in the entire area without the regard to its observability by a particular camera. We describe these events int terms of space and time, rather than a dirrect appearance of it in a camera view. This approach allows us to solve many problems of large area tracking and ebent detection very efficiently. We use our network of 250+ sensors and 6 PTZ cameras to monitor the entire MERL office with its ~80 occupants for a period of over a year and search it in a matter of seconds. It allows us to judge global office-wide events and activity patterns as well as find and track individuals during the day.
In addition to the effecient search algorithms we have developed large scale visualization techniques and a user interface that allow us to specify events of interest "on-the-fly" and translate simple intuitive gestures into search constraints and query statements. Visualization solutions allow us to display the search results such that complex spatio-temporal relationships are understood pre-attentively.
Future Direction: We plan to integrate the search and forensic tracking system with live event detection module that allows fast autonomous control of cameras, parallel information gathering and best shot detection for recording applications.
Contacts:
Yuri Ivanov
Christopher R. Wren
| Technical Reports: | |
| Tracking People in Mixed Modality Systems | |
| Weighted Ensemble Boosting for Robust Activity Recognition in Video | |
| Toward Spatial Queries for Spatial Surveillance Tasks | |
Technology Areas:
Imaging
Computer Vision
Modification Date: August 1, 2007
