Learning Normal Activity and Detecting Anomalies
At present, you attend to the computer. In the future, the computer should attend to you. This computer vision system watches you and learns your normal work habits and basic activities. It can summarize your day, adjust your environment for different of activity, or signal an alarm when something unusual or disruptive happens.
Background & Objective: We are interested in the use of computer vision to "keep an eye on things" while you're busy or away. This application, a personal security device, might be used to keep an eye on a child at play, a hospital patient, or an elderly parent.
Technical Discussion: The basic approach is to learn a probabilistic model of the scene dynamics. The demonstration system extracts a motion-based feature vector from each video frame, then learns a summary of signal dynamics in the form of a hidden Markov model. An entropy-minimization algorithm estimates both model structure and parameters, yielding a small, sparse, accurate model that generalizes very well to new video. Due to its sparsity, the states in the model are highly correlated with meaningful partitions of the video, e.g., activities. The resulting model reads like a flowchart of a normal day's work activities. We can use the model to detect when you are in particular states, e.g., working at the whiteboard, and have the computer adjust your environment appropriately, e.g., brightening the lights. The system is also very good at detecting anomalous activity-the laboratory prototype proved quite adept at detecting coffee buzz an hour after several espressos.
Publications:
| Technical Reports: | |
| Learning concise models of human activity from ambient video via a structure-inducing M-step estimator | |
Technology Areas:
Computer Vision
Artificial Intelligence
Modification Date: July 9, 2001