Video Summarization for PVRs
Personal Video Recorders have increasingly large storage capacity extending beyond 100 hours of content. Video Summarization is therefore essential to enable the consumer to skim through the content and view the content in differing detail depending on preference. We have developed a suite of summarization algorithms that are based on rapid audio-visual analysis in the compressed domain, and work well across diverse content genres. We are investigating the usability issues in a practical PVR application as well. We define tasks such as browsing a collection of programs, skimming through a program, watching the highlights, etc., and how summarization and video segmentation technologies apply.
Background & Objective: In this project we emphasize the Personal Video Recorder application, which provides the user with the content he wants when he wants it by storing a large volume of content recorded from broadcast and then providing effective navigation of the stored content using summarization and indexing. Our summarization algorithms are based on compressed domain analysis of both the audio and the video. Since such analysis is fast, our algorithms have been easy to realize on our target platforms. Our target products include personal video recorders such as DVD recorders. Our sports highlights detection was featured in Mitsubishi Electric's DVR-HE50W DVD Recorder, the world's first DVD recorder with sports highlights playback. We also study the usability issues and integration of summarization and browsing with PVR user interfaces.
Technical Discussion: The technical challenge lies on two broad fronts. The first is audio-visual content analysis techniques that enable accurate content summarization over a broad range of content genres. An important constraint is feasibility on our target platforms. Our algorithms therefore have to be computationally simple and robust to the high variation in broadcast video. ;Recent progress on this front includes the development of a genre-independent scene-change detector that uses a support vector machine classifier trained on hand-labeled examples of scene changes from a wide variety of broadcast genres. The second front is the usability of video summarization and browsing in PVR applications. The technology should be seamlessly integrated with the typical tasks that a PVR user have, such as browsing through a large number of programs, deciding what to watch, locating desired part of a program, or watching a summary of a program, etc. We plan to meet the flexibility requirement by developing scalable summarization algorithms that generate summaries of varying lengths. We will collaborate with our MERL colleagues in developing convenient user interfaces for the PVR application and running user studies to test the effectiveness of the techniques.
Future Direction: We will refine our genre-independent framework and work to incorporate new types of audio and video features into our scene change classifier. We will develop a holistic approach to application of video analysis, summarization and browsing technologies to PVRs that centers around user tasks.
Contacts:
Ajay Divakaran
Samuel Shipman
Kevin W. Wilson
Publications:
Divakaran, A.; Peker, K.A.; Radharkishnan, R.; Xiong, Z.; Cabasson, R., "Video Summarization Using MPEG-7 Motion Activity and Audio Descriptors", Video Mining, Rosenfeld, A.; Doermann, D.; DeMenthon, D., October 2003 (Kluwer Academic Publishers, TR2003-034)
| Technical Reports: | |
| An Enhanced Video Summarization System Using Audio Features for a Personal Video Recorder | |
| A Content-Adaptive Analysis and Representation Framework for Audio Event Discovery from | |
| A Highlight Scene Detection and Video Summarization System Using Audio Feature for a Personal Video Recorder | |
| Video Mining: Pattern Discovery Versus Pattern Recognition | |
| A Unified Framework for Video Summarization, Browsing and Retrieval | |
Technology Area: Sensor and Data Systems
Modification Date: October 2, 2007
