Visualization & Layout for Image Libraries
We have developed a prototype system for visualization and layout  with an intuitive browser for retrieval and navigation in large photo libraries. Optimized layouts reflect mutual similarities as displayed on a 2-D screen, hence providing a perceptually intuitive visualization as compared to traditional sequential 1-D content-based image retrieval systems. A framework for user modeling also allows our system to learn and adapt to a user's preferences.
Background & Objective: Traditional image database retrieval systems display query results as a list, sorted by similarity to the query. This presents one major drawback: relevant images can appear at separate places in the ordered list. The purpose of our proposed content-based visualization is augmenting a user's perception so as to visualize a large information space that cannot be easily perceived by traditional sequential array. The retrieved images are displayed not only in ranked order of similarity from the query but also according to their mutual pair-wise similarities, so that similar images are grouped together (see figure).
Technical Discussion: Our system represents an image as a 37-dimensional vector of visual features corresponding to color, texture and structure. An ensemble of images can then be projected to the 2-D screen based on Principle Component Analysis (PCA). PCA is a very fast linear transformation that achieves the maximum distance preservation from the original high dimensional feature space to 2-D space among all the linear transformations. This visual layout is denoted as a "PCA Splat" (see above figure). The layout is further optimized for maximal visibility by adjusting the size and position of each image to minimize overlap while relating other dimensions of information (such as relevance/similarity to the query). The mathematical technique used with "PCA Splats" easily lends itself to modeling a user's style and preference. This is achieved by using a weighted subspace projection model which automatically computes the relevance of the key visual features such as color, texture and structure in a given layout.
Technology Area: Computer Vision
Modification Date: July 7, 2008