Shader Lamps: Animating Real Objects with Projectors
Traditionally, 3D computer graphics is used to modify 3D models using a program and the result is seen on a computer monitor or screen. The typical surface modifications are applying color, changing texture or modifying material properties (bidirectional reflectance) of the object. This invention describes an idea to change the surface appearance of a real object by projecting images. The images themselves are generated using a 3D graphics program and then fed to the projector.
Background & Objective: Consider a white clay vase sitting on a table. Can we make that vase look like it is made up of gold or marble? Can we change the design pattern on the vase? Can we make the vase to appear as if it is rotating? The perceived color of an object is a function of illumination, surface reflectance and the geometric relationship between the light source(s), surface orientation and viewer location. We can rearrange the factors along the path. For example, a green object illuminated with white light will look the same as white object illuminated with green light. We exploit this idea using a complete 3D geometric representation of the real object and then illuminate it with images with appropriate image intensities.
Technical Discussion: We use a graphics model, which shares the geometry of the physical object but has assigned virtual material properties, for real time rendering. The resultant images are projected on the physical object. Our algorithms solve three fundamental problems involved when a complex object is illuminated by multiple projectors. First, we achieve alignment between the physical object and projected images, semi-automatically using projector calibration techniques. Second, we use 3D graphics hardware to change not just the texture of the object, but also the view dependent appearance. This allows us to make, for example, an object appear shiny with specular highlights. Third, we solve the problem of seamlessly merging images from multiple projectors in the presence of occlusions and self-shadows.
Contact: Joseph Katz
Publications:
Raskar, R.; Low, K-L, "Interacting with Spatially Augmented Reality", ACM International Conference on Virtual Reality, Computer Graphics and Visualization in Africa (AFRIGRAPH), November 2001 (TR2001-051)
Bandyopadhyay, D.; Raskar, R.; Fuchs, H., "Dynamic Shader Lamps: Painting on Movable Objects", IEEE and ACM International Symposium on Augmented Reality, pp. 207-216, October 2001 (IEEE Xplore, TR2001-050)
Raskar, R.; Welch, G.; Low, K-L.; Bandyopadhyay, D., "Shader Lamps: Animating Real Objects with Image Based Illumination", Eurographics Workshop on Rendering, June 2001 (EGWR 2001, TR2001-021)
Technology Area: Graphics
Modification Date: July 7, 2008