64. VIRTUALIZEME: INTERACTIVE MODEL RECONSTRUCTION FROM STEREO VIDEO STREAMS.
Department: Computer Science & Engineering
Research Institute Affiliation: California Institute for Telecommunications and Information Technology (Calit2)
Faculty Advisor(s):
Falko Kuester
Primary Student
Name: Daniel M Knoblauch
Email: dknoblau@ucsd.edu
Phone: 949-355-1018
Grad Year: 2011
Abstract
Tele-Immersion depends on face-to-face, viewpoint corrected, stereoscopic, virtual environments, allowing users to naturally interact with each other and the digital environment surrounding them, via realistic avatars. This project introduces a scalable technique for interactive 3D modeling from stereo video streams. Based on background extraction and disparity map focusing a versatile avatar representation can be reconstructed at interactive rates. Disparity map focusing is achieved by computing the object disparity based on silhouette centroids. By knowing the approximate distance to the camera pair the disparity values exploration can be focused around the object allowing avatar reconstruction with only 16 disparity values. An interpolation between similarity costs in each pixel is introduced to allow sub-pixel disparity values in the resulting disparity map and reconstruction. This extension allows a smoother reconstruction of 3D objects. The objects are reconstructed as point clouds in world coordinates based on a prior camera calibration. This coordinate space supports remote rendering of the avatar, agnostic to the scene and its camera arrangements. On average the whole system runs at about 15 frames per second, with a lower bound for the worst case scene configuration of 7 frames per second. Multi-user reconstruction is possible as long as the entire depth range of the users does not exceed the corresponding available disparity range. By combining several disparity map reconstructions from different viewing angles a realistic, high resolution avatar can be optained that allows the interaction with remote user and the virtual environment.
Related Links:
All Content Copyright 2008 Regents of the University of California. All rights reserved.
Official Web page of the University of California, San Diego.
