Szeliski Being There

Information about Szeliski Being There

Published on February 28, 2008

Author: Toni

Source: authorstream.com

Content

Being There: Capturing and Experiencing a Sense of Place:  Being There: Capturing and Experiencing a Sense of Place Richard Szeliski Microsoft Research Symposium on Computational Photography and Video Early art: events:  Early art: events Lascaux Early art: events:  Early art: events Early art: events:  Early art: events Bayeux Tapestries Renaissance art: events:  Renaissance art: events Da Vinci Baroque art: portraits:  Baroque art: portraits Rembrandt Baroque art: landscapes:  Baroque art: landscapes Rembrandt Classical art: landscapes:  Classical art: landscapes Corot Impressionism: landscapes:  Impressionism: landscapes Monet Modern art: multi-perspective:  Modern art: multi-perspective Picasso Impressionism: landscapes:  Impressionism: landscapes Monet Photography: events:  Photography: events Henri Cartier-Bresson Photography: landscapes:  Photography: landscapes Ansel Adams Photography: landscapes:  Photography: landscapes Galen Rowell Photography: landscapes:  Photography: landscapes Art Wolfe Interactive Visual Media:  Interactive Visual Media Branching storytelling Games: problem solving, skill acquisition sense of place, navigation Real-world “immersive” media:  Real-world “immersive” media Recapturing the sense of place QTVR: the first real breakthrough in modeling from reality? Fairly universal for high-end home and car sales, virtual tourism Limitations: mostly static, discrete jumps Increasing realism:  Increasing realism Better: field of view and resolution Demo: Space Needle dynamic range Additional cues/modalities: motion / movement Demo: Deception Falls sound VR Tools:  VR Tools Fully automated stitch discovery:  Fully automated stitch discovery Demo Image stitching — open issues:  Image stitching — open issues Fully automated assembly (no ordering) Full 2D stitching (multiple rows) Double image fix-up (de-ghosting) Merging different exposures Automated grouping/clustering (detection) HDR: merging exposures:  HDR: merging exposures Inputs Tonemapped output: no motion compensation or consistency check HDR: merging exposures:  Inputs Tonemapped output: global+local motion compensation HDR: merging exposures Video Textures:  Video Textures How can we turn a short video clip into continuous video? Enhance sense of liveness Use in games and presentations Video Textures:  Video Textures Find cyclic structure in the video Play frames with random shuffle Optional region-based analysis Smooth over discontinuities (morph) Animating Stills:  Animating Stills What if we only have a single beautiful photo (or painting)? Can we add so liveness to a photo/still? Animating Stills:  Animating Stills Increasing “immersion”:  Increasing “immersion” Add continuous movement Demo Bellevue Botanical Garden:  Demo Bellevue Botanical Garden Demo HDR Home Walkthrough:  Demo HDR Home Walkthrough Video-based walkthroughs:  Video-based walkthroughs Add continuous movement How did we do this? Is this 3D? 2D? Graphics? Video? Dimensionality and storage? Limits of Video-Based Walkthroughs:  Limits of Video-Based Walkthroughs Video-based walkthroughs are “rail experiences” of a space-time slice How do we break the bounds? Can’t move from side to side If there is motion in the scene, don’t see it Capturing all points of view everywhere in space at all times is impractical A Practical 3D Video Camera:  A Practical 3D Video Camera Working volume? Walls of a room: Virtualized Reality 2D “window”: LightField Array 1D “rail”: Virtual Viewpoint Video “3D” video:  “3D” video Virtual Viewpoint Video:  Virtual Viewpoint Video Capture multiple synchronized video streams Key to view interpolation: Geometry:  Key to view interpolation: Geometry Stereo Camera 1 Camera 2 Image 1 Image 2 Virtual Camera Don’t match pixels – match segments:  Don’t match pixels – match segments Segments contain more information, so they’re easier to match. Iteratively update depths:  Iteratively update depths Depth through time::  Depth through time: Matting:  Matting Some pixels get influence for multiple surfaces. Background Surface Foreground Surface Image Camera Close up of real image: Multiple colors and depths at boundary pixels… Find matting information::  Find matting information: Find boundary strips using depth. 2. Within boundary strips compute the colors and depths of the foreground and background object. Background Foreground Strip Width Why matting is important:  Why matting is important Matting No Matting Representation:  Representation Color Depth MAIN LAYER Color Depth Matting information BOUNDARY LAYER Rendering pipeline (GPU):  Rendering pipeline (GPU) Render Main Layer Render Boundary Layer Composite Render Main Layer Render Boundary Layer Real-time viewpoint control:  Real-time viewpoint control “Matrix”-like effects – on demand! (With much, much less hardware.) Wrapping up…:  Wrapping up… Light Field Rendering:  Light Field Rendering Sample and synthesize (capture and render) from a 4D ray space [Lightfield, Levoy & Hanrahan; Lumigraph, Gortler et al., SIGGRAPH’96] Slow Glass:  Slow Glass What if we could capture all the photons and play them all back at a later time? Light of Other Days Bob Shaw, © 1969 http://www.scifi.com/scifiction/classics/classics_archive/shaw/shaw1.htm Slow Glass:  Slow Glass One could stand the glass beside, say, a woodland lake until the scene emerged, perhaps a year later. If the glass was then removed and installed in a dismal city flat, the flat would—for that year—appear to overlook the woodland lake. During the year it wouldn't be merely a very realistic but still picture—the water would ripple in sunlight, silent animals would come to drink, birds would cross the sky, night would follow day, season would follow season. Apart from its stupendous novelty value, the commercial success of slow glass was founded on the fact that having a scenedow was the exact emotional equivalent of owning land. The meanest cave dweller could look out on misty parks—and who was to say they weren't his? A man who really owns tailored gardens and estates doesn't spend his time proving his ownership by crawling on his ground, feeling, smelling, tasting it. All he receives from the land are light patterns… Being There:  Being There Artists (and all of us) have always wanted to capture a sense of being there. Computational photography and video bring us a lot closer: realism (field of view, resolution, contrast) movement and sound immersion and exploration What does the future hold?

Related presentations


Other presentations created by Toni

The 5 Regions of Georgia
16. 01. 2008
0 views

The 5 Regions of Georgia

Multithreading
19. 03. 2008
0 views

Multithreading

MieleGlassWasherBene fits
08. 01. 2008
0 views

MieleGlassWasherBene fits

Homemade Bread
10. 01. 2008
0 views

Homemade Bread

BEARAWARE
14. 01. 2008
0 views

BEARAWARE

Animal World9 ppt1
15. 01. 2008
0 views

Animal World9 ppt1

SADP Hunan
19. 01. 2008
0 views

SADP Hunan

11 Andrew Jones
22. 01. 2008
0 views

11 Andrew Jones

ScottHorowitz112806
21. 01. 2008
0 views

ScottHorowitz112806

L8 Rebirth Without A Soul
05. 02. 2008
0 views

L8 Rebirth Without A Soul

kaplan
05. 02. 2008
0 views

kaplan

2 1 07 11 00am Ashley
10. 01. 2008
0 views

2 1 07 11 00am Ashley

ADHDintheClassroom1
29. 01. 2008
0 views

ADHDintheClassroom1

6 FredStarr
06. 02. 2008
0 views

6 FredStarr

Nickel BMO
14. 02. 2008
0 views

Nickel BMO

ERGONMCS
05. 03. 2008
0 views

ERGONMCS

eggers
11. 03. 2008
0 views

eggers

leading diversity
15. 03. 2008
0 views

leading diversity

06 ORBs
21. 03. 2008
0 views

06 ORBs

science olympics
07. 02. 2008
0 views

science olympics

webcast 2006 07 21
08. 04. 2008
0 views

webcast 2006 07 21

LGBT Health Curriculum Final
04. 02. 2008
0 views

LGBT Health Curriculum Final

3977
16. 04. 2008
0 views

3977

A New Coursefor KS3 French
24. 04. 2008
0 views

A New Coursefor KS3 French

Non country DPMM 11 05
22. 01. 2008
0 views

Non country DPMM 11 05

27 Help Ive Got ELLs 218832 7
10. 01. 2008
0 views

27 Help Ive Got ELLs 218832 7

1751
08. 05. 2008
0 views

1751

200610gloream paris mircea
20. 02. 2008
0 views

200610gloream paris mircea

BOB PURGASON PRESENTATION
24. 01. 2008
0 views

BOB PURGASON PRESENTATION

Schoenborn 2BK 3AK 3BK 5BK
18. 02. 2008
0 views

Schoenborn 2BK 3AK 3BK 5BK

Group C Tara Toellner
11. 02. 2008
0 views

Group C Tara Toellner

moorea
04. 02. 2008
0 views

moorea

1 abuse HK 2007
17. 01. 2008
0 views

1 abuse HK 2007