Computer Graphics

Capture.PNG

3D and Computational Photography

Prof. James Davis

Tue/Thur 11:40am-1:15pm

CMPS260 Computer Graphics is going to focus on 3D and Computational Photography in the Winter 2021 session. 3D cameras are integral to some of the hottest tech trends, for example AR/VR systems and self-driving cars. Your mobile phone takes pictures rivaling an expensive pro camera in large part because of the computation that happens whenever you take a picture.

In addition to lectures you will read research papers in this area, some classic, some recent.

Piazza Link: https://piazza.com/class/kjc5ekwna0w3z6

TA: Fahim Hasan Khan (fkhan4@ucsc.edu)

TA Office hour: Mon 3:30 - 4:30 pm (Zoom link)

Learning Objectives:

  • Understanding of a sampling of classic 3D acquisition methods .e.g. Time of Flight, Stereo, Structured Light, Photogrammetry / Structure from Motion
  • Understanding of a sampling of classic Computational Photography methods .e.g. Flash no flash, Panoramas, Lightfields, RTI
  • A broad but shallow survey of what is hot currently (papers on these topics at SIGGRAPH in the last few years)

Homework/grading:

  • Read (up to) 4 papers a week (for exposure to research ideas) (20%) (about 2-5hrs/week)
    • Write short responses to each paper 
  • Participation (20%) (about 3hrs/week)
    • You have to come to class
    • You have to participate in paper discussions 
  • Assignments (60%) (about 0-10hrs/week, highly variable depending on assignments)
    • Some assignments build on others, some are completely different
    • Some assignments require coding, some do not

Topics: (not complete and in sorted order, just to provide examples)

  • Camera: sensors, bayer mosaic, assorted pixels, how cameras work, flash no-flash
  • Image: RSME vs perceptual error metrics; color spaces YUV vs RGB, JPG vs RAW
  • Light ray: Lightfields, Coded aperture, Ptyography
  • Stereo3D: Stereo, Structured Light, ToF
  • ML: PCA, dim reduction, tex. synth, vector quant, CGAN, Image Analogies
  • RTI: PTM, Photometric Stereo
  • Classic Application: Panoramas (10 years ago)
  • Classic Application: Deblurring/denoising: Photo stacking vs Deconvolution (now)
  • Classic Application: Background blur, refocusing (still works poorly)
  • The above are classic topics, we will also have lots of recent papers

 

Class time schedule 

  • Jan 5 - Intro lecture and course description
  • Jan 7 - How a camera works high level
  • Jan 12 - Look at results from A1. Introduce A2.
  • Jan 14 - Camera sensors
  • Jan 19 - Sensor Demosaicing & Noise.
  • Jan 21 -  Range Scanning Pipeline
  • Jan 26 - Panoramic Mosaics, (supplement: how to ICP)
  • Jan 28 - Meshes, About Assignment 2
  • Feb 2 - Light Fields
  • Feb 4 - Time of Flight
  • Feb 9 - Results of A2, Introduce A3
  • Feb 11 - Passive stereo, triangulation
  • Feb 16 - Triangulation with active lighting
  • Feb 18 - Photometric stereo
  • Feb 23 - Relighting
  • Feb 25 - Discuss progress on A3

Directions for reading responses and how class discussion will work:

Readings:
(Note response due the day before discussion)

Course Summary:

Date Details Due