Computer Graphics

Jump to Today

Capture.PNG

3D and Computational Photography

Prof. James Davis

Soc Sci 2 Rm 071 – MW 5:20-6:55pm

CMPS260 Computer Graphics is going to focus on 3D and Computational Photography in the Fall 2019 session. 3D cameras are integral to some of the hottest tech trends, for example AR/VR systems and self-driving cars. Your mobile phone takes pictures rivaling an expensive pro camera in large part because of the computation that happens whenever you take a picture.

You will be exposed to 80 research papers in this area, reading 40 and seeing another 40 in class.

You will do a research project and write a paper. In the past, some have published or been MS projects. We will explicitly use class time discussing how to perform and communicate research properly.

First 45 min of each class is – ‘research topic lecture or paper presentation’

Second 45 min of each class is – ‘logistics of class, how to write a paper, and the projects you are doing’

I expect the style of the class to appeal to PhD students and MS students interested in research.

Learning Objectives:

  • Understanding of a sampling of classic 3D acquisition methods .e.g. Time of Flight, Stereo, Structured Light, Photogrammetry / Structure from Motion
  • Understanding of a sampling of classic Computational Photography methods .e.g. Flash no flash, Panoramas, Lightfields, RTI, ML for images
  • A broad but shallow survey of what is hot currently (all papers on these topics at SIGGRAPH in the last two years)
  • How to perform and communicate research - picking topics, how to discuss related work, how to make a good figure, what is most important, how to review a paper, etc

Homework/grading:

  • Read 4 papers a week (for exposure to research ideas) (20%)
    • Write short responses to 3 papers a week
    • Write a “review” to 1 paper a week
  • Prepare presentations of research papers, about 10-15 minutes long (~2 times) (20%)
  • Full quarter team research project – background research, code, experiment, write, (50%)
  • Participate (come to class) (10%)

Topics:

  • Basics: Camera: senors, bayer mosaic, assorted pixels, how cameras work, flash no-flash
  • Basics: Image: RSME vs perceptual error metrics; color spaces YUV vs RGB
  • Basics: Light ray: Lightfields, Coded aperture, Ptyography
  • Project Background:Stereo3D: Stereo, Structured Light, ToF
  • Project Background:ML: PCA, dim reduction, tex. synth, vector quant, CGAN, Image Analogies
  • Project Background:RTI: PTM, Photometric Stereo
  • Classic Application: Panoramas (10 years ago)
  • Classic Application: Deblurring/denoising: Photo stacking vs Deconvolution (now)
  • Classic Application: Background blur, refocusing (still works poorly)
  • Paper writing: Mistakes not to make, sample review forms, what makes a strong paper
  • The above are classic topics, we will also have lots of recent papers as short presentations

Homework details:

  • Read 4 papers a week (for exposure to research ideas) (20%)
    • One picked by me – on the weekly topic area
      • Write a “review” following a standard conference review method
    • One more picked by me
    • Two picked by you – related work to the papers you are writing
      • Write a short ½ page journal entry including summary, what is the contribution of this paper, how did they prove it, what are your ideas for new papers
    • Prepare presentations of research papers, about 10-15 minutes long (1-2 times during the qtr) (20%)
      • Most classes will have 2 papers presented by students
      • When its your turn, you’ll have a presentation to prep in addition to your journal
      • Your goal is to communicate the goal, the contribution, the results, and a high level of key methods in your 15 minute slot
      • Instructor rating of your presentation 10%
      • Your peers will be asked to rate your presentation 10%
    • Full quarter team research project – background research, code, experiment, write, … (50%)
      • Background research – Search for existing papers to answer: Has someone done this exactly before? Has someone done something close? What are the categories of related work that already exist?
        • Turned in assignments, individual (10%)
      • Experiment/code – First make an explicit list of the results that you hope to obtain (before coding). Second, do what is needed to get these results.
        • Team portion of grade based on demos and results (10%)
        • Individual portion grade based on peer review from your team-mates (10%)
      • Write – We will talk about sections in a paper one at a time, using the class projects as examples. Expect homework of the form “produce a ‘Results’ section this week with fake draft figures showing what you hope to eventually get working for real”. Then we talk about those in class.
        • Individual writing 10%
        • Peer review of other peoples sections 5%.
      • Figures/captions – Figures, tables, plots, images all take much longer than words to get right. Expect multiple revisions as we talk about how to make these convincingly. They typically are the main “proof” that you have of your method working. 5%
    • Participation (10%)
      • Come to class. Its expected, but missing when sick or you have a conference or interview is ok (you only lose 0.5%/day).

Class discussion and paper writing timeline:

Most classes first 1/2: Prof presents slides or leads discussion on 1 assigned paper, Students present 2 random modern papers
Most classes second 1/2: Prof presents or leads discussion on how to write this week’s paper section.
Every class: Homework 1 assigned paper and 1 you pick (related to your project or presentation)
Homework related to paper writing to follow roughly the schedule below

Sep 30 –                                            

  • Intro research topics
  • Intro syllabus, HW, paper writing and HW
  • HW (2 day):: Find related work that might force a change to a project

Oct 2                                                           

  • (research topics, exact order determined later according to project choices, repeats first ½ every class)
  • Pick projects and teams
  • HW (3 day): Find more related work, focus on the projects chosen

Oct 7     

  • Outline of all needed sections, tools, data, plots, to get a complete paper ready
  • (change teams/projects if needed)
  • HW (2 day): Make an outline plan for your paper, make an outline plan for internal deadlines you want to hit

Oct 9     

  • What makes a good figure, types of figures
  • HW (3 day): Plan for results figures (completely fake, hand drawn, no real results)

Oct 14                                                

  • What goes in the related work, how to write this section defensively, bibliography
  • HW (2 day):: Draft related work section (with whatever you know now)

Oct 16

  • What goes in an introduction section, framing your contribution
  • HW (3 day): Draft introduction and contribution statement (compatible with hypothesized related work and results)

Oct 21

  • What goes in the method section, which things to describe and which to leave out
  • HW (2 day):. Draft method section

Oct 23

  • What makes a good figure part 2
  • HW (3 day): At least one figure with real data (although maybe not the best result yet)

Oct 28

  • What goes in the conclusion
  • HW (2 day):: Conclusion section

Oct 30

  • Putting it all together: Contribution -> Related work shows novel -> Results shows it works
  • HW (3 day): Complete paper

Nov 4

  • (complete paper is due)
  • HW (2 day):: Review classmates papers

Nov 6                                                 

  • (reviews of classmates due)
  • HW (5 day): Fix comments from classmates
  • HW: If submitting to CVPR, merge as needed

Nov 11 – Holiday                                                                                                                   

Nov 13                                               

  • (final paper is due)
  • What goes in a video, common problems in videos, other supplemental material
  • HW (5 day): Prepare a video, other supplemental

----Cvpr deadline Fri Nov 15 ----

Nov 18 

Nov 20

  • (video and supplemental due) 
  • In class: Comment on supplemental materials

-  CVPR supplemental material deadline Nov 22

Nov 25 – Thanksgiving week (possible class canceled)

Nov 27 – Thanksgiving week (possible class canceled)

Dec 2

  • What goes in a conference presentation
  • HW (2 day): Prep your presentation

Dec 4    

  • Present your papers to the class

Possible Class Projects:  (longer PDF doc with details)

F19_CSE260_Syllabus.pdf

Recent papers that someone in the class will present (as many as we can fit in the class)

F19_CSE260_Syllabus.pdf

Presentation Signup:

https://docs.google.com/spreadsheets/d/1Hjg7r1_-I7KaaIvgueJS0j5kuradMTgZaS-Z7mYLgps/edit#gid=0

Readings:

Course Summary:

Date Details Due