Computer Graphics
3D and Computational Photography
Prof. James Davis
Soc Sci 2 Rm 071 – MW 5:20-6:55pm
CMPS260 Computer Graphics is going to focus on 3D and Computational Photography in the Fall 2019 session. 3D cameras are integral to some of the hottest tech trends, for example AR/VR systems and self-driving cars. Your mobile phone takes pictures rivaling an expensive pro camera in large part because of the computation that happens whenever you take a picture.
You will be exposed to 80 research papers in this area, reading 40 and seeing another 40 in class.
You will do a research project and write a paper. In the past, some have published or been MS projects. We will explicitly use class time discussing how to perform and communicate research properly.
First 45 min of each class is – ‘research topic lecture or paper presentation’
Second 45 min of each class is – ‘logistics of class, how to write a paper, and the projects you are doing’
I expect the style of the class to appeal to PhD students and MS students interested in research.
Learning Objectives:
- Understanding of a sampling of classic 3D acquisition methods .e.g. Time of Flight, Stereo, Structured Light, Photogrammetry / Structure from Motion
- Understanding of a sampling of classic Computational Photography methods .e.g. Flash no flash, Panoramas, Lightfields, RTI, ML for images
- A broad but shallow survey of what is hot currently (all papers on these topics at SIGGRAPH in the last two years)
- How to perform and communicate research - picking topics, how to discuss related work, how to make a good figure, what is most important, how to review a paper, etc
Homework/grading:
- Read 4 papers a week (for exposure to research ideas) (20%)
- Write short responses to 3 papers a week
- Write a “review” to 1 paper a week
- Prepare presentations of research papers, about 10-15 minutes long (~2 times) (20%)
- Full quarter team research project – background research, code, experiment, write, (50%)
- Participate (come to class) (10%)
Topics:
- Basics: Camera: senors, bayer mosaic, assorted pixels, how cameras work, flash no-flash
- Basics: Image: RSME vs perceptual error metrics; color spaces YUV vs RGB
- Basics: Light ray: Lightfields, Coded aperture, Ptyography
- Project Background:Stereo3D: Stereo, Structured Light, ToF
- Project Background:ML: PCA, dim reduction, tex. synth, vector quant, CGAN, Image Analogies
- Project Background:RTI: PTM, Photometric Stereo
- Classic Application: Panoramas (10 years ago)
- Classic Application: Deblurring/denoising: Photo stacking vs Deconvolution (now)
- Classic Application: Background blur, refocusing (still works poorly)
- Paper writing: Mistakes not to make, sample review forms, what makes a strong paper
- The above are classic topics, we will also have lots of recent papers as short presentations
Homework details:
- Read 4 papers a week (for exposure to research ideas) (20%)
- One picked by me – on the weekly topic area
- Write a “review” following a standard conference review method
- One more picked by me
- Two picked by you – related work to the papers you are writing
- Write a short ½ page journal entry including summary, what is the contribution of this paper, how did they prove it, what are your ideas for new papers
- Prepare presentations of research papers, about 10-15 minutes long (1-2 times during the qtr) (20%)
- Most classes will have 2 papers presented by students
- When its your turn, you’ll have a presentation to prep in addition to your journal
- Your goal is to communicate the goal, the contribution, the results, and a high level of key methods in your 15 minute slot
- Instructor rating of your presentation 10%
- Your peers will be asked to rate your presentation 10%
- Full quarter team research project – background research, code, experiment, write, … (50%)
- Background research – Search for existing papers to answer: Has someone done this exactly before? Has someone done something close? What are the categories of related work that already exist?
- Turned in assignments, individual (10%)
- Experiment/code – First make an explicit list of the results that you hope to obtain (before coding). Second, do what is needed to get these results.
- Team portion of grade based on demos and results (10%)
- Individual portion grade based on peer review from your team-mates (10%)
- Write – We will talk about sections in a paper one at a time, using the class projects as examples. Expect homework of the form “produce a ‘Results’ section this week with fake draft figures showing what you hope to eventually get working for real”. Then we talk about those in class.
- Individual writing 10%
- Peer review of other peoples sections 5%.
- Figures/captions – Figures, tables, plots, images all take much longer than words to get right. Expect multiple revisions as we talk about how to make these convincingly. They typically are the main “proof” that you have of your method working. 5%
- Background research – Search for existing papers to answer: Has someone done this exactly before? Has someone done something close? What are the categories of related work that already exist?
- Participation (10%)
- Come to class. Its expected, but missing when sick or you have a conference or interview is ok (you only lose 0.5%/day).
- One picked by me – on the weekly topic area
Class discussion and paper writing timeline:
Most classes first 1/2: Prof presents slides or leads discussion on 1 assigned paper, Students present 2 random modern papers
Most classes second 1/2: Prof presents or leads discussion on how to write this week’s paper section.
Every class: Homework 1 assigned paper and 1 you pick (related to your project or presentation)
Homework related to paper writing to follow roughly the schedule below
Sep 30 –
- Intro research topics
- Intro syllabus, HW, paper writing and HW
- HW (2 day):: Find related work that might force a change to a project
Oct 2
- (research topics, exact order determined later according to project choices, repeats first ½ every class)
- Pick projects and teams
- HW (3 day): Find more related work, focus on the projects chosen
Oct 7
- Outline of all needed sections, tools, data, plots, to get a complete paper ready
- (change teams/projects if needed)
- HW (2 day): Make an outline plan for your paper, make an outline plan for internal deadlines you want to hit
Oct 9
- What makes a good figure, types of figures
- HW (3 day): Plan for results figures (completely fake, hand drawn, no real results)
Oct 14
- What goes in the related work, how to write this section defensively, bibliography
- HW (2 day):: Draft related work section (with whatever you know now)
Oct 16
- What goes in an introduction section, framing your contribution
- HW (3 day): Draft introduction and contribution statement (compatible with hypothesized related work and results)
Oct 21
- What goes in the method section, which things to describe and which to leave out
- HW (2 day):. Draft method section
Oct 23
- What makes a good figure part 2
- HW (3 day): At least one figure with real data (although maybe not the best result yet)
Oct 28
- What goes in the conclusion
- HW (2 day):: Conclusion section
Oct 30
- Putting it all together: Contribution -> Related work shows novel -> Results shows it works
- HW (3 day): Complete paper
Nov 4
- (complete paper is due)
- HW (2 day):: Review classmates papers
Nov 6
- (reviews of classmates due)
- HW (5 day): Fix comments from classmates
- HW: If submitting to CVPR, merge as needed
Nov 11 – Holiday
Nov 13
- (final paper is due)
- What goes in a video, common problems in videos, other supplemental material
- HW (5 day): Prepare a video, other supplemental
----Cvpr deadline Fri Nov 15 ----
Nov 18
Nov 20
- (video and supplemental due)
- In class: Comment on supplemental materials
- CVPR supplemental material deadline Nov 22
Nov 25 – Thanksgiving week (possible class canceled)
Nov 27 – Thanksgiving week (possible class canceled)
Dec 2
- What goes in a conference presentation
- HW (2 day): Prep your presentation
Dec 4
- Present your papers to the class
Possible Class Projects: (longer PDF doc with details)
Recent papers that someone in the class will present (as many as we can fit in the class)
Presentation Signup:
https://docs.google.com/spreadsheets/d/1Hjg7r1_-I7KaaIvgueJS0j5kuradMTgZaS-Z7mYLgps/edit#gid=0
Readings:
- Due Oct 2
- Tom Malzbender, Dan Gelb, and Hans Wolters. 2001. Polynomial texture maps. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH '01).
- Due Oct 7
- James Davis, Ravi Ramamoothi, Szymon Rusinkiewicz. Spacetime Stereo : A Unifying Framework for Depth from Triangulation, IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition (CVPR), 2003.
- Due Oct 9 (power out, update to Oct 14)
- Due Oct 16
- Due Oct 21
- Due Oct 23
- Due Oct 28
- Jacob Telleen, Anne Sullivan, Jerry Yee, Prabath Gunawardane, Oliver Wang, Ian Collins, James Davis, Synthetic Shutter Speed Imaging, Computer Graphics Forum 26(3), Eurographics 2007
- https://ai.googleblog.com/2018/11/night-sight-seeing-in-dark-on-pixel.html
- Due Nov 6
- Due Nov 13
- Due Nov 20
- Shree K. Nayar and Srinivasa G. Narasimhan. 2002. Assorted Pixels: Multi-sampled Imaging with Structural Models. In Proceedings of the 7th European Conference on Computer Vision-(ECCV '02)
- Due Nov 25
- Due Dec 2
Course Summary:
Date | Details | Due |
---|---|---|