Class Discussion Summary (Feb 11)

Put the citations to papers discussed here:

• Thabo Beeler, Bernd Bickel, Paul Beardsley, Bob Sumner, and Markus Gross. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 4, Article 40 (July 2010), 9 pages (Links to an external site.) Links to an external site.
• A Morphable Model for the Synthesis of 3D Faces, Volker Blanz and Thomas Vetter, SIGGRAPH'99 Conference Proceedings Links to an external site.

Scribes:

Andy Vitek

Chuangbo Tong

Abdullah Al-Omari

Yu Ji

Ishaan Paranjape

Write the Class Discussion Summary Here:

1. Could the proposed multi-camera method be applied to a multi camera smartphone, or even with a single camera taking a burst of pictures while moving?

Considering that the paper is showing a consumer 3D device with two simple lenses, it looks like it would be possible to extend the method to a multi camera smartphone without excessive technical issues.

As of using a single camera, it would only be possible to do so if the burst shots were guaranteed to preserve focus while moving, as it is a key component of the algorithm.

It's also possible to take photos simultaneously, deblur with matured picture processing technologies, then combine them all together.

While many issues regarding deblurring due to camera shake can be resolved, there may be possible issues with using a multi-camera smartphone for depth calculation; this is due to having two cameras close to each other in one plane, compared to the cameras being wide apart along a hemisphere as shown in the paper.

2. Can the single shot capture method be improved on with a database of faces like the other method?

Yes, the single shot capture method can be improved on with a database of faces. In some of the examples shown in the single shot capture method paper, some of the models contained noise and reduced image quality due to the small sensor size. With the aid of a database, perhaps this missing information can be substituted with data from the database, allowing for a more complete model, albeit a less truthful one.

However, it is worth noting that the faces in that database don’t have enough facial features such as race and age, and other changes. The reliance on a database limits whatever constructions might occur by the quality and completeness of the models in the database, so constant improvements would need to be made there as well in order to further improve on this.

3. This technique needs a very precious laser scan 3D face dataset. But perhaps image features can be learned directly with AI, and dispense with the 3D scans.  Is this paper out of date?

Because of machine learning’s ability to learn features from a few annotated examples, it is possible that the extensive 3D faces dataset would not be required to achieve similar high quality results; an algorithm could be trained on a few representative 2D faces for facial expressions and facial features, and then ran on even more examples to build a more extensive database. Recent research has also shown that it is possible to generate the 3D face from 2D representations.

It's also worth noting that for some situations with AI, not more data is better.

However, we must make sure to account for the data that the ML models are not being trained on; data that can be present in 3D scans. This information can be added to the ML model separately (with different datasets, etc) to further enhance the results.