Eye Reconstruction and Modeling for Digital Humans

Abstract

The creation of digital humans is a long-standing challenge of computer graphics. Digital humans are tremendously important for applications in visual effects and virtual reality. The traditional way to generate digital humans is through scanning. Facial scanning in general has become ubiquitous in digital media, but most efforts have focused on reconstructing the skin only. The most important part of a digital human are arguably the eyes. Even though the human eye is one of the central features of an individual's appearance, its shape and motion have so far been mostly approximated in the computer graphics community with gross simplifications. To fill this gap, we investigate in this thesis methods for the creation of eyes for digital humans. We present algorithms for the reconstruction, the modeling, and the rigging of eyes for computer animation and tracking applications. To faithfully reproduce all the intricacies of the human eye we propose a novel capture system that is capable of accurately reconstructing all the visible parts of the eye: the white sclera, the transparent cornea and the non-rigidly deforming colored iris. These components exhibit very different appearance properties and thus we propose a hybrid reconstruction method that addresses them individually, resulting in a complete model of both spatio-temporal shape and texture at an unprecedented level of detail. This capture system is time-consuming to use and cumbersome for the actor making it impractical for general use. To address these constraints we present the first approach for high-quality lightweight eye capture, which leverages a database of pre-captured eyes to guide the reconstruction of new eyes from much less constrained inputs, such as traditional single-shot face scanners or even a single photo from the internet. This is accomplished with a new parametric model of the eye built from the database, and a novel image-based model fitting algorithm. For eye animation we present a novel eye rig informed by ophthalmology findings and based on accurate measurements from a new multi-view imaging system that can reconstruct eye poses at submillimeter accuracy. Our goal is to raise the awareness in the computer graphics and vision communities that eye movement is more complex than typically assumed, and provide a new eye rig for animation that models this complexity. Finally, we believe that the findings of this thesis will alter current assumptions in computer graphics regarding human eyes, and our work has the potential to significantly impact the way that eyes of digital humans will be modelled in the future.