Metadata only
Date
2022-07Type
- Conference Paper
ETH Bibliography
yes
Altmetrics
Abstract
Recent research work has developed powerful generative models (e.g., StyleGAN2) that can synthesize complete human head images with impressive photorealism, enabling applications such as photorealistically editing real photographs. While these models can be trained on large collections of unposed images, their lack of explicit 3D knowledge makes it difficult to achieve even basic control over 3D viewpoint without unintentionally altering identity. On the other hand, recent Neural Radiance Field (NeRF) methods have already achieved multiview-consistent, photorealistic renderings but they are so far limited to a single facial identity. In this paper, we propose a new Morphable Radiance Field (MoRF) method that extends a NeRF into a generative neural model that can realistically synthesize multiview-consistent images of complete human heads, with variable and controllable identity. MoRF allows for morphing between particular identities, synthesizing arbitrary new identities, or quickly generating a NeRF from few images of a new subject, all while providing realistic and consistent rendering under novel viewpoints. We train MoRF in a supervised fashion by leveraging a high-quality database of multiview portrait images of several people, captured in studio with polarization-based separation of diffuse and specular reflection. Here, we demonstrate how MoRF is a strong new step forwards towards generative NeRFs for 3D neural head modeling. Show more
Publication status
publishedExternal links
Book title
SIGGRAPH '22: ACM SIGGRAPH 2022 Conference ProceedingsPages / Article No.
Publisher
Association for Computing MachineryEvent
Subject
neural rendering; novel view synthesis; photoreal human synthesis; generative models; neural radiance fieldsOrganisational unit
03420 - Gross, Markus / Gross, Markus
More
Show all metadata
ETH Bibliography
yes
Altmetrics