Light Source Estimation via Intrinsic Decomposition for Novel View Synthesis
Open access
Author
Date
2023Type
- Master Thesis
ETH Bibliography
yes
Altmetrics
Abstract
In this thesis, we propose a method for light source estimation via intrinsic decomposition within a Neural Radiance Field-like setting. We begin by exploring a learned method to recover light sources from intrinsic components - reflectance, shading, and surface normals. As demonstrated, the learned mapping is unable to properly learn the relationship between the different intrinsic components, and instead overfits some components while ignoring the rest. However, inference of local SH lighting, while somewhat spatially discontinuous, shows promise for future extensions. For these reasons, we later pivot our approach to directly optimize global spherical harmonics lighting, in which the in-scene radiance is represented by coefficients of a single global basis function. With this optimization-based approach, we successfully reconstruct a coarse estimate of the lighting of our scenes. To validate and support our method, we introduce a novel synthetic dataset, which consists of intrinsic components of several scenes. For each scene in our dataset, we provide geometry, shading, reflectance, and fully diffuse and rendered passes. We hope that extensions of our work could serve as modules of future NeRF models, enabling the joint optimization and refinement of lighting, geometry, and radiance. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000645817Publication status
publishedContributors
Examiner: Van Gool, Luc
Examiner: Oswald, Martin Ralf
Examiner: Das, Partha
Examiner: Sandström, Erik
Publisher
ETH ZurichOrganisational unit
03514 - Van Gool, Luc / Van Gool, Luc
More
Show all metadata
ETH Bibliography
yes
Altmetrics