arXiv Analytics

Sign in

arXiv:2111.15490 [cs.CV]AbstractReferencesReviewsResources

FENeRF: Face Editing in Neural Radiance Fields

Jingxiang Sun, Xuan Wang, Yong Zhang, Xiaoyu Li, Qi Zhang, Yebin Liu, Jue Wang

Published 2021-11-30, updated 2022-03-20Version 2

Previous portrait image generation methods roughly fall into two categories: 2D GANs and 3D-aware GANs. 2D GANs can generate high fidelity portraits but with low view consistency. 3D-aware GAN methods can maintain view consistency but their generated images are not locally editable. To overcome these limitations, we propose FENeRF, a 3D-aware generator that can produce view-consistent and locally-editable portrait images. Our method uses two decoupled latent codes to generate corresponding facial semantics and texture in a spatial aligned 3D volume with shared geometry. Benefiting from such underlying 3D representation, FENeRF can jointly render the boundary-aligned image and semantic mask and use the semantic mask to edit the 3D volume via GAN inversion. We further show such 3D representation can be learned from widely available monocular image and semantic mask pairs. Moreover, we reveal that joint learning semantics and texture helps to generate finer geometry. Our experiments demonstrate that FENeRF outperforms state-of-the-art methods in various face editing tasks.

Comments: Accepted to CVPR 2022. Project: https://mrtornado24.github.io/FENeRF/
Categories: cs.CV
Related articles: Most relevant | Search more
arXiv:2209.05277 [cs.CV] (Published 2022-09-12)
StructNeRF: Neural Radiance Fields for Indoor Scenes with Structural Hints
arXiv:2112.05139 [cs.CV] (Published 2021-12-09, updated 2022-02-04)
CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields
arXiv:2008.02268 [cs.CV] (Published 2020-08-05)
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections