Plausible 3D Face Wrinkle Generation Using Variational Autoencoders

Authors: Qixin Deng, Luming Ma, Aobo Jin, Huikun Bi, Binh Huy Le, Zhigang Deng

Publication Date: 2022

Published in: IEEE Transactions on Visualization and Computer Graphics, 28(9), 3113-3125

Publication link: https://ieeexplore.ieee.org/document/9321747

Abstract: Realistic 3D facial modeling and animation have been increasingly used in many graphics, animation, and virtual reality applications. However, generating realistic fine-scale wrinkles on 3D faces, in particular, on animated 3D faces, is still a challenging problem that is far away from being resolved. In this paper we propose an end-to-end system to automatically augment coarse-scale 3D faces with synthesized fine-scale geometric wrinkles. By formulating the wrinkle generation problem as a supervised generation task, we implicitly model the continuous space of face wrinkles via a compact generative model, such that plausible face wrinkles can be generated through effective sampling and interpolation in the space. We also introduce a complete pipeline to transfer the synthesized wrinkles between faces with different shapes and topologies. Through many experiments, we demonstrate our method can robustly synthesize plausible fine-scale wrinkles on a variety of coarse-scale 3D faces with different shapes and expressions.