We propose to synthesize high-quality 3D face models from natural language descriptions including both concrete and abstract descriptions. Describe3D dataset is established with large-scale 3D faces and fine-grained text descriptions for text-to-3D face generation task.
CVPR 2023    Project Page   Code

We propose a novel approach for the single-view 3D face reconstruction task in a non-parametric scheme. Our method gets rid of the heavy dependence on the statistic model and, therefore, its limitations and achieves state-of-the-art performance by learning from our created pseudo 2D&3D datasets.
AAAI 2023 (Oral)    Project Page    Code

We propose a parametric model that maps free-view images into a vector space of coded facial shape, expression and appearance using a neural radiance field, namely Morphable Facial NeRF. MoFaNeRF can be used to synthesize free-view images by fitting to a single image or generating from a random code. The synthesized face is morphable that can be rigged to any expressions and be edited to any shapes or appearances.
ECCV 2022    Project Page    Code

we propose a novel framework to efficiently recover detailed facial geometry from calibrated multi-view images. The cost regression and multi-view fusion is solved by learning an implicit function in our network, which is proved to be more accurate and time-saving.
AAAI 2022 (Oral)    Code

This paper presents a novel method to recover detailed avatar from a single image. A learning-based framework is proposed to combine the robustness of the parametric model with the flexibility of free-form 3D deformation. The neural networks are used to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, and restore detailed human body shapes with complete textures beyond skinned models.
TPAMI 2022

We explore a novel direction to apply the learningbased framework, consisting of a pre-processing module for point cloud voxelization, scaling and partition, compression network for rate-distortion optimized representation, and a post-processing module for point cloud
reconstruction and rendering, to represent point clouds geometry using compact features with the state-of-the-art compression efficiency.
TCSVT 2021    Code

We present FaceScape, a large-scale detailed 3D face dataset consisting of 18,760 textured 3D face model with pore-level geometry. By learning dynamic details from FaceScape, We present a novel algorithm to predict from a single image a detailed rigged 3D face model that can generate various expressions with high geometric details.
CVPR 2020    Code & Dataset

This paper presents a self-supervised method that can be trained on videos without known depth, which makes training data collection simple and improves the generalization of the learned network. The self-supervised learning is achieved by minimizing a photo-consistency loss between a video frame and its neighboring frames.
CVPR 2020    Code

We propose a novel approach to convert given speech audio to a photo-realistic speaking video of a specific person, where the output video has synchronized, realistic, and expressive rich body dynamics. The system is achieved by first generating 3D skeleton movements from the audio using a RNN, then synthesizing the output video via a conditional GAN.
ACCV 2020

We propose a novel learning based framework that combines the robustness of parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and shading information.
CVPR 2019 (oral)    Code    Project Page