This paper presents a novel method to recover detailed avatar from a single image. A learning-based framework is proposed to combine the robustness of the parametric model with the flexibility of free-form 3D deformation. The neural networks are used to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, and restore detailed human body shapes with complete textures beyond skinned models.
TPAMI 2022

We explore a novel direction to apply the learningbased framework, consisting of a pre-processing module for point cloud voxelization, scaling and partition, compression network for rate-distortion optimized representation, and a post-processing module for point cloud
reconstruction and rendering, to represent point clouds geometry using compact features with the state-of-the-art compression efficiency.
TCSVT 2021    Code

We present FaceScape, a large-scale detailed 3D face dataset consisting of 18,760 textured 3D face model with pore-level geometry. By learning dynamic details from FaceScape, We present a novel algorithm to predict from a single image a detailed rigged 3D face model that can generate various expressions with high geometric details.
CVPR 2020    Code & Dataset

This paper presents a self-supervised method that can be trained on videos without known depth, which makes training data collection simple and improves the generalization of the learned network. The self-supervised learning is achieved by minimizing a photo-consistency loss between a video frame and its neighboring frames.
CVPR 2020    Code

We propose a novel approach to convert given speech audio to a photo-realistic speaking video of a specific person, where the output video has synchronized, realistic, and expressive rich body dynamics. The system is achieved by first generating 3D skeleton movements from the audio using a RNN, then synthesizing the output video via a conditional GAN.
ACCV 2020

We propose a novel learning based framework that combines the robustness of parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and shading information.
CVPR 2019 (oral)    Code    Project Page

We study how to synthesize novel views of human body from a single image. Our new pipeline is a composition of a shape estimation network and an image generation network, and at the interface a perspective transformation is applied to generate a forward flow for pixel value transportation.
CVPR 2018    Project Page

We propose a very convenient system of scanning a human body using only a conventional video camera without the aid of special sensor or controlled illumination. The point cloud reinforcement is proposed to detect and adjust the conflict point data for the slender and shaky body part, which achieves reasonable and plausible mesh reconstruction.
TCSVT 2017    Project Page

Hi3D system automatically reconstructs high-quality 3D mesh model using single camera and a turntable. The system aims at modeling small-scale still object with low-cost hardware. Several refining algorithms are involved to make the end-to-end system robust and accurate.
Project Page

The system capture the multi-view images of the object to reconstruct the high quality 3D mesh model. We make modifications according to different kind of objects, utilizing various prior to formulate a robust and accurate 3D modeling scheme. The method aiming at human face efficiently recover the 3D face model with detailed geometry, and has been used in movie and TV industries.
Project Page