We propose a parametric model that maps free-view images into a vector space of coded facial shape, expression and appearance using a neural radiance field, namely Morphable Facial NeRF. MoFaNeRF can be used to synthesize free-view images by fitting to a single image or generating from a random code. The synthesized face is morphable that can be rigged to any expressions and be edited to any shapes or appearances.
ECCV 2022    Project Page    Code

This paper presents a novel method to recover detailed avatar from a single image. A learning-based framework is proposed to combine the robustness of the parametric model with the flexibility of free-form 3D deformation. The neural networks are used to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, and restore detailed human body shapes with complete textures beyond skinned models.
TPAMI 2022

We explore a novel direction to apply the learningbased framework, consisting of a pre-processing module for point cloud voxelization, scaling and partition, compression network for rate-distortion optimized representation, and a post-processing module for point cloud
reconstruction and rendering, to represent point clouds geometry using compact features with the state-of-the-art compression efficiency.
TCSVT 2021    Code

This paper presents a self-supervised method that can be trained on videos without known depth, which makes training data collection simple and improves the generalization of the learned network. The self-supervised learning is achieved by minimizing a photo-consistency loss between a video frame and its neighboring frames.
CVPR 2020    Code

We propose a very convenient system of scanning a human body using only a conventional video camera without the aid of special sensor or controlled illumination. The point cloud reinforcement is proposed to detect and adjust the conflict point data for the slender and shaky body part, which achieves reasonable and plausible mesh reconstruction.
TCSVT 2017    Project Page

Author's picture

Hao Zhu

NJU-3DV Lab, Nanjing University
E-mail: zh@nju.edu.cn

Assistant Professor, PhD Advisor

Nanjing, China