We introduce a method for animating human images, using the SMPL 3D human parametric model within a latent diffusion framework to improve shape alignment and motion guidance. By incorporating various maps and skeleton-based guidance, we enrich the model with detailed 3D shape and pose attributes, fusing them via a multi-layer motion fusion module with self-attention mechanisms.
arXiv 2024 Paper    Project Page

We present a novel differentiable point-based rendering framework for material and lighting decomposition from multi-view images, enabling editing, ray-tracing, and real-time relighting of the 3D point cloud. Our framework showcases the potential to revolutionize the mesh-based graphics pipeline with a relightable, traceable, and editable rendering pipeline solely based on point cloud.
arXiv 2023 Paper    Project Page   Code

We propose a parametric model that maps free-view images into a vector space of coded facial shape, expression and appearance using a neural radiance field, namely Morphable Facial NeRF. MoFaNeRF can be used to synthesize free-view images by fitting to a single image or generating from a random code. The synthesized face is morphable that can be rigged to any expressions and be edited to any shapes or appearances.
ECCV 2022 Paper    Project Page    Code

Author's picture

Hao Zhu

CITE Lab - 3DV Group, Nanjing University
E-mail: zhuhaoese@nju.edu.cn

Associate Researcher

Nanjing, China