Distribution-Aligned Diffusion for Human Mesh Recovery

Foo, Lin Geng and Gong, Jia and Rahmani, Hossein and Liu, Jun (2024) Distribution-Aligned Diffusion for Human Mesh Recovery. In: 2023 IEEE/CVF International Conference on Computer Vision (ICCV) :. Proceedings of the IEEE International Conference on Computer Vision . Institute of Electrical and Electronics Engineers Inc., FRA, pp. 9187-9198. ISBN 9798350307184

Text (Foo_Distribution-Aligned_Diffusion_for_Human_Mesh_Recovery_ICCV_2023_paper)
Foo_Distribution-Aligned_Diffusion_for_Human_Mesh_Recovery_ICCV_2023_paper.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (2MB)

Abstract

Recovering a 3D human mesh from a single RGB image is a challenging task due to depth ambiguity and self-occlusion, resulting in a high degree of uncertainty. Meanwhile, diffusion models have recently seen much success in generating high-quality outputs by progressively denoising noisy inputs. Inspired by their capability, we explore a diffusion-based approach for human mesh recovery, and propose a Human Mesh Diffusion (HMDiff) framework which frames mesh recovery as a reverse diffusion process. We also propose a Distribution Alignment Technique (DAT) that injects input-specific distribution information into the diffusion process, and provides useful prior knowledge to simplify the mesh recovery task. Our method achieves state-of-the-art performance on three widely used datasets. Project page: https://gongjia0208.github.io/HMDiff/.