We propose the first compression approach for image-to-shape Diffusion Transformers (DiTs) that substantially reduces model size while preserving geometric fidelity. Despite remarkable progress in 3D shape generation, large DiT-based models remain computationally prohibitive in resource-constrained settings. Furthermore, it is difficult to directly transfer existing diffusion model compression strategies developed for different domains to 3D generation. Prior 3D efficiency approaches primarily focus on inference speed rather than backbone compression. To address this limitation, we build a geometry-aware compression framework tailored to image-to-shape DiTs. Guided by the observation that 3D DiT layers exhibit non-uniform importance for geometry synthesis, we introduce a vitality-guided framework integrating structured pruning, adaptive quantization, and targeted fine-tuning. Our method achieves up to 66% model-size reduction across state-of-the-art image-to-3D models while maintaining synthesis fidelity comparable to full-sized counterparts. These results highlight the potential of our framework as a plug-and-play solution for efficient 3D shape generation across diverse models.
Vitality-Guided Compression. We estimate the geometric importance, or vitality, of each DiT layer by removing one layer at a time and comparing the resulting point cloud with the full model’s output using Earth Mover’s Distance. This layer-wise analysis identifies which layers are essential for preserving global structure and fine geometric details, and which layers contribute little to the final shape. Since double- and single-block layers exhibit different sensitivity patterns, we prune low-vitality layers using separate thresholds for the two modules. We then adaptively quantize the retained layers, assigning 8-bit precision to highly vital layers and 4-bit precision to less vital ones. Together, vitality-guided pruning and quantization substantially reduce the model size while maintaining geometric fidelity.
Targeted Fine-tuning. We refine the compressed student by matching its conditional and unconditional predictions to those of the full teacher along the teacher’s flow-sampling trajectory. Rather than updating the entire network, we fine-tune only the lowest-vital retained layer in each double- and single-block module while keeping the remaining layers frozen. This selective distillation provides stable and efficient recovery from compression-induced quality loss.
Pick an example below, then toggle between baselines on the left.
Step through the models, then pick an example to compare each compression stage.
We additionally apply our approach to the SSF DiT module of TRELLIS. Our method reduces the module from 1.078GB to 0.320GB (about 70.3% model size reduction), which emphasizes the applicability of our method.
@article{lee2026vitality,
title = {Vitality-Aware Compression for Efficient Image-to-Shape Diffusion Transformers},
author = {Lee, Jaeah and Kim, Hyunjin and Cho, Jaewoong and Kwon, Gihyun},
journal = {arXiv preprint arXiv:2607.00382},
year = {2026}
}