DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling

Authors: Linqi Zhou, Andy Shih, Chenlin Meng, Stefano Ermon

What

This paper introduces DreamPropeller, a method for accelerating text-to-3D generation using score distillation by generalizing Picard iterations to handle complex computation graphs and leveraging parallel compute.

Why

The paper addresses the slow generation time of existing text-to-3D methods that utilize score distillation, which hinders their practical use despite high generation quality.

How

The authors generalize Picard iterations, a parallel ODE solving technique, to handle the intricacies of 3D generation, such as momentum-based updates and varying dimensionality, and apply this generalized framework to accelerate existing score distillation methods.

Result

DreamPropeller achieves up to 4.7x speedup across various 3D representations and score distillation techniques, including NeRF, DMTet, SDF, and 3D Gaussian Splatting, with negligible drop in generation quality measured by CLIP R-Precision and FID.

LF

The paper acknowledges limitations in perfectly matching baseline quality due to the fixed-point error and suggests exploring alternative distance metrics or adaptive optimization strategies for further improvement. Future work may also involve investigating the application of DreamPropeller to other domains beyond 3D generation.

Abstract

Recent methods such as Score Distillation Sampling (SDS) and Variational Score Distillation (VSD) using 2D diffusion models for text-to-3D generation have demonstrated impressive generation quality. However, the long generation time of such algorithms significantly degrades the user experience. To tackle this problem, we propose DreamPropeller, a drop-in acceleration algorithm that can be wrapped around any existing text-to-3D generation pipeline based on score distillation. Our framework generalizes Picard iterations, a classical algorithm for parallel sampling an ODE path, and can account for non-ODE paths such as momentum-based gradient updates and changes in dimensions during the optimization process as in many cases of 3D generation. We show that our algorithm trades parallel compute for wallclock time and empirically achieves up to 4.7x speedup with a negligible drop in generation quality for all tested frameworks.