Stability AI launches model "Stable Zero123" for generating 3D objects from images.

2023-12-15

Stable AI has launched its latest innovative product, Stable Zero123. This generative AI model, trained internally by the company, is capable of creating 3D images from ordinary photos with improved quality and efficiency. According to the company, this newly released model is an improvement upon the Zero1-to-3 and Zero123-XL models, utilizing advanced training datasets and techniques. Unlike its predecessors, Stable Zero123 demonstrates a deep understanding of objects and can generate high-quality images from different perspectives. The company's blog post highlights that Stable Zero123 is based on Stable Diffusion 1.5, which utilizes video random-access memory (VRAM) similar to generating a novel perspective. However, Stability AI explicitly states that generating 3D objects with this model requires more time and memory, recommending a minimum of 24GB VRAM for optimal performance. It is important to note from the statement that this model is intended for non-commercial and research purposes only, as the company aims to foster innovation within the scientific community. The company announces that researchers and enthusiasts can now access Stable Zero123 on Hugging Face, facilitating experimentation and exploration of its capabilities. Setting a new standard for 3D image generation Through Stable Zero123, Stability AI aims to advance the field of computer-generated imagery, providing researchers with a tool to explore the possibilities of 3D image generation. To achieve this, the company has enhanced the training dataset of Stable Zero123, utilizing filtered training data from Objaverse to focus on preserving high-quality 3D objects. The company has employed realistic rendering techniques on these objects, surpassing previous methods. During training and inference, the generative AI model benefits from highly conditioned inputs. By providing estimated camera angles to the model, it can make more informed and high-quality predictions, resulting in superior visual outcomes. Furthermore, the combination of precomputed datasets (precomputed latent variables) and an improved data loader has accelerated training efficiency by 40% compared to its predecessor, Zero123-XL. To encourage open research in the field of 3D object generation, Stability AI has improved the open-source code of the threestudio project to support Zero123 and Stable Zero123. A simplified version of the Stable 3D process is currently in private preview, utilizing Score Distillation Sampling (SDS) to optimize the use of Stable Zero123 in Neural Radiance Fields (NeRF). However, it is important to note that this release is strictly for research purposes and not intended for commercial use, as emphasized by the company.