Luma Secures $43 Million to Advance 3D AI Model Technology

2024-01-10

Luma announced that it will begin utilizing a computing cluster consisting of approximately 3,000 Nvidia A100 GPUs to train new AI models that can "see, understand, display, and explain, ultimately interacting with the world." The first phase of this initiative involves creating models that can generate 3D objects based on textual descriptions; earlier this year, Luma introduced a model called Genie on its Discord server. The next step will be developing "next-generation" generative AI models to address the limitations of the current GenAI. "We believe multimodality is crucial for intelligence. To go beyond language models, the next breakthrough will come from vision," said Yu in an email interview. "However, AI needs to become smarter to unlock the potential of what it sees in the world." To realize this vision, Luma raised $43 million in Series B funding, with participation from established and new supporters such as Andreessen Horowitz. According to insiders familiar with the matter, this funding round valued Luma between $200 million and $300 million. Luma's current focus, launching AI models that create 3D models, is a highly competitive field. There are object creation platforms like 3DFY and Scenario, as well as startups such as Hypothetic, Kaedim, Auctoria, and Mirage. Stability AI recently introduced a standalone 3D model generation tool, and the new startup Atlas did the same. Even established companies like Autodesk and Nvidia are expanding their footprint in this area, with the release of the Get3D app that converts images into 3D models and ClipForge, which generates models from textual descriptions. So how will Luma's tool stand out? The key is fidelity, according to Yu. "Current models are trained on 2D images and when asked to generate scenes, they break spatial, body, and motion coherence," he said. "It's hard to generate anything coherent and usable in the first few attempts, limiting where you can use the output... We are bringing state-of-the-art generative realism techniques into an intuitive application." Considering it is still early in Luma's ambitious roadmap, there is hope. An improved version of Genie was launched today, but there is still a long way to go for more powerful generative AI models. However, Luma has wasted no time and plans to double its 24-person team and integrate a model running server cluster consisting of "thousands" of GPUs by the end of next year. Perhaps progress will indeed be made; time will tell. Jain said, "We have been developing teams in generative AI research, engineering, design, and product to turn our vision into reality and plan to accelerate this aspect significantly after this funding round. With Genie, AI has made it possible for the first time to create 3D objects at scale, growing to 100,000 users in just four weeks... But we aim to build more powerful, smarter, and more useful visual models for our users."