Hugging Face launches open-source LeRobot robot project

2024-05-07

Earlier this year, Hugging Face successfully attracted Remi Cadene, a former scientist at Tesla, and appointed him to lead a new open-source robotics project, which immediately drew widespread attention in the industry. The reason why this news is so eye-catching is not only because of Tesla's outstanding position in the technology and robotics field - Tesla has been committed to developing a humanoid robot model called Optimus recently, but also because the field of robotics has made great progress and breakthroughs in recent months by integrating large-scale language models (LLMs) based on transformers and more extensive generative AI, which happen to be Hugging Face's expertise. Now, we have a glimpse of Hugging Face's open-source robotics vision for the first time: the company has announced a new project called LeRobot - an open-source toolkit aimed at popularizing AI robots and inspiring a new generation of robotics experts, which is now available on Github. Cadene posted a series of long articles on his personal account X (formerly Twitter), stating, "The next step in AI development is to apply it to our physical world. Therefore, we are building a community-driven project around AI robots and opening it to everyone!" Cadene also provided a link to a free Discord chat server for those interested in joining, using open-source code, and discussing the LeRobot project. So, what exactly is LeRobot? LeRobot is a powerful framework that can be called the "Transformers" of the robotics field. This toolkit is not only a software package but also a comprehensive platform that includes a versatile library for sharing and visualizing data and training state-of-the-art models. Users can also access a large number of pre-trained models to quickly start their projects. In addition, LeRobot seamlessly integrates with physical simulators, allowing enthusiasts and developers without physical robot hardware to simulate and test their AI models in a virtual environment. Features and Highlights The functionality of this toolkit is not limited to the software level. It is designed to handle various robot hardware - from simple robot arms used in education and research to more complex humanoid robots seen in advanced AI research facilities. The goal of LeRobot is to provide an AI system that adapts to and controls any form of robot, enhancing the flexibility and scalability of robots. Cadene posted some code examples that are now available in the LeRobot library on X, demonstrating some of the robot's capabilities, such as training the robot to navigate unknown spaces, grabbing objects from videos, and how to pass objects between two robot grippers/arms. Open-source Philosophy The decision to make LeRobot open-source is strategically significant, aiming to avoid the monopoly of power and innovation by a few companies. By providing this toolkit for free, Hugging Face invites developers, researchers, and enthusiast communities worldwide to contribute to the collective progress of AI robots and benefit from it. Building Community and Data Repository The cornerstone of supporting the development of LeRobot is the creation of the largest crowdsourced robot dataset in history. By collaborating with universities, startups, large tech companies, and individual enthusiasts, Hugging Face is driving the establishment of a massive data repository. This includes a large number of in-car video recordings formatted using the lightweight LeRobotDataset, which can be quickly uploaded and downloaded through the Hugging Face platform. Cadene stated on X, "We are building a diverse community from different backgrounds, software, and hardware to develop the next generation of intelligent robots in the real world." This reflects the inclusive vision of the project. By lowering the entry barrier and promoting an environment of sharing knowledge and resources, Hugging Face aims to cultivate a community that can redefine the landscape of the AI robotics field.