Israeli startup Mentee Robotics, co-founded by Amnon Shashua, the founder of Mobileye and AI21 Labs, has finally unveiled its secret project after two years of development - a humanoid robot named Menteebot.
Although Menteebot is still in the prototype stage, it is already targeting two major application areas: homes and warehouses. This robot utilizes artificial intelligence technologies, including popular large language models (LLMs) such as OpenAI's ChatGPT, which are integrated into all aspects of its operations.
Mentee Robotics describes Menteebot as an AI-driven robot capable of performing complex tasks end-to-end. Unlike most players in the industry who gradually incorporate AI into their products, Mentee Robotics has built this robot with AI as its core concept from the beginning.
Mentee has also released a video demonstrating how this AI robot responds to verbal commands, handles tasks, and performs actions related to motion, scene understanding, object detection and localization, and grasping. Please watch the following video for more details:
What makes Menteebot unique?
While humanoid robots have existed for many years, most research has focused on improving the interaction between robots and the physical world, including mimicking human movements and dexterity.
In the past, most robots were either pre-programmed or controlled through software platforms to perform specific tasks, such as moving boxes in a controlled environment.
However, with the emergence of language and body learning models, the robotics industry has undergone a new transformation. Many robot manufacturers have quickly adopted this technology (mainly through collaborations with relevant companies), enabling their robots to understand questions posed by users in natural language and perform tasks through learning.
Mentee is doing something similar, but instead of simply integrating AI technology into existing humanoid robots under development, they are committed to incorporating AI technology into all aspects of their operations to create a completely new humanoid robot.
The three levels of Menteebot
According to the company, the Menteebot prototype translates human instructions into complex real-world actions using three main levels of AI technology.
First, it uses transformer-based large language models (LLMs) to parse instructions and "think" about the steps required to complete tasks.
Next, it utilizes NeRF-based algorithms to dynamically build a cognitive 3D map of the environment, including semantic information about objects and items, and locates itself within the map while planning dynamic paths to avoid obstacles.
Finally, it employs a Sim2Real machine learning approach to execute planned steps on the path, defining the required movements in a simulated environment and implementing them in the real world through gaits and hand actions.
"We are at the convergence of computer vision, natural language understanding, powerful and detailed simulators, and methodologies for transitioning from simulation to the real world," said Shashua in a statement. "At Mentee Robotics, we see this convergence as the starting point for designing future general-purpose bipedal robots that can move like humans, have a brain for executing household tasks, and learn to perform tasks they have not been trained on through imitation learning."
Although the robot demonstrated in the video seems to be able to perform basic tasks such as entering the kitchen and moving fruits from one place to another, it is worth noting that it does not complete this task with a single command. The user first instructs the robot to enter the kitchen and wait, and then gives another command for it to pick up and place the fruits in another location. We still need to observe whether the robot can perform the same task in one go.
Nevertheless, considering that this is just a prototype, we can expect the robot to continuously improve over time and acquire the ability to handle complex commands without step-by-step guidance. This is crucial for practical applications in homes and warehouses.
Mentee stated that the final production-ready version of the humanoid robot will rely solely on camera perception, proprietary motors with unprecedented dexterity, and fully integrated AI technology. It is expected to be ready for deployment in the first quarter of 2025, although the company has not confirmed which market segment it will initially target.
Other companies dedicated to developing AI-driven humanoid robots
Although Shashua and his team's expertise in computer vision and large language models (LLMs) give Mentee an advantage in this field, it does not mean that the company will easily win. Several companies, including Tesla led by Elon Musk, Figure AI supported by OpenAI, and 1X Technologies, are actively entering the AI humanoid robot field.
NVIDIA has also launched Project GR00T, a general-purpose base model for humanoid robots, and has made it available to multiple companies in the industry, including Agility Robotics, Apptronik, Boston Dynamics, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics.
As for Boston Dynamics, the long-standing robotics research company is now owned by Hyundai Motor Group and has also introduced a new fully electric Atlas humanoid robot primarily for automotive and industrial applications.
In this fierce competition, it will be interesting to see how Mentee effectively and rapidly deploys its AI-driven humanoid robot.
