Google releases Gemini Nano: a large language model that can run on mobile phones

2023-12-07

Gemini may be the largest and most powerful large language model (LLM) developed by Google, but it can only run in data centers and is not suitable for mobile devices. Therefore, Google has introduced Gemini Nano, a smaller version of Gemini that can run locally and offline on your device. However, currently only Pixel 8 Pro can use Nano, and Google believes that this new model will be a core part of Android's future. If you have a Pixel 8 Pro, there are two things on your phone that now use Gemini Nano: the automatic summarization feature of the Recorder app and the Smart Reply section of the Gboard keyboard. Both of these are part of Pixel's December feature drop. Since the model runs on the device, they can both work offline and are fast and native. Google's start with Gemini Nano is small. The Smart Reply feature is only used in WhatsApp with Gemini, but Google says there will be more applications next year. Gemini is only launched in English, so many users cannot use it. Your Pixel 8 Pro may not feel like an upgrade - although it may in the future if Gemini is as good as Google claims. Next year, Google will bring Bard, powered by Gemini, to the Assistant on Pixel phones, giving you more Gemini experiences. Nano is the smallest version of the Gemini model (shh), but Demis Hassabis, CEO of Google DeepMind, says it is still powerful. "It has to fit in a footprint, right?" he said. "The footprint of Pixel phones is small. So there are memory constraints, speed constraints, and so on. It's an incredible model, considering its size - and it can benefit from larger models through distillation and that kind of thing." The goal of Nano is to create a version of Gemini with the highest possible capabilities without taking up storage space or heating up the processor to the temperature of the sun. Currently, Google's Tensor 3 processor seems to be the only processor capable of running the model. However, Google is also working to integrate Nano into Android: it has introduced a new system service called AICore, which developers can use to add Gemini-powered features to their apps. Your phone still needs a high-end chip to use it, but Google mentions companies like Qualcomm, Samsung, and MediaTek in its blog post announcing the feature, indicating that they manufacture compatible processors. Developers can now access Google's Early Access Program. In recent years, Google has been saying that its Pixel phones are AI devices. With the Tensor chip and close integration with all of Google's services, they should become better and smarter. With Gemini Nano, this could become a reality for many high-end Android devices. For now, this is one of the best reasons to invest in a Pixel 8 Pro.