Google Unveils Gemma: World's Strongest Open-Source Large Language Model, Surpassing Llama with 13 Billion Parameters

2024-02-22

Google has announced the release of two new artificial intelligence models, Gemma 2B and Gemma 7B, and has released "variants of pre-training and instruction tuning" for these models. During the press conference, Tris Warkentin, the head of Google DeepMind, stated that these new models are "a series of the latest small models" that will allow developers to use tools based on Gemini research and technology for research and AI development in open domains. To ensure the safety of AI applications, Google has also released a "Responsible AI Toolkit" that provides guidance and tools for developers to "create safer AI applications" using Gemma. According to Google, Gemma has surpassed Meta's Llama 2 and Mistral models in benchmark tests. Warkentin pointed out that they are collaborating with Nvidia and Hugging Face, so these benchmark tests will be included in the open LLM leaderboard from the day of release. However, when asked about the specific meaning of the term "open," Jeanine Banks, Google's Vice President and General Manager of X and DevRel, explained that it does not mean open source. She noted that the commonly used term "open" in the industry usually refers to open weight models, which means that developers and researchers have broad access, customization, and fine-tuning capabilities for the models. However, she emphasized that the terms of use may vary depending on the specific terms of each model. In the later part of the press conference, someone asked if the Gemma models can be used for commercial purposes. Banks responded that Google provides a "commercially licensed license" for Gemma, which means there are no restrictions based on organization type, organization size, or number of product users. She added that Google will monitor how developers and researchers use these tools to ensure they are not used to promote harm. Furthermore, Google will provide support for popular frameworks such as JAX, PyTorch, and TensorFlow through native Keras 3.0. Developers can also take advantage of readily available Colab and Kaggle notebooks, as well as integrations with tools such as Hugging Face, MaxText, and NVIDIA NeMo. It is worth mentioning that these pre-trained and instruction-tuned Gemma models can run on laptops, workstations, or Google Cloud, with deployment options including Vertex AI and Google Kubernetes Engine (GKE). When asked about the potential applications of these small models, Warkentin stated that they have a wide range of applications. He noted that from the perspective of open models, the 7B size is suitable for text generation and understanding applications. He also added that the "generation quality" of these models has "significantly improved" in the past year, so many use cases no longer require the use of large language models (LLMs). Finally, Warkentin emphasized that these models provide new ways for developing AI applications, including the ability to run inference and training on local developer desktops or laptops. He said, "This completely opens up new ways for us to develop AI applications that we're very excited about, where you can use your RTX GPU or on a single host in GCP, or you can use cloud GPUs."