Meta has announced the release of a new version of its code generation AI model, Code Llama. This model has enhanced processing power, higher accuracy, and supports specific programming languages such as Python, C++, Java, and PHP.
Code Llama 70B is a derivative product of Meta's open-source Llama 2 large-scale language model, specifically designed to generate code based on natural language prompts. Code generation is often considered the most viable use case for generative AI. By assisting in generating code at scale, generative AI can make software development more efficient and accessible, while enabling developers to be more innovative.
The challenge with code generation is that the code must be precise, strict, follow strict rules and syntax, and it must also be predictable and function as expected by users. However, software code can often be lengthy and complex, which means AI models must have a significant amount of context and logic to understand and generate it. To achieve this, AI models must have access to substantial computational power, which is where Code Llama 70B shines.
As one of the largest open-source code generation models ever, Code Llama 70B is expected to become a crucial benchmark in this specific subset of AI. It has tremendous potential as many existing code generation AIs have struggled to be reliable and secure in automating software development.
Meta states that Code Llama 70B has been trained on over 500 billion tokens of code and related data, making it more powerful and robust than earlier models. Additionally, it has a larger context window (100,000 tokens), enabling it to generate more complex code.
Meta's CEO, Mark Zuckerberg, is thrilled about the release of Code Llama 70B to the point that he decided to personally announce it on Facebook. He said, "Writing and editing code has become one of the most important use cases for AI models today. Coding ability has also proven to be crucial for AI models to handle information in other domains rigorously and logically. I'm proud of the progress here and look forward to incorporating these advancements into Llama 3 and future models."
The company is offering several variants of Code Llama 70B to the public, including a version called CodeLlama-70B-Instruct, which has been fine-tuned to handle code requests in natural language. Additionally, CodeLlama-70B-Python is an optimized version specifically for generating Python code. It is said to have an additional 100 billion tokens based on Python code to ensure greater accuracy and fluency in that language.
The Code Llama 70B model is now available for download on platforms such as Hugging Face, PyTorch, Jupyter Notebook, and TensorFlow.
