"Meta to Launch Two Smaller Llama 3 Language Models Next Week, Targeting Multimodal Capabilities of GPT-4"
According to The Information, internal employees at Meta revealed that Meta plans to release two smaller-scale Llama 3 language models (LLM) next week as a prelude to the official version of Llama 3 for the summer.
The launch of these two smaller models aims to lay a solid foundation for the grand debut of Llama 3. Last July, Meta successfully released Llama 2, and since then, companies such as Google, xAI under Musk, and Mistral have also launched their own open-source large language models, intensifying market competition.
Llama 3 directly targets OpenAI's GPT-4, which is a powerful multimodal model capable of handling long texts and processing image inputs.
It is revealed that the official version of Llama 3 will also support multimodal processing, meaning it will be able to understand and generate both text and images simultaneously. However, the upcoming small versions will not have this feature for now.
Generally, smaller models are more cost-effective and faster in terms of operation speed. This advantage becomes more apparent, especially in the current context of high operating costs for large models. Additionally, smaller models are easier for developers to use in developing AI applications on mobile devices.
Previously, Meta has released three versions of Llama 2, with the largest version having up to 70 billion parameters, while the other two versions have 13 billion and 7 billion parameters, respectively.
The largest version of Llama 3 may have over 140 billion parameters, showcasing its powerful computing capability and ability to handle complex tasks.
Furthermore, Meta plans to address the issue of Llama 2 being too conservative when dealing with controversial topics in Llama 3. Researchers will relax the restrictions of large models in this aspect, allowing them to interact more actively with users and provide richer background information instead of simply avoiding the issues.