According to the latest ranking from Hugging Face, one of the leading platforms for natural language processing (NLP) research and applications, a new open-source language model has claimed the top spot among the world's best language models.
This model, named "Smaug-72B," was publicly released by the startup company Abacus AI, which is dedicated to helping businesses tackle challenges in the field of artificial intelligence and machine learning. Technically, "Smaug-72B" is a fine-tuned version of "Qwen-72B," another powerful language model that was released a few months ago by the research team at Alibaba Group.
What's particularly noteworthy about this release is that Smaug-72B outperforms GPT-3.5 and Mistral Medium, two state-of-the-art proprietary large-scale language models developed by OpenAI and Mistral, respectively, in several popular benchmark tests. Smaug-72B also significantly surpasses its derivative model, Qwen-72B, in many evaluations.
According to the Hugging Face Open LLM leaderboard, which measures the performance of open-source language models in various natural language understanding and generation tasks, Smaug-72B is currently the first and only open-source model to achieve an average score of over 80 in all major LLM evaluations.
While the model's average score has not yet reached the human level of 90-100, its emergence indicates that open-source artificial intelligence may soon rival the capabilities of major tech companies, whose abilities have long been shrouded in secrecy. In short, the release of Smaug-72B could fundamentally reshape the development of artificial intelligence and tap into the intelligence of individuals outside wealthy corporations.
Advantages of Open Source
"Smaug-72B from Abacus AI is now available on Hugging Face, ranking at the top of the LLM leaderboard as the first model to achieve an average score of 80! In other words, it is the best open-source base model in the world," said Bindu Reddy, CEO of Abacus AI, in an article published on X.
She added, "Our next goal is to publish these techniques as research papers and apply them to some of the best Mistral models, including miqu (the 70B fine-toothed LLama-2). The techniques we use are specifically tailored for reasoning and mathematical skills, which is why GSM8K scores high! Our upcoming papers will provide more explanations on this."
With the release of the model, Smaug-72B becomes the first open-source model to achieve an average score of 80 on the Hugging Face open LLM leaderboard, which is considered a remarkable achievement in the field of natural language processing and open-source artificial intelligence.
Smaug-72B excels particularly in reasoning and mathematical tasks, thanks to the techniques employed by Abacus AI during the fine-tuning process. These techniques, which address the weaknesses of large-scale language models, will be detailed in an upcoming research paper, enhancing their capabilities.
Smaug-72B is not the only open-source language model making headlines recently. The team behind Qwen-72B, Qwen, has also released Qwen 1.5, a set of small yet powerful language models with parameters ranging from 0.5B to 72B.
Qwen 1.5 outperforms popular proprietary models like Mistral-Medium and GPT-3.5, with a context length of 32k, enabling fast and on-device inference with various tools and platforms. Qwen has also open-sourced Qwen-VL-Max, a new large-scale vision-language model that rivals Gemini Ultra and GPT-4V, developed by Google and OpenAI, respectively, which are the most advanced proprietary vision-language models.
Impact on the Future of Artificial Intelligence
The emergence of Smaug-72B and Qwen 1.5 has sparked great excitement and discussion within the artificial intelligence community and beyond. Many experts and influential figures have praised the achievements of Abacus AI and Qwen and expressed appreciation for their contributions to open-source artificial intelligence.
In a LinkedIn post, AI influencer and analyst Sahar Mor said, "It's hard to believe that less than a year ago, we were excited about models like Dolly." She marvels at the progress made by open-source models in the past year.
Smaug-72B and Qwen 1.5 are now available for download, use, and modification on Hugging Face. Abacus AI and Qwen have also announced plans to submit their models to the llmsys human evaluation leaderboard. They hint at future projects and goals, including the creation of more open-source models and their application in various domains and applications.
Smaug-72B and Qwen 1.5 are just the latest examples of the rapid development of open-source artificial intelligence this year. They represent a new wave of AI innovation and democratization, challenging the dominance and monopoly of large tech companies and bringing new possibilities and opportunities to everyone. Only time will tell how long Smaug-72B can maintain its leading position on the Hugging Face leaderboard, but it is certain that open-source artificial intelligence is experiencing a significant moment as we enter a new year.
