Orion Star Unveils a 1.4 Billion Parameter Model, Operable with a Mid-range Graphics Card

2024-01-22

Cheetah Mobile Chairman and CEO, Fu Sheng, also Chairman of Liehu Xingkong, held an AI open class with the theme "Innovation in the AI Wave". He shared his innovative thinking in the AI wave and released several products, including the enterprise-level base model Orin-14B. This base model can run on a thousand-yuan graphics card and is completely free, open source, and commercially available, according to the official statement. Orin-14B has 14 billion parameters and is based on the MoE multi-expert model architecture, with performance close to that of a hundred-billion-parameter large model. Fu Sheng demonstrated its high accuracy using the vertical model application in the social security and provident fund field as an example, showing an accuracy rate of 60% without fine-tuning and as high as 97% after fine-tuning. Fu Sheng emphasized the good balance achieved between performance and cost by the hundred-billion-parameter model, expressing his concern about it. Official data shows that after quantization, the Liehu Xingkong large model INT4 reduces parameters by 70%, improves inference speed by 30%, and has a performance loss of less than 1%. In addition, this model supports long-text processing with 320K contexts, processing 450,000 Chinese characters in one go and extracting key summary information. Fu Sheng predicts that hundred-billion-parameter large models will gradually decline as society does not need too many large models. In addition to the base model, Liehu Xingkong also launched the "Fine-tuning Toolbox", which includes seven suites for fine-tuning. Fu Sheng believes that internet companies have the opportunity to achieve tenfold growth by reconstructing their business lines with large models. Microsoft is a typical example, and in the future, Microsoft may become the first $10 trillion company. Finally, Fu Sheng emphasized the innovative concept of "Think different" and regarded enterprise-owned large models as another embodiment of innovation. He reviewed the internet entrepreneurship wave and the current AI technology wave, stating that only by daring to be different can one truly innovate.