Groq announced on Monday that it has secured a $1.5 billion investment commitment from Saudi Arabia to expand its artificial intelligence chip operations in the country.
This deal comes approximately six months after the company raised $640 million from investors including Samsung Electronics Co., Cisco Investments, and others, valuing Groq at $2.8 billion.
Last year, Groq revealed plans to build an AI data center in Dammam, Saudi Arabia. The facility is optimized for inference tasks—operations that run neural networks in production after training. According to Reuters, this week's $1.5 billion commitment will be used to expand the data center.
The facility will be powered by Groq's flagship LPU (Language Processing Unit) chips. The company claims its processors are 10 times more energy-efficient than graphics processing units. Additionally, Groq states that the LPU is easier to program, reducing the time and custom code required to deploy AI workloads on the chip.
NVIDIA's GPUs can handle large language models as well as a variety of other workloads. In contrast, Groq's LPU is specifically optimized for large language models, which contributes to its efficiency. When engineers design chips for specific purposes, they can eliminate components found in general-purpose processors like GPUs, thereby reducing power consumption.
Graphics cards break down AI processing tasks into simpler steps. Once a step is completed, the hardware resources used for the computation can be immediately reassigned to the next task. However, in practice, reassigning these resources often slows down due to technical issues.
Groq claims its LPU simplifies this process. The chip features a mechanism that automatically determines which data chunks a given circuit group should process, how to process them, and where the output should be sent. This setup enables AI workloads to better utilize the LPU's onboard computational resources, according to Groq.
Another way the company aims to boost efficiency is by improving how chips exchange data within AI clusters.
Large language models typically run across multiple processors rather than a single one. To coordinate their work, these processors regularly exchange data, a process that requires dedicated networking chips. Groq asserts that its LPU design reduces the need for external networking components, lowering costs and making AI clusters driven by the chip easier to program.
Groq offers its LPU alongside an internally developed compiler. The compiler converts customers' AI models into a format that the chip can process more easily. During this process, it optimizes the models to better leverage the underlying hardware—a task that developers usually have to perform manually.
Groq sells its chips as part of a system called GroqRack, which includes eight servers, each equipped with eight LPUs. The processors are interconnected via RealScale, an internally developed interconnect that eliminates the need for external switches.
A single GroqRack can deliver 12 petaflops of performance when processing FP16 data points, commonly used to store information in AI models. One petaflop equals 1 quadrillion calculations per second.
Groq also provides its chips through GroqCloud, a managed cloud platform. The company recently updated the platform to allow customers to run workloads in its new Dammam data center.
