Microsoft unveils cloud and AI custom chips: Azure Maia AI Accelerator and Azure Cobalt CPU

2023-11-27

At the latest Ignite conference, Microsoft unveiled two chips tailored for its cloud infrastructure: Microsoft Azure Maia AI Accelerator (Athena), optimized for artificial intelligence (AI) tasks and AI generation, and Microsoft Azure Cobalt CPU, a processor designed for general computing workloads on Microsoft Cloud based on the Arm architecture. Azure Maia AI and Azure Cobalt chips will enter Microsoft's data centers in 2024, initially powering services such as Microsoft Copilot or Azure OpenAI Service. According to the company's press release, the Maia AI chip is manufactured using a 5-nanometer process and features 105 billion transistors. This chip is specifically designed for the Azure hardware stack and will power some of the largest internal AI workloads running on Microsoft Azure. Meanwhile, Azure Cobalt is a 128-core chip built on the Arm Neoverse CSS design and customized for Microsoft. Wes McCullough, Vice President of Microsoft's Hardware Product Development, stated in the press release: "Choosing Arm technology is a key element of Microsoft's sustainability goals. It aims to optimize 'performance per watt' in its data centers, achieving more computational power per unit of energy consumption." In the past, Microsoft has reduced costs by building servers and racks and optimizing server capacity and heat dissipation efficiency by incorporating chips. The latter case is driven by Microsoft's mission to achieve carbon-negative emissions by 2030. Microsoft's competitors, Google and AWS, have already developed their own chips for AI workloads. For example, Google announced the Tensor Processing Unit at the 2016 Google I/O conference, which is used in its data centers to support neural networks for machine learning frameworks like TensorFlow. Additionally, AWS introduced its Graviton Arm chip and Inferentia AI processor in 2018, and in 2020, announced Trainium for model training. A responder on the Hacker News forum, Aromasin, commented: "Note that Microsoft has been using 'custom chips' for years, they were just FPGA-based rather than ASIC. They developed IPs that accelerate various processes. Now their scale is growing at a crazy pace, proving that the additional expense of designing ASICs is acceptable compared to using off-the-shelf parts." Microsoft has a closer collaboration with silicon vendors AMD (AMD MI300X Accelerated VMs) and Nvidia (NVIDIA H200 Tensor Core GPUs). However, unlike AMD and Nvidia, Microsoft does not allow customers to purchase servers with its chips. Scott Guthrie, Executive Vice President of Microsoft's Cloud and AI Group, tweeted: "Microsoft has rebuilt our infrastructure with a comprehensive systems approach to meet our customers' unique AI and cloud needs. With the introduction of our new AI accelerator Azure Maia and cloud-native CPU Azure Cobalt, along with our ongoing collaboration with silicon vendors, we can now offer more choices and performance." Finally, Microsoft plans to expand its range of choices in the future and continue efforts to develop the second generation of Azure Maia AI Accelerator and Azure Cobalt CPU.