Anthropic CEO Dario Amodei revealed in a podcast interview that the AI model currently under intensive development could cost an astonishing $1 billion to train. He compared it to cutting-edge models like ChatGPT-4, which currently maintains a training cost of around $100 million. However, looking ahead to the next three years, he predicts that this number will skyrocket to $10 billion or even $100 billion.
"At this stage, the training cost for a single model is approximately $100 million. And for the models currently under development, their cost is nearing $1 billion," emphasized Amodei. "If the cost really climbs to $10 billion or $100 billion, I believe this will be achieved in the not-too-distant future, around 2025, 2026, or 2027. With the dual driving forces of algorithm optimization and chip technology advancement, we are likely to have AI models that surpass human intelligence in most fields."
When discussing the path from generative AI, such as ChatGPT, to artificial general intelligence (AGI), Amodei believes that this transition is not an overnight process but a gradual development. New models will iterate and evolve on the shoulders of their predecessors, similar to the pattern of a child's learning and growth.
He further analyzed that if the capabilities of AI models continue to grow at a rate of tenfold each year, the hardware performance supporting them will also need to increase by at least tenfold. Therefore, hardware costs are likely to be a key factor driving the skyrocketing training costs of AI. According to data from 2023, ChatGPT's operation already consumed over 30,000 GPUs, and Sam Altman confirmed that the training cost of ChatGPT-4 reached $100 million.
Looking back at last year, global data centers received over 3.8 million GPUs. Considering that the latest B200 AI chip from NVIDIA has a unit price of around $30,000 to $40,000, Amodei's estimated training cost of $1 billion is expected to be achieved in 2024. Unless more efficient technologies like Sohu AI chips can quickly become popular, the demand for hardware will also surge in sync with the progress of model optimization and quantitative research.
In fact, this exponential growth trend has already begun to emerge. Elon Musk plans to purchase 300,000 B200 AI chips, and OpenAI and Microsoft have jointly planned a $100 billion AI data center project. Faced with such enormous demand, if suppliers like NVIDIA can keep up with the market pace, the delivery volume of GPU data centers is expected to increase to 38 million next year.
However, while pursuing AI technological leaps, the supply of electricity and infrastructure construction cannot be ignored. The electricity consumed by data center GPUs sold just last year is enough to meet the daily power needs of approximately 1.3 million households. If the power demand of data centers continues to climb exponentially, power shortages and price increases may become severe challenges. In addition to expanding power plants, a completely new power grid system needs to be built to cope with the enormous power load generated by AI chip operations. In light of this, tech giants like Microsoft have begun considering innovative energy solutions such as modular nuclear power for their data centers.
In conclusion, the rapid development of artificial intelligence technology is leading the wave of hardware innovation. While the projected $100 billion training cost by Anthropic may seem distant, with the support of companies like NVIDIA, AMD, and Intel, this vision is gradually becoming a reality. However, as AI technology progresses at an astonishing pace, its profound impact on society's future is also worth contemplating.
