How fast is the compute needed for given AI capability falling?
Core argument: Estimates put AI algorithmic progress at 4x per year, meaning 4x less compute is needed annually for the same capability, while Anthropic CEO Dario argues advances run faster, yielding up to 10x improvement.
Estimates put algorithmic progress at 4x per year, meaning that for every passing year, 4x less compute is needed to achieve the same capability. Dario, CEO of Anthropic argues that algorithmic advancements are even faster and can yield a 10x improvement. When investigating the cost for GPT-4, in this case, we see algorithmic improvements and optimizations creating a 10x decrease in cost and increase in capability. To be clear DeepSeek is unique in that they achieved this level of cost and capabilities first. They are unique in having released open weights, but prior Mistral and Llama models have done this in the past too. DeepSeek has achieved this level of cost but by the end of the year do not be shocked if costs fall another 5x. The Bank of China has announced a $140B (1T Yuan) subsidy over the next 5 years for the AI Industry Chain, one day after meeting the CEO and founder of DeepSeek. The goal of this subsidy is explicitly to empower China to be completely self-reliant in science and technology.





