A seismic shift is underway that could redefine the future of AI development and deployment. The big question is: Will models trend towards commoditization, or will closed models like OpenAI and Anthropic capture all the margin with premium services?
Just a year ago, proprietary AI models like GPT-4 and Claude 2 held a massive advantage over competitors. Open-source models were hardly usable, including the initial version of Meta's Llama. Unlike proprietary model companies like OpenAI, Anthropic and Cohere, open-source players lacked the billions in funding needed for AI data centers.
Times are changing! Last week, Llama 3.1 closed the performance gap with proprietary model vendors OpenAI and Anthropic.
Let’s look at the background of this major moment in AI development, examine the recent progress of the biggest companies and talk about what it all might mean for the future.
2022 and 2023: The race for compute
The dominance of closed-source models such as GPT-4 and Claude 2 was unchallenged in 2022 and 2023, with hardly any of the top AI practitioners using open-source models.
Training these very large models was insanely capital-intensive — early movers like OpenAI raised $11 billion, and Anthropic raised $7.3 billion. Open-source models were underfunded and lacked the computing capacity needed to build accurate models. NVIDIA A/H100 chips were sold out many quarters in advance, with AI developers scrambling to CoreWeave, Crusoe, Lambda and others to procure GPU supply.
Shortages divided the tech world into “haves” and “have nots” with GPUs — or, as we put it, “the GPU Rich and the GPU Poor.”
2024: The rise of Meta, Mistral and open-source with meaningful compute
In 2024, open-source AI became GPU rich. Meta has invested $20 billion in AI-focused data centers to gain independence from third-party AI vendors, nearly matching OpenAI’s funding from Microsoft and others. Mistral, with nearly $1 billion in funding, has become one of the top model developers, giving Europe a horse in the race and bolstering open-source overall.
Additionally, companies like Hugging Face and Databricks have magnified the open-source AI community by supporting and developing on these models.
As enterprises move from tinkering to deploying models in production, they face three main concerns:
- Latency: Critical for time-sensitive AI applications.
- Cost: Significant for compute-intensive apps.
- Security: Important for data residency and preventing third-party models from ingesting private data.
Open-source models offer advantages in these areas, providing greater control, cost efficiency and enhanced security.
Model developer platforms like Fireworks.ai, Gradient.ai, Baseten, Together.ai and Modal are enhancing the accessibility and usability of open-source AI for enterprises, making these technologies more practical and widespread. In fact, four of the 20 early-stage and mid-stage Enterprise Tech 30 companies were model developer platforms in 2024.
Today: A comparative analysis of closed-source versus open-source models
The performance gap between types of models is closing, while other advantages of open-source are becoming more apparent — notably security, latency and cost. However, there are still challenges with open-source in terms of scalability and enterprise-grade reliability, which has given rise to the model platforms mentioned above.
Recent data and performance metrics indicate that open-weight models are rapidly catching up with their closed-source counterparts. The latest Llama 3.1 flagship model is competitive with leading foundation models across most tasks, including GPT-4, GPT-4o and Claude 3.5.
Across the ecosystem, model inference prices are dropping significantly. OpenAI is cutting aggressively, and the cost of GPT-4o is 90% cheaper than GPT-4 a year ago. The company achieved these reductions alongside improvements in context length, latency and knowledge cut-off dates — meaning customers are getting more value, not just lower prices.
Despite these price cuts, OpenAI’s API revenue ($1.25 billion through OpenAI and $1 billion via Azure-OAI) has grown more than 10x in the past year.
Per public benchmarks done by Artificial Analysis, GPT-4o and Llama 3.1 405B are tied on quality and roughly the same on speed — but Llama 3.1 405B is more than 7x cheaper. Further, Meta’s small 8B model is approximately 50% faster than OpenAI’s fastest model, GPT-4o Mini, and 50% less expensive than OpenAI’s cheapest model.
It’s worth noting that the major open-source vendors that are competing with heavyweights OpenAI and Anthropic aren’t exactly grassroots projects. Rather, both are multibillion-dollar initiatives backed by companies that just so happen to have open-model architectures. Capital intensity and the GPU still remain supreme in AI development of cutting-edge models.
Implications for the future
Who will win — open-source or proprietary models? That’s the ultimate question in the future of software and AI.
Are we reaching diminishing marginal returns on each incremental model? Or will capital intensity and talent prevail, leading to a monopoly or duopoly situation of proprietary vendors?
This year, OpenAI’s training and inference costs are projected to reach $7 billion, while Anthropic will spend $2.5 billion. OpenAI’s computing expenses are more than 5x its employee costs, while for Anthropic, they are nearly 10x.
Looking ahead, training costs are expected to stabilize, and inference costs are predicted to drop significantly, potentially leading to better margins. Currently, computing costs exceed total revenue for both leading model developers, which is unsustainable.
Despite the massive cash burn required to compete in AI, open-source models have caught up, and Meta and Mistral are leading the way. Both have substantial funds to stay competitive, with Meta allocating $20 billion and Mistral raising nearly $1 billion. It’s clear that open-source AI is no longer constrained by GPU availability. OpenAI and Anthropic will need to excel in areas beyond sheer compute volume to stay competitive.
The future of AI as open-source narrows the gap
As we've seen, AI development is evolving fast. Open-source models, backed by tech giants like Meta and well-funded startups like Mistral, are starting to perform on par with proprietary leaders like OpenAI and Anthropic. This shift suggests a trend toward commoditization of base AI capabilities. However, the massive investments and ongoing innovations from proprietary model developers indicate that premium services and specialized applications will still provide significant value.
As computational costs stabilize (and potentially decrease), the next battleground in AI may well be in areas like data quality, model fine-tuning and innovative applications that go beyond raw performance metrics.
Subscribe to my Substack newsletter for additional insights on the data and tech infrastructure.
