Deep Learning After DeepSeek & What’s Next
TLDR: As we’ve been saying for years –OpenAI’s Sam Altman has been collectively deceiving us– and DeepSeek proved that both OpenAI’s defensibility as a business and their “scaling to AGI/ASI” hype narrative are seriously flawed. The industry is reeling –tech stocks are down, starting with Nvidia– and, after a recovery, the market will want alternative AI technologies like what we are building at AO Labs.
What happened
DeepSeek, a side-project of a Chinese quant firm, released R1, an open source language model configured and tuned as a reasoning agent comparable to OpenAI’s o1 and soon-to-be-released o3 models while using vastly (orders of magnitude) less compute. While there has been much hype building up to o3 as one would expect from OpenAI, DeepSeek’s R1 crushed o1 (and models by others like Anthropic) across almost all benchmarks. Adding salt to the wound, DeepSeek claims to have done this with about $6M of compute, a few hundred billion shy of expectations OpenAI recently set with the US government (project Stargate).
Important points:
DeepSeek seems to not believe in ASI (artificial super intelligence)– they don’t have it as part of their mission like OpenAI, don’t mention it at all, and that they open sourced their product seems to suggest they think open core (open source + paid managed API) is a bigger or more realistic business than shooting for ASI like OpenAI
After DeepSeek’s release, OpenAI says they’ll release o3 for free…
DeepSeek R1’s improved performance and efficiency on benchmarks has not yet translated into real-world reliability improvements, and it might not (in other words, R1 does not solve hallucination)
In 2023, a Google memo leaked saying that “OpenAI has no moat and neither do we”-- DeepSeek has proven this true
What this means for AO Labs
AO Labs is building AI that learns like humans do. Since our inception we have been operating out of an awareness of the limitations of deep learning, which is not at all how humans learn. DeepSeek, while impressive, remains a variant of deep learning that’s revealing that the emperor has no clothes; the race-to-the-bottom phase of deep learning will continue and we are going to continue avoiding that bloodbath.
While it happens, we’ll build and grow what we need for reliable, human-like learning AGIs, for the market to continue to come to us as the OpenAI hype deflates. In that sense, DeepSeek has strengthened our position and made more people, both developers and customers, hungry again for new and alternative AI technologies .
Even our research is predicated on a different set of assumptions than OpenAI –see this post from 2023 about the new “bitter lesson”– assumptions which are now being proven out.
Why does our approach matter? Humans, each of us, have our own definitions and understanding of language that remains dynamic as we learn and communicate– we can understand what the other means. Deep learning involves pre-training models on large amounts of data, which results in a token bottleneck from causing a lack of reliability when interacting with AI systems like ChatGPT and limiting our ability to retrain or teach those systems in real-time. We call this the language gap.
Our path ahead is as clear as always and now even brighter than ever. As deep learning continues to provide diminishing results, it’s time to capture the next-generation of talent and interest in AI for us to build the next part of the stack. Reach out to dig deeper.
Sapere aude. It is time for us to think for ourselves again.