By Jerome Manuel
Copyright yourstory
Every founder today is asking the same question: where is AI headed, and how do I build for it?
Artificial intelligence is no longer confined to research labs, it’s reshaping industries, jobs, and even the way we think. Few people understand this better than Prateek Jain, Principal Scientist and Director at Google DeepMind, who has spent two decades at the heart of AI’s biggest breakthroughs.
In this conversation, Jain explains why long reinforcement learning may be the most underrated shift, what bottlenecks still threaten progress, and how Indian entrepreneurs can position themselves to ride the wave rather than be crushed by it.
From Backprop to Transformers
Jain’s first encounter with AI was almost accidental: “When I started, actually, I was in college and I came across this book by Geoffrey Hinton who had an algorithm named Backprop there. I tried it out, and it seemed to work. It was interesting and surprising what’s going on here.”
His curiosity led him deeper into machine learning at UT Austin, where the field was still in its infancy. Early models like decision trees and support vector machines required domain experts to carefully design features. “Nothing was given at that time because you needed to handcraft basically the input that goes into the model,” he recalls.
The turning point came in 2011 with AlexNet. “That was the first time a purely neural network-based model was able to do better than fairly carefully designed computer vision models for image classification, and that essentially opened the floodgates.”
Soon after, NLP followed with breakthroughs like Word2Vec and sequence-to-sequence models. The arrival of Transformers in 2017 unlocked parallel training at scale, aided by GPUs and TPUs, setting the stage for today’s large language models.
The surprise of long reinforcement learning
Even as generative AI dominates headlines, Jain points to a more recent breakthrough on long reinforcement learning.
By letting models “think longer” before committing to an answer, researchers found that performance didn’t just improve on narrow tasks, it also generalised. “If we run a lot of reinforcement learning on math problems, then the models start to become good overall in general. They started to do better reasoning overall,” Jain explains.
What’s striking is the analogy he draws to human learning: “It seems very similar to what Indian parents stress that you should learn your math very carefully and then the rest of the subjects will fall in place. Somehow that seemed to happen for models very naturally.”
The bottlenecks ahead
For all the excitement around AI’s rapid progress, Jain is clear about the roadblocks that remain. He points to three areas where today’s models still fall short and where the next breakthroughs must come.
The first is latency. “The models currently are autoregressive, which means that they produce one word or one token at a time. So they have a sequential nature to them which then makes utilising our very fast parallel accelerators in a very deep manner [hard].” In other words, even as GPUs and TPUs get more powerful, today’s architectures are not designed to fully exploit that parallelism. Exploring alternatives like diffusion models, he suggests, could help overcome this bottleneck.
The second challenge is data. Scale has powered most of AI’s gains so far, but that well may be running dry. “We might be running out of the data that we can train our models on. So figuring out more data sources, and also figuring out maybe if we can have synthetic data sources to augment our core training, that might be another very interesting direction.” For startups, this is a reminder that proprietary datasets, curated annotations, and domain-specific signals could become invaluable moats as general web-scale data hits its limits.
The third is opacity and safety. Despite their power, large models remain mysterious even to their creators. “We routinely come across various scenarios where a model which is as powerful as one that won an IMO Gold Medal many times also makes very silly mistakes. And right now it’s not very clear where that originates from, how we can fix them.” This lack of interpretability is not just a scientific curiosity, it’s a security risk. “The opaqueness in the models is a problem especially because as these models become more powerful, more used, there is also a severe risk that they will be used by bad actors, jailbreaks, attacks, poisoning.”
India’s AI opportunity
Asked whether India is doing enough deep AI research, Jain is candid: “We should not be just consumers… we should be getting that expertise built in house. For a country of this size and diversity, it is very, very critical.”
He stresses that India must play the long game by investing in people. “We need to build a very strong base of AI engineers, AI app developers, AI researchers, building fundamentals has been a strong suit of India. So we should invest very heavily in it.”
Open source, he agrees, can accelerate progress, but “many times the bottleneck is not the model, we are lacking significantly in terms of very deep AI infrastructure talent, AI engineering talent.”
Advice to founders
For Indian entrepreneurs navigating the AI wave, Jain has a simple but urgent reminder: build for where models will be, not where they are today.
“These models have been improving at an exponential rate. If your differentiator is cost or a gap in today’s models, it might evaporate in three to six months,” he cautions. The risk, he explains, is that an entire startup strategy could be wiped out overnight by the release of a cheaper, better foundation model. Betting on temporary inefficiencies is like building on sand.
So where should founders focus? Workflows. “Whatever domain you pick, you should understand that domain’s workflow very well. And if you can design an architecture or a workflow that can exploit that domain workflow, then you’ll be in a good shape.” This means the moat isn’t just the model, but the integration, evaluation, and guardrails that wrap around it. In industries like healthcare, education, or finance, that domain depth becomes the true differentiator.
Listen to the full podcast episode to gain more insights.
00:00 – Introduction
03:00 – How Prateek Jain stumbled into AI research in college
07:00 – The AlexNet moment and why it mattered
14:30 – GPUs, TPUs & scaling AI
16:20 – The ChatGPT moment & mass adoption
21:40 – Long reinforcement learning: the biggest surprise
24:00 – Bottlenecks in AI: data, efficiency, alignment
26:49 – India’s deep AI research challenge
50:44 – Founder advice for building with AI
(Edited by Swetha Kannan)
