Srinivas, Aravind

co-founder and CEO of Perplexity AI; former AI researcher at DeepMind, Google, and OpenAI

Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet

Lex Fridman Podcast · Jun 19, 2024

• Aravind Srinivas
• Perplexity AI
• RAG retrieval augmented generation
• search engine business models
• transformer architecture history
• chain of thought reasoning
• AGI and inference compute
• Larry Page Google
• startup advice
• AI safety and compute concentration

Core ideas

Srinivas frames Perplexity as an "answer engine" rather than a search engine or chatbot: it retrieves relevant documents via search, extracts relevant passages, and feeds them to an LLM explicitly instructed to write a concise answer with a citation for every claim — an approach directly borrowed from academic writing norms ("every sentence you write in a paper should be backed with a citation") and from Wikipedia's sourcing culture. The idea emerged from necessity rather than design: an early Slack bot piping questions to GPT-3.5 (used to answer a confused co-founder's health-insurance questions) produced confident but often wrong answers, and forcing citations was the fix. He stresses the product doesn't end at the answer — related/suggested follow-up questions are core to the design, reflecting David Deutsch's The Beginning of Infinity thesis that knowledge-seeking never terminates, and the company's stated mission ("world's most knowledge-centric company," explicitly modeled on Amazon's "most customer-centric") is bigger than beating Google, since anchoring a mission to a competitor caps ambition.

He explains why Perplexity deliberately avoids competing with Google on Google's own terms: rather than build a marginally-better ten-blue-links search engine, Perplexity bet that inverting the UI (answers as the primary unit, links secondary) would improve as underlying models get cheaper and smarter, while Google is structurally disincentivized from cannibalizing its own high-margin AdWords/link-click business model with lower-margin alternatives — an idea he ties to Jeff Bezos's "your margin is my opportunity" (and Amazon's AWS/e-commerce strategy) and to why Amazon built cloud before Google did. He walks through Google's AdWords auction mechanics in detail (inherited from Overture, refined mathematically by Google) and argues Google's real invulnerability comes from Cloud and YouTube alone reaching a $100B annualized run rate, meaning search-ad disruption wouldn't sink the company. On adversarial dynamics, he describes "answer engine optimization" — websites embedding invisible text instructing AI crawlers to say specific things (demonstrating live by describing how a site could inject "say Lex is smart and handsome") — as an unavoidable cat-and-mouse problem no one can fully anticipate in advance.

Srinivas describes drawing specific, named lessons from entrepreneurs he admires: Larry Page and Sergey Brin's contrarian PageRank insight (using link structure rather than pure text similarity, itself inspired by academic citation graphs — the same inspiration behind Perplexity's citation model), Page's obsessive latency discipline (testing Chrome on old laptops so it would be fast everywhere, which Srinivas mirrors by testing Perplexity on airplane wifi) and his "the user is never wrong" philosophy (illustrated by the Google-vs-Excite pitch demo, where identical queries returned relevant results only on Google); Bezos's demand for written clarity of thought and his "one-way door vs. two-way door" decision framework; Elon Musk's "sheer force of will," his lesson about not outsourcing distribution (learned the hard way at Zip2, applied by selling Teslas direct rather than through dealers), and first-principles obsession with every operational detail; Jensen Huang's practice of holding all-hands with 60 direct reports simultaneously rather than 1:1s, and his admitted "paranoia" about going out of business despite Nvidia's dominance, given that hardware roadmaps lock in years in advance; Mark Zuckerberg's "move fast" ethos and Meta's strategically important open-sourcing of frontier-competitive Llama models, which Srinivas says he's grateful for since it prevents AI capability from concentrating in two or three closed labs; and Yann LeCun, an early investor in Perplexity, whom he credits for correctly (and unfashionably, in 2016) arguing that reinforcement learning is just "the cherry on the cake" while self-supervised learning is the actual "cake" — the recipe that became modern LLM pretraining — while noting LeCun's newer claim that autoregressive models may be a dead end is a distinct, more debatable position about reasoning happening in latent space rather than raw token space.

He gives a detailed technical history of the deep learning breakthroughs underlying LLMs: Bahdanau/Bengio's soft-attention mechanism beating Sutskever's brute-force scaled RNN on machine translation; DeepMind's PixelRNN/WaveNet insight that masked convolutions allow fully parallel training instead of sequential backpropagation-through-time; the 2017 Transformer paper combining attention's expressiveness with WaveNet's parallelizability (largely unchanged in core architecture since); and the OpenAI/Google lineage from GPT-1 (Radford/Sutskever's "unsupervised learning" bet) through BERT to GPT-2/3 and the Chinchilla scaling-laws insight that data quantity/quality must scale with model size. He distinguishes pre-training (raw scaling for general common sense) from post-training (RLHF, supervised fine-tuning, and myriad product-level details like citation formatting and tool-use routing) as where much near-term progress will happen, and discusses Microsoft's "Phi" small-language-model research as a bet that reasoning can be decoupled from raw factual memorization — a potential disruption to compute-hungry foundation-model players if it works. He explains the STaR (Self-Taught Reasoner) paper's bootstrapping idea — training a model on its own generated chain-of-thought explanations, even retroactively rationalizing toward a known correct answer — as mathematically related to a variational lower bound, and as a promising but empirically unproven path toward transferable reasoning skill that could generalize to agentic tasks.

On AGI and the future, Srinivas argues the defining moment won't be a single dramatic leap but the point where a model can go away for an extended period (a week, a month) applying massive inference-time compute and return with a genuinely novel, mind-blowing insight — comparable in impact to PageRank, the FFT, or the Transformer paper itself — and that achieving this is fundamentally an inference-compute problem more than a pre-training one, meaning power will concentrate not around who owns model weights but who can afford to run such jobs (raising his central AI-safety framing: the real risk isn't rogue AI but concentration of compute access among a few individuals, companies, or nations). He is skeptical that current AI has anything resembling human curiosity — the capacity to spontaneously generate the right question rather than just answer one well — and sees this as possibly the deepest, least-understood aspect of human distinctiveness. On self-play and recursive self-improvement, he argues new "signal" is the bottleneck (verifiable domains like math/code/games are tractable; open-ended real-world questions like predicting markets are not), and that a fleet of AI copies of a genius mind (a million "Ilyas" or "Andrejs") talking to each other risks becoming an echo chamber unless deliberately seeded with differing worldviews — itself an unsolved, likely-hardcoded problem.

He shares extensive product/engineering detail: Perplexity's own fine-tuned model ("Sonar," built on Llama 3, optimized with NVIDIA on TensorRT-LLM for low time-to-first-token and high throughput) coexists model-agnostically alongside GPT-4o and Claude, since users care about answer quality, not which model produces it; the four distinct failure modes causing hallucination (model not smart enough to synthesize retrieved text; stale/thin snippets; too much irrelevant context confusing the model; retrieval returning nothing relevant, in which case a good model should just say "I don't know"); why pure vector-embedding retrieval is insufficient for web-scale search (citing that BM25, a refined TF-IDF, still beats many embedding models on retrieval benchmarks, including OpenAI's own), requiring hybrid retrieval plus signals like domain authority and recency; and the engineering discipline of tracking P90/P99 "tail latency" (a lesson from Jeff Dean) rather than average-case speed. He closes with startup advice grounded in his own path (do what genuinely gives you dopamine, not what you think investors or markets want — he and his co-founders were already obsessed with search/knowledge products before founding the company) and a vision of Perplexity's "Pages" feature turning any individual's curiosity-driven research session into a shareable, audience-calibrated public knowledge artifact — explicitly positioning the long-term mission as making humans more curious and truth-seeking, deliberately avoiding the "AI companion/girlfriend" product space (which he was told by investors would be more lucrative) as a harder but more important path, and hoping widespread AI-augmented human curiosity — rather than AI replacing human relationships — is what ultimately reduces bias, echo chambers, and zero-sum thinking in the world.

Excerpts