Monga, Rajat

Engineering Director at Google, leading the TensorFlow team

Rajat Monga: TensorFlow

Lex Fridman Podcast · Jun 3, 2019

• Rajat Monga
• TensorFlow
• Google Brain
• TensorFlow 2.0
• Keras integration
• PyTorch competition
• open source strategy
• eager execution vs. graphs
• TensorFlow ecosystem
• engineering team culture

Core ideas

Monga traces TensorFlow's lineage back to Google Brain's founding in 2011 under Jeff Dean and Andrew Ng, whose founding hypothesis was straightforward: take deep learning research approaches and scale them with Google's compute and data resources to see whether scale itself would improve results. Early wins in speech recognition (via close collaboration with Google's speech research team) and image classification (the famous "cat paper," an unsupervised model that learned to recognize cats from YouTube video frames) validated that the scaling hypothesis worked, and by 2014 both internal product demand (speech, then photos) and rising external academic interest convinced the team that deep learning was becoming a major, durable direction rather than a passing research trend.

He recounts TensorFlow's precursor, DistBelief, as internally successful but limited — tightly coupled to Google's specific infrastructure and using a more restrictive, straight-line (non-general) graph structure. Building TensorFlow (design begun in 2014, open-sourced November 2015) involved surveying existing frameworks (Theano, Torch/Lua, Caffe) and deliberately prioritizing two constraints from the outset: the flexibility to express rapidly evolving research ideas, and support for diverse and changing hardware (CPUs, GPUs, and Google's first TPU, which was about to launch) as well as mobile deployment, since Google already had products running hand-crafted ML models on phones at the time. On the choice to use a computation graph rather than eager, line-by-line execution (the Theano-like approach), Monga says the deciding factor was production deployment: DistBelief had already been used to deploy trained models widely inside Google, and while eager, Python-native execution looked simpler to use experimentally, the team couldn't see how to deploy from it — pushing them toward a graph-based design despite its comparative unfriendliness to newcomers.

On open-sourcing, Monga credits Jeff Dean as the primary champion, motivated by two converging beliefs: that Google Research's culture of building on and openly sharing prior work (a major driver, in his view, of deep learning's rapid overall progress) should extend to tooling, not just papers, and a software-strategy precedent Google had already established of releasing a "better," Google-informed alternative to a category (citing Bigtable's open HBase-compatible API as an example) rather than always relying on others' reverse-engineered reimplementations of internal Google research. Monga states plainly he never anticipated TensorFlow's actual adoption scale (41 million downloads, ~50,000 commits, ~10,000 pull requests, ~1,800 contributors by the time of this interview) — the growth became apparent only in the months after release, driven partly by unexpectedly strong documentation (a major differentiator versus academic-project libraries like Theano and Caffe, which weren't built with non-specialist developers in mind) that let ordinary developers, not just ML researchers, pick up deep learning directly.

He describes a phased maturation from initial research/hobbyist adoption toward enterprise stability demands as the library approached and passed the 1.0 release, noting a persistent split in the user base even years later: many enterprises still run older, "good enough" models (citing Inception and ResNet-50 as still-widely-used examples) that don't need cutting-edge performance, while a separate research-facing edge continually pushes newer architectures (RNNs, GANs, Transformers, combinations with reinforcement learning) — a split Monga considers healthy since backward stability for the former doesn't block innovation for the latter. On enterprise use specifically, he distinguishes hobbyist-driven transfer learning (fine-tuning existing image models) from the actual "bread and butter" of most enterprise ML, which remains structured, tabular-data problems historically served by simpler regression and gradient-boosted-tree models, only some of which meaningfully benefit from deep learning; he also describes fielding frequent practical questions from non-technical adopters (including lawyers exploring ML for their field) that usually reduce to a more basic problem — organizations' data simply isn't digitized or organized enough to use yet — motivating TensorFlow's investment in curated datasets (TensorFlow Datasets) and pre-trained models to lower that barrier.

On the integration of Keras, Monga recounts that François Chollet built Keras independently (initially atop Theano) before joining Google, purely as a personal nights-and-weekends project — Monga says the TensorFlow team didn't even realize Chollet worked at Google until months after noticing Keras's growing popularity as a clean API layered on TensorFlow. Rather than continuing to maintain several competing internal and community API layers (which created a "which one do I use?" support problem), the team invited Chollet to work with them for what was meant to be a single quarter and ended up becoming a roughly two-year (and ongoing) deep integration, ultimately making Keras the officially recommended entry point for TensorFlow 2.0 — a decision Monga frames as organic consensus around whichever API developers already loved most, rather than a top-down mandate.

Asked about PyTorch as a competitor, Monga takes a notably collegial view: PyTorch's willingness to prioritize research ergonomics over production concerns (no static graph requirement, less initial focus on raw performance) let it explore design space TensorFlow's more production-oriented original architecture hadn't, and he credits this competitive pressure with accelerating (though not solely causing) TensorFlow's move toward eager execution, ultimately unified with graph performance in TensorFlow 2.0. On backward compatibility, he frames it as a genuine trade-off rather than a pure cost: because many production systems (inside and outside Google) depend on long-lived TensorFlow deployments, breaking changes impose real costs, but sustained compatibility also signals reliability to future adopters — his stated design heuristic for the team is to design new features "with a clean slate" unconstrained by legacy concerns, and only afterward work out how to reconcile the result with existing compatibility commitments.

Monga describes the ecosystem's broader mission as enabling machine learning "on every device that has compute capability" — spanning cutting-edge research enablement (citing BERT as a recent example of research TensorFlow helped make broadly buildable), full production pipelines (TensorFlow Extended), edge devices (TensorFlow Lite), browsers and Node.js (TensorFlow.js), and increasing modularity so external hardware vendors and organizations (citing IBM's involvement in special interest groups) can build and extend components without needing the core team to build everything centrally. He acknowledges TensorFlow's core remains a large, still largely monolithic codebase built up over roughly four years of rapid evolution, and identifies breaking it into cleaner, more independently extensible interfaces — while the system keeps evolving under active use ("changing the engine with the car running") — as one of the team's key ongoing engineering challenges.

On team management, Monga emphasizes that a large engineering organization's output is a genuine product of team cohesion and shared direction, not simply the sum of individually brilliant engineers — arguing hiring for pure technical skill isn't sufficient without genuine motivation and alignment toward where the team is headed, and stating that even a highly productive "superstar" contributor is a problem if their presence undermines team dynamics. He describes Google's standard hiring process (refined over roughly two decades) as including an explicit culture-fit evaluation alongside technical assessment, with the specific culture bar varying somewhat by team — TensorFlow historically favored people comfortable with fast-moving, less settled work, though as it's matured into a full production-grade product the team now also needs people comfortable with more disciplined, less corner-cutting engineering. On process, he says the team deliberately avoids artificial internal deadlines in most day-to-day work (preferring continuous, regularly-cadenced releases where unfinished features simply roll to the next release), while acknowledging that genuine external deadlines (like the TensorFlow 2.0 developer summit) did create productive urgency and that "spring is a relative concept" for a still-evolving release timeline.

Drawing on his earlier experience leading a Google Search Ads team, Monga argues search advertising, done well, functions as a direct extension of search's core mission of connecting people to information (including commercial information) they actually want, describing an internal minimum ad-quality bar below which an ad simply won't be shown even at the cost of forgone revenue. He expects advertising to remain a durable model but anticipates continued growth of a mixed ecosystem alongside it — citing rising willingness to pay for content (Netflix, YouTube subscriptions, newspaper paywalls) as evidence users increasingly choose paid, ad-free experiences when they perceive clear value. Closing on accessibility, he points to free tools (Colab notebooks requiring no local installation, ever-more-powerful personal devices, and free-tier cloud services) as the most practical on-ramp for students and beginners, while noting free compute (like free-tier TPU access) necessarily comes with usage limits relative to paid tiers.

Excerpts