MNEMOSYNEv0.1.2563

Podcasts/Lex Fridman Podcast/Oliphant, Travis

Oliphant, Travis

creator of NumPy and SciPy; co-founder of Anaconda and Quansite

Travis Oliphant: NumPy, SciPy, Anaconda, Python & Scientific Programming

Lex Fridman Podcast · Sep 23, 2021

▶ Watch on YouTube

Core ideas

Oliphant traces his path into scientific Python back to 1997-98 as a biomedical engineering PhD student at Mayo Clinic, drawn to Python because, unlike Perl, code he wrote remained readable months later, and because an early array library called Numeric (created by Jim Hugunin) gave Python n-dimensional arrays and complex numbers — abstractions he considered essential for engineering work but largely missing or poorly supported elsewhere. He describes writing SciPy starting in 1998 as an act of "scratching his own itch" (needing ODE solvers, integration, and optimization routines he'd previously used in MATLAB), wrapping decades-old Fortran routines from Netlib, and crediting the broader open-source ethos — inspired by Linux and Linus Torvalds — for his instinct to share the resulting packages publicly rather than keep them private, despite having no formal support to do so as a graduate student and later assistant professor.

He recounts the pivotal, deliberately unglamorous origin of NumPy in 2005: the scientific Python community had split into two incompatible array libraries (the aging Numeric and a newer, more capable rival called Numarray, created in part for Hubble Space Telescope image processing), and rather than wait for someone else to unify them, Oliphant used a semester with no students to teach take on the project himself, merging both codebases' features into a single array object over about four months of intense work (roughly fourteen months total to reach general usability) — a decision he says was risky for his tenure case at BYU and one he frames as an important, underappreciated model of individual initiative resolving early-stage ecosystem fragmentation before entrenched factions make cooperation much harder (drawing an explicit parallel to today's TensorFlow/PyTorch split). He's candid about design regrets — notably NumPy's array-scalar type system, built on a "Python 1-era" typing model rather than Python 2's richer extensible type system, which he says has made later innovations (like proper GPU array support) harder than they needed to be, requiring separate libraries like CuPy rather than native GPU support in NumPy itself.

Oliphant discusses the practical, community-driven "stewardship not ownership" culture that emerged among early scientific Python maintainers (citing John Hunter's creation of Matplotlib, Fernando Pérez's IPython, and Wes McKinney's Pandas, whose data-frame concept traces back to NumPy's record arrays) as key to the ecosystem's growth, and reflects candidly on the economics of open-source creation — including funding his graduate student partly through direct sales of a self-published "Guide to NumPy" book (which he priced with an informal plan to make it free once it earned $250,000 or after five years, ultimately opening it early once a community documentation effort needed the material) and his broader, still-unresolved quest (discussed at length regarding his current ventures Quansite, Quansite Labs, and the "OpenTeams" enterprise-software marketplace concept) to find sustainable mechanisms connecting open-source contribution to entrepreneurial income, informed by his personal study of economics (citing Ludwig von Mises's "economic calculation problem" argument about the necessity of price signals) after realizing he'd built widely-used tools without understanding how to financially sustain the work or himself.

He describes founding Anaconda (originally Continuum Analytics) in 2007 after leaving a tenure-track BYU faculty position, driven by a desire to scale Python-based data analytics and prevent the web from bypassing the Python community; from that effort came Numba (a JIT compiler translating a typed subset of Python to LLVM, aimed at making explicit for-loops fast without sacrificing NumPy-array semantics, funded partly through a commercial GPU-targeting version and DARPA grants), Conda (a cross-platform, cross-language package manager designed to solve binary dependency problems — like compiled Fortran/C++ code in SciPy or scikit-learn — that pip, by his account, still doesn't properly address, though he acknowledges Conda's own community-governance shortcomings let pip's simpler mindshare dominate despite gaps like poor binary-dependency handling), and other projects like Bokeh, Dask, and Jupyter-adjacent tools. He reflects on his professional relationship with Guido van Rossum (praising his openness to community input and willingness to defer on scientific-computing questions, while noting van Rossum's general indifference to Python's packaging problem, memorably telling Oliphant's team at a 2012 Google meeting to "go fix it yourselves"), and on the long, feature-thin early years of Python 3 as an illustration of how difficult it is to move an established user base absent compelling new capability.

Oliphant closes with reflections on programming craft (valuing curiosity, decomposing problems into manageable pieces, learning from — but not blindly copying — others' code, and being wary of hype cycles) and hiring (favoring people with genuine open-source affinity while noting a real tension between strong open-source contributors and strong product-focused engineers, and valuing humility and a demonstrated hunger to keep learning above credentials). He also discusses his belief in projecting positive intent onto others' motives (contrasting this with social media's incentive to reward negativity and assume bad faith), and offers life advice emphasizing anchoring oneself in genuine relationships and commitments, remaining open to changing one's views over a decade of intellectual development, and preferring to build alternatives rather than simply attack what one dislikes.

Excerpts

On why he decided to personally write NumPy to resolve the Numeric/Numarray split:

“I said, well, you know, I'm writing tools for my own use in a PhD... I thought, well, there's just a few things missing... nobody else is going to do this. So it's sort of out of a sense of duty and passion, knowing that I don't think my academic advisor, I don't think the department here is going to appreciate this — but it's the right thing to do.”

On his design regret around NumPy's type system:

“In retrospect I wouldn't have created those — I'd have improved the type system... made the type system actually a Python type system, as opposed to currently it's a Python-one-level type system... it's hard to create new types. You want a quaternion type, or you want to add a new posit type... it's clumsy.”

On why he believes projecting positive intent matters in open-source communities:

“There's a tendency, and social media makes this worse — when you don't have perfect information about a situation, you tend to fill the gaps with the worst possible story. I think it's good to live life filling the gaps with the good, the best, the hopeful explanation of why you see this... and when you project that positive intent, people step up to that too.”