MNEMOSYNEv0.1.2563

Podcasts/Lex Fridman Podcast/Knuth, Donald

Knuth, Donald

Turing Award-winning computer scientist; author of "The Art of Computer Programming"; creator of TeX

Donald Knuth: Algorithms, Complexity, and The Art of Computer Programming

Lex Fridman Podcast · Dec 30, 2019

▶ Watch on YouTube

Core ideas

Knuth traces his love of computing to his freshman year encountering an IBM 650 (2,000 words of memory, a rotating drum with no random-access memory until his senior year), and uses this origin story to unpack what he calls "geek" thinking — a trait he estimates at roughly 2% of the population and defines by two specific abilities rather than raw intelligence: fluently jumping between levels of abstraction (moving unconsciously between "what does the computer's drum timing require" and "what does this algorithm accomplish"), and comfort with non-uniform, many-case systems (an algorithm with ten different special-case steps doesn't bother him, whereas much of pure mathematics prizes reducing everything to one or two universal rules). He extends this into a running theme of embracing dualities rather than picking a side: he describes his own life as "a convex combination of English and mathematics," citing his admiration for authors who "dot the i's and cross the t's" (Tolstoy, Herman Wouk, Raymond Chandler) prose stylists over ones who don't polish (Dostoevsky), and frames his invention of literate programming — writing code interleaved with natural-language exposition, explaining the "why" of a program the way a teacher explains a concept — as an application of the general principle that understanding deepens when the same idea is presented both formally and informally.

On The Art of Computer Programming's own history, Knuth recounts that the 1962 table of contents was originally scoped as a single book about how to write compilers, with combinatorics thrown in as one minor chapter "just for fun" — only for combinatorics to explode into its own massive subfield through the 1970s, eventually consuming what is now the multi-volume Volume 4. He walks through the books' overall arc (Volume 1: fundamentals, data structures, and the mathematical tools to analyze algorithms' running time, not just whether they work; Volume 2: numerical algorithms — arithmetic, random numbers, power series; Volume 3: sorting and searching; Volume 4: combinatorial algorithms, his personal favorite domain because a single good idea can produce million-fold speedups) and describes being repeatedly forced to rewrite large sections as the field advanced past him — most dramatically when Binary Decision Diagrams (1986) and SAT solvers (circa 2000) each turned into wholly new thirty-plus-year subfields he had to learn from scratch and integrate, adding hundreds of pages to Volume 4B. He describes a granular daily writing process — drafting by hand in pencil in a specific chair, writing roughly five test programs a week to verify every technique before describing it in prose, and revising obsessively for rhythm at the typing stage, "like haiku" — and recalls deliberately cutting a favorite fully-developed baseball example from the book once he realized it would alienate readers unfamiliar with baseball (leaving in a smaller, less risky bowling reference instead), illustrating his broader concern for the specific reader's experience over the writer's own attachment to material.

On theoretical computer science, Knuth explains why he intuits (without proof) that P may equal NP, yet insists this would carry little practical weight even if proven — because the overwhelming majority of the space of possible algorithms is incomprehensibly vast, so an existence proof gives no route to actually finding a usable algorithm, a point he illustrates via the Robertson–Seymour graph minor theorem (which guarantees a polynomial-time algorithm exists for testing membership in any minor-closed graph family, without telling you what that algorithm is) and via combinatorial games like Hex (where a winning strategy for one player is provable to exist purely from the impossibility of a draw, without anyone knowing what it is). He is skeptical of contemporary AI/deep learning specifically because it produces results without anyone — including its own builders — understanding what has actually been learned, contrasting this with his own preference for techniques (like literate programming or BDDs) that yield genuine insight into why something works; he states he personally finds the pursuit of "echoes of intelligence" in algorithms less captivating than most of his field does, worrying more about people mistaking the appearance of understanding for the real thing than about AI's ultimate limits.

On mortality, meaning, and religion, Knuth describes the month-long grief process after his father's death that temporarily made him unable to sing or do original research, and reflects on his 2006 prostate cancer diagnosis as a moment he made peace with an ending he felt he had no right to complain about, given a life of good fortune — he had, at that point, only one unfulfilled personal goal (composing an original piece of music), which he finished and premiered on his 80th birthday, after which the only agenda item left is finishing The Art of Computer Programming. He describes a personal philosophy of "point eight is enough" — the idea that a well-designed organism should be happy roughly 80% of the time, since 100% happiness would be dysfunctional — while noting his TeX/typography work, where he sought genuine excellence for its own sake, pushed him closer to "point nine eight." Discussing his 1999 MIT lectures on religion (based on a rigorous, randomly-sampled study of Bible verses), he says he deliberately treats God's existence as something he prefers to leave unproven, since a definitive proof would foreclose an ongoing source of meaning and mystery in his life, and connects this directly to his comfort with deep unsolved problems generally: he finds genuine joy in incremental progress on things he'll never fully understand (P vs NP, the nature of cognition, the Bible) rather than needing final answers.

Excerpts

On the two traits he thinks define "geek" thinking:

“One is this ability to jump levels of abstraction, so you see something in the large and you see something in the small, and can you pass between those unconsciously... The other thing is that it's more of a talent to be able to deal with non-uniformity, where there's case one, case two, case three, instead of having one or two rules that govern everything.”

On why proving P=NP wouldn't hand anyone a usable algorithm:

“Many more algorithms exist than anybody can understand or ever discover... we have situations now where we know that an algorithm exists, but we don't have the foggiest idea what the algorithm is... I have to rule out so many possible algorithms that have not been working, and that's the reason for my intuition, but by no means a proof.”

On his "point eight is enough" philosophy of happiness:

“I say point eight is enough, which means — if I can have a way of rating happiness — I think it's good design to have an organism that's happy about eighty percent of the time, and if it was a hundred percent of the time, it would be like everybody's on drugs, and everything collapses, nothing works, because everybody's just too happy.”