Chollet, François
▶ Watch on YouTubeCore ideas
Chollet opens by tracing his intellectual influences — developmental psychologist Jean Piaget (intelligence as something constructed by children in stages, not fixed) and neuroscientist Jeff Hawkins (the mind as a multi-scale hierarchy of prediction/compression modules) — before developing a view of language as an "operating system" layered on top of cognition rather than cognition's foundation: a tool for querying and even reprogramming one's own memory, with pre-linguistic thought (in babies) likely grounded in visual/motion-based mental representations. He draws a distinction, borrowed loosely from math, between topological thinking (are two concepts connected, like a subway map) and the geometric/vector-distance thinking that deep learning is forced into (since differentiability requires continuous space), suggesting the brain may rely more on the former than current neural networks do.
The episode's core is Chollet's formal definition of intelligence, elaborated from his paper "On the Measure of Intelligence": intelligence is not skill or knowledge itself but the efficiency with which a system acquires new skills at genuinely novel tasks it wasn't specifically prepared for — the ratio between how much new "problem space" a system can competently cover and how little prior knowledge plus experience it needed to get there. He sharply distinguishes the process that produces a skill (intelligence) from the output artifact of that process (a program, or a trained model), using the analogy of a road-building company versus one specific road: a chess program written by a smart human encodes that human's intelligence but is not itself intelligent, since it cannot adapt to a new domain like Go. This grounds his critique of GPT-3: while genuinely novel in its apparent few-shot task learning, he argues it's more likely pattern-matching new prompts against similar patterns already present in its enormous training corpus than developing an actual model of a task, evidenced by its sensitivity to adversarial rephrasing and lack of self-consistency or factual grounding — and he predicts scaling GPT-family models further will improve textual plausibility but not fix this, since the real bottleneck is available training data (already near "the entire web"), not compute or parameter count. He similarly argues full self-driving is fundamentally a scale/data problem rather than a test of intelligence, since sufficiently exhaustive training data could in principle brute-force it without any generalizable understanding — contrasting this with a human's ability to "zero-shot" driving on the opposite side of the road in a foreign country.
Chollet lays out a hierarchy of generalization — robustness (handling known unknowns within an anticipated distribution), flexibility (handling unknown unknowns within one fixed domain, e.g. the long tail of self-driving edge cases), and extreme generalization (adapting across arbitrary, unrelated domains, as humans can learn gardening after being "trained" on cooking) — and situates human intelligence as an instance of extreme generalization, while stressing (via the "physical fitness" analogy: fit humans still can't fly or breathe underwater) that human generality, though remarkable, is not universal but a byproduct of evolving in a highly unpredictable environment. He credits Elizabeth Spelke's "core knowledge" theory (four innate/early-acquired priors: objectness, agentness/goal-directedness, basic geometry/navigation, and small-number sense) as the foundation for his ARC (Abstraction and Reasoning Corpus) benchmark — a set of ~1,000 novel visual grid puzzles designed so solving them requires only these core priors (no language, no cultural knowledge), with a hidden private test set specifically to prevent developers from simply hard-coding solutions to known tasks, which would just relocate the intelligence into the human engineer rather than the system. He reports that after its Kaggle competition debut, ARC performance moved from 0% to roughly 20% within limited time, which he considers evidence the benchmark resists shortcuts while still being tractable — unlike the Turing test, which he rejects as scientifically unproductive because it outsources the definition of intelligence entirely to potentially unreliable, biased human judges, rewards deceptive "parlor tricks" over genuine progress (comparing it to stage magic), and targets human-likeness, which he considers the very last milestone of generally intelligent AI rather than a useful early one.
He also rejects the popular idea (associated with Marcus Hutter's compression-based intelligence prize) that cognition simply is compression: while compression is a useful tool the mind employs for efficiency, true cognition must retain seemingly "useless" information and diverse behavioral options precisely because it must operate under future uncertainty that a maximally compressed model of the past would discard — analogous to a diversified investment portfolio being wiser than a strategy purely optimized on historical stock data. On brain-computer interfaces (Neuralink), Chollet is skeptical that raw input/output bandwidth is actually the bottleneck limiting human cognition, arguing that culture, language, and external tools (books, Wikipedia, computers) already function as scalable "externalized cognition," while the brain's own processing speed (not bandwidth) is closer to its true limiting constraint. He closes on meaning: since even our most personal thoughts are built from concepts, words, and ideas inherited from others, he frames human identity as fundamentally cultural, and argues that any positive contribution — an idea, a piece of art, an act of kindness — propagates as "ripples" into future minds indefinitely, making that propagation itself a form of immortality and, in his view, the closest thing to a meaning of life.
Excerpts
On why intelligence must be measured as a process, not an output:
“The intelligence is the process of developing that program, but the program itself is just encoding the output artifact of that process — the program itself is not intelligent... we should not confuse the output and the process. It's the same as, do not confuse a road-building company and one specific road.”
On why the Turing test rewards the wrong thing:
“If your goal is to trick, to convince a panel of human judges that they're talking to a human, then you have an incentive to rely on tricks... it's not physics, it's not making any progress in understanding of the universe... being very human-like, being indistinguishable from a human, is actually the very last step in the creation of machine intelligence.”
On why cognition cannot simply be compression:
“Cognition is about being able to operate in future situations that include fundamental uncertainty and novelty... a good model of the world is going to include all kinds of things that are completely useless, actually, just because just in case — because you need diversity in the same way that in your portfolio you need all kinds of stocks that may not have performed well so far.”