Hawkins, Jeff

founder of the Redwood Center for Theoretical Neuroscience and Numenta; author of On Intelligence; creator of Hierarchical Temporal Memory and the Thousand Brains Theory of Intelligence

Jeff Hawkins: Thousand Brains Theory of Intelligence

Lex Fridman Podcast · Jul 1, 2019

• Jeff Hawkins
• neocortex
• Thousand Brains Theory
• hierarchical temporal memory
• reference frames
• cortical columns
• sparse representations
• biological vs. artificial neurons
• AI existential risk
• consciousness and qualia

Core ideas

Hawkins states his primary interest is understanding the human brain itself, not merely building intelligent machines by any means, but argues these goals are inseparable: he believes true machine intelligence is unreachable without first understanding the brain's actual operating principles, making neuroscience the fastest path to AI rather than a detour from it. He rejects the idea that there's a fundamental "wall" beyond which the brain can't be understood, pointing to historical precedent (vitalism's collapse once "living vs. non-living matter" stopped seeming mysterious) and states — notably, on record — that he no longer considers the neocortex's operation an open mystery in the way it was even five years earlier, crediting a specific theoretical breakthrough his small research lab made roughly three years before the interview.

He describes the neocortex (70-75% of human brain volume, uniquely large relative to other mammals, and the substrate for all high-level perception and cognition) as strikingly uniform in structure across sensory modalities and even across species — an observation first articulated by Vernon Mountcastle in 1978, supported by experiments showing that rewiring the optic nerve to an auditory brain region turns that region visual, and that a blind person's dormant visual cortex gets repurposed for other tasks. This uniformity implies a single "common cortical algorithm" underlies vision, hearing, touch, language, and mathematics alike — meaning discovering what that shared circuit does would explain all cognitive functions at once, rather than requiring separate theories per domain. Hawkins frames his research approach as fundamentally constraint-driven rather than speculative: because a century of neuroscience data acts as a dense set of empirical constraints a correct theory must satisfy simultaneously, a theory that satisfies all of them at once carries strong evidentiary weight almost by mathematical necessity — and he compares the resulting "aha" experience to Copernicus, Darwin, and Watson and Crick's discovery of DNA's structure (recounting a personal anecdote about presenting his theory to Watson himself at Cold Spring Harbor).

He traces his earlier Hierarchical Temporal Memory (HTM) framework as emphasizing three underappreciated aspects of brain function: time (the brain fundamentally processes continuously changing patterns, not static snapshots — unlike, he argues, most classical vision theories and even convolutional neural networks, which process fixed images), memory (the brain builds and stores an actual predictive model of the world rather than just processing input), and hierarchy (regions of the uniform neocortex are organized in rough hierarchical connectivity). He now considers HTM's vocabulary insufficient to capture what his lab has since learned, in favor of the newer Thousand Brains Theory.

The core discovery underlying the Thousand Brains Theory, per Hawkins's own account, arose from a simple introspective observation: predicting what his finger would feel when moving across a coffee cup requires the brain to know both what object it's touching and the finger's location relative to that object's own reference frame (not the body's) — implying every patch of cortex processing sensory input must maintain object-centered reference frames, essentially like a CAD engineer's coordinate system. This led to the theory that the brain does not process sensory data as a pipeline extracting successively abstract features (the deep-learning model), but instead constantly assigns all input to reference frames, with thousands of these frames active simultaneously. Crucially, each cortical column (roughly a "spaghetti-strand"-sized unit, of which there are about 150,000) can, given enough time and movement, build a complete model of an entire object from just its own local, partial sensory access — meaning many redundant, semi-independent models of any given object (a coffee cup, a cell phone) exist simultaneously across the cortex.

This resolves the classical "sensor fusion problem" (how disparate sensory streams recombine into one unified percept) by rejecting its premise: rather than models merging into one central representation, Hawkins argues thousands of local columnar models simply vote via long-range cortical connections, using an associative mechanism (not full Bayesian probability, but closer to intersecting sets of live possibilities) that rapidly "crystallizes" onto a single consensus interpretation once enough columns agree — at which point the whole system acts as if it definitively knows what it's perceiving, even though no single column ever directly experienced the whole object. Hawkins argues this same reference-frame mechanism likely extends far beyond physical objects into abstract thought and language, since cortical regions processing language, mathematics, and other high-level cognition look anatomically similar to sensory regions — a claim he supports with an fMRI study showing subjects reasoning about abstract bird categories exhibit "grid cell"-like activity patterns structurally identical to those used for physical spatial navigation. He extends this to mathematical reasoning itself, framing the act of proving a conjecture as literally navigating a path through a mathematical reference frame from known axioms toward a target result, and connects it to the classical "method of loci" memory technique (mentally placing items around a familiar physical space), arguing this technique works precisely because it exploits the brain's underlying preference for reference-frame-based storage and recall rather than being a mere mnemonic trick.

He describes ongoing open problems his lab is still working through — most notably how the brain integrates orientation (not just location) information into these reference frames, a problem with an analog in "head direction cells" from rodent neuroscience — and describes an "attentional" mechanism for handling context and compositional structure: an object like a coffee cup is itself a component nested within larger reference frames (a table, a room), and attention lets a person move fluidly up and down this nested hierarchy of reference frames without losing surrounding context, a process he says he now finds himself consciously monitoring during ordinary moments (walking to the bathroom at night, deliberately testing his own dead-reckoning/grid-cell accuracy).

On artificial neural networks, Hawkins credits deep learning's real practical successes while arguing today's "point neurons" are a crude, non-biological simplification: real neurons have 5,000-30,000 synapses, of which only a small fraction (near the cell body) can directly trigger firing, while the vast majority (far out on dendrites) instead detect coincident patterns of roughly 10-20 simultaneously active synapses within a tiny dendritic region, producing "dendritic spikes" that slightly depolarize the neuron — priming it to fire fractionally sooner than it otherwise would, which triggers competitive lateral inhibition via fast inhibitory neurons, ultimately producing sparse population codes rather than backpropagation-trained weights. He argues this makes every real neuron an intrinsic predictive/temporal engine (dozens to hundreds of recognized input patterns per neuron) rather than a static summing unit, and that learning in the brain proceeds via synaptogenesis (forming entirely new, localized synaptic connections on specific dendritic segments) and "silent synapses" flipping to active (enabling near-instant short-term memory formation) — a fundamentally Hebbian, error-propagation-free mechanism ("cells that fired together wire together") incompatible with backpropagation, and one that allows continuous, simultaneous learning and inference (unlike most current deep networks, which separate training and inference into distinct phases).

Hawkins pushes back on Richard Sutton's "bitter lesson" thesis (that scale and compute alone, not clever hand-engineered priors, ultimately win in AI) by arguing that simply scaling up today's convolutional architectures cannot reach the kind of intelligence brains exhibit, since current large networks remain qualitatively similar to earlier, smaller ones. He describes Numenta's own pragmatic, incremental strategy for bridging brain theory into mainstream machine learning (calling it "the entrepreneur's dilemma" — having a long-term vision but needing viable interim steps): first, enforcing sparse representations and sparse connectivity within convolutional networks, which his team empirically showed increases robustness to adversarial examples (since sparse, near-orthogonal population codes require an attacker to alter far more units to cross a decision boundary, unlike dense representations vulnerable to small perturbations); and second, exploring "thousand brains"-style voting among multiple simple sub-models as a route toward faster-learning, more multimodal, more robust systems. He is critical of the machine learning field's benchmark culture, arguing standard computer vision/NLP benchmarks optimize for narrow, often compute-and-data-intensive competitive metrics that don't capture — and sometimes actively work against — the properties (continuous learning, robustness, sample efficiency) his brain-based approach targets, and expresses mild disappointment with "human-centered AI" institutional initiatives (at MIT and Stanford) for potentially conflating human-AI interaction design with the more fundamental question of what intelligence actually is.

On AI risk, Hawkins is skeptical of existential-risk framings from figures who, in his view, lack a grounded theory of what intelligence actually is; he argues intelligence (a property of the neocortex specifically) can and should be built without also building the old brain's emotional, self-replicating, or survival-driven machinery — comparing the more genuinely existential threat of engineered pathogens or antibiotic-resistant bacteria (already-existing, self-replicating, uncontrollable threats) favorably (in terms of being a known, already-managed category of risk) against speculative superintelligence scenarios, arguing that adding "intelligence" to a system actually tends to constrain its potential harm compared to purely self-replicating biological threats, since intelligent systems don't need to want survival or reproduction unless deliberately designed to. He argues genuine general "human-level intelligence" is a poorly framed goal in the first place — the aim instead should be identifying intelligence's core principles so they can be applied at any scale, in any physical form or distributed configuration — but does insist some form of embodiment (movement through physical space, mathematical space, or even a network of hyperlinks) is a necessary component of any system that builds genuine reference-frame-based models of its domain.

On consciousness, Hawkins separates self-awareness (which he considers straightforwardly explicable — a system with a body model plus continuous memory of that body's trajectory through time and the ability to recall it) from qualia (the harder, unresolved question of why sensations like color or pain feel like anything at all, illustrated with the phantom-limb phenomenon as evidence that even bodily sensation is fundamentally a brain-constructed model rather than a direct readout of reality). He states plainly that he doesn't consider qualia relevant to building intelligent machines and doesn't need to resolve it for his research program to succeed, while remaining personally curious about it. Closing on humanity's legacy, Hawkins argues what's genuinely unique about humans isn't biology or genetics (no more special than other primates') but accumulated knowledge extending beyond direct individual sensation (e.g., knowledge of the far side of the moon or other galaxies), and frames "estate planning for humanity" — preserving and extending that knowledge via intelligent machines that could outlast the species and even the solar system — as what he considers the most meaningful legacy project available, expressing confidence that intelligent life has almost certainly existed elsewhere in the universe's history even if it's rare or short-lived per civilization.

Excerpts