George, Dileep

co-founder of Vicarious with Scott Phoenix; formerly co-founder of Numenta with Jeff Hawkins and Donna Dubinsky; researcher at the intersection of neuroscience and AI

Dileep George: Brain-Inspired AI

Lex Fridman Podcast · Aug 14, 2020

• Dileep George
• brain-inspired AI
• Recursive Cortical Network
• cortical microcircuits
• feedback connections
• CAPTCHA
• GPT-3
• episodic memory
• Vicarious
• probabilistic graphical models

Core ideas

George argues that understanding the brain functionally is a prerequisite to building one, contrasting his approach with Henry Markram's Blue Brain Project, which simulates highly detailed biophysical neuron models (Hodgkin-Huxley-style ion-channel dynamics) wired according to observed connectivity statistics without a governing theory of what the resulting circuit is supposed to compute. He compares this to modeling a single transistor perfectly and then wiring millions of them together by guesswork, hoping a working microprocessor emerges without ever understanding Boolean logic — arguing you need a functional, information-processing theory first, built by taking neuroscience's hypothesis-driven experimental findings (which he calls genuinely rigorous and detailed, not random probing) and synthesizing them into working computational models, which can then be tested, refined, and fed back to generate new neuroscience hypotheses in an iterative loop.

A central theme is that the brain's visual cortex is not a simple feed-forward cascade (as popularized by deep learning intuitions) but has far more feedback connections than feed-forward ones, implementing a continuous inference process: the brain builds and projects an internal model of the world onto incoming sensory evidence, and perception is the resulting best explanation reconciling model and evidence — illustrated by experiments showing edge/contour representations settle before interior-surface representations in figure-ground segregation, and by the Kanizsa triangle illusion, where neurons signaling illusory (hallucinated) edges fire later than neurons signaling real edges, consistent with the illusory signal arising from feedback rather than direct input. He explains this using probabilistic graphical models and the concept of "explaining away" (e.g., a burglar alarm's evidence for a burglary is explained away by news of an earthquake) as the mechanism by which competing causal hypotheses in the brain resolve — hypothesized to be implemented via cortical columns (each roughly encoding a variable/concept, not necessarily human-interpretable) connected to each other and to the thalamus, detailed in an upcoming paper on cortical microcircuits.

George describes his and his team's Recursive Cortical Network (RCN), published in Science, as a functional vision model built from these neuroscience-derived principles: unlike convolutional neural networks (which share translation-invariant weights everywhere, something the fovea-based, non-uniform biological visual system doesn't actually do), RCN factorizes scenes into separate models of foreground/background and shape/texture, uses lateral connections between nodes to enforce global coherence across local feature detectors, and performs genuine dynamic inference rather than "amortized" inference baked in via massive training data — demonstrated by cracking text CAPTCHAs (a problem chosen specifically because its whole design goal is resisting exactly the kind of brute-force pattern learning that deep nets excel at) using orders of magnitude less training data than deep learning approaches, and by strong few-shot performance on inverted MNIST benchmarks. He recounts wry frustration that despite the paper explicitly arguing for graphical models and inference over deep learning, Science's own press office described it in a release as "a new deep learning model" — illustrating how hype dynamics distort science communication regardless of researchers' intent, and defends brain-inspired AI as a serious research program (not marketing) while cautioning that fidelity to biology must sometimes be deliberately abandoned for good engineering (citing convolution itself as a useful trick absent in the actual, foveated, non-translation-invariant visual cortex).

On language and reasoning, George argues cognition can't be reduced to text-based pattern completion: answering a question like "was the nail horizontal or vertical?" after being told "Sally pounded a nail into the ceiling" requires mentally simulating the physical scene using grounded, pre-verbal concepts built from lifelong sensorimotor experience — not information recoverable from text alone, since almost no one writes down the ordinary-seeming physical facts (e.g., "it's easier to walk forward than backward") a language model would need. He explicitly rejects Chomsky's view of language as foundational to cognition, arguing visual/motor concept learning is a prerequisite for real language understanding, and frames GPT-3-style models as sophisticated statistical models of "the text world" rather than models of the physical world — comparing extremely large transformers to a more sophisticated Markov chain of English text (per Shannon), useful and impressive but architecturally incapable of causal/counterfactual reasoning or maintaining consistent world-state, a limitation he expects to persist even as scale increases. He describes his own program (schema networks for controllable world-dynamics modeling, then a "cognitive programs" system for pre-verbal concept learning from action sequences, and now early work grounding simple language in these simulations) as building toward that missing piece, and separately outlines a hypothesis for episodic memory in which the hippocampus stores an indexed timeline of one-off experiences that can be "replayed" back into the cortex's statistical/causal world model to evaluate similarity to new situations (a proposed mechanism for déjà vu).

On brain-computer interfaces, George considers near-term goals (prosthetic limb control, restoring function for quadriplegics) reasonable and promising, but expects safety and long-term biological unknowns (how the brain adapts to implanted electrodes) to be harder problems than the communication-bandwidth engineering itself, and favors an approach where the brain adapts to a fixed external interface (much like it adapts to a keyboard) rather than attempting bidirectional co-adaptation between brain and machine. Asked about consciousness, he connects it to Vicarious's namesake concept — an organism's world-modeling apparatus, originally built to simulate the external world or other people's minds, turned reflexively on itself — and argues human mortality-driven urgency has no logical necessity for artificial systems, since AI systems (unlike humans) can in principle be copied without losing their accumulated experience, undercutting the assumption that fear of death is a fundamental prerequisite for general intelligence. He closes recommending Judea Pearl's Probabilistic Reasoning in Intelligent Systems, Hofstadter and Dennett's The Mind's Eye, and a history of the Wright brothers, using the latter to make the point that studying how birds fly (specifically, observing that soaring birds solve control before propulsion) was directly useful to solving powered flight — a rebuttal to the common claim that "planes don't flap their wings" proves nature-inspired engineering doesn't work.

Excerpts