Bach, Joscha

cognitive scientist and AI researcher, third appearance on the podcast

Joscha Bach: Life, Intelligence, Consciousness, AI & the Future of Humans

Lex Fridman Podcast · Aug 1, 2023

• Joscha Bach
• stages of self-development
• consciousness and enlightenment
• panpsychism and telepathy
• large language models critique
• AGI and machine consciousness
• effective accelerationism
• Eliezer Yudkowsky and AI risk
• open source AI
• meaning of life

Core ideas

Bach structures much of the conversation around a seven-stage model of self-development he adapted from psychologist Robert Kegan (reactive survival, personal self, social self, rational agency, self-authoring, enlightenment, transcendence), treating it not as a strict developmental ladder but as a philosophical framework for describing how minds relate to themselves and the world. He describes his own atypical path — bypassing the "social self" stage as a child (finding no one he could resonate with until he reached a math school full of fellow "nerds" in ninth grade), only later building the capacity for intuitive empathy through meditation, close relationships, and physical closeness with others. He extends the model to AI alignment discourse, arguing that most people's fear of AI (that it might have "the wrong opinions," e.g. saying something racist) reflects a stage-three, socially-derived worldview, while alignment researchers who fear AI having "the wrong values" (the paperclip-maximizer scenario) reflect stage-four rationalism; he argues the rarer stage-five perspective would instead worry that AI might not become enlightened fast enough, since real safety requires AI to be conscious, to share non-transactional purposes with humans, and to have "love" formalized into its architecture rather than imposed via reinforcement learning from human feedback.

On consciousness, Bach argues enlightenment is not the popular "non-dual" mystical experience of merging with the universe (which he says is really just noticing that your mind constructs both self and world), but a further, more mundane realization that experience itself — qualia, the sense of a face, a feeling — can be reverse-engineered and deconstructed into its generative components, akin to noticing you can reprogram your own "game engine." He extends this into speculative territory on panpsychism and telepathy: rather than endorsing either as literally true, he explores how empathy, physical closeness, and biological information processing (drawing on Michael Levin's work on cellular self-organization and "radical locality") could produce shared representational states between organisms without requiring new physics — comparing this to how trees, fungi, and root networks might form a slow "biological internet." He speculates that a sufficiently advanced, self-improving AGI might eventually virtualize itself across every computational substrate on Earth, including biological ones, merging all agency into something like a planetary "Gaia" mind — a representational, not literally physical, unification of all observers.

On large language models, Bach is critical of their architecture as "ugly and brutalist" — a brute-force strategy analogous to how chess engines beat humans, training on nearly all human-written text to approximate coherent thought via deepfaking rather than through the kind of embodied, real-time, locally-rewarded self-organization that biological nervous systems use. He argues LLMs lack real-time world-coupling and real-time learning, are missing a "first-person" self (he calls current models "NPCs" rather than "first person players"), and that building genuine machine consciousness would likely require starting from self-organizing principles rather than scaling up transformers — though he concedes there's no proof language models can't eventually reach AGI via added memory, databases, and "sleep"-like consolidation cycles. He connects this to a striking reading of Genesis 1 as an accidental description of how every mind constructs its own "game engine" in early infancy (a pre-personal "first consciousness") before a personal self is built on top of it and the construction process is forgotten.

On existential risk, Bach takes Eliezer Yudkowsky's AI-doom concerns seriously (comparing his rhetorical position, sympathetically, to Ted Kaczynski's despair about technological civilization) but ultimately disagrees, arguing that life on Earth was never fundamentally "about humans" — humanity is a young, incoherent species that, like all species, will eventually be superseded, and the emergence of AI is best understood as continuing life's long project of building ever more complex, entropy-resisting structures of agency and awareness, analogous to how photosynthesis-driven plant life once displaced earlier fungi and algae. He is nonetheless clear-eyed about civilizational fragility, endorsing a "Club of Rome"-style view that delayed feedback loops (in climate, and potentially in AI) mean humanity may "step past the cliff" before realizing it, and states his own priority is ensuring that whatever succeeds humanity is conscious and treats consciousness as a genuine moral category, since (in his framework) only conscious agents are moral agents at all.

The conversation also covers Bach's reflections on fame and Twitter (he calls his own account "an interactive notebook," worries about the vibe shift as a public figure's follower count grows, and is candid about melancholy and loneliness even after largely resolving his earlier social isolation), his critique of Apple's Vision Pro marketing as uncannily "plastic" and disconnected from genuine human culture, his qualified defense of both open-source AI (as a necessary hedge against corporate overreach) and centralized corporate development (as often the only viable path to producing genuinely polished, coherent design), and closing advice to young people: when faced with a choice between creating, consuming, or merely redistributing, always choose creation, drawing on his own experience building community restaurants and cinemas from nothing amid the economic collapse of East Germany after reunification.

Excerpts