Bostrom, Nick

philosopher at Oxford; director of the Future of Humanity Institute; author of "Superintelligence"

Nick Bostrom: Simulation and Superintelligence

Lex Fridman Podcast · Mar 26, 2020

• Nick Bostrom
• simulation argument
• simulation hypothesis
• Doomsday Argument
• anthropic reasoning
• superintelligence
• AI alignment
• technological maturity
• experience machine
• existential risk

Core ideas

Bostrom distinguishes the "simulation hypothesis" (the literal claim that we live inside a computer simulation run by an advanced civilization) from the broader "simulation argument," which shows that at least one of three propositions must be true: (1) almost all civilizations at our stage of development go extinct before reaching "technological maturity"; (2) civilizations that do reach maturity lose interest in running large numbers of detailed "ancestor simulations" (simulations containing conscious beings with human-like experiences); or (3) we are almost certainly living in a simulation. He argues technological maturity is a coherent, roughly attainable ceiling — pointing to Eric Drexler's theoretical work on molecular nanotechnology and known physical limits on space travel as evidence we can already characterize a lower bound on what a mature civilization could achieve, computationally and otherwise — and that the timescale to reach it (decades vs. millions of years) doesn't affect the argument's logical structure, only how probability is distributed among the three alternatives. He suggests that reaching technological maturity plausibly requires developing machine superintelligence first, which could itself radically reshape a civilization's motivations (for instance, gaining direct control over one's own mental states might eliminate many of today's indirect, instrumentally-driven behaviors) in ways that make predicting whether mature civilizations "lose interest" in simulating ancestors genuinely uncertain.

On the philosophy-of-mind premises underlying the argument, Bostrom holds that consciousness likely depends on computational structure rather than substrate (biological neurons vs. silicon), and that a simulation replicating a human brain's neuron-level computation would very plausibly be conscious, though how much simplification is possible remains an open question; he further notes that a simulated environment likely wouldn't need to render unobserved details (analogous to a video game only rendering what's in view) to seem fully real to its inhabitants. Discussing Robert Nozick's "experience machine" thought experiment, he explores why people generally wouldn't choose to permanently enter a perfect pleasure-generating simulation, suggesting this reveals that people value real-world connection, consequence, and identity-continuity beyond mere subjective experience — while noting a status-quo bias may explain some of this reluctance, and that adding richer simulated relationships and real-world consequences to the thought experiment makes it progressively more appealing, converging toward ordinary reality. He and Lex debate whether a very convincing simulated conversational partner would necessarily require instantiating a second genuine consciousness, with Bostrom suggesting that sufficiently rich, open-ended interaction (unlike a simple pre-recorded response) might be very hard to fake without actually generating something with its own subjective experience.

Bostrom explains the "bland principle of indifference" used in the final step of the simulation argument: given a set of observers with subjectively identical experiences, some simulated and some not, and no internal way to distinguish which you are, you should assign probability proportional to the relative sizes of those groups — a much weaker and less controversial assumption, he argues, than the "self-sampling assumption" underlying the Doomsday Argument (which he also explains in detail, using an urn analogy: your unusually early birth-rank among all humans who will ever live is evidence, by the same logic used to update on a low-numbered ball drawn from an urn, that humanity's total population will be smaller rather than astronomically larger). He notes the Doomsday Argument's stronger assumption (treating yourself as a random sample from all humans who will ever exist) produces a widely-doubted but hard-to-refute conclusion, whereas the simulation argument's weaker assumption is less objectionable, and offers a limiting-case intuition: if literally 100% of observers with your experiences were simulated, you could deduce with certainty you're simulated, so the probability should scale continuously as that fraction approaches 100% rather than jumping discretely. He also observes, half-playfully, that a particularly unusual or consequential person (naming Elon Musk as example) might have modestly elevated reason to suspect they're in a simulation, if simulators are more likely to run simulations centered on remarkable individuals.

Turning to superintelligence, Bostrom defines it as general cognitive capacity — faster learning, better reasoning, more effective planning — far exceeding humans', and states that whether it need be physically embodied is unimportant since even limited actuators (like text output) could allow enormous real-world impact. He stresses that his 2014 book "Superintelligence" intentionally emphasized risks because the potential upsides were, at the time, comparatively neglected, and clarifies that near-term AI concerns (algorithmic bias, self-driving cars) and long-term concerns (transformative or existential outcomes from AGI) are different conversations often unhelpfully conflated. He expresses a personal view that some period of rapid, "explosive" AI progress around human-equivalent cognitive ability is fairly likely, considers it very unlikely that human-level cognition represents any kind of hard ceiling, and pushes back on the intuition that surpassing human intelligence must mean humans losing control — noting that maintaining alignment between superintelligent systems and human values, not merely "losing control," is the actual technical challenge that AI safety research aims to solve. He also relativizes the sense of losing our unique cognitive status, comparing collective structures like the global scientific community or search engines (superhuman in narrow retrieval tasks but "zero" elsewhere, much like a calculator or Deep Blue) to how a general, self-improving learning system like AlphaZero already feels intuitively "more intelligent" due to its domain-general learning capacity rather than narrow performance alone.

Asked to envision a positive long-term outcome, Bostrom argues AGI is best understood as a general-purpose technology with no single "killer app," capable of amplifying human capability across virtually every field, though it wouldn't automatically resolve conflicts requiring political coordination rather than raw material abundance. He suggests that the vastly expanded "option space" opened by mature AI-enabled technology should prompt fundamental first-principles rethinking of what humans actually value, and proposes that instead of assuming irreducible value trade-offs, humanity should first aim to find outcomes that score highly across multiple different ethical and value frameworks simultaneously (rather than immediately picking one framework and treating the rest as zero), since a much larger space of possible futures may make such broadly-satisfying outcomes newly attainable. He closes by distinguishing happiness from meaning (a pleasurable but unreflective activity like watching TV reruns may carry little meaning, while a difficult, self-sacrificial achievement may carry great meaning with little immediate pleasure), suggesting that navigating a future of radical abundance will require clearly distinguishing these different sources of value before trying to combine them.

Excerpts