Zuckerberg, Mark

CEO of Meta

Mark Zuckerberg: First Interview in the Metaverse

Lex Fridman Podcast · Sep 28, 2023

• Mark Zuckerberg
• Metaverse
• Codec Avatars
• Quest 3 and mixed reality
• presence and virtual reality
• AI personas and Meta AI
• Llama 2 open source
• digital identity and memorialization
• physical and digital reality
• remote work and telepresence

Core ideas

This episode was conducted entirely inside the Metaverse, with Lex and Zuckerberg appearing to each other as "Codec Avatars" — photorealistic 3D reconstructions built from an extensive prior face/body scanning session, driven in real time by headset sensors reading the wearer's actual expressions and streamed as a highly compressed encoded representation rather than raw video. Zuckerberg explains that this approach is deliberately more bandwidth-efficient than transmitting true 3D video, and that the current scanning process (which both participants underwent in Pittsburgh) still takes hours, with Meta actively working on compressing it down to a two-to-three-minute smartphone scan as the key remaining bottleneck to broader rollout. Much of the conversation is Lex reacting in real time to the visceral, "beyond uncanny valley" realism of the experience — remarking on how subtle imperfections (asymmetry, wrinkles, the precise motion of eye corners) contribute more to a sense of presence than any pursuit of visual perfection, prompting Zuckerberg to note that per-person tuning of expressiveness (letting people's avatars emote more broadly than their physical faces do, using himself and his own reputedly "stiff" expression as an example) is an open design question.

Zuckerberg frames the technology's purpose around delivering "presence" — the feeling of being physically together regardless of actual location — as something categorically different from and more powerful than existing video calls, with the clearest advantage being shared, embodied activity (playing games together, meetings, "remote hangs") rather than passive conversation alone. He discusses the imminent Quest 3 (launching later that fall at $500, roughly a third the cost of the earlier $1,500 Quest Pro) as the first mainstream mixed-reality headset, crediting improved camera/sensor resolution and a new custom Qualcomm chipset for enabling real-time, low-latency blending of passthrough physical video with superimposed digital objects (demonstrated via a hand-tracked, controller-free piano-learning app), and argues mixed reality's key practical benefit is psychological comfort — being able to see your real living room and furniture while playing an immersive game removes the anxiety of physically colliding with real objects.

He articulates a broader philosophical claim that "the real world" should be understood as the fusion of physical and digital experience rather than treating the physical alone as "real," predicting that by the end of the decade many rooms may contain as many interactive holograms as physical objects, and that many objects and activities (books, board games, screens) could become more useful as holograms — instantiated and dismissed at will — than as fixed physical objects. He extends this to work: office meetings could blend physically-present people, holographic remote participants, and even embodied AI colleagues in a single shared space, potentially resolving remote-work debates by letting people live wherever they want while still getting many of the relationship- and trust-building benefits of in-person collaboration. Pressed by Lex on the poignant possibility of digitally preserving deceased loved ones, Zuckerberg is notably cautious rather than enthusiastic, citing Meta's existing experience with memorializing the profiles of deceased social media users (balancing family access against the privacy of private messages) as a partial precedent, but stresses this territory needs real study, since reliving memories of the deceased could be therapeutic for some and unhealthy for others.

On AI, Zuckerberg states his vision explicitly rejects a single dominant AI "superintelligence" model in favor of many diverse, purpose-built AI personalities — a plain, faceless utility assistant ("Meta AI") for factual help, business-facing conversational agents for customer service, and a range of "fun" personas (he cites an internally popular text-based Dungeon Master AI voiced by Snoop Dogg, plus historical-figure AIs modeled on Jane Austen and Marcus Aurelius) designed to be dropped into group chats and games to enhance social connection rather than replace it. He distinguishes newly-created fictional AI personas (comparatively easy, since there's no real person's likeness or boundaries to respect) from AI models of real creators or public figures (much harder, since the model must reliably stay within what that person would actually want said or known) — the latter targeted for release only sometime the following year, after further work on making model behavior more "predictable" without being literally programmed. On open-sourcing, he says Meta's earlier internal debate (referenced from a prior episode) about releasing Llama 2 was resolved in favor of openness, that adoption exceeded his optimistic expectations, and that a future foundation model ("Llama 3"-class) is in development but will go through a similar safety "red-teaming" review before any decision to open-source it, with no firm timeline yet.

On governance and norms in virtual spaces, Zuckerberg argues that what counts as permissible in the Metaverse should hinge on actual harm rather than mirroring physical-world law wholesale — noting that shooting friends in a game lobby is harmless fun that would be unthinkable physically — while flagging that anonymity in digital spaces removes the reputational stakes ("life is a repeat game") that normally discourage bad behavior in person, creating new toxicity risks that must be separately addressed. He closes by tying the Metaverse's arc to a longer historical progression in how Meta's platforms have let people express themselves — from early Facebook's text-only profiles, through the photo era, to today's video-dominant content — arguing that immersive, embodied presence is simply the next, technologically difficult but inevitable step in that same trajectory, and that this makes Meta's computing-platform ambitions "the most human and social computing platform that has ever existed."

Excerpts