Karaman, Sertac
▶ Watch on YouTubeCore ideas
Karaman argues autonomous flight is easier than autonomous driving at small/consumer scale (explaining its earlier commercial takeoff) but far harder at the large scale needed for real transportation/logistics impact, because the core unsolved challenge across all robotics isn't isolated autonomy (robots on Mars, in factories) but deploying dense fleets of autonomous vehicles into human-occupied environments never designed for robots, where humans aren't trained to interact with them and may not even know they're autonomous — a challenge he expects will first be solved at scale for cars before drones, partly because safety is easier to manage on the ground. He's optimistic about future demand for short-hop urban air transport (e.g., Boston to New York in 90 minutes) and about "agile airspace" — the underused altitude band too high to hit with a thrown rock but below commercial aircraft — while cautioning that building software for that space is orders of magnitude more complex than current aviation autonomy, especially since much of that complexity must be learned from data rather than hand-coded.
On simulation, Karaman highlights that simulating vehicle dynamics and "interoceptive" sensors (inertial sensing) has long been tractable, and simulating cameras via ray-tracing has recently become good enough to fool people — except when human figures are rendered, since human vision is specifically evolved to detect subtle anomalies in human movement (unlike built environments, which came "after" human evolutionary history) — making realistic simulation of human behavior, not just appearance, one of the hardest remaining problems, with real consequences for training autonomous vehicles that must predict what pedestrians and other drivers will do next. He identifies a research gap in existing self-driving development: after early focus on localization/mapping (SLAM) and then perception (deep learning identifying "what everybody else is"), the field still lacks robust prediction of what other agents will do, and almost entirely ignores the game-theoretic dimension of how an autonomous vehicle's own behavior (aggressive vs. cautious) shapes others' responses — noting that a visibly empty, driverless vehicle gets treated more like an inert "thing" that can be abused than a human driver would, and that this raises real design tradeoffs between transportation efficiency and neighborhood livability/sustainability that become newly explicit and optimizable once autonomy is introduced.
Karaman contrasts Waymo, Tesla, and his own company Optimus Ride as pursuing largely orthogonal strategies rather than simply differing in "caution" versus "aggression": Waymo, in his reading, is building a general-purpose "AI engine" for understanding self/other/prediction with an unrestricted A-to-B mission and something closer to a research orientation; Tesla is selling a real, valued product (Autopilot) that people pay extra for today even with human oversight still required, benefiting from rapid iterative deployment and strong brand/visionary appeal; and Optimus Ride deliberately targets geofenced, transportation-deprived environments (like the Brooklyn Navy Yard) with small, low-speed vehicles, explicitly using human "AV controllers" (not teleoperators) who set high-level intent for many vehicles at once rather than directly driving them — a model Karaman argues will let the ratio shift from "one person operates one vehicle" toward "ten people operate fifty vehicles." He explains Optimus Ride's preference for many small (4–6 seat) vehicles over large shuttles as reducing wait times and improving agility, and frames the potential to reclaim parking-lot land and reduce car-dependent urban design as major, underappreciated societal payoffs alongside pure transportation efficiency, while candidly noting that predicting when any of these approaches reach mass deployment is close to impossible, given a "time dilation" effect where distant-but-simple technologies get compressed into near-term hype while genuinely hard, near AI-complete problems get similarly mispredicted.
On sensing, Karaman states he believes camera-only autonomy will eventually be viable (pushing back gently on Elon Musk's "lidar is a crutch" framing while agreeing with its spirit), but argues most current lidar-heavy approaches reflect an engineering-convenience bias (lidar is simply easier to build usable demos with) rather than proof that cameras can't work — and predicts that lidar hardware may become cheap enough that skipping it becomes economically foolish even after camera-only computer vision matures, given lidar's potential to reduce the compute burden. He describes his research group's work on "high-throughput computing" for aggressive drone flight — building cameras and processing pipelines that operate far faster than human visual-reaction timescales (humans see roughly around 100Hz; his systems aim for kilohertz-scale perception) — as directly relevant to reducing traffic fatalities (many crashes occur in split-second, recoverable-if-detected-fast-enough scenarios) and to enabling extreme drone racing maneuvers, and explains a genuine physical bottleneck: copper-wire data-transfer limits (the Shannon limit) and speed-of-light constraints on clock synchronization across a chip both force redesigning how camera data and chips are architected, rather than simply running existing designs faster. Discussing the Lockheed Martin AlphaPilot autonomous drone racing challenge, he notes machines currently beat humans on repeated precision (executing identically every lap) while humans still have an edge in one-off runs and strategic adaptation, and values embracing frequent (engineered, safe-to-recover) crashes as essential to rapid iterative learning, contrasting this with the slower, more cautious experimentation typical of academic robotics labs.
Karaman closes by naming Bellman's equation — the foundation of dynamic programming and modern reinforcement learning — as the single idea in robotics that has most captivated him: a strikingly simple, teachable formula that simultaneously reveals the "curse of dimensionality" (the number of possible joint states/decisions across many variables can exceed the number of atoms in the universe), making him marvel that decision-making problems provably intractable in the worst case nonetheless work remarkably well in practice — a pattern he likens to how little we understand of deep foundational questions generally (the beginning of time, the consistency of mathematical axioms) even as everyday practice muddles through successfully.
Excerpts
On why deploying robots among humans, not isolated autonomy, is the real challenge of this era:
“The real challenge of our time is to take these vehicles and put them into places where humans are present... these humans will not potentially be even trained to interact with them, they may not even be using the services provided by these vehicles, they may not even know that they're autonomous — they're just doing their thing, living in environments that are designed for humans, not for robots.”
On why an empty autonomous vehicle gets treated differently than a human driver:
“If you see a vehicle that's completely empty and it's trying to do something, all of a sudden it becomes a thing... people kind of abusing these autonomous vehicles is a real issue in some sense, and so when you're an ego vehicle trying to coordinate your way, it's actually kind of harder than being a human — you not only need to be as smart as humans are, but you also, you're a thing, so they're gonna abuse you a little bit.”
On what Bellman's equation reveals about the limits of computation:
“It both kind of tells you the simplicity — because it's a single equation that anyone can write, we can teach it in the first course on decision-making — at the same time it tells you how computationally hard the problem is... as the number of variables grows, the number of decisions you can make grows rapidly — if you have a hundred variables each taking ten values, all possible assignments is more than the number of atoms in the universe.”