Kumar, Vijay

roboticist; professor and Dean of Engineering at the University of Pennsylvania; former director of the GRASP Lab

Vijay Kumar: Flying Robots

Lex Fridman Podcast · Sep 8, 2019

• Vijay Kumar
• aerial robots
• robot swarms
• quadrotor design
• IMU sensors
• autonomous flight vs. driving
• computer vision scaling limits
• drone delivery
• human-robot collaboration
• counter-swarm defense

Core ideas

Kumar traces his path from building a 7,000-pound hydraulic hexapod in graduate school (coordinated by 19 separate half-megahertz processors, requiring what he believes was one of the first multiprocessor operating systems) to today's small, agile aerial robots — reflecting that "bigger means grander" is largely a psychological bias, since modern miniaturized robots achieve far more with far less. He identifies his proudest work as 2011-era research enabling small UAVs to coordinate into deformable three-dimensional formations in midair, extending earlier 2D ground-robot pattern formation into a genuinely new capability: sculpting and reshaping 3D objects in open space using swarms of individually agile units. He draws deep inspiration from ants — individually simple but remarkably resilient (a six-legged ant missing a leg keeps functioning without apparent disruption) and collectively capable of complex emergent behaviors (chaining together to cross gaps, recruiting help to move large food, reaching group consensus without centralized communication) — arguing engineers should design individual robot "primitives" cleverly enough that the swarm's collective behavior can be reasoned about abstractly in low-dimensional terms, since modeling many individually complex units directly produces an intractable combinatorial explosion.

He draws a sharp distinction between how nature organizes swarms and how engineered robot swarms typically need to operate: natural swarms coordinate almost entirely through local, indirect interactions oriented around survival, shelter, and reproduction, whereas engineered swarms are usually given missions defined in a human-specified global coordinate frame (e.g., "surround this building"), requiring individual robots to be simultaneously locally aware of neighbors and globally cognizant of a shared reference frame — a fundamentally more expensive and "unnatural" (in nature's sense) computational task, though falling unit costs are gradually making individual robot loss less economically significant, edging engineered swarms closer to nature's tolerance for losing individual units.

On autonomy, Kumar distinguishes the limited, infrastructure-dependent autonomy of most current military and commercial UAVs (reliant on GPS, external communications, or human piloting — all of which are fragile: GPS fails in urban canyons, communications can be jammed) from "true autonomy," which he defines as navigation without any prior map, external positioning, or communications infrastructure. He explains why quadrotor configurations (settled on after his lab's own early, less successful experiments with an eight-rotor omnidirectional spherical design that lacked sufficient thrust-to-weight ratio) became the dominant small-UAV design: four motors provide exactly enough degrees of freedom (via differential rotor speeds generating both thrust and reaction-moment-based rotation) to control all six degrees of freedom of flight, a remarkably efficient and versatile configuration despite initially seeming underactuated. He credits a surprising cross-industry dependency for enabling small agile drones at all: the automotive industry's mandated adoption of airbags drove mass production of cheap accelerometers, which in turn commoditized the inertial measurement units (IMUs) — combined with the 2007-era leap in embedded computing power (the "iPhone/cloud" inflection point) — that made onboard sensing and control for small UAVs newly feasible.

Kumar pushes back on the idea that his field has avoided machine learning, noting that iterative, model-free learning has long been used to compensate for aerodynamic effects that are essentially impossible to fully model in advance — including blade flapping (rotor blades bending slightly rather than staying perfectly rigid, which changes angular acceleration at high speeds), ground effect, ceiling suction effects, and wall/train-passing-style buffeting from nearby surfaces. He characterizes the current state of learning in robotics as strongly asymmetric: computer vision perception (object detection, classification, tagging) has advanced dramatically via machine learning, but genuinely successful end-to-end learned action/control policies (pixels directly to motor commands) remain, in his view, an unrealized "holy grail," with the practical path forward being hybrid systems combining data-driven perception with model-based planning and control rather than pure end-to-end learning.

On computer-vision-only approaches to autonomy (referencing the Tesla/Musk camera-first position), Kumar is skeptical for safety-critical domains: from his experience with a spinoff company operating drones in dark, dusty underground mines (where vision fails and lidar is necessary), he argues the accuracy-versus-data curve for vision-based perception is likely exponential rather than linear — going from 90% to 99% detection accuracy requires substantially more data, and 99% to 99.9% (needed for genuinely safety-critical applications) requires even more still — a scaling relationship he doubts is sustainable, compounded by the genuinely enormous energy cost of the data infrastructure required (citing that data centers consumed roughly 2% of total U.S. electricity as of 2014). Comparing autonomous flight to autonomous driving, he argues flight has a structural advantage (a straightforward, obstacle-minimal vertical-up/horizontal/vertical-down trajectory is much safer and more mathematically tractable than 2D ground routes dense with obstacles) but a corresponding disadvantage in needing to model and reason about a full three-dimensional environment plus complex, hard-to-fully-model aerodynamic effects.

He sees strong near-term potential for drone delivery in specific high-friction contexts — congested cities where ground travel is disproportionately slow (Hong Kong river crossings), and remote regions lacking road infrastructure (citing Wake Robotics' work in the Peruvian Amazon, where river travel that takes five hours by boat could take thirty minutes by drone, useful for drug delivery and medical sample retrieval) — framing the primary barriers as economic rather than purely technological, particularly the field's persistently limited progress in battery energy and power density (noting vertical takeoff for small drones costs roughly 200 watts per kilogram, versus the human brain's ~20 watts total, a comparison he finds almost absurd). On flying cars, he's cautiously optimistic about near-term autonomous capability but skeptical of "clean" (all-electric) claims given current battery limitations, citing a company building small, efficient jet engines (Jetoptera) achieving useful speed and payload as a more near-term-viable, if louder, alternative to purely electric designs.

On human-robot collaboration, Kumar rejects the idea that robots should be designed to exclude humans from consideration, distinguishing three interaction modes — command, true collaboration (e.g., a robot jointly carrying an object with a human), and robots treating humans as bystanders to be acknowledged (as in autonomous driving) — and argues no single unified theory covers all cases. He contrasts the relatively low-stakes, boredom-driven collaboration challenge in semi-autonomous highway driving (Tesla's model of periodic human re-engagement versus Toyota's shared/collaborative-autonomy paradigm) with the far more consequential, unstructured collaboration needed in search-and-rescue, where human first responders and robots must jointly and rapidly navigate genuinely dangerous, time-critical situations — and proposes an intriguing reversal of the usual human-modeling-the-car dynamic: cars equipped with driver-facing sensors modeling the human's attention state (via eye tracking, saccadic movement) could make driving meaningfully safer, much as smartphones already model user behavior and intent.

On robotics' societal risks, Kumar acknowledges weaponized swarms as a genuine concern (noting that a swarm attack only requires one unit out of many to succeed, unlike defensive/cooperative swarm tasks that require full coordination), but argues the appropriate response is for engineers with deep technical understanding of swarm autonomy to actively work on counter-swarm defense (citing a National Academies report on counter-UAS technology) rather than ceding the field to purely political oversight, framing "technology as the new liberal art" that politicians and citizens alike need literacy in. He identifies the central open problem in robotics as generalization across unstructured, variable environments and tasks — noting that today's systems (self-driving cars in a parking lot, or an AI beating humans at board games) succeed in highly structured settings and that mastering a structured game like chess or Go tells us very little about general intelligence, since it's analogous to the well-defined structure of an assembly line rather than the genuinely open-ended unpredictability of streets in Naples or Mumbai. He closes with career advice for aspiring roboticists: expect the field to be unrecognizable within a few years and cultivate breadth rather than narrow specialization; maintain grounding in mathematical foundations and representation (arguing that even progress toward explainable AI ultimately depends on having understandable representations, not just found packages or fitted neural networks); and situate engineering within a broader liberal-arts context, since engineers should never lose sight of the societal dimension of their work.

Excerpts