Categories
Uncategorized

Architectural Gaps in Physical AI: When’s the Big Moment?

In some specific robotics use cases, the magic moment has clearly already happened. Autonomous driving had its moment in 2025. That’s why in 2026 you can ride a wheeled robot, a Waymo, all over SF with minimum hassle.

In Ukraine, autonomous drones now rule the sky (and the battlefield). At Amazon, more than 1 million robots help humans to move, package, sort, and pick, and we take it for granted that the stuff we ordered earlier today shows up at our doorstep the next morning. In China, humanoids now outrun Usain Bolt, the fastest-sprinting human.

But for many other use cases, it seems robots aren’t quite there yet. Is this just a question of needing to patiently wait until robots outperform humans, task by task, or are there deeper structural gaps? Based on growing real world deployments of our robots in SF and NYC, we see two gaps over and over again.

1/ We are using the wrong brain architecture

Engineering is about making choices. Remember the MIT saying – sleep, friends, grades – choose two? The same applies to robots. If you operate in the real world, you have a finite energy, constrained mass, and finite ability to dissipate heat. And if you want people to buy your robots, they need to be reasonably priced (unlike, say, anything ever manufactured by Boston Dynamics).

What this means for robot brains is that trying to build one all-capable model, the one to rule them all, is a fool’s errand. Sure you can prepend “universal”, “all powerful”, or “grand unified” to your approach, but that’s just a labeling exercise. The solution is to operate a smart router (with context) as the core of the brain, and then dynamically (re)optimize which data go to which models, as a function of the robot’s task and environment.

Need to balance on your toe? Focus all your resources on that stability task. Listening to a lonely elderly parent in the park? Well then use all your compute for empathetic personalized conversation and gesture generation. Helping a lost child find their parents at SFO? Now you care about path planning and scanning human (parent) faces for expression of concern.

As a capable robot, each of those tasks deserves your focus, and that’s why, as a robot, you should be constantly adjusting which models (and compute and actuators and sensors) you should be using in this very instant for optimal performance. At OpenMind, we are placing our bet on intelligent routing/attention as the foundational brain function of a robot, the goal being to use the best task-optimized model for the immediate task, whatever it may be.

2/ We left out the physics

As the name implies, physical AI (and robotics) is about physics, not just bits. Let’s say you want a machine to perform a specific task in the physical world, such as fly. Presumably, you would not start by collecting videos of humans flapping their arms. A more promising strategy is to focus on what actually constrains and enables that capability – Navier-Stokes allows you to compute what an efficient wing looks like. For human readers, don’t take this (too) personally, but the way in which you drive cars, fold t-shirts, do patient intake in the hospital ED, or pick up apples, may not be generally optimal given the underlying physics. The relevant question is – given a specific desired physical task/action, and given your current physical form (One arm? Two wings? Three feet? Two wheels?), and your current battery level, what is the optimal sequence of movements to accomplish that task? That’s a nonlinear optimization problem, not a “replay memorized motion sequences” task. The framing as nonlinear optimization gracefully accommodates changes in your current physical form and capabilities. If one of your arms falls off, no problem – you just change how you move, without more than a moment’s recalculation.

OpenMind’s definition of the Physical Turing Test

The physical Turing test (by Nvidia’s Jim Fan) builds on Alan Turing’s test and refers to the ability of a robot to operate in the real world without it being apparent to observing humans that they are seeing a robot rather than one of their own. This is one good way to define a discrete watershed, but it’s still very much centered on our self-reflection.

At OpenMind, we have a more general definition with three basic ingredients. We think that the “ChatGPT” moment for physical AI will be when robots have these three attributes:

1/ [Rapid learning of extreme dexterity] Given only the laws of physics and knowledge of its internal architecture and force generating capabilities, a robot must learn to move, run, jump, and manipulate diverse physical objects within 24 hours of being allowed to freely explore the real world. Further, the robot will gracefully and quickly overcome loss of one major sensor (e.g. a camera) or actuator (e.g. a finger, wheel, or foot) without external help.

2/ [Success with diverse cognitive, planning, and physical tasks] The robot will complete a succession of diverse tasks, encompassing verbal, emotional, reasoning, coordination, spatial navigation, manual/physical, and learning, without prior knowledge of which specific tasks it will face, and in which order.

3/ [Safety/Constitutional Robotics]. By virtue of its core design, the robot will reconcile the above capabilities with an internal constitution that minimizes risks of the technology for humans. You might well wonder how this can be possible – TLDR it’s probably not possible with our current approach to AI since we have very limited understanding of how frontier reasoning models actually work. It’s way too easy to hide behaviors inside of these models and they almost certainly will try to solve problems in ways that harm humans.