Inside the start-up aiming for a giant leap in robot intelligence

In San Francisco, inside a warehouse coated with gleaming steel panels, I am handed a fresh cup of coffee made entirely by a robot. This fact alone is unimpressive — robots have been making coffee for more than a decade — but the robotic brain that made this coffee is no one-trick pony. It has also learned how to do many other tasks, such as folding clothes, peeling vegetables and cleaning kitchens, in the time that most toddlers barely learn how to walk.

Physical Intelligence, a start-up founded in 2024, is betting that a robot brain that can learn how to do many different tasks will, in the not-too-distant future, enable robots to become enmeshed in our daily lives. Instead of focusing on a single machine, like the humanoid robots built by Tesla or Boston Dynamics or the factory robots used by Amazon, the company wants to build an adaptable control system that can perform many tasks with many different machines.

A general-purpose robotic intelligence isn’t a new idea: many roboticists would say it has been a long-term goal for decades. But, just as the early 2020s saw a flourishing of the large language models (LLMs) that power AI chatbots thanks to the right combination of computing power, data and algorithmic advances, Physical Intelligence is hoping to conjure a similar leap of progress in general robotics.

“In most domains, solving more problems only makes things harder, but in AI, it actually makes it easier, because then you have more diverse sources of knowledge to learn from,” says Sergey Levine at the University of California, Berkeley, who is one of the firm’s founders.

The success of LLMs has led to a new kind of robotic AI, called a vision-language-action (VLA) model, that underpins much of Physical Intelligence’s research. Instead of teaching a robot one skill at a time, a VLA uses the broad knowledge of an LLM to translate general requests into specific actions, enabling robots to follow instructions and carry out many different tasks. “(VLAs) are probably the most direct translation of the excitement that we have from large language models,” says Ingmar Posner at the University of Oxford. Rather than predict the next word, these systems predict the next robotic move needed to complete a particular task, he says.

Free newsletter

Sign up to The Daily

The latest on what’s new in science and why it matters each day.

Sign up to newsletter

One big challenge for training robots is that there is a near-infinite number of real-world variations for any task, and very little data for robots to learn on. Automating the learning – teaching robots to teach themselves – is a possible solution, but most robot developers have shied away from doing this because gathering enough data is a tall order, says Levine. “Even though, in principle, it should be automatic, in practice, the amount of work required to get the data for your particular application was larger than the work needed to just do everything by hand.”

Levine and his colleagues are hoping that, powered by VLAs, they will need considerably less data to succeed. Beneath the boardroom where I spoke with Levine, a fleet of employees were teaching the robots to do what appeared to be a banal series of tasks: folding shirts, placing pillows on shelves, cutting bows on present boxes. Around the corner, I learned, were two warehouses containing fake supermarkets, bedrooms and kitchens, which were renovated each week, where Physical Intelligence’s robots and AI models could learn to cope with a variety of settings. The company was also rolling its robots out to real, lived-in homes to test how they might cope with the mess of the real world.

This variety is one of the key ingredients that has led to a surprising amount of progress, including robots learning to generalise beyond tasks they have seen before. A recent model, called π0.7, was able to cook sweet potatoes in an air fryer with step-by-step verbal instructions from a human, despite never having used an air fryer before.

The speed of progress in the two years that Physical Intelligence has been operating has surprised Levine. “It’s actually gone quite a bit quicker than we thought,” he says.

Other companies are taking notice. A slew of start-ups with billions of dollars in funding, as well as more established companies, like Amazon and Google DeepMind, are attempting to develop their own general-purpose robots.

Despite the rapid advances so far, it is hard to predict how fast the field will move forward. Progress for AI companies like OpenAI and Anthropic has famously been rapid, but the going is often slower for robotics companies. Every robotics researcher will be familiar with Moravec’s paradox: computer scientist Hans Moravec observed in 1988 that it is easy for robots to master games like chess or score highly on IQ tests, yet it is “difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility”.

It is still unclear just how much data Physical Intelligence will need to prepare its robots for real-world use, says Posner. “I would say right now we’re at early signs that something interesting might be happening, but whether that’s really the route to go is a different question.”

He thinks success in the real world is still a long way off, partly because users will push robots to their limits. “Humans are adversarial. They like messing with a robot, if nothing else, just because it’s fun,” says Posner. “Do I believe this stuff is going to get deployed at scale anytime soon, with a business model that actually makes money? No, definitely not. I would find that very difficult to believe.”