Existing methods for training robotic AI models, such as having algorithms learn to control a particular system through teleoperation or in simulation, do not generate enough data, Pathak says.
Skild’s approach is to instead have a single algorithm learn to control a large number of different physical robots across a wide range of tasks. Over time, this produces a model which the company calls Skild Brain, with a more general ability to adapt to different physical forms—including ones it has never seen before. The researchers created a smaller version of the model, called LocoFormer, for an academic paper outlining its approach.
The model is also designed to adapt quickly to a new situation, such as missing leg or treacherous new terrain, figuring out how to apply what it has learned to its new predicament. Pathak compares the approach to the way large language models can take on particularly challenging problems by breaking it down and feeding its deliberations back into its own context window—an approach known as in-context learning.
Other companies, including the Toyota Research Institute and a rival startup called Physical Intelligence, are also racing to develop more generally capable robot AI models. Skild is unusual, however, in how it is building models that generalize across so many different kinds of hardware.
In one experiment, the Skild team trained their algorithm to control a large number of walking robots of different shapes. When the algorithm was then run on real two- and four-legged robots—systems not included in the training data—it was able to control their movements and have them walk around.
At one point, the team found that a four-legged robot running the company’s omni-bodied brain will quickly adapt when it is placed on its hind legs. Because it senses the ground beneath its hind legs, the algorithm operates the robot dog as if it were a humanoid, having it stroll around on its hind legs.
The generalist algorithm could also adapt extreme changes to a robot’s shape—when, for example, its legs were tied together, cut off, or modified to become longer. The team also tried deactivating two of the motors on a quadruped robot with wheels as well as legs. The robot was able to adapt by balancing on two wheels like an unsteady bicycle.
Skild is testing the same approach for robot manipulation. It trained Skild Brain on a range of simulated robot arms and found that the resulting model could control unfamiliar hardware and adapt to sudden changes in its environment like a reduction in lighting. The startup is already working with some companies that use robot arms, Pathak says. In 2024 the company raised $300 million in a round that valued the company at $1.5 billion.
Pathak says the results might seem creepy to some, but to him they show the sparks of a kind of physical superintelligence for robots. “It is so exciting to me personally, dude,” he says.
What do you think of Skild’s multitalented robot brain? Send an email to [email protected] to let me know.
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.
