I recently visited a robotics startup in San Francisco. They are trying to build generally intelligent robots. General intelligence means building a general model. They are basically building the first robotics foundation model. This model should be competent across a diverse set of motor tasks.

When the LLM was being built, AI companies used the internet to pre-train the models. This is a large set of human data that was broad and high quality.

What is the equivalent to the internet for robotics? Where is there a rich set of data that allows models to become generally competent in the physical world? This is a data set of representations over light, sounds, forces, and other forms of physical data.

The startup I visited is trying one answer: create the internet of data yourself. So they have these big warehouses where dozens of humans are collecting physical data for the robots, basically 24/7. It is a massive data farm. They have taken loads of Venture money and invested it in relatively high-capex data collection centers (where costs scale somewhat linearly as labor is the highest cost). They are now in a waiting game. They are waiting for a non-linear improvement in the quality of the models.

Can they do it? Probably. We saw in LLMs that relatively simple training architectures can be used to develop rich generalizations about human languages when you have a lot of data. I don’t see why the laws which regulate the physical world are categorically different from the laws which regulate the structure of language.

So the real question is: can they do it first? There is a lot at stake here and thus a very healthy dose of competition. Maybe investing in algorithms and/or more compute is the better bet.