Language models were trained on text that already existed. Cleaning it was work. Producing it was not, because nobody wrote the internet so that a transformer could read it. Robot policies have no equivalent corpus, and the gap is not a rounding factor. The dataset the field calls large-scale and in-the-wild runs to 350 hours of robot interaction, and fifty people on three continents spent a year producing it. That is not a technical footnote. It is the cost structure.

What “large-scale” means here

The DROID paper describes its dataset as “76k demonstration trajectories or 350 hours of interaction data, collected across 564 scenes and 84 tasks by 50 data collectors in North America, Asia, and Europe over the course of 12 months.”

Three hundred and fifty hours. On the single shift I used for battery swaps, eight hours a day and 250 days a year, that is 44 shifts. The whole dataset is under a fifth of what one robot does in one ordinary working year.

Divide the other way and each of the fifty collectors accounts for seven hours of usable interaction across twelve months. The paper does not say what fraction of anyone’s time this was, so read that as yield per participant rather than a work rate.

A trajectory is one recorded attempt at one task. At 76,000 of them in 350 hours, the average runs about seventeen seconds.

The corpus that made this look easy

Meta pre-trained the flagship Llama 3 on “15.6T text tokens”, against about 1.8T for Llama 2. The difference between one generation and the next is roughly fourteen trillion tokens. Making them usable was real work: the paper builds the set “from a variety of data sources” and applies “several de-duplication methods and data cleaning mechanisms” to each one. That is the cost of cleaning a supply, not of creating one.

Nobody performed it. Every cost below follows from that.

The field’s own answer is a consortium

The Open X-Embodiment authors make this comparison themselves, and more bluntly. Vision and language, they note, “can leverage large datasets sourced from the web,” while “comparably large and broad datasets for robotic interaction are hard to come by. Even the largest data collection efforts still end up with datasets that are a fraction of the size and diversity of benchmark datasets in vision (5-18M) and NLP (1.5B-4.5B).”

Their answer to that scarcity is to pool. Open X-Embodiment holds “1M+ real robot trajectories spanning 22 robot embodiments, from single robot arms to bi-manual robots and quadrupeds”, and the paper is direct about where they came from: the dataset was built by “pooling 60 existing robot datasets from 34 robotic research labs”, through a collaboration between 21 institutions.

It was not collected. It was gathered up. Twenty-one institutions had to agree to put what they already had into one pile to reach a million trajectories.

For a sense of what one such effort costs, the RT-1 dataset was “gathered over the course of 17 months with a fleet of 13 robots” and holds about 130,000 episodes. Spread evenly that is roughly 590 episodes per robot per month, though the paper does not say the fleet ran continuously, so read it as a scale marker not a rate.

My read is the simple one: nobody assembles a twenty-one-way consortium around an input that is cheap to make.

What transfer fixes, and what it does not

The real argument against all of this is in the same paper. RT-X, trained across the pooled data, “exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms.” If experience on one robot pays for another, the per-robot cost stops compounding.

It works, and more broadly than I first read it. In the small-data domains the pooled model beats the specialist on four of five datasets. In the large-data setting “the RT-1-X model does not outperform the RT-1 baseline trained on only the embodiment-specific dataset,” but the paper puts that down to “underfitting for that model class” and reports that “the larger RT-2-X model outperforms both the Original Method and RT-1 suggesting that X-robot training can improve performance in the data-rich domains, but only when utilizing a sufficiently high-capacity architecture.” So transfer is not a rescue for the data-poor alone. Given enough model, it reaches the data-rich too.

What it has not been shown to do is reach a robot that was never in the pool. The authors are explicit about that. The paper lists its own limits: “it does not consider robots with very different sensing and actuation modalities, it does not study generalization to new robots, and it does not provide a decision criterion for when positive transfer does or does not happen.”

So the mechanism that would turn robot data into a one-time cost is real, and stronger than I expected. It has been shown among the twenty-two embodiments already in the corpus, with no published rule for counting on it outside them.

What it costs

For a language model company the corpus behaves like a fixed cost, and much of it is reachable by any competitor willing to scrape and clean the same web. Capital goes into compute. For a robot company a real share of capital goes into manufacturing the corpus, with robots, operators and scenes, and it recurs whenever the task list, the environment or the hardware changes.

Two things follow.

The first is where the money lands. This is the same human-hours problem I found in robot operator job ads, arriving one stage earlier. There, people were covering for autonomy in deployment. Here, they are manufacturing the experience that autonomy is supposed to be learned from. Both are headcount that grows with ambition, and neither is a one-time data licence.

The second is the moat, and the transfer result sharpens it rather than softening it. The better pooled data trains, the more valuable the pool is relative to anything held privately. The field’s answer to the cost, so far, has been to pool the asset. If a company’s data advantage sits inside Open X-Embodiment, it is not that company’s advantage, it is the field’s. The question to put to a management team is which trajectories they hold that are not in the open pool, and how many hours those represent.

Where I could be wrong

Transfer may already be further along than this. The limits I quoted describe what the experiments covered, not a finding that transfer fails, and the RT-2-X result shows the ceiling moves with model size. If a later result shows clean generalization to robots outside the pool, the per-embodiment half of my argument goes.

Simulation is not in this piece. If sim-to-real closes for manipulation, the cost structure changes completely and most of this goes with it. I have not sourced that question here and I am not claiming the answer.

These are research datasets, not company balance sheets. Commercial fleets may hold far more than anything above. None of it is published, which is exactly why the academic numbers are the ones you can check.

The timestamps matter. RT-1’s collection ran 17 months and the paper is from 2022, Open X-Embodiment from 2023, DROID from 2024. Read every figure against those dates rather than against today.

Success rates here are benchmark numbers. RT-1 reports a 97% success rate on over 700 training instructions, and what a number like that means on a real floor is a separate question I have already taken up.

The corpus behind language models was inherited. Robotics has to produce its own, hour by hour, and then share it to reach a useful size. Any model of a robot company that books data as a one-time acquisition is missing a line that recurs.