Real world datafor physicalintelligence

Skilled people, recorded at work on our hardware. Data primed for robotic policy training.

Humans do countless tasks. Robots have a lot to learn from them.

From a pair of hands to a policy

What the cap sees, what the hands do, what they feel. Then the scene as a measurement, as a simulation, and as a robot doing the task.

Capture01capture

Capture

A cap with two cameras, worn all day by people who know the job.

Hands02hands

Hands

Twenty-one points per hand, every frame. How a grip forms, and how it lets go.

Touch03touch

Touch

Where the fingers press, how hard, and for how long.

Reconstruct04reconstruct

Reconstruct

Two views become depth. The bench becomes a measurement.

Simulate05simulate

Simulate

The same scene, rebuilt where a policy can fail a thousand times for free.

Transfer06transfer

Transfer

Trained on people. Checked in simulation. Handed to a robot.

100K+

Hours collected and sold

5 to 10K

Hours a day of collection capacity

3

Regions: USA, India and Southeast Asia

Stereo

Our own capture hardware, in the field

  • Tailoring
  • Auto repair
  • Bakery
  • Street food
  • Incense rolling
  • Restaurant kitchen
  • Plastics packing
  • Print shop
  • Autorickshaw garage
  • and many more
Tailoring: first person frameAuto repair: first person frameBakery: first person frameStreet food: first person frameIncense rolling: first person frameRestaurant kitchen: first person framePlastics packing: first person framePrint shop: first person frameAutorickshaw garage: first person framefaces blurred before delivery
Open a sample

Recorded where the work happens

faces blurred before delivery
Tailoring

01 / 10

Every clip ships with every layer

Real recordings, with the signals a model trains on. Recording, episode, environment, device, operator.

Aari embroidery on a frame with beads and gold thread, first person frame
aari embroidery · frame 01 / 04stereo cap · 30 fps · 1 of 2 streams

recording

fps30
timezoneasia/kolkata

episode

taskaari embroidery
consentyes

environment

settingtailoring workshop
sectorclothing and fashion

device

kitstereo cap
cameras2 synced

operator

idpseudonymous
handright

What we are building

Data is the product. Capture, hands and simulation are what make it compound.

Capture hardware

capture hardware

A stereo camera cap people can wear all day.

stereo pair30 fpsall day battery
Hands and touch

hands and touch

Hand pose and contact, read from the recordings.

21 point posecontact estimateboth hands
Real to sim

real to sim

Real scenes rebuilt as simulations.

stereo depthscene rebuildphysics
Sim to real

sim to real

The next step: policies back onto robots.

plannedrobot handsclosed loop

The same moment, three times

A recording becomes a measured scene. A measured scene becomes a simulation.

First person view of two hands threading a nut onto a bolt at a workbench
The same bench as a point cloud
The same bench as a simulation, with robot hands
bench.0217 · stereoscene rebuilt · metric