One bit of reward, our robots.
98.0–99.7% on a 20-part board, in simulation.
We re-implemented NVIDIA's AlphaNIST recipe, one RL policy that learns every assembly task on a NIST task board from a binary success signal, on a board small enough to iterate on in an afternoon. Then we checked which of the paper's findings hold up and put the whole thing on Rosie arms with our SendCutSend gripper.
- Rosie 1400 + 1430 adapter
- 99.7%
- Rosie 1000 + 20 mm spacer
- 98.4%
- Dense reward shaping
- 0.03
- Shared vs specialists
- 0.97 vs 0.59
All 20 parts, every kind of start.
94% from loose parts on the table.
Collapsed, as the paper warns.
RJ45: one policy for all parts wins.
The paper in one paragraph.
AlphaNIST trains one PPO policy across all 20 assembly pairs of NIST Assembly Task Board #1 (pegs, gears, nuts, connectors) with no demonstrations, no task stages and no per-task tuning. The reward is a single bit: the part is within 1 mm of fully assembled, or it isn't.
It reaches 98.2% training success at 64K parallel environments, beats twenty separately trained specialists that used 39× the GPU-hours, and discovers regrasps, pivots and one-sided nut tapping that nobody scripted.
Reset bank
Cached starts for every part: loose on the table, in hand, near the socket, half assembled. No stage label is given to the policy.
Beta sampling
Starts the policy solves about half the time are drawn most; nothing is ever starved.
qi ∝ ε + √(pi(1 − pi))
Binary reward
1 once the assembly error drops below 1 mm, then the episode ends. A small control cost, nothing that measures progress.
The authors' code isn't out yet, and their setup is Isaac Lab on racks of NVIDIA GPUs. So we rebuilt the recipe from the paper and ran it on something smaller.
A board we can afford.
NIST-lite: the same 20 part families, simplified contact, fast enough to iterate on a laptop.
NIST-lite has 4/8/12/16 mm rods and square pegs, three gears, M4–M16 nuts, USB-A, D-sub, RJ45, a keyed waterproof connector and a BNC twist-lock. Contact is kinematic and rule-based, but the rules keep what makes the real board hard.
The policy sees point clouds of the part and its socket plus its own joint state, never a task id. Before training anything we wrote a privileged scripted controller to prove every part is solvable inside the 6-second episode from every kind of start (100%), and checked that a random policy only succeeds from half-assembled starts: exactly the foothold the reset bank is meant to give.
- GraspsThe fingers must straddle the part and match its symmetry; the height you grabbed it at is remembered.
- InsertionEach pair has a clearance and a yaw: keyed connectors go in one way, gears need their tooth phase.
- Fingers vs boardFingers can't pass through the board, so a peg grabbed too low can't be seated: regrasp higher, or let go and push it home.
- ThreadingNuts only advance by turning while gripped, and the wrist runs out of travel, so threading needs release, wind back, regrasp.
The paper's ablations, on rented GPUs.
Same PPO schedule as the paper (32 steps × 5 epochs × 4 minibatches), evaluated on one frozen set of starts with stochastic actions, again as in the paper.
| Variant | Final success | From loose parts | Steps to 50% |
|---|---|---|---|
| Beta sampling + binary reward (AlphaNIST), 16K envs | 0.943 | 0.90 | 42M |
| Uniform sampling + binary reward, 16K envs | 0.932 | 0.87 | 52M |
| Beta sampling + dense shaping (Eq. 7) | 0.029 | 0.01 | peaks at 0.50, collapses |
| Uniform sampling + dense shaping | 0.042 | 0.00 | never |
| Beta sampling + binary reward, 4K envs | 0.994 | 0.98 | 13M |
Shared policy vs specialists.
Same budget per run: one policy trained on all twenty parts against five policies that each train on one. The specialists for the M4 nut, RJ45 and BNC never solve a single loose-part start; the shared policy solves them almost every time. What the hard parts are missing is grasping, carrying and aligning, and the shared policy learns those on the easy parts.
The two findings that reproduced cleanly are the least intuitive: shaping hurts, and more variety makes the hard tasks easier.
What didn't reproduce: beta sampling's edge over uniform. It gets to 50% about 20% sooner, then uniform catches up. The paper's gap shows up at 256K environments and 25K iterations; we stopped at 16K × 1K. With ε = 0.05, starts nobody has solved yet get ten times less practice than frontier starts, which slows the hardest family early on.
What the policy figured out.
Nothing in the reward, the observation or the reset bank describes any of these manoeuvres.
- Threading in strokes3.6 grasps per M4 nut from a loose start: turn until the wrist runs out, let go, wind back, regrasp. Coarser nuts need fewer (2.0 for M16).
- Regrasp to align1.3–1.7 grasps per D-sub, the same behaviour as the paper's Fig. 8b.
- Grasping for laterFrom loose starts, every peg and rod is picked up exactly once, high enough to be seated in one go.
- Pushing homeWhen it starts holding a peg too low, it lets go and pushes the part down with closed fingers.
Does it survive our hardware?
The arms from rosie-sim-kits, our gripper's CAD and dimensions, and every move through the arm.
The SendCutSend compact gripper: 0–60 mm stroke, 105 mm/s per jaw, 25 mm V-fingers, on the 20 mm ISO spacer for the 600/1000 or the 22.4 mm 1430 adapter for the 1400.
The policy still commands the gripper in board coordinates, but each step the command is tracked with the tool held vertical by damped least squares inside every joint's speed and position limits, and only what the arm achieved reaches the contact rules. Reach, joint speeds and J6's ±180° range are the robot's own; J6 is now the wrist limit that forces the nut regrasps.
| Arm + mount | Success | From loose parts | Steps to 50% / 80% | Arm tracking error (eval) |
|---|---|---|---|---|
| Rosie 600 + 20 mm spacer | 0.980 | 0.93 | 24M / 45M | max 0.10 mm |
| Rosie 1000 + 20 mm spacer | 0.984 | 0.94 | 26M / 50M | max 0.19 mm |
| Rosie 1400 + 22.4 mm 1430 adapter | 0.997 | 0.99 | 21M / 39M | max 0.06 mm |
4,096 environments × 1,500 iterations each, about half an hour and a dollar per arm. All three learn at least as fast as the free gripper did.
How faithful is this?
Close on the learning algorithm, loose on the world it learns in. That distinction matters more than any number on this page.
| Piece | Paper | Here | |
|---|---|---|---|
| Reset bank | Cached sim states, all 20 pairs, no stage label | Same families, generated from the rules | Same |
| Beta sampling | ε + √(p(1−p)), last L outcomes | Same rule; ε and L were our choice (unpublished) | Same |
| Reward | 1 mm binary + cost on action, joint speed, torque | 1 mm binary (+ twist for nuts/BNC) + action cost only | Close |
| Policy | Separate SimBa actor/critic, point clouds incl. robot | Same shape, smaller, no robot cloud | Close |
| PPO schedule | 32 steps × 5 epochs × 4 minibatches, 25 Hz | Same | Same |
| Physics | Isaac Lab contact on real board CAD | Kinematic rules: no forces, friction, tilt, jamming | Different |
| Robot & actions | Franka, relative joint targets, impedance control | Rosie kinematics, Cartesian deltas, tool kept vertical | Different |
| Scale | 64K–256K envs, 25K iterations, 410 GPU-h | 4K–16K envs, 1K–1.5K iterations, one seed | Different |
| Multi-part episodes | Up to 10 pairs, reward only when all done | Not attempted | Missing |
The recipe works on a kinematic stand-in driven through our arms' real kinematics, limits and gripper geometry. It is not yet evidence that a Rosie can assemble the physical board.
That needs contact physics with the real part and gripper meshes. The algorithm code would carry over unchanged.
What bit us.
- Write the scripted solver first. It found an off-by-equality grasp bug (fingers closing in 10 mm steps land exactly on a 10 mm part) and showed some connector yaws need a put-down-and-regrasp, before a single GPU-minute was spent.
- Checkpoint everything. All nine GPU containers in a sweep were interrupted at once at iteration ~750 and restarted from scratch. Training now saves and resumes every 50 iterations.
- One seed hides a lot. The 4K-vs-16K comparison was decided by when one run happened to discover the D-sub, not by scale.
- IK near 180° is a trap. Small-angle orientation errors vanish there, and J6 pinned itself at a limit for 2–4% of targets. Re-seeding J6 from the required yaw and solving twice fixed it.
- Log the consistency you assume. A per-step "where the env thinks the gripper is vs where the joints put it" metric showed early exploration briefly reaching 8 mm on the 600, even though trained policies stay under 0.2 mm.
Cost and what's next.
~$22 for the ablation sweep (a third of it the interrupted attempt) and ~$4 for the three Rosie arms. The CPU runs and all the debugging ran on a laptop.
- PhysicsReal contact with the part and gripper meshes (MuJoCo on GPU or Isaac Lab), so grasps can slip and nuts can cross-thread.
- Seeds and scaleThree seeds per variant and a sweep of ε before claiming anything about beta vs uniform.
- Multi-part boardsSeveral parts per episode, reward only when all are seated, which is where the paper's planning-like behaviour appears.
- Joint-space actionsRelative joint targets and tool tilt, closer to how the paper drives its Franka.