Crystal Lin

It starts with seeing

A 40 mm ball crosses a 2.74 m table in a fraction of a second. Before a humanoid can return it, something has to see it.

So I built the eyes

Cameras, calibration and tracking, designed and modelled end to end.

40 mm of matte orange

Found in 3D every few milliseconds.

One bounce, then the robot has to be there

A regulation table is 2.74 by 1.525 m. The ball crosses it in a fraction of a second.

Four cameras, four corners

The rig I proposed: global-shutter cameras on 2.7 m stands around a 5.2 m square, all aimed just above the table centre.

226 frames a second, in sync

One hardware trigger fires every camera at once, so their rays meet at a single point. Every point of the flight volume is seen by at least three cameras.

1.2 mm median error

And 2.1 mm at the 95th percentile, modelled over the whole flight volume. In a simulated 500-ball match the robot returns as many balls with this rig as with the commercial motion-capture system it would replace.

Ball motion capture, built from scratch

On real hardware, pingtrack turns two off-the-shelf cameras and a laptop into a live 3D tracker at about 60 fps. Print three AprilTags and it calibrates itself.

Two tracks

Track M learns whole-body table tennis from human players and is on hold while we focus on a near-term deadline. The simplified track ships now, adding reach to the team’s controller without changing it.

Learning from human players

A whole-body tracker learns to imitate professional strokes from motion capture, retargeted onto all 29 of the G1’s joints, while holding a calm, balanced stance between them.

A space of skills

The tracker is distilled into a compact skill space. A learned prior keeps every choice human-like and balanced, so a policy can pick a stroke instead of steering 29 joints.

Returning the ball from skill space

A return policy chooses skills inside a safety barrier around the prior, plus small wrist corrections, to meet each incoming ball. It is built and waiting for training.

40,000 robots per GPU

Track M trains in Isaac Lab on Newton physics with motor delays and torque limits modelled. A contact guard stops training at any physics fault instead of hiding it.

A small learned residual

On the simplified track, a PPO policy nudges eight arm and waist joints at 48 Hz on top of the team’s controller and never touches the legs. It kept zero falls and roughly halved foot slip.

Strokes invented on a GPU

About 1,280 copies of the controller run side by side in MuJoCo Warp while a search proposes new strokes. The full controller then decides which ones to keep. The current set adds 474 cm² of reach and 172 more clean returns, with zero falls.

Visual odometry as one input

cuVSLAM tracks image features and reports how the camera moved. It feeds odometry next to leg motion and the gyro, but never sets the field pose. On a mirror-image pitch a loop closure could pick the wrong half.

A particle filter on the field map

Up to 3,000 pose guesses, 500 once tracking, each scored by how well the field lines seen by the RGB-D camera fit the map. From an unknown start it converged in 1.07 s.

Smooth and drift-free

The filter corrects the odometry without making it jump. While walking, the error was 1.3 cm median and 3.1 cm at the 95th percentile.