How it was made

The teacher is an implicit cloth solver written for this page. It models stretch, shear and bending, friction, and contact with the table, balls, rods and the cloth itself, using barrier functions that never let anything pass through. Each frame takes ten substeps, and each substep is a small optimisation solved with Newton’s method. It is accurate and slow.

The student is a graph neural network in the style of MeshGraphNets, with a three-level hierarchy borrowed from HOOD: messages pass between neighbouring vertices, between every second vertex and between every fourth, so a tug on one side reaches the far side in a handful of rounds instead of thirty. Vertices that come close in space but lie far apart on the cloth get extra contact edges. Each frame it reads positions, velocities, grippers and nearby obstacles, and predicts every vertex’s acceleration.

Training used about 750 five-second clips of towels being dropped, dragged, folded, flung, shaken and draped, simulated by the teacher on a laptop GPU. The network learns from the teacher’s frames, and also from the teacher’s physics directly: on its own rollouts it is scored by the energy the solver minimises, so it learns to recover from its own mistakes rather than drift into states the teacher never showed it.

How close it gets. On 16 held-out clips, the network’s towel is on average 5 cm from the teacher’s after half a second and 15 cm after five seconds. The buttons under “Solver vs network” replay three of them side by side. It errs most when a towel that is partly lying on the table is let go: the lifted part falls more slowly than it should.