LEARNING TO QWOP
Highlighted keys show controller inputs. Press Q, W, O or P to take control.
Control a simulated runner with Q, W, O and P. Compare a scripted controller with three stages of neural network training: behavior cloning, DAgger and PPO. The final model runs 100 m in 18.6 seconds.
How it was trained
Training uses a custom rigid-body simulator in Python. This page uses a matching JavaScript version. Each neural network has 77 inputs and two hidden layers with tanh activations. It selects one of nine key combinations 30 times per second.
PPO reached 21.4 seconds per 100 m after 300 million steps on a laptop CPU. Another 4.4 billion steps on a GPU reduced the time to 18.6 seconds. At roughly 130 million steps, PPO initialized from DAgger reached 22.1 seconds, compared with 38.0 seconds from random weights. This comparison used one run per setup. Penalizing knee contact produced an upright gait at about one-third the speed.
Statistics use 2,048 runs per controller with randomized starting poses and a 100-second limit. Training code: qwop/train.