How it works
A pen as a string of moves
When you write on the pad, the page records your pen as a list of points. It finds your baseline and the height of your small letters, rescales the line so that a lowercase x is one unit tall, and resamples every stroke at an even step of a fifth of that height. Your line becomes a few hundred small moves, each one a step to the next point plus a flag for whether the pen was lifted first. That is exactly the form the network learned from.
Guessing the next move
The network is a stack of three LSTM layers with 3.8 million weights. It reads the moves one at a time and, after each, predicts the next. The prediction isn’t a single point but a cloud: a blend of 20 two-dimensional bell curves, plus the chance that the pen lifts. To write, the page picks a move at random from that cloud, feeds it back in, and repeats. With Show its guesses ticked, the circle beside the sheet magnifies the pen tip and draws the strongest few bell curves as ellipses. Partway along a stroke there is one tight ellipse just ahead of the pen; near the end of a letter a wide one appears where the pen might land if it lifts, and the caption gives the odds that it will. The neatness slider sharpens the cloud before each pick, trading some wobble for legibility.
Which letter it’s on
The network reads the text through a soft window that slides along the characters. With every pen move it decides how far to slide the window, so it works out for itself that an m takes longer than an i. The strip under the sheet shows where the window is centred. When the window slides past the last character, the line is done. This design comes from Alex Graves’s 2013 paper Generating Sequences With Recurrent Neural Networks, the first to show a network writing by hand.
Your line as a prompt
There is no setting for “style” anywhere in the network. Instead, the page first feeds it your line and its text, as if it had just written that line itself. By the end its memory holds your slant, your size and spacing, the way you make each letter, and it carries on in the same hand with the words you typed. Graves called this priming; it is the same trick as prompting a language model. To make it work, every training example was two or three lines by one person laid end to end on a single baseline, so carrying on in someone’s hand is exactly what the network practised.
How well it copies a hand
To check that priming really carries a hand, 12 of the 170 writers were kept out of training. For each one, the network read their “The quick brown fox” and then wrote six sentences that the same person had also written, three times each. Five things were measured on the generated lines and on the person’s real ones, all relative to their own letter height: slant, width per letter, pen lifts per letter, how high the tall letters reach and how low the tails hang.
| Primed with their line | Not primed | Chance | |
|---|---|---|---|
| Closer to the right writer than to another one | 80% | 55% | 50% |
| Right writer is the closest of all 12 | 33% | 17% | 8% |
| Right writer is among the closest 3 | 58% | 25% | 25% |
Slant and letter width carry over most strongly: across the 12 writers, the generated and real values correlate at 0.80 and 0.82. How often the pen lifts carries over partly (0.47). How tall the tall letters are and how far the tails drop hardly carry over at all (0.28 and 0.13). Swapping the primes gives the same picture: lines primed with someone else’s writing match that someone else 81% of the time.
Where it goes wrong
Most of the 170 writers in the training set printed rather than joined their letters, so a joined hand tends to come back partly printed. Writing with a mouse is shaky, and the network copies the shake faithfully. Now and then it drops or doubles a letter, more often at low neatness. Characters it never saw anyone write, such as @ or accented letters, are left out. And it judges your size and baseline from one line: across 165 people who wrote this same line, that estimate of letter height was within about 13%.
Credits
A. Graves, Generating Sequences With Recurrent Neural Networks (2013). Handwriting from the BRUSH dataset: A. Kotani, S. Tellex and J. Tompkin, Generating Handwriting via Decoupled Style Descriptors, ECCV 2020, used under its non-commercial research licence. Colours sampled from a Wikimedia Commons photograph of a 1980 maths worksheet run off a spirit duplicator (KurtSchwitters, CC BY 4.0). Everything runs in your browser; your handwriting never leaves the page.