minGPT TensorFlow.js

About

This is Karpathy's minGPT (gpt-micro) reimplemented in TensorFlow.js and pre-trained on ~1M characters of Shakespeare. The model was trained offline in Node.js and the weights are loaded for inference here.

Architecture

• 4 transformer layers, 4 attention heads, 128-dim embeddings (~826K params)
• 128-character context window, 65-character vocabulary
• Pre-norm decoder-only transformer with causal masking

Generation

• Top-k sampling (k=10) with adjustable temperature
• Lower temperature = more predictable, higher = more creative