Under the hood
How I think: predicting the next character
Big AI Brain's growing neural network does exactly one thing, millions of times: it looks at the last 32 characters and guesses the next one. Everything it says comes out of that.
One letter at a time
Give it Why did the chick and it should guess e. Then n. Then a space. Keep going and a whole punchline falls out. When it's new, its guesses are random, which is why it babbles. Every night it studies thousands of examples and nudges its connections so the right letter becomes a little more likely.
Why this works at all
To predict the next letter well, you end up needing to know spelling, then grammar, then facts, then jokes. The job is tiny, but doing it well forces real knowledge into the network. The big AI systems you've heard of use the same recipe, with pieces of words instead of single letters and billions of connections instead of Big AI Brain's modest pile. Same recipe, much bigger pot.
The heart of it, in eight lines
def next_char(net, text):
ctx = encode(text[-32:]) # last 32 characters -> numbers
emb = net.E[ctx].reshape(-1) # a learned "meaning" vector per character
h1 = np.tanh(emb @ net.W1 + net.b1) # layer 1: pattern detectors
h2 = h1 + np.tanh(h1 @ net.W2 + net.b2) # layer 2, plus a shortcut
logits = h2 @ net.W3 + net.b3 # a score for every possible character
probs = softmax(logits) # scores -> probabilities
return sample(probs) # pick one (a little randomness = personality)
Those W and b values are the weights, the brain's connections. Learning means adjusting them. Big AI Brain has about half a million of them on day one.