From the internet to your answer

What this lesson shows

Trained once on filtered internet text (guess, miss, nudge, repeat), then frozen; live answers come from tools feeding the frozen model.

The question it opens with

It answers about today's weather, but it was built months ago. How was it made, and how does it know NOW?

The walkthrough, in the lesson's own words

• It answers about right now. Tap where you think that answer comes from.
• First: how the model was made, from raw internet to frozen numbers.
• Then: how your question gets answered today, with the model frozen.
• The model was built months ago, yet the weather answer is right. Where did it come from?
• Right. The model itself is frozen, its training ended months ago. The app fetched today's weather and pasted it next to your question. Let's watch both halves of that story.
• It can't have: training ended months before today, so today's weather was never in the data. The app fetched it and pasted it next to your question. Let's watch both halves of that story.
• No sensors. The model is just frozen numbers. The app fetched today's weather with a tool and pasted it next to your question. Let's watch both halves of that story.
• One arc: filtered internet text taught a guessing machine by trial and error, the finished numbers were frozen, and apps bolt tools onto that frozen machine so it can answer about right now.
• ChatGPT looks answers up in a big database of facts, and searches the web whenever it wants.
• There is no fact database inside: only frozen numbers tuned during training. When it cites something current, the APP ran a search tool and pasted the results into the conversation for the model to read.
• Where exactly does the training data come from?
• Web crawls, digitized books, code repositories, licensed article archives, and question-answer sites. Most of what is collected gets thrown away: duplicates, spam, broken text, and content filtered out for quality or safety. You will drive that exact cleaning funnel yourself in the data curation lab.
• That is a second, shorter phase called post-training: humans write example answers and rank the model's attempts, and those preferences tune the same frozen-to-be numbers. It is why the raw next-word guesser behaves like an assistant. Act 2 shows the training loop up close, and the post-training lesson shows this polishing step.
• ChatGPT quotes a news article published this morning, but its training ended months ago. How is that possible?
• A search tool. The app noticed the question needs fresh facts, searched the web, and pasted the article text into the conversation. The frozen model then wrote a summary of what it was handed. Training knowledge ends at a cutoff; everything newer arrives through tools.
• A real, correct, up-to-the-minute answer, from a model that finished training months ago.
• It starts as a huge slice of the internet: pages, books, code, forums.
• Most of it is thrown away: duplicates, spam, junk. Quality in, quality out.
• Then the game from 0.1, in reverse: show a sentence, hide the next word, let the model guess, nudge its numbers when it misses. Repeated trillions of times, for months, on thousands of GPUs.
• When training stops, the numbers are frozen. This is the model that ships.
• Frozen. From this moment the numbers never change, no matter what you ask it.
• Your question reaches the app. The model alone can't answer: its knowledge stops at its training cutoff, months ago.
• So the app (not the model) runs a tool: a live weather lookup.
• The result is pasted into the conversation, right next to your question.
• Now the model does the only thing it ever does, predict the next word, but reading fresh facts. That loop writes your answer.

Key takeaway

The whole pipeline in one picture: data → training → frozen model → tools → your answer.

What you can do after this lesson

You can sketch the full arc: where the training data comes from, what training changes, why the model is frozen, and how apps answer live questions anyway.

Prefer reading first? This lesson has a companion explainer: Training vs inference

Where it leads: But how can a machine rank words by meaning? It has to turn them into numbers.

This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.