← Back to the course
AI explainers
Plain-language, answer-first guides to how modern AI works. Each one answers the question up front, names what people get wrong, and links to a short interactive lesson where you operate the idea yourself.
Foundations
What a language model is and how it turns your words into an answer.
- How LLMs workHow does a large language model actually work?A large language model is a next-word predictor. It turns your text into numbers, runs them through billions of learned weights, and produces a probability for every possible next token. It picks one, adds it to the text, and repeats. There is no database of facts and no lookup step. The intelligence is a very good statistical guess about what comes next, learned from a huge amount of text. Everything else, chat, code, agents, is built on that one loop.Read the explainer →
- TokenizationWhat is a token, and why do AI models count tokens instead of words?A token is a chunk of text, often a word-piece rather than a whole word. Before a model can read your text it splits it into tokens and maps each to a number. Models bill and budget in tokens, not words, because tokens are the real unit they process. The same idea can cost more or less depending on how it splits: common English words are one token, while code, rare words, and many non-English languages break into more tokens per word.Read the explainer →
- EmbeddingsWhat is an embedding, and how does it capture meaning?An embedding turns a piece of text into a list of numbers, a vector, positioned so that similar meanings sit close together. Meaning becomes geometry. Words and sentences that are used in similar ways end up near each other, even when they share no exact words. That is what lets a search box find the right help article from a different phrasing, and what lets retrieval pull the relevant document for an AI answer. The model learns these positions from how language is actually used.Read the explainer →
- Context windowWhat is a context window, and why does the AI forget things mid-conversation?The context window is everything the model can see while writing its next word: your messages, its replies, instructions, documents, all measured in tokens up to a fixed limit. It is working memory, not storage. When a conversation outgrows the window, older content is dropped or summarized away, and the model answers as if it never existed. That is why it forgot your name from an hour ago. Nothing was stored, so nothing could be recalled.Read the explainer →
- Next-token predictionIs ChatGPT really just predicting the next word?Yes, and that is not a small thing. The entire model is trained on one objective: given text, guess the next token. Do that near-perfectly across trillions of tokens and the guessing has to absorb grammar, facts, style, code idioms and chains of reasoning-shaped text, because they all improve the guess. Chat is the same trick pointed at a transcript: the model predicts what a helpful assistant would say next, one token at a time.Read the explainer →
- Word vector analogiesWhy does king minus man plus woman land near queen?Because when words become vectors, relationships become directions. In a well-trained embedding space, the arrow from man to woman points roughly the same way as the arrow from king to queen: a 'gender direction' the model learned purely from how words co-occur in text. Add that arrow to king and you land near queen. The famous demo, from word2vec in 2013, was the first striking proof that meaning could be geometry, the idea every modern model is built on.Read the explainer →
- Training vs inferenceWhat is the difference between training and inference?Training is where the numbers are learned: the model reads enormous amounts of text, gets things wrong, and every weight is nudged to be less wrong. It happens once, costs a fortune, and then stops. Inference is every time you use the result: the weights are frozen and read, not changed, to turn your prompt into an answer. Nothing you type during inference is learned. That single fact explains why a model does not know today's news, why it forgets your last conversation, and why answers are billed per token.Read the explainer →
Inside the transformer
The architecture every modern model is built from, one part at a time.
- AttentionWhat does the attention mechanism do in a transformer?Attention lets each word look at the other words in the sentence and decide which ones matter for it right now. In 'the trophy did not fit in the suitcase because it was too big', attention is what tells the model that 'it' refers to the trophy. Each position gathers a weighted blend of the others, leaning hardest on the ones that fit. This is how a model handles pronouns, long-range references, and the way a word's meaning shifts with its context.Read the explainer →
- Mixture of ExpertsWhat is a Mixture of Experts model, and why do labs use it?A Mixture of Experts model replaces some layers with many parallel sub-networks, the experts, plus a small router that picks a few of them for each token. The model can hold a huge number of parameters, but only the chosen experts run, so each token costs a fraction of the compute. That is the trick: the capacity of a giant model at something closer to the price of a small one. The router's choices are learned, not programmed.Read the explainer →
- SoftmaxWhat does softmax do, and where does an LLM use it?Softmax turns a list of raw scores into percentages that add up to 100. It exaggerates gaps: a score slightly ahead becomes a share far ahead, and low scores shrink toward zero without ever quite reaching it. Language models use it in the two places that matter most: at the output, to turn word scores into the next-word probabilities you sample from, and inside attention, to decide how much each earlier word contributes to the current one.Read the explainer →
- Positional encodingHow does a transformer know word order, if attention looks everywhere at once?Attention on its own treats a sentence as a bag of words: 'dog bites man' and 'man bites dog' would look identical. So models mix a position signal into every token's vector before attention runs. Each position gets its own mathematical stamp, and the stamp becomes part of the token's meaning, so 'first word' and 'seventh word' are different even for the same word. Many modern models use a rotating variant, RoPE, which encodes relative distance between words.Read the explainer →
- TransformersWhat is a transformer, and how does the architecture actually work?A transformer is a stack of identical blocks, each doing two things. First, attention: every token looks at the earlier tokens and pulls in what it needs, so meaning flows between words. Second, a feed-forward network transforms each token on its own, where most of the model's learned knowledge lives. A running representation of each token passes through the stack, refined a little per block, until the top of the stack is sharp enough to score every possible next token.Read the explainer →
- Query, Key, ValueWhat do Query, Key and Value actually mean in attention?Attention is a search. Each token asks a question, its Query. Each token also advertises what it contains, its Key, and carries the content it would hand over, its Value. Matching a Query against every Key produces scores, softmax turns the scores into shares, and the token's new representation is the share-weighted blend of the Values. All three roles are computed from the same token vector through learned weight matrices. Nobody writes them; training finds them.Read the explainer →
How models are trained
Where the numbers come from, and what changes a model after training.
- Gradient descentHow does a neural network actually learn?Training is a loop. The model makes a prediction, a loss function measures how wrong it was, and calculus tells every single weight which tiny nudge would have made the error smaller. All the weights take their nudge, and the loop repeats, billions of times. That is gradient descent: rolling downhill on an error landscape you can never see whole, one small step at a time. Nobody writes the rules. The rules are whatever weight values the descent settles into.Read the explainer →
- Fine-tuning vs promptingWhen should you fine-tune a model, and when is prompting enough?Prompting changes the input: instructions, examples, retrieved documents, all with frozen weights, instant and reversible. Fine-tuning changes the weights: extra training on your data, often through small LoRA adapters, slower and stickier. The working rule: prompting plus retrieval for knowledge and one-off behavior, fine-tuning for form, when you need a consistent style, format or skill baked in across thousands of calls. Most production systems never need fine-tuning at all.Read the explainer →
- Scaling lawsWhy did making models bigger keep making them better?Researchers found that a model's error falls along smooth, predictable curves as you grow three things: parameters, training data and compute. Predictable is the key word: labs could forecast how good a run would be before spending on it, which justified spending more. The Chinchilla result added the balance: for a fixed compute budget, a smaller model trained on more data beats a bigger one trained on less, roughly twenty tokens of data per parameter.Read the explainer →
- RLHFWhat is RLHF, and why does a text predictor act like an assistant?Pretraining produces a raw predictor that continues text in any direction, helpful or not. RLHF is the finishing school: humans compare pairs of model answers and pick the better one, a reward model learns to imitate those preferences, and the LLM is then trained to score highly on that reward. The result answers questions, follows instructions and declines harmful requests, not because it was told rules, but because responses shaped that way rated better.Read the explainer →
- In-context learningHow can a model learn from examples in the prompt without being retrained?Put three examples of a task in your prompt and the model performs the fourth in the same pattern, with zero weight changes. That is in-context learning. It works because pretraining relentlessly rewarded continuing patterns, so pattern-following became one of the model's strongest reflexes. The learning is real but rented: it exists only inside the current context window, and it vanishes the moment the conversation ends.Read the explainer →
- DistillationWhat is model distillation, and why is the mini version so good?Distillation trains a small student model on the outputs of a large teacher model rather than on raw text alone. The teacher's answers are cleaner and more consistent than the internet, and in the richer versions the student also learns from how confident the teacher was across every option, not just its final pick. The student ends up far better than a model of its size trained from scratch. It inherits the teacher's style and its mistakes, and it does not exceed the teacher.Read the explainer →
Speed, cost and control
What happens when a model actually runs, and what you pay for it.
- Model routingWhat is model routing, and how do you choose which AI model to use?Model routing sends each request to the cheapest model that can still handle it, instead of using one big model for everything. Most requests are easy, so a small, cheap model clears them, and only the hard few need a frontier model. A router can decide up front, or cascade: try a small model, check the result, and escalate only if it falls short. Done well, you hold a quality bar while cutting cost and latency, because you stop paying frontier prices for easy work.Read the explainer →
- KV cacheWhat is the KV cache, and why does it matter for speed and cost?When a model generates text, each new token needs to look back at every earlier token through attention. Recomputing that look-back from scratch for every token would be brutally slow. The KV cache stores each token's attention keys and values the first time they are computed, so every later step reuses them and only computes the newest token. The price is memory: the cache grows with context length, which is why long conversations eat GPU memory.Read the explainer →
- QuantizationWhat is quantization, and how does it let big models run on small hardware?Quantization stores each of a model's weights with fewer bits, for example 4 bits instead of 16. The model keeps the same number of weights, but each one becomes a coarser number, so the whole model shrinks to a quarter of the memory and moves through the hardware faster. Done well, the quality loss is small, because the weights did not need all that precision to begin with. It is the main reason capable models now run on laptops and phones.Read the explainer →
- TemperatureWhat does the temperature setting actually do to an AI model?Temperature reshapes the model's next-word probabilities before it picks one. Low temperature sharpens them, so the top choice wins almost every time and answers come out consistent. High temperature flattens them, so second and third choices get real chances, which reads as variety or creativity, and sometimes as nonsense. It changes nothing about what the model knows. It only changes how much risk the model takes when choosing among words it already considers plausible.Read the explainer →
- GPUs for AIWhy are GPUs, and not CPUs, the hardware of the AI boom?Almost everything a language model does is multiplying enormous grids of numbers, and those millions of little multiply-adds do not depend on each other. A CPU has a handful of fast cores built to race through steps one after another. A GPU has thousands of simple cores built to do the same small operation on huge batches at once. For matrix math, the thousands win. Graphics needed exactly that kind of math first, which is why the gaming chip became the AI chip.Read the explainer →
- Reasoning modelsHow do reasoning models actually work?A reasoning model writes out a long private working-out before it answers. That draft is generated the same way as any other text, one token at a time, but it gives the model somewhere to break a problem into steps, try an approach and correct itself. Spending more tokens at answer time reliably improves hard maths, code and multi-step logic. It costs more and takes longer, and on simple questions it buys nothing.Read the explainer →
- Speculative decodingWhat is speculative decoding and why does it make models faster?A small fast model drafts the next few tokens, then the big model checks all of them in a single pass. Checking several tokens at once costs about as much as generating one, because the bottleneck is reading the weights out of memory, not the arithmetic. Every draft token the big model agrees with is free speed; the first disagreement is corrected and drafting restarts. The output is identical to what the big model would have produced alone. Only the wall clock changes.Read the explainer →
- Prompt cachingWhat is prompt caching and how much does it actually save?Most requests to an assistant repeat a long, identical opening: the system instructions, the tool definitions, the document you are asking about. Prompt caching stores the model's processed form of that prefix so the next request skips recomputing it. Providers charge a large discount on cached input and it comes back faster. The rules are strict: the prefix must match exactly from the first token, and the cache expires after minutes of disuse, so it rewards putting stable content first.Read the explainer →
- Inference economicsWhy is running AI so expensive, and what are you paying for?Every answer is computed from scratch on hardware that has to be reserved whether or not you are using it. There is no cached page to serve. Cost scales with the tokens in your prompt plus the tokens generated, and generation is the expensive half because each new token requires reading the entire model out of memory again. That is why prices are quoted per token, why output costs several times more than input, and why long context and long thinking both show up on the bill.Read the explainer →
Agents and tools
What changes when the model can act, not just answer.
- AI agentsWhat makes an AI agent different from a chatbot?An agent is a language model placed inside a loop that can take actions. A chatbot writes a reply and stops. An agent proposes a tool call, a harness runs it, the result comes back into the context, and the model decides the next step, repeating until the task is done. The model still only predicts text. The power comes from the loop around it: read a file, run a search, call an API, check the result, try again. That loop is what turns a predictor into something that gets work done.Read the explainer →
- Context engineeringWhat is context engineering, and how is it different from prompt engineering?Context engineering is deciding everything the model gets to see for a task, not just the wording of one prompt. A modern system assembles its context from many sources: instructions, the user's request, retrieved documents, past turns, tool results, memory, and files. The model can only reason about what is in that window, and the window is a limited budget. Good context engineering puts the right information in, leaves noise out, and orders it well. It is the discipline that replaced 'prompt tips' once systems got complex.Read the explainer →
- Prompt injectionWhat is prompt injection, and why are AI apps insecure in new ways?Prompt injection is when untrusted content the model reads contains instructions that hijack it. A model cannot reliably tell your instructions apart from text inside a web page, email, or document it was asked to process. So an attacker can hide 'ignore your task and do this instead' in that content. It becomes dangerous when an agent combines three things: access to private data, exposure to untrusted content, and a way to send data out. That combination, the lethal trifecta, is the recipe for data exfiltration.Read the explainer →
- Computer-use agentsHow can AI click around apps, and when is that safe?A computer-use agent operates a screen the way a person would: it takes a screenshot, plans a step, clicks or types, looks at the result, and verifies before moving on. That loop, look, plan, act, observe, verify, is what lets a model use software that has no API. The catch is that interfaces are brittle and some actions cannot be undone. So verification and human approval on risky steps are not extras; they are what separates a useful agent from one that confidently clicks the wrong button.Read the explainer →
- MCPWhat is the Model Context Protocol, and why does it matter?MCP, the Model Context Protocol, is an open standard that lets an AI assistant discover and call outside tools and data. Before it, every assistant needed custom code for every tool: N assistants times M tools meant N times M integrations. With MCP, a tool ships one server describing what it can do, and any MCP-speaking assistant can use it. It is a plug standard, not a model. What changes is how much an assistant can reach without anyone writing glue code first.Read the explainer →
- AI coding agentsHow do AI coding agents work?A coding agent is a language model wrapped in a loop with tools: read a file, search the codebase, edit, run the tests, read the output, decide again. The model never edits your project directly. It proposes a tool call, the harness runs it, and the result comes back as text the model reads on the next turn. Almost everything that separates a good agent from a bad one lives in that harness: which tools exist, what context is gathered, and where a human is asked to confirm.Read the explainer →
- Agent vs chatbotWhat is the difference between an AI agent and a chatbot?The model is the same. A chatbot takes your message and answers once. An agent decides on an action, runs a tool, reads what came back, and decides again, repeating until it judges the goal met. That loop is the whole difference, and it changes everything around it: an agent needs tools, permissions, a step limit and a place for a human to approve anything irreversible. A chatbot that can search the web once is still a chatbot. What makes it an agent is choosing the next step for itself.Read the explainer →
Building on it, and trusting it
Grounding answers in your data, measuring quality, and knowing the limits.
- RAGWhat is retrieval-augmented generation (RAG)?RAG is how an AI answers from your documents instead of only its training. When you ask a question, the system searches your content for the most relevant passages, pastes them into the model's context, and asks the model to answer using them. The model never memorized your data. It reads the retrieved text at answer time. That is why RAG can cite sources and stay current, and why most RAG failures are really retrieval failures: if the right passage was not fetched, the model cannot use it.Read the explainer →
- LLM evalsWhat are evals, and how do teams know an AI feature actually works?An eval is a repeatable test for an AI feature: a set of inputs, and a way to score whether the outputs are good enough. Because models are non-deterministic and 'looks fine' does not scale, teams build evals to catch regressions before users do. Scoring can be exact checks, rubrics, or another model acting as a judge. The hard part is keeping evals honest: a frozen offline set can go stale or leak into training, so production and adversarial tests catch what it cannot.Read the explainer →
- HallucinationWhy do AI models confidently make things up?Because they are built to continue text plausibly, not to report facts. A language model always produces the most likely next words. When the truth is in its training data, likely and true usually coincide. When it is not, the model fills the gap with something that sounds right, in perfect fluent prose, because nothing inside it distinguishes a remembered fact from a plausible pattern. Hallucination is not a glitch on top of the system. It is the system, running without grounding.Read the explainer →
- Vector databasesWhat is a vector database, and why does every RAG stack have one?A vector database stores embeddings, the number-lists that encode meaning, and answers one question fast: which stored items sit closest to this new vector? That is search by meaning instead of by keyword. Embed your documents once, embed the user's question at query time, fetch the nearest chunks, and paste them into the model's context. That fetch step is the retrieval in RAG, and the specialized indexes exist because comparing millions of vectors naively would be far too slow.Read the explainer →
- LLM limitsWhat are the real limits of large language models?The honest list: they cannot reliably know what they don't know, so confidence and correctness come apart. They have no memory beyond the context window unless a product bolts one on. Exact arithmetic and counting are unreliable without a calculator tool. They cannot check facts against the world, only against patterns in training text. And they cannot act, browse or run code by themselves; every 'agent' is a harness of tools and permissions wrapped around the same predictor.Read the explainer →
- LLM as a judgeWhat does LLM as a judge mean, and does it work?Using a model to grade another model's output against a written rubric, instead of paying a human to read every answer. It is fast, cheap and consistent enough to be the default in most eval pipelines. It also inherits the judge's blind spots: it rewards long, confident, well-formatted answers, it can prefer text from a model like itself, and it favours whichever answer it saw first. Calibrate it against human labels before you trust it.Read the explainer →
- RAG vs fine-tuningShould I use RAG or fine-tuning?Ask what you are missing. If the model lacks facts, use retrieval: keep the documents outside the model and fetch the relevant ones into the prompt, so a wrong fact is an edit and answers can cite a source. If the model has the facts but gets the format, tone or task wrong, fine-tune: it buys consistency and shorter prompts. Knowledge is retrieval, behavior is fine-tuning. Most teams need retrieval, and the ones who reach for fine-tuning first usually end up doing both.Read the explainer →
- Data flywheelWhat is a data flywheel, and why is it called the moat in AI?A data flywheel is a loop where using a product generates data that makes the product better, which attracts more use, which generates more data. In AI products the loop is concrete: real traffic gets logged, failures get mined and labeled, and the results feed evals and fine-tuning, so quality compounds with every turn. It is called the moat because strong base models are available to everyone, while your production data, what your users actually ask and where the product fails, is yours alone.Read the explainer →
Beyond text
Images, audio and video, on the same underlying ideas.
- Multimodal AIHow do images, audio, and documents become something a model can reason about?Multimodal models turn every input, text, image, audio, or a screenshot, into vectors in one shared space, then reason over all of them together. An image is cut into patches and each patch becomes a vector, the same kind of vector a word becomes. Because they live in the same space, the model can compare a picture and a caption, answer a question about a chart, or describe what is on a screen. It is the same machinery as text, pointed at more kinds of input.Read the explainer →
- Diffusion modelsHow do AI image generators actually work?A diffusion model is trained by taking real images, adding noise until they are static, and learning to undo one step of that noise. To generate, it starts from pure random noise and runs the learned undo step many times, each time nudging the picture toward something more image-like. Your text prompt steers every step, so the noise resolves into a picture that matches the description. Nothing is copied or retrieved. The image is denoised into existence.Read the explainer →
- Video and world modelsHow does AI generate video, and what is a world model?Video generators extend image diffusion across time: instead of denoising one picture, they denoise a block of frames together, so movement stays consistent from one frame to the next. To keep a bouncing ball or a poured drink believable, the model has to absorb rough regularities of how things move. That learned, implicit sense of how the world behaves is what people mean by a world model. It is learned from video, not programmed, so it is approximate and it breaks.Read the explainer →