Plain-language explainer
AI video, and the physics a model picks up by watching
How does AI generate video, and what is a world model?
Video generators extend image diffusion across time: instead of denoising one picture, they denoise a block of frames together, so movement stays consistent from one frame to the next. To keep a bouncing ball or a poured drink believable, the model has to absorb rough regularities of how things move. That learned, implicit sense of how the world behaves is what people mean by a world model. It is learned from video, not programmed, so it is approximate and it breaks.
Last reviewed
Reading is the slow way. Start with a free lesson you can operate right now.
Start free: From the internet to your answer →Free, no code, no signup.
Then go deeper: Video and world models Locked
What people get wrong
- It renders frames one at a time like an animator. It generates a chunk of frames jointly, which is what keeps motion coherent.
- It runs a physics engine. There are no rules for gravity or collision inside it, only patterns absorbed from footage, which is why objects sometimes pass through each other.
- A world model means the system understands the world. It means it predicts what usually happens next on screen, which is a much narrower claim.
Where you see it in real products
- Text-to-video tools such as Sora and Veo.
- Storyboards, ads and short clips generated from a written scene.
- Robotics and self-driving research, where predicting the next frame is used to plan actions.
Common questions
- Why are AI videos usually short?
- Cost and consistency both work against length. Frames are generated together, so compute grows quickly with duration, and small drifts in a character's face or the lighting accumulate. Most tools stitch several short, well-conditioned clips instead of generating one long take.
- Does the model understand physics?
- Not as rules. It has seen enough falling, splashing and colliding to reproduce it convincingly in familiar situations. Ask for something unusual and the failure is telling: liquids that do not conserve volume, objects that merge, shadows that ignore the light.
- Why do people care about world models beyond video?
- Because a system that can predict what happens next can also evaluate a plan before acting. That is directly useful in robotics and self-driving, where trying an action in the real world is expensive, and it is why labs treat video generation as more than an entertainment feature.
Related explainers
One idea at a time, in your inbox
New lessons and explainers, written the way these pages are. Now and then, not daily, and never a sales sequence.
We email you a confirmation link first. Unsubscribe in one click, any time. Privacy.
Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.