Parameters become capability · 7 min
What a model carries
A model is a learned function whose parameters, or weights, shape how inputs become outputs.
Plain language, the core causal flow, and required checks.
What a model actually is
A model is a fixed architecture plus a very large collection of learned numbers called weights or parameters. Text is split into tokens and converted to token IDs. The architecture repeatedly combines those IDs with the weights and produces scores for the token that could come next.
It is not a database of memorized sentences and it does not contain handwritten rules for every answer. Its capability comes from learning patterns that make next-token prediction accurate across many kinds of text.
One prediction at a time
The model produces a distribution over its vocabulary. A decoding policy selects one candidate, appends it to the context, and runs the model again. That loop is how a response grows.
Interactive explanation
Explore a next-token distribution
Next-token prediction
Watch a next-token distribution change
A model does not retrieve an answer. It recomputes a distribution over possible next tokens at every position.
The production model predicts the next ___
Lower temperature concentrates probability on the leading choices. Higher temperature spreads probability across more candidates, making selection less predictable.
Temperature does not make the model more knowledgeable. It reshapes the choice: lower values concentrate probability on leading candidates; higher values spread probability across more candidates. The three tabs above are simply different sentence examples—not different model or GPU modes.
Architecture, weights, checkpoint, and context
- Architecture defines the operations and connections.
- Weights are the learned numeric settings used by those operations.
- Checkpoint stores weight tensors and reconstruction metadata.
- Tokenizer maps text fragments to and from token IDs.
- Context window is the text supplied for the current computation; it is not permanent memory.
During ordinary inference, the weights remain fixed. The system repeatedly reads them; it does not learn from the conversation unless another process later trains or adapts the model.
Where billions of parameters come from
Depth adds layers roughly linearly. Width appears in square projection matrices, so increasing width grows parameters much faster. Vocabulary embeddings matter proportionally more in small models, while feed-forward blocks occupy a large share of many dense large models.
Interactive explanation
Build a transformer parameter budget
Parameter and memory budget
Build a transformer and watch width bite
Depth grows parameter count linearly. Width appears inside square matrices, so a modest increase changes memory far faster.
Choose layer count, hidden width, vocabulary size, and feed-forward expansion. These dimensions define the shapes of the learned tensors.
Keep the three levels connected
The orientation map is not another inference animation. It shows where the current lesson sits across model creation, inference-system preparation, and GPU execution. The final replay later runs one prompt through the same stages.
Interactive explanation
Orient across the weights-to-token journey
Course orientation
Three connected views of one token-producing system
Begin with the model, follow one inference request, then zoom inside the GPU work that prefill and decode repeatedly invoke.
Produces the reusable model artifact.
- What weights carry
- How training changes them
- How the artifact is packaged
- Distinguish architecture, parameters, and weights.
- Explain why a checkpoint is not the same as a running service.
Think of the architecture as a machine design and the weights as billions of learned settings inside that machine.
Core check
During inference, what happens to the trained weights?
Terms in this chapter
- Parameter
- A numeric value used by a model operation. Trainable parameters are adjusted during learning.
- Checkpoint
- A saved snapshot of model tensors and associated training or configuration state.
Primary references
- Transformers model loading ↗Hugging Face
Primary runtime documentation for model configuration and state dictionaries.