Choose how to adapt

Four ways to get a model that knows your business

Choose the least expensive intervention that solves the actual problem.

Approach	What changes	Best fit	Business example
Prompting	Instructions and examples sent with each request	Clarifying a task, format, role, or policy	Ask for an incident summary in the company’s standard five headings
Retrieval (RAG)	Current source material is added to the request context	Changing facts that must be cited or updated	Retrieve the latest product catalog and support policy before answering
Fine-tuning	Some or all model weights change	Stable behavior, tone, format, or a narrow task	Train on approved escalation examples so outputs follow the house workflow
Pretraining from scratch	A new base model and its broad capabilities are created	Exceptional scale, sovereignty, language, or genuinely unique data domains	Build a sovereign-language foundation model when no suitable base can be used

Most organizations should begin with prompting, add retrieval for current knowledge, and change weights only when behavior—not merely facts—must change. Pretraining from scratch is rarely the right business adaptation strategy.

Prompting and retrieval leave the base weights alone

Prompting is instant and reversible. Retrieval keeps knowledge outside the model, so updating or rolling back the source updates behavior without a training run. Retrieval still needs good chunking, ranking, permissions, citations, and evaluation; it is not automatically grounded just because documents were fetched.

Fine-tuning changes behavior

Full fine-tuning updates every trainable weight and can achieve the highest adaptation ceiling, but it consumes substantial memory and creates a complete model variant. Narrow data can also cause catastrophic forgetting.

LoRA freezes the base and learns two thin matrices whose product forms a full-shaped update. That makes one base reusable across many small adapters.

Interactive explanation

Inspect a LoRA update

Core + Expert

Low-rank adaptation

Keep the base fixed and learn two thin matrices

interactive explanation

LoRA does not retrain every value in W. It learns A and B, multiplies them into a full-shaped adjustment, and adds that adjustment to the frozen base.

01Full fine-tuning02Freeze the base03Insert A and B04Build delta W05Merge or attach

Base matrix W · train every value

gradients update W

A · 8 × 4096thin

×

B · 4096 × 8thin

Inserted after the base is frozen

Full-shaped adjustment

ΔW = B × A

Effective weightsW

Full fine-tuning changes the complete base matrix.

Rank8Hold the frozen base at 4-bit (QLoRA)

Trainable parameters≈ 7,000M

Share of 7B model100%

Base storage14 GB

Compare fine-tuning methods

Interactive explanation

Compare fine-tuning methods

Core + Expert

Adaptation method comparison

See where each adaptation method intervenes

interactive explanation

Compare what moves, what stays frozen, how much state training owns, and what cost remains during inference.

01Full fine-tuning02LoRA03QLoRA04Adapters05Prefix tuning

Embeddings

Attention

Feed-forward

Layer norms

Output head

Trainable share100%

Training memoryVery high

Inference costNone

Best fitMaximum capacity

• QLoRA keeps the frozen base quantized while training LoRA parameters.
• Adapters insert small permanent modules and therefore add inference work.
• Prefix or prompt tuning learns virtual input tokens but consumes context.
• Head-only training is a useful low-cost baseline.

Core check

Which approach adds changing knowledge without modifying weights?

ARetrieval-augmented generationBFull fine-tuningCTraining from scratch