Foundation models can be adapted at very different costs: change the prompt, feed better context, retrieve documents, or retrain weights. These sections cover which approach fits which problem and how tuning works.
Choosing a customization approach
Customization runs from cheap to expensive: better prompts, richer context, retrieval (RAG), fine-tuning, continued pre-training and, rarely, training a model from scratch. Start at the cheap end and climb only when results demand it.
How it works
| Approach | Changes the model? | Cost | Use when |
|---|---|---|---|
| Prompt engineering / in-context examples | No | Lowest | Task is clear; examples help |
| Context engineering | No | Low | Model needs user data, memory, tools |
| RAG | No | Low to medium | Answers need private or fresh documents |
| Fine-tuning | Yes | Medium to high | New task or style; labeled data |
| Continued pre-training | Yes | High | Domain language; unlabeled data |
| Distillation | Yes (smaller model) | Medium | Keep quality, cut serving cost |
| Pre-training | New model | Highest | Building a foundation model |
- Try prompts first, adding examples (in-context learning).
- Add context: instructions, history, user profile, tools and retrieved knowledge.
- Fine-tune or continue pre-training only for gaps that context can’t close.
Exam cues
- If the question says least effort or cost, think prompt engineering.
- If the question says private, changing documents and cited sources, think RAG.
- If the question says style or task performance with labeled data, think fine-tuning.
- If the question says domain vocabulary from unlabeled text, think continued pre-training.
Remember
- Prompting, context and RAG leave the weights alone; fine-tuning and pre-training change them.
- RAG and fine-tuning can be combined.
Trap: Fine-tuning doesn’t keep a model current. Facts baked in by training go stale; RAG brings fresh data at query time.
From the AWS docs (8)
- Prioritize prompt engineering and RAG before model customization; start with the least resource-intensive option and move to more advanced methods only if needed.
- Choose continued pre-training or fine-tuning when a specific task needs better performance, you have labeled data for it, or the model must understand domain-specific language such as medical or legal terms.
- RAG builds question answering over custom documents without fine-tuning, can take in new documents in minutes, and returns references to its sources.
- For question answering over custom documents, start with RAG; use fine-tuning when the model must perform additional tasks or match an organization's style. The two can be combined.
- Pre-training builds a foundation model from scratch and requires GPU clusters running continuously for weeks.
- Context engineering treats the context window as a dynamic workspace; prompt engineering focuses on what to ask, while context engineering focuses on what to show the model.
- An engineered context combines the system prompt or instructions, the user query, the user profile, memory of the conversation, tool definitions and MCP servers, and knowledge retrieved with RAG.
- Few-shot prompting puts examples of the task in the prompt itself, improving answers without changing the model.
Fine-tuning methods and training data
Fine-tuning keeps training a pre-trained model on your data, so its weights change. The data decides the result: accurate, representative and varied examples matter more than volume.
How it works
| Method | Data | Purpose |
|---|---|---|
| Instruction tuning | Prompt and response pairs | Follow task instructions |
| Domain adaptation | Domain text | Learn a field’s language |
| Transfer learning | Small dataset for a related task | Reuse what the model already knows |
| RLHF | Human rankings of responses | Align with human preferences |
| LoRA | Same data, small adapters | Cheaper, faster tuning |
- Curate data: remove duplicates and errors, label consistently, remove PII, cover every case the model will see.
- Split into training and validation sets.
- Train, evaluate against a baseline, and repeat.
Exam cues
- If the question says prompt and response examples teach the model a task, think instruction-based fine-tuning.
- If the question says humans rank responses to improve the model, think RLHF.
- If the question says reuse a model trained on a related task, think transfer learning.
- If the question says the tuned model fails on cases missing from training data, think unrepresentative data; add those examples.
Remember
- Fine-tuning changes weights; it needs curated, labeled data.
- Nova SFT: 2,000 to 10,000 samples recommended, 200 minimum.
Trap: RLHF trains the model on human preferences. A human evaluation job only rates models and changes nothing.
From the AWS docs (9)
- Fine-tuning further trains a pre-trained foundation model on your own smaller dataset and changes the model's weights.
- SageMaker JumpStart offers two main fine-tuning approaches, domain adaptation on domain-specific data and instruction-based fine-tuning on prompt and response examples.
- Domain adaptation fine-tuning adapts a pre-trained model with limited domain-specific data when prompt engineering doesn't give enough customization.
- Transfer learning fine-tunes a model pre-trained on one task for a new, related task, so it adapts faster and with less data and compute than training from scratch.
- Reinforcement learning from human feedback (RLHF) has humans score or rank model responses, and that feedback trains the model toward responses people prefer.
- LoRA (Low-Rank Adaptation) trains a small set of adapter weights for lower cost and faster training, while full fine-tuning updates all weights for deeper customization.
- Fine-tuning data should favor quality over quantity, use accurate labels, closely reflect production use cases, and include diverse examples, difficulty levels and edge cases to prevent overfitting.
- For supervised fine-tuning of Amazon Nova models, the recommended dataset size is 2,000 to 10,000 samples, with a minimum of 200.
- RLHF collects human prompts and ideal answers, fine-tunes a pretrained model, trains a separate reward model from human preferences between responses, and then uses that reward model to optimize the language model with reinforcement learning.
Flashcards
Nothing to review here right now. Pick other topics or another filter.
·
All 15 cards as a list
Order these customization approaches from least to most cost and effort - fine-tuning, prompt engineering, continued pre-training, RAG, pre-training from scratch.
Prompt engineering, RAG, fine-tuning, continued pre-training, pre-training from scratch.
A company wants answers grounded in product manuals that are updated every week. Which approach fits best?
RAG, which picks up new documents in minutes and cites sources, with no retraining.
A model must write in a company's distinctive style for a task it does poorly, and the company has many labeled examples. Which approach fits?
Fine-tuning on the labeled examples (after trying prompt engineering and RAG).
What does context engineering add beyond prompt engineering?
It assembles everything the model needs in its context window: instructions, the query, user profile, conversation memory, tool definitions and retrieved knowledge. Prompting decides what to ask; context engineering decides what to show.
When does training a foundation model from scratch make sense for a typical company?
Almost never. Pre-training needs GPU clusters for weeks; adapting an existing FM is far cheaper.
A company's generic prompts work, but the model lacks medical vocabulary and the company has a large unlabeled corpus of clinical text. Which approach fits?
Continued pre-training (domain adaptation) on the unlabeled domain text.
Is fine-tuning the first thing to try when a model's answers are off?
No. Start with prompt engineering, then RAG; fine-tune only if those fall short.
A team trains a model on prompt-and-response pairs that show how to follow instructions for its tasks. Which fine-tuning approach is this?
Instruction-based fine-tuning (instruction tuning).
A chatbot's answers are accurate but sound robotic. Human reviewers rank pairs of answers, and the rankings train the model. What is this technique?
Reinforcement learning from human feedback (RLHF).
A model trained to recognize animals is adapted to recognize dog breeds with a small dataset. Which ML concept is this?
Transfer learning, which reuses a pre-trained model's knowledge for a related task.
Name four qualities of a good fine-tuning dataset.
Any four of: accurate labels, representative of production use, diverse (full range of inputs and edge cases), consistent output format, enough samples, curated and free of contradictions.
A fine-tuned model performs well on one product line and poorly on four others. What is the likely data problem and fix?
The training data wasn't representative. Add labeled examples from all product lines and fine-tune again.
Which fine-tuning training type is cheaper and faster because it trains only small adapter weights?
LoRA (Low-Rank Adaptation). Full fine-tuning updates every weight.
Does a bigger fine-tuning dataset always beat a smaller, carefully curated one?
No. Quality comes first: accurate, representative, diverse examples matter more than raw volume.
In RLHF, what turns human preferences into something reinforcement learning can optimize automatically?
A separate reward model trained on human choices between responses. It then acts as the reward function that optimizes the language model.