EC2 runs virtual servers called instances. Use Amazon Machine Images (AMIs) to launch their software, instance store for temporary local data, and automatic recovery to restore supported instances after an underlying host failure.
Accelerated computing for AI
Training and serving large models needs accelerators. AWS offers its own chips (Trainium for training, Inferentia for inference) next to NVIDIA GPU instances.
How it works
| Accelerator | Instances | Main job |
|---|---|---|
| AWS Trainium | Trn2, Trn3 | Training (and serving) |
| AWS Inferentia | Inf2 | Cost-efficient inference |
| NVIDIA GPUs | P5, P6, G6e | Training and inference, graphics |
Exam cues
- If the question says cost-efficient training on AWS-designed chips, think Trainium.
- If the question says cost-efficient inference on AWS-designed chips, think Inferentia.
Remember
- Trainium and Inferentia use the AWS Neuron SDK.
Trap: Trainium is for training, Inferentia for inference. The names give it away.
From the AWS docs (4)
- AWS Trainium is an AWS-designed AI chip for training and serving models; Amazon EC2 Trn2 and Trn3 UltraServers use it for large-scale training and inference.
- AWS Inferentia is an AWS-designed AI chip for inference; Amazon EC2 Inf2 instances use it for high-performance, cost-efficient deep learning and generative AI inference.
- NVIDIA GPU instances such as Amazon EC2 P5 and P6 serve demanding training and inference, and G6e instances serve cost-efficient inference and graphics workloads.
- AWS Trainium and AWS Inferentia use the AWS Neuron SDK, which supports PyTorch, JAX and vLLM.
Flashcards
Nothing to review here right now. Pick other topics or another filter.
·
All 4 cards as a list
A company wants the most cost-efficient AWS-designed chips to train its own large models. Which option fits?
AWS Trainium (Amazon EC2 Trn2 or Trn3 instances).
A company wants low-cost, high-throughput inference for a deep learning model on AWS-designed chips. Which EC2 instances fit?
Amazon EC2 Inf2 instances, powered by AWS Inferentia.
A graphics application needs NVIDIA acceleration on Amazon EC2. Which type of compute should the company choose?
NVIDIA GPU instances, such as Amazon EC2 G6e.
Is AWS Inferentia meant for training large models?
No. Inferentia targets inference; Trainium targets training (and also serves models).