System Design
Table of Contents
1. AI Engineer System Design Resources
Ideas:
- Inference Engines: Learn the difference between vLLM, TGI, and Ollama.
- RAG Stack: Understand Vector DBs (Pinecone/Qdrant) and Embedding strategies.
- Observability: Learn how to monitor Data Drift (classic) and Hallucinations/Faithfulness (LLMs).
- Optimization: Understand Quantization (FP8, INT4) and why it’s necessary for cost-effective usage.
1.1. Full Stack Deep Learning
- Lecture 5: Deployment
- Lab 5: Troubleshooting & Testing
- Lecture 7: Foundation Models
1.2. DeepLearning.ai
https://www.deeplearning.ai/courses/
This has short courses on specific things you want to build and specific topics in DeepLearning.
Examples:
1.3. Books
Designing Machine Learning Systems by Chip Huyen
The definitive guide for building reliable ML pipelines. It covers everything from data labeling to monitoring drift in production.
Machine Learning System Design Interview by Ali Aminian & Alex Xu
Follows the ByteByteGo style but applied to ML problems like Recommendation Systems and Search Ranking.
The LLM Engineering Handbook by Paul Iusztin & Maxime Labonne
Focuses on the "modern stack": RAG, fine-tuning, and deploying LLMs at scale.
2. Software Engineer System Design Resources
Designing Data-Intensive Applications (2nd Edition) by Martin Kleppmann & Chris Riccomini (2026)
Still the "Gold Standard." The new edition adds vital context on cloud-native patterns and modern stream processing.
- The best resources for learning the template of a design interview.
- Volume 2 covers complex topics like S3, Proximity Services, and Payment Systems.
Building Microservices by Sam Newman
For understanding service boundaries, communication protocols (gRPC, REST), and distributed transactions (Saga pattern).