System Design

1. Steps in System Design

Ideas:

Inference Engines: Learn the difference between vLLM, TGI, and Ollama.
RAG Stack: Understand Vector DBs (Pinecone/Qdrant) and Embedding strategies.
Observability: Learn how to monitor Data Drift (classic) and Hallucinations/Faithfulness (LLMs).
Optimization: Understand Quantization (FP8, INT4) and why it’s necessary for cost-effective usage.

This has short courses on specific things you want to build and specific topics in DeepLearning.

Examples:

Designing Machine Learning Systems by Chip Huyen

The definitive guide for building reliable ML pipelines. It covers everything from data labeling to monitoring drift in production.
Machine Learning System Design Interview by Ali Aminian & Alex Xu

Follows the ByteByteGo style but applied to ML problems like Recommendation Systems and Search Ranking.
The LLM Engineering Handbook by Paul Iusztin & Maxime Labonne

Focuses on the "modern stack": RAG, fine-tuning, and deploying LLMs at scale.

Designing Data-Intensive Applications (2nd Edition) by Martin Kleppmann & Chris Riccomini (2026)

Still the "Gold Standard." The new edition adds vital context on cloud-native patterns and modern stream processing.
- The best resources for learning the template of a design interview.
- Volume 2 covers complex topics like S3, Proximity Services, and Payment Systems.
Building Microservices by Sam Newman

For understanding service boundaries, communication protocols (gRPC, REST), and distributed transactions (Saga pattern).

LRU Cache
Queue from stacks: Implement a FIFO queue using only stack operations.

(Two stacks (in + out) lazy transfer. Amortized $O(1)$ per op)
Min stack: Stack that also returns its current minimum in O(1)

(Main stack + auxillary min stack. O(1) per op. O(n) space.)
O(1) random set: insert, remove, getRandom all in average O(1)

(Hash map to create a set and an array of items for random. When item is removed from hashmap swap with last item in array.)
FIFO queue with fixed size

(Use circular buffer)
Rate Limiter

(Token bucket: A counter that fills (lazily on each request) with a fixed rate)
KV store with TTL: each entry expires after a time-to-live

(Hashmap from key to (value and expiry timestamp) and a min heap for eviction)
Time based KV store: For each key, store multiple values with timestamps; get(key, timestamp_max) should return the latest value with timestamp less <= timestamp_max. timestamps arrive in increasing order.

(Hashmap to a sorted array. Insert is just append at the end and find is binary search. O(lg n) get O(1) set)