Designing an Extensible Multi-Tier Rate Limiter
Rate limiters test your ability to structure reusable library code, apply object-oriented design patterns (Strategy, Registry), manage memory efficiency, and perform time-based calculations without spinning background threads.
4 Questions to Establish Leadership Scope
“Are we throttling based on client IP address, authenticated user ID, or an API token header?”
“Should the architecture allow plugging in different algorithms (e.g. Token Bucket vs. Sliding Window) via an interface?”
“Should exceeded requests be immediately rejected with an HTTP 429 status code, or queued for deferred execution?”
“Is this an in-memory library for a single edge gateway process, or should we design the storage interface for Redis clustering?”
Stores only 2 numbers per client: `currentTokens` and `lastRefillTimestamp`. Tokens are refilled lazily on each request. Zero background threads needed. Handles sudden traffic bursts gracefully.
Stores an array of timestamps for every request in the rolling window. Perfectly accurate, but under high traffic (e.g. 5,000 requests/sec), memory footprint explodes.
Full 200-line code is collapsed to preserve screen focus. Click expand to inspect token bucket math, concurrency locks, and unit tests.