DEPT·F2 — TRANSPORT & TELEMETRY2026
Redis Toolkit
Rate-limiting toolkit for LLM gateways: atomic token-bucket quotas, pluggable stores, honest failure modes — with a Javalin gateway demo exposing an OpenAI-style endpoint.
The brief
LLM inference is expensive: unbounded traffic is a DoS on your own wallet. Most limiters race under concurrency or hard-fail when Redis does.
What was built
Every quota decision computed atomically — never a read-modify-write that slips a token past the limit.
On Redis failure: graceful local fallback + loud metrics, not mass 5xx.
Contract test suite shared by in-memory and Redis implementations, plus concurrency and parity tests.
Standard X-RateLimit-* headers and correct 429 + Retry-After.
Metrics
429+ Retry-After, always
2stores, one contract suite
STACK
JavaJavalinRedisTestcontainers