2026 · jordimarsal/redis-toolkit

redis-toolkit

Rate-limiting toolkit for LLM gateways: atomic token-bucket quotas, pluggable stores, honest failure modes — with a Javalin gateway demo exposing an OpenAI-style endpoint.

Problem

LLM inference is expensive: unbounded traffic is a DoS on your own wallet. Most limiters race under concurrency or hard-fail when Redis does.

Highlights

• Every quota decision computed atomically — never a read-modify-write that slips a token past the limit.
• On Redis failure: graceful local fallback + loud metrics, not mass 5xx.
• Contract test suite shared by in-memory and Redis implementations, plus concurrency and parity tests.
• Standard X-RateLimit-* headers and correct 429 + Retry-After.

Stack

• Java
• Javalin
• Redis
• Testcontainers

Metrics

429+Retry-After

correct rate-limit contract

2

stores, one contract suite