Rate limiting algorithms that protect users and dependencies

Rate limiting is not a single number applied to every route. A cheap metadata read, a password attempt, a webhook delivery, and an expensive report consume different resources and create different risks. Define the protected resource first: CPU, database queries, provider quota, account fairness, or abuse resistance. Then decide the identity dimension, such as account, API key, IP, tenant, route, or a combination. A limit that is fair to one dimension can be unfair or ineffective at another.

Publish the limit as part of the API contract when clients are expected to cooperate. Include the limit window, remaining allowance, reset hint, and a retry guidance header where possible. Do not reveal internal capacity details that help attackers tune abuse. The public number should reflect a product promise, while the internal limiter can add an emergency guardrail when the dependency is unhealthy.

A fixed-window limiter counts requests in buckets such as one minute and resets the count at the boundary. It is easy to implement and explain, but a client can send a burst at the end of one window and another burst at the start of the next. The service experiences twice the nominal rate in a short period. This may be acceptable for a low-cost endpoint, but it is dangerous for a fragile provider or an expensive query.

If you use fixed windows, add a separate burst or concurrency guard and choose a window that matches the operation. Store counters with an atomic increment and an expiration so concurrent instances do not race. Make reset behavior consistent across regions or document that limits are per region. A fixed-window response should not promise a precision the distributed counter cannot provide.

A sliding-window counter estimates requests across the current interval and the previous interval, weighting the previous count by how much of it overlaps. It smooths the boundary burst without storing every request. A sliding log stores timestamps and is more exact but uses more memory and cleanup work. Choose the approach based on the traffic volume and the fairness requirement, not because one algorithm sounds more advanced.

Distributed sliding windows require a shared clock and atomic operations. Clock skew between instances can change the effective boundary, so use a centralized data store's time or tolerate a small error. Keep the key cardinality bounded by expiring inactive identities. If a client can choose an arbitrary tenant or header for the key, validate that identity before incrementing to prevent an attacker from creating unlimited limiter entries.

A token bucket refills at a steady rate up to a maximum capacity. Each request consumes one or more tokens, so the bucket allows a controlled burst while preserving an average rate. This is useful for API clients that batch work or for a webhook receiver that needs to absorb a short provider retry burst. The capacity and refill rate should represent the dependency's safe envelope, not an arbitrary number that makes a dashboard green.

Token buckets need a monotonic time source and an atomic update. Store the last refill time and token count together so concurrent requests cannot both spend the same token. When the shared store is unavailable, fail open only for low-risk reads and fail closed or use a local emergency budget for abuse-sensitive actions. Be explicit about that degraded behavior because it changes the protection boundary during an outage.

A request-per-second limit does not protect a service when requests take minutes. Ten long-running requests can consume more memory and connections than a hundred quick reads. A concurrency limiter controls the number of in-flight operations and releases a permit when work finishes or is cancelled. Combine it with a rate limit when both arrival bursts and work duration matter. Return a retryable response or queue the work rather than holding a connection while waiting for a permit.

Apply concurrency limits at the narrowest dependency boundary. A global semaphore can let one noisy tenant block everyone, while a per-tenant limit may fail to protect a shared database. Partition by the failure domain and keep a small reserve for health and administrative traffic. Measure wait time and rejection reason so operators can tell whether clients are arriving too quickly or the service has become slow.

Authentication gives you a stronger identity than IP address, but unauthenticated routes still need protection against shared networks and rotating addresses. Use layered limits: a broad edge limit, a credential or tenant limit, and a route-specific cost. Assign heavier costs to operations that trigger expensive work. Do not let a client choose a trusted identity header. Resolve the identity from verified authentication or a server-side mapping.

An error response should tell a well-behaved client how to recover without helping an attacker enumerate accounts or limits. Use a consistent status such as 429, a bounded Retry-After, and a request ID. Avoid exposing remaining quota for another identity. Record internal categories for blocked IP, blocked account, dependency protection, and concurrency saturation so product teams can adjust the right policy.

Test exact boundaries, bursts, concurrent increments, refill behavior, cancellation, shared-store latency, store outages, and multiple regions. Use a fake clock for deterministic unit tests and a controllable load test for integration behavior. Assert that one logical request cannot obtain multiple permits through retries. Verify that a 429 response is not cached as a successful resource response and that clients honor the retry guidance.

Rate limiting is a control system, not a punishment system. Protect the resource, select the right identity, choose an algorithm that matches duration and burst behavior, and expose enough information for legitimate clients to adapt. Review limits after real incidents. A limiter that blocks customers during a provider outage may need a dependency-aware mode rather than a larger static quota.

Publish the identity key, window, burst allowance, response status, and Retry-After behavior for every public limit. Test clock skew, multiple instances, a provider outage, and a client that retries too aggressively. If a user can legitimately perform a long-running operation, rate-limit admission separately from completion. Operators should be able to distinguish an abusive caller from a capacity limit and tune the policy without changing business logic.

Select a token bucket, sliding window, concurrency limit, or combination based on the protected resource. Key the limit by the identity that needs fairness, add a global dependency guardrail, and return a bounded Retry-After hint. Keep counters and decisions atomic across instances so a scale-out event cannot multiply the allowance.

Test bursts, sustained traffic, multiple instances, clock skew, anonymous callers, authenticated tenants, and dependency degradation. Assert that allowed, rejected, and retried requests match the documented policy. Inspect limit key, remaining allowance, reset time, queue depth, and provider quota together; a local limiter can hide a downstream limit.

Monitor rejection rate, hot keys, counter-store latency, and legitimate-user impact. Keep a conservative emergency limit when the counter store is unavailable, and document whether each route fails open or closed. Adjust limits through reviewed configuration with an owner and expiry rather than editing a running instance during an incident.

Compare token bucket and sliding-window literature, RFC 6585's 429 status, and the dependency's quota documentation. Record the cost unit and identity dimension for every limiter.