API rate limiter

Fixed window counts per clock minute; it is cheap but allows a burst of twice the limit at the boundary. Sliding window (a log or a weighted counter) smooths that edge. Token bucket refills tokens at a fixed rate and allows bursts up to the bucket size. Leaky bucket drains a queue at a constant rate, so output is smooth but requests wait. Token bucket is the usual default for public APIs.

As close to the edge as possible: in the API gateway or a middleware layer, so rejected requests do not consume backend resources. Different layers can limit differently: per IP at the CDN or WAF, per API key or user at the gateway, and per tenant or resource deeper in services.

A sliding window log stores each request's timestamp (for example in a sorted set) and counts those within the last window: precise but memory-heavy. A sliding window counter keeps counts for the current and previous fixed windows and weights the previous one by overlap: approximate but cheap.

Store tokens and the last refill time per key. On each request, a Lua script calculates tokens added since the last refill (rate × elapsed), caps at capacity, subtracts one if available and saves the new state atomically, returning allowed or denied plus remaining tokens.

Keep the counters in a shared store such as Redis and update them atomically: INCR with an expiry, or a Lua script that refills and takes a token in one step. For very high traffic, each node can take a small local allowance and sync it periodically, accepting a little inaccuracy for speed.

Status 429 Too Many Requests, a Retry-After header saying when to retry, and rate-limit headers such as RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset so well-behaved clients can back off.

By what you want to protect and who you trust: authenticated user or API key for fairness, IP for anonymous traffic (beware shared NAT and IPv6 ranges), tenant for multi-tenant plans, and endpoint or cost for expensive operations.

Assign weights to operations (a search costs 5, a read 1, or token counts for LLM calls) and subtract the weight from the bucket instead of one. This matches limits to real resource use.

Use the central store's clock (for example Redis TIME inside the Lua script) rather than each server's clock, or algorithms that rely on store-side expiry, so skew between app servers does not change limits.

Keep the store close (same region), use pipelining or Lua for a single round trip, cache denials briefly locally, or use local token buckets that sync with the central count periodically, accepting slight over-admission.

Store limits per plan in configuration, look up the caller's plan from the API key (cached), apply the matching bucket size and refill rate, and expose current usage in headers and a dashboard so customers can see when they will be limited.

Rate limiting rejects requests beyond a limit. Throttling slows them down, for example queueing or delaying requests to smooth bursts. Both protect capacity; throttling suits background or internal traffic where waiting is acceptable.

Compute a query cost before execution from depth, list sizes and field weights, reject queries above a maximum cost, and charge the cost against a per-client budget in a token bucket. Return the cost and remaining budget in the response extensions so clients can tune their queries.

Limit connection attempts per user and IP at the gateway, cap concurrent connections per account, and apply a token bucket to messages inside each connection. Close or slow abusive connections with a clear close code rather than silently dropping messages.

Usually fail open, with a conservative local in-memory limit per node, so a cache outage does not become an API outage. Sensitive endpoints such as login or OTP can fail closed instead. Either way, alert on it.

Limit attempts per account and per IP with escalating delays, add CAPTCHA after several failures, alert on credential-stuffing patterns across many accounts, and avoid revealing whether the account exists.

Unit-test the algorithm with a fake clock, load-test with tools like k6 to check limits under concurrency, verify headers and 429 responses, and test behaviour when the store is slow or down.

Read Retry-After, back off exponentially with jitter, avoid retry storms, queue non-urgent work, and surface a clear message to users when they hit a hard limit.

Smoothing output to a constant rate: requests enter a queue that drains at a fixed pace, which suits protecting downstream systems that cannot handle bursts, at the cost of added latency or drops when the queue is full.

A client can send the full limit at the end of one window and again at the start of the next, doubling the rate briefly. Sliding windows or token buckets smooth this.

Use a semaphore or counter of in-flight requests per key, incremented on start and decremented on finish (with a TTL safety net), rejecting new ones above the maximum. This protects slow, expensive endpoints.

Use allowlists for service accounts or internal networks, separate higher limits for partners, and authenticate internal calls so exemptions cannot be spoofed by headers.

Per-region limits are fast and simple but let a client exceed the global total by spreading across regions. Global limits need cross-region coordination or periodic syncing; many systems accept per-region limits sized as a share of the global one.

Decide a failure mode per endpoint: fail open with a generous local in-memory limit for most APIs, fail closed for sensitive ones like login. Use short timeouts and a circuit breaker so the limiter never becomes the outage itself.

Put a shared token bucket in front of the client library sized to the provider's quota, queue or delay calls when tokens run out, respect the provider's Retry-After, and spread the quota across workers through a central store.

Use per-tenant limits plus fair queuing: each tenant gets its own queue or weight, and a scheduler takes work round-robin or by weight, so one tenant's backlog waits behind its own share instead of blocking everyone.

Run it in shadow mode first, logging who would have been limited without rejecting. Review the impact, notify affected customers, then enforce it gradually with a feature flag, watching 429 rates and support tickets.

Track allowed and rejected counts per key, endpoint and plan, top limited clients, limiter latency and store errors, and alert on sudden spikes in 429s that may indicate attacks or misconfigured limits.

Adjusting limits automatically based on system health, such as lowering limits when latency or error rates rise (load shedding), so the system protects itself during incidents.

429 means this client exceeded its limit. 503 means the service as a whole is overloaded or shedding load, regardless of the client's usage.

Publish limits per plan and endpoint, the headers returned, recommended retry behaviour and how to request higher limits, and keep them versioned with the API docs.

Edge limits stop floods before they reach your servers and absorb volumetric attacks cheaply, but they see less context, usually only IP, path and headers, so per-user or per-plan limits still belong in the API gateway or application.

1. A503 Service Unavailable
2. B409 Conflict
3. C403 Forbidden
4. D429 Too Many RequestsCorrect answer

Why: 429 is the standard status, ideally with a Retry-After header saying when to try again.

1. AAfter the business logic
2. BAt the gateway or edgeCorrect answer
3. COnly in the database
4. DIn the browser

Why: Rejected requests should not reach the backend.

1. ACache-Control
2. BX-Retry
3. CExpires
4. DRetry-AfterCorrect answer

Why: Sent with 429 responses.

1. AServer clocks can driftCorrect answer
2. BIt is faster
3. CIt is required by HTTP
4. DTo save memory

Why: Skew would change the effective limit.

1. ARound robin
2. BFixed window counter
3. CToken bucketCorrect answer
4. DLeaky bucket

Why: Tokens accumulate up to the bucket size, so a burst can spend them, while the refill rate caps the average.

1. AToken bucket
2. BLeaky bucket
3. CSliding window logCorrect answer
4. DFixed window counter

Why: Precise but memory-heavy.

1. AThey need Lua
2. BThey never reset
3. CThey are too precise
4. DBursts at window boundariesCorrect answer

Why: Up to double the limit around the boundary.

1. ATime of day
2. BRequest path only
3. CUser or API keyCorrect answer
4. DBrowser type

Why: IPs may be shared by many users.

1. ALeaky bucketCorrect answer
2. BFixed window
3. CSliding log
4. DRandom drop

Why: Requests drain from a queue at a fixed pace.

1. ABy IP only
2. BThey should not be limited
3. CBy tokens or cost, not just requestsCorrect answer
4. DBy request count only

Why: Requests vary hugely in cost.

1. AResponse size
2. BHeaders
3. CThe number of in-flight requestsCorrect answer
4. DRequests per day

Why: Protects slow endpoints.

1. ACount HTTP requests only
2. BBlock all nested queries
3. CLimit by response size after sending
4. DCharge a computed query cost against a budgetCorrect answer

Why: One GraphQL request can cost far more than another, so cost is the fair unit.

1. ALogging who would be limited without rejectingCorrect answer
2. BHiding the limit from documentation
3. CLimiting only at night
4. DApplying it to internal users only

Why: It shows the real impact before anyone is blocked.

1. AIt avoids storing keys
2. BLua is faster than C
3. CThe check and the update run atomically in one stepCorrect answer
4. DIt encrypts the counters

Why: Redis runs a script atomically, so no other request can sneak in between reading the count and updating it.

1. ANothing
2. BRandom jitterCorrect answer
3. CFixed 1 ms waits
4. DImmediate retries

Why: Avoids synchronised retry storms.

1. A404
2. B503Correct answer
3. C429
4. D401

Why: 429 means one client hit its limit.

1. ANo limits
2. BCaching responses
3. CPer-account limits and CAPTCHA after failuresCorrect answer
4. DHigher limits

Why: Defends against brute force.

1. AIgnoring bursts
2. BDelaying requests instead of rejectingCorrect answer
3. CAlways rejecting
4. DOnly applying to GET

Why: Requests wait rather than fail.

1. Ak6Correct answer
2. BPrettier
3. CWebpack
4. DESLint

Why: Generates concurrent traffic.

1. ASystem health like latency and errorsCorrect answer
2. BTime zones
3. CCode style
4. DBrowser versions

Why: Sheds load during incidents.

1. ABypass authentication
2. BUse more memory
3. CExceed the global total across regionsCorrect answer
4. DNever send requests

Why: Coordination is needed for strict global limits.

1. ASynchronous logging
2. BA Lua script or pipeliningCorrect answer
3. CMore app servers
4. DBigger payloads

Why: Read and write happen together.

1. AFail open with no limit
2. BReturn 200 for everyone
3. CDisable passwords
4. DFail closedCorrect answer

Why: Brute-force protection matters more than availability for login.

1. AFair queuing with per-tenant sharesCorrect answer
2. BA single FIFO queue
3. CBigger servers only
4. DDisabling retries

Why: Each tenant waits behind its own work, not everyone's.

1. ASent as fast as possible
2. BRetried immediately on 429
3. CMade from the browser
4. DShaped by a shared token bucket sized to the quotaCorrect answer

Why: Staying under the provider's quota avoids their 429s and bans.

1. AA per-connection token bucketCorrect answer
2. BThe HTTP 429 status
3. CDNS TTLs
4. DThe TLS handshake

Why: Messages arrive on one connection, so the limit lives inside it.

1. AOnly CPU
2. BOnly disk
3. CNothing
4. DAllowed and rejected counts per keyCorrect answer

Why: Find abusive or misconfigured clients.

1. AA spoofable header
2. BNo authentication
3. CBy user agent
4. DAuthenticated service identities or allowlistsCorrect answer

Why: Exemptions must not be forgeable.

1. AContent-Length
2. BAccept
3. CRateLimit-Remaining and RateLimit-ResetCorrect answer
4. DETag

Why: Standardised rate-limit headers.

1. AIn the client
2. BIn DNS
3. CConfiguration looked up by API keyCorrect answer
4. DHard-coded in each handler

Why: Easy to change per plan.

1. AOnly IPv4
2. BMAC addresses
3. CA prefix such as /64Correct answer
4. DEach full address

Why: One user may hold many IPv6 addresses.

1. AThe user's plan and account identityCorrect answer
2. BThe client IP
3. CThe request path
4. DRequest rate per IP

Why: Per-plan limits need application context the edge lacks.