Rate limiting failed authentication: the flood nobody counts

A while back we published a post arguing that you should rate limit by API key, not by IP. It is good advice. Bucketing on the caller's network address punishes every customer behind the same corporate NAT, and it hands anyone with a header-rewriting proxy a free bypass.

It also, if you follow it literally, opens a hole you can drive a botnet through. Rate limiting failed authentication attempts is the one thing that advice cannot do, because a request with a bad key has no key to bucket on. The requests you most want to stop are precisely the ones your limiter never counts.

I spent a morning measuring how bad that gets. The short version: 300 requests carrying 300 different guessed keys, all from one address, produced 300 responses and not a single 429.

The measurement: 300 guessed keys, zero rate limits

Here is the shape almost everyone lands on after reading the by-key advice. Authenticate, then rate limit the customer you just identified:

import express from 'express'
import { rateLimit } from 'express-rate-limit'

const app = express()
app.set('trust proxy', 1)

app.use(authenticate)                    // 401 if the key is unknown
app.use(rateLimit({
  windowMs: 60_000,
  limit: 3,
  standardHeaders: 'draft-8',
  keyGenerator: (req) => req.consumer,   // the identity auth just resolved
}))

app.get('/v1/thing', (req, res) => res.json({ ok: true }))

That is a correct per-customer limiter. Each paying customer gets their own bucket, nobody shares, and the header-rotation bypass is gone. Now point 300 requests at it from a single address, each carrying a different randomly generated key, and count what comes back.

flood tally:            { "401": 300 }
first 429 at request:   null
authenticate() ran:     302 times
rate limiter ran:       2 times
valid key, same IP:     200

Zero 429s. The limiter ran twice in the entire run, and both times were the two legitimate requests I sent afterwards to check the app still worked. Three hundred guesses went straight through the expensive part of the stack and out the other side as 401s, at whatever rate the attacker felt like sending them.

Measured on Express 5.2.1, express-rate-limit 8.6.1, Node 26.0.0.

Why rate limiting failed authentication does not happen by itself

The mechanism is dull, which is exactly why it survives code review. Your auth middleware returns a 401. Returning a response ends the request. Everything registered after it never executes, and your rate limiter is registered after it.

So the counter only ever sees requests that already passed authentication. It is a limiter for people who are allowed in, positioned to protect your handlers, doing its job perfectly, while the front door absorbs an unbounded flood.

This is not an Express quirk. We have now probed the ordering in five frameworks, and the failure shows up in every one that lets you move the limiter behind auth. Fastify gets there without you doing anything at all: @fastify/rate-limit attaches as a route-level hook, so a global auth hook runs first by default. A 500-request flood against a 3-per-minute limit there returned 500 × 401 and, again, zero 429s.

The uncomfortable part is that both placements are defensible. In front of auth, the limiter can only bucket on IP, which is the identifier you were told to stop using. Behind auth, it buckets on the right identity but never sees an attack. There is no single position that solves both, and pretending otherwise is how you end up with one limiter and a false sense of coverage.

Can I just bucket on the raw API key header?

No, and this one is worse than doing nothing. An attacker guessing keys sends a different key on every request, so keying the limiter on the raw header mints a brand new bucket each time. Every request has a fresh counter at 1 of 3, forever. You have built a rate limiter that cannot rate limit, and as a bonus you are filling your limiter's store with attacker-supplied cardinality.

Bucket on identity you resolved and trust, never on a string the caller handed you. That rule holds whether the string is X-Forwarded-For or X-API-Key.

What the flood actually costs you

"It returns 401, so who cares" is the natural next thought. Here is who cares: your key lookup is the most expensive thing in that request path, and it is the only thing that runs.

Validating a key means, at minimum, a hash and a store round trip. If you followed the standard advice and are hashing keys before storage, that is real CPU per attempt. In my probe I stood in for the store with a deliberately non-trivial hash, and the 300 rejected requests burned about 15 ms of CPU doing nothing but proving that random strings are not valid keys. Swap the local hash for a database query or a network hop to an auth service and that number stops being funny.

Then there is the second bill. If your validation layer is a metered service, every guess is a billable event. ReqKey's own pricing counts a request as one key validation or one logged API call, so an unfiltered flood of garbage keys is a flood of validations you pay for. I would rather tell you that plainly than have you discover it from an invoice.

Response-based counting: only failures fill the bucket

The fix has a name, and it comes from the WAF world rather than the framework world. Cloudflare's rate-limiting guidance recommends configuring the counting expression separately from the matching expression, so that you "count only requests that return error responses (such as 401 or 403)". Their credential-stuffing example increments only when the response code is in (401, 403).

The idea transfers cleanly to the application layer, and express-rate-limit ships it as skipSuccessfulRequests. Put a coarse, address-keyed limiter in front of auth that only counts failures, and keep the precise per-customer limiter behind it:

import { rateLimit, ipKeyGenerator } from 'express-rate-limit'

// 1. Coarse gate. Generous, address-keyed, counts failures only.
app.use(rateLimit({
  windowMs: 60_000,
  limit: 30,
  standardHeaders: 'draft-8',
  skipSuccessfulRequests: true,
  keyGenerator: (req) => ipKeyGenerator(req.ip),
}))

// 2. Now identify the caller.
app.use(authenticate)

// 3. Precise per-customer limit, unchanged.
app.use(rateLimit({
  windowMs: 60_000,
  limit: 3,
  standardHeaders: 'draft-8',
  keyGenerator: (req) => req.consumer,
}))

Same 300-request flood, same single address:

flood tally:            { "401": 30, "429": 270 }
first 429 at request:   31
authenticate() ran:     31 times   (was 302)
CPU spent on garbage:   ~1 ms      (was ~15 ms)

Thirty guesses get a real answer, the remaining 270 get bounced before the key lookup runs at all. Your expensive path did a fifteenth of the work. That is the whole trick.

And skipSuccessfulRequests really does keep legitimate traffic out of that bucket. Ten valid requests against a limit of five, with the flag on, all returned 200. With the flag off, the same ten returned 200 five times and then 429 five times. Successes do not accumulate.

The part nobody writes down: the fix bills a paying customer

Here is where I stopped feeling clever. skipSuccessfulRequests skips the increment. It does not skip the check.

So once an attacker has spent that address's failure budget, the bucket is empty for everyone on that address, including customers holding perfectly valid keys. I measured it directly, with the limit dropped to five to keep the output readable:

attacker, 5 x bad key      →  401, 401, 401, 401, 401
valid key, SAME address    →  429   RateLimit: "5-in-1min"; r=0; t=60
                                   Retry-After: 60
valid key, DIFFERENT addr  →  200   RateLimit: "5-in-1min"; r=4; t=60

A customer who did nothing wrong, whose key is valid, whose own per-customer bucket is untouched, gets a 429 and a one-minute cooldown because somebody sharing their egress address was guessing keys. Their traffic never contributed a single increment. They are collateral damage of a control that exists to protect them.

This is the same shared-address problem that made per-key limiting the right answer in the first place, and it has come back around to bite the mitigation. Carrier-grade NAT, corporate egress, a customer's Lambda fleet on a handful of NAT gateway addresses: any of these puts your paying users behind the same counter as an attacker.

Which leads to the honest conclusion. The coarse gate is a damage limiter, not a security control. Set it generously enough that no plausible customer ever trips it, treat it as protection for your infrastructure rather than protection for your keys, and accept that a determined attacker spreading guesses across a residential proxy pool stays under it comfortably. If you need to actually stop distributed key guessing, that job belongs at an edge that has device and reputation signals your application does not.

How shipped APIs handle it

Two public examples worth copying, both of which treat invalid credentials as a distinct traffic class rather than an afterthought.

Etherscan throttles unauthenticated and malformed traffic separately from real traffic. A request with a missing or placeholder key comes back with "OK-Missing/Invalid API Key, rate limit of 1/5sec applied", and sustained bad-key traffic eventually gets "Too many invalid api key attempts, please try again later". Notice the design: the bad-key path is not blocked, it is given its own much slower lane.

GitLab ships a failed-authentication ban for Git and container registry traffic. Their docs put it plainly: GitLab returns HTTP status code 403 for 1 hour if 30 failed authentication requests were received in a 3-minute period from a single IP address. It is disabled by default, which tells you something about how confident they are in that address-based heuristic.

That 403 is also a small design choice worth arguing with. A ban is a rate limit, and 429 with a Retry-After header tells a well-behaved client exactly when to come back. A 403 tells it to give up, which is fine for an attacker and unhelpful for the developer who fat-fingered a token and is now staring at a permission error.

A ladder that actually works

Layer	Buckets on	Stops	Costs you
Format rejection	Nothing	Malformed keys, before any lookup	Nothing. Do this first.
Coarse failure gate	Address	Single-source floods	Collateral damage on shared addresses
Per-customer limit	Resolved consumer	One customer overrunning their share	Nothing, but sees no failed auth
Edge / WAF rules	Reputation, device, ASN	Distributed guessing	Money, and another system to run

The cheapest win on that list is the first row, and it is the one people skip. If your keys carry a prefix and a fixed length, a regex rejects most garbage in microseconds without touching your store. It will not stop an attacker who read your docs, but it deletes the entire background noise of internet scanners probing with random strings, and it costs you one if statement.

If you want the fundamentals of limiter design underneath all this, this walkthrough is a solid 14 minutes:

Where a managed key layer helps, and where it does not

Moving validation to a service like ReqKey changes the economics of the flood without changing its shape.

What it fixes: rejection is cheap and it never touches your money. POST /key/validate runs its checks in a documented order with the credit deduction last, so an unknown, revoked, expired or wrong-project key is rejected before anything is charged. An unknown key comes back as HTTP 200 with {"valid": false, "message": "Key not found"}, and a rate-limited one comes back as 429 with {"valid": false, "rateLimited": true} plus Retry-After. Per the error docs, a rate-limited request consumes no credits and no rate-limit quota, so a throttled client recovers the moment it slows down.

What it does not fix, and I would rather say this than let you find out in production: ReqKey's rateLimit is consumer-level. It is {limit, window} attached to a consumer and shared by all that consumer's keys. A guessed key has no consumer, so it has no bucket. The structural problem in this post is not something a key-management vendor can wave away, and any vendor telling you otherwise is selling you the per-customer limiter and calling it brute-force protection.

The coarse gate stays your job, in front of the validation call. That is true whether the thing behind it is ReqKey, a competitor, or your own Postgres table.

Key takeaways

• Your per-customer rate limiter has never seen an attack. It sits behind authentication, and failed authentication short-circuits before it. Measure it on your own service: send 100 requests with garbage keys and count the 429s. If the answer is zero, this post is about you.
• Failed authentication needs its own counter, in front of auth, keyed on address. There is no identity to bucket on yet, so address is what you have. Use response-based counting so successes do not fill it.
• That counter will eventually punish an innocent customer. A valid key from a shared egress address got a 429 in my probe after an attacker spent the bucket. Set the limit generously and treat it as infrastructure protection, not as key security.
• Never bucket on a string the caller controls. Keying on the raw API key header gives every guess a fresh bucket, which is strictly worse than keying on address.
• Reject on format before you reject on lookup. A prefix and length check costs one comparison and removes most scanner noise before it reaches your hash, your database, or your bill.

If you want the per-customer half of this handled for you, that is what ReqKey does: {limit, window} on a consumer, enforced at the edge, with 429s that cost the caller nothing. The free tier runs 100,000 requests a month, which is plenty to point a flood at a staging key and see for yourself which requests get counted. Build the coarse gate yourself. Now you know why it has to be there.