Designing Rate Limiting for Multi-Tenant SaaS APIs on AWS

A SaaS application can work perfectly during early development and still run into an interesting problem as soon as real customers start using it.

Imagine you have built a SaaS platform with a shared API. Tenants use that API to manage their data, generate reports, trigger workflows and integrate their own applications.

Most customers make a reasonable number of requests. Then one customer starts running a script that sends hundreds of requests every few seconds. Another customer schedules a bulk import during business hours. Suddenly, response times increase, database connections become busy and other customers begin experiencing the same slowdown.

The problem is not necessarily that your infrastructure is too small. The problem is that one tenant can consume a disproportionate share of a shared resource.

This is where rate limiting becomes an important part of SaaS architecture.

In this article, I will explain how I would approach rate limiting for a multi-tenant SaaS application on AWS, where AWS-managed throttling fits, how a fixed-window counter works, and what changes when rate limits must be enforced independently for each tenant.

1. Why a SaaS application needs rate limiting

Rate limiting controls how many requests a client or tenant can make within a defined period.

Consider a SaaS application with three subscription plans. The following limits are illustrative examples, not recommended defaults for every application.

Subscription plan	Example request limit
Starter	60 requests per minute
Professional	300 requests per minute
Enterprise	Custom limit

Without tenant-aware rate limiting, one customer could make enough requests to affect every other customer sharing the same API, compute capacity or database.

This is commonly described as a noisy-neighbor problem.

Rate limiting helps control request volume, protect downstream services and support subscription-based usage policies. It does not replace authentication, authorization, tenant isolation, concurrency controls or capacity planning.

It is also important to separate two different requirements:

Platform protection: Prevent excessive traffic from overwhelming the API or application.
Tenant enforcement: Apply the appropriate request allowance to each authenticated customer.

These requirements overlap, but they are not the same problem.

2. Where rate limiting fits in an AWS architecture

AWS provides several layers at which requests can be controlled. I would use more than one layer when the application’s availability and isolation requirements justify it.

A typical request flow might look like this:

Client
  |
  v
Amazon CloudFront (if required)
  |
  v
AWS WAF
  |
  v
Amazon API Gateway
  |
  v
Authentication and tenant resolution
  |
  v
Tenant-aware rate limiter
  |
  v
Lambda or ECS application services
  |
  v
Database and downstream dependencies

Not every SaaS application needs every component. For example, an application running behind an Application Load Balancer might perform authentication and tenant-aware rate limiting in its application middleware rather than use API Gateway.

The important design principle is that excessive traffic should be rejected as early as practical, while subscription-specific decisions should use a trusted tenant identity.

AWS WAF: protect the public entry point

AWS WAF supports rate-based rules that count requests and apply actions when traffic exceeds configured thresholds.

For example, a rule can help protect a login endpoint against repeated requests from a source IP address. Rate-based rules can also use supported request aggregation criteria and scope-down statements to target particular traffic.

This is useful for reducing abusive traffic before it reaches the application.

However, WAF rate-based rules are designed for approximate traffic control, not precise per-customer quota accounting. An IP address is also not necessarily equivalent to a SaaS tenant: many legitimate users may share an IP address, while one tenant may connect from several networks.

Reference: Using rate-based rules in AWS WAF.

Amazon API Gateway: protect APIs and routes

API Gateway supports account-level, API-level and route- or method-level throttling, depending on the API type.

For HTTP APIs, you can configure route-level rate and burst settings. For REST APIs, usage plans and API keys can provide client-specific throttling targets and request quotas.

These controls are useful for setting a baseline across the API before requests reach application code.

There is an important limitation: AWS documents API Gateway throttling and usage-plan quotas as best-effort targets rather than guaranteed hard limits. API keys should not be treated as a replacement for authentication or authorization.

Reference: Throttling HTTP APIs and Usage plans and API keys.

Application-level enforcement: apply tenant-specific policies

Suppose two customers use the same API route:

Tenant A has a Starter subscription with a limit of 60 requests per minute.
Tenant B has a Professional subscription with a limit of 300 requests per minute.

A single global API limit cannot express those policies independently.

The application needs to identify the authenticated tenant, resolve its applicable plan and enforce its own tenant-specific limit.

For this layer, a distributed counter stored in Redis is one possible implementation. On AWS, a managed Redis-compatible service such as Amazon ElastiCache for Valkey can provide shared counter state to application instances.

The exact choice depends on expected traffic, latency requirements, availability requirements and operational cost.

3. Understanding the fixed-window algorithm

A fixed-window rate limiter divides time into consecutive windows of a known duration and counts requests within each window.

For example, assume a tenant may make 60 requests per minute.

The application uses a 60-second window and associates requests with the appropriate minute-sized bucket. Each accepted request increments the bucket’s counter. When the configured allowance has been consumed, further requests are rejected until the next window begins.

The algorithm is straightforward:

Resolve the tenant and its rate limit.
Determine the current time window.
Increment the counter associated with that tenant and window.
Compare the resulting count with the allowed request limit.
Allow the request if it is within the limit; otherwise, return HTTP 429.

The counter must be shared across all application instances. A counter held only in a process’s memory is insufficient when the application runs across multiple containers, Lambda execution environments or EC2 instances.

What the counter represents

For a 60-request-per-minute policy, an illustrative counter key could be:

rate-limit:tenant-123:reports:29400000

The components represent:

tenant-123: the authenticated tenant.
reports: the API route or operation category.
29400000: the time-window identifier.

The window identifier can be calculated by dividing the Unix timestamp in seconds by the window duration and taking the integer portion.

This produces a distinct bucket for each time window.

Including the tenant and operation category lets the application apply different policies to different customers and expensive operations. If all routes share one allowance, the route category can instead represent a common tenant-wide bucket.

4. A practical implementation using Redis

Consider a Node.js service running on ECS that receives authenticated API requests. A Redis-compatible store holds shared counters, and application middleware checks the counter before executing the business operation.

The flow is:

Validate the user’s access token.
Resolve the trusted tenant ID from the verified identity or a server-side API-key mapping.
Determine the tenant’s rate policy.
Calculate the time-window key.
Increment the counter atomically.
Return HTTP 429 if the counter exceeds the allowed limit.
Continue to the business handler when the request is allowed.

Never trust a tenant ID supplied only through an arbitrary client-controlled header. The application must establish the tenant identity through authenticated claims, an authorization decision or a trusted server-side mapping.

Keep the counter update atomic

Two requests could arrive at almost exactly the same time. If each request reads a counter and independently writes back an incremented value, one increment could be lost.

Redis supports Lua scripts so that the counter update and expiry logic can run atomically.

For a fixed-window key, a simple script is:

local count = redis.call('INCR', KEYS[1])

if count == 1 then
  redis.call('EXPIRE', KEYS[1], tonumber(ARGV[1]))
end

return count

The script increments the counter and sets an expiration when the key is first created. The application passes the Redis key and an expiration duration as parameters.

The key itself already contains the window identifier, so requests in the next window use a different key. The expiry duration can be longer than the window duration to allow old buckets to be cleaned up safely.

After the script returns, the application compares the counter with the tenant’s configured limit.

The logic is equivalent to:

if counter <= tenantLimit:
    allow request
else:
    reject with HTTP 429

This is a minimal example rather than a complete production middleware implementation. Production code should also validate configuration, handle Redis errors, apply connection and timeout settings, emit metrics, and establish an explicit failure policy for when the counter service is unavailable.

Return a useful response to clients

When a tenant exceeds its allowance, return HTTP 429 Too Many Requests.

For example:

{
  "error": "rate_limit_exceeded",
  "message": "The request limit for this operation has been exceeded.",
  "retryAfterSeconds": 12
}

The retry duration should reflect when the current window ends, rather than always returning the entire window duration.

A response may also include a Retry-After header and documented rate-limit headers if those are part of the API contract.

Clients should respect the retry instruction and use bounded backoff where appropriate. Retrying immediately in a tight loop can make the problem worse.

5. The fixed-window algorithm has an important weakness

The fixed-window approach is simple and inexpensive, but it can permit bursts at window boundaries.

Imagine a tenant has a limit of 60 requests per minute.

The tenant sends 60 requests just before a minute boundary, and then another 60 immediately after the boundary.

Both groups fall within separate windows, so the limiter can accept 120 requests within a very short interval.

The system is enforcing its configured count correctly, but the result may not match the application’s intention of smoothing traffic.

This distinction matters when the downstream operation is expensive or when a burst can overwhelm a database.

There are several alternatives:

Fixed window: Simple and efficient, but permits boundary bursts.
Sliding-window counter: Estimates request volume over a continuously moving interval and generally reduces the fixed-window boundary problem.
Token bucket: Allows a controlled burst while replenishing capacity at a configured rate.

Amazon API Gateway uses a token-bucket approach for throttling. This is not identical to implementing a fixed-window counter in application code.

Choose the algorithm according to the requirement. A fixed-window counter can be suitable for a simple per-minute API allowance. A token bucket may be more appropriate when the application needs controlled bursts and smoother traffic. Neither choice alone solves every concurrency or resource-protection problem.

6. Rate limits should reflect the operation, not just the tenant

Not every endpoint has the same cost.

A lightweight request that retrieves a small record should not necessarily consume the same allowance as a report that queries a large dataset or triggers a long-running workflow.

I would consider separate policies for:

Standard read operations.
Expensive report generation.
Bulk imports and exports.
Authentication and password-reset endpoints.
Operations that trigger external services or paid resources.

For example, a tenant may be allowed 300 ordinary API requests per minute but only a smaller number of concurrent report-generation jobs.

Rate limits and concurrency limits solve different problems. A request-rate limit controls how frequently work may be submitted; a concurrency limit controls how much work may execute simultaneously.

Long-running jobs may be better handled through a queue, such as Amazon SQS, with controlled worker concurrency instead of processing every request synchronously.

7. Rate limiting is not the same as SaaS billing

A SaaS subscription might include a monthly allowance of 100,000 API requests, alongside a short-term limit of 300 requests per minute.

These are separate controls.

The per-minute limiter protects service capacity and enforces short-term traffic policies. The monthly allowance is a metering and entitlement requirement.

I would not use a best-effort API Gateway quota as the only source of truth for billing or contractual usage limits.

For billable usage, record usage events or counters in a durable design appropriate to the business requirements. Account for retries, duplicate processing, delayed events and the definition of a billable operation. The metering system may use an event queue and persistent storage, while a fast rate limiter independently protects the request path.

A system that rejects excessive requests reliably is not automatically a system that measures customer usage accurately.

8. What changes when the SaaS application scales?

A distributed limiter introduces a dependency of its own. Before adding it to a production request path, consider its failure modes.

If the counter store is unavailable: Choose deliberately whether requests should fail closed or be allowed under a conservative fallback policy. The answer depends on the operation. A non-critical read and an expensive operation that triggers paid downstream work may require different treatment.

If traffic spans multiple AWS Regions: Independent regional counters can permit a tenant to exceed a globally intended quota. A global limit needs a design that accounts for cross-region coordination, latency, availability and consistency.

If there are many tenants: Avoid turning every tenant ID into a high-cardinality CloudWatch metric dimension. Use aggregate metrics for the main dashboard and structured logs or sampled diagnostics for tenant-level investigations.

If one tenant has many concurrent requests: Make the counter operation atomic, and test boundary cases and concurrent traffic rather than testing only sequential requests.

If subscription limits change: Define when an updated policy takes effect and how the application refreshes cached plan configuration.

These considerations are often more important than the counter algorithm itself.

9. How I would test this design

Before releasing rate limiting, I would test it against an environment with the same key infrastructure components as production.

For each tenant, verify that requests within the configured limit are accepted and requests beyond it receive HTTP 429. Send requests from multiple application instances to confirm that they share the same counter. Test the exact window boundary, simultaneous requests, expiration, Redis timeouts and recovery after an outage.

Also test tenant isolation explicitly: a tenant must not be able to evade its policy by changing an arbitrary request header, and one tenant’s counter must not consume another tenant’s allowance.

Monitor accepted and rejected requests, Redis latency and errors, API Gateway throttles, downstream latency and resource saturation. Use alerts that help distinguish a misconfigured tenant policy from a genuine capacity problem or an attack.

The purpose of these tests is not merely to prove that a counter increments. It is to verify that the policy behaves correctly under the concurrency and failure conditions the SaaS application will encounter.

Conclusion

Rate limiting is a small algorithm with consequences across the entire SaaS architecture.

On AWS, I would use WAF for public-edge abuse controls where appropriate, API Gateway throttling for API-level protection, and an application-level limiter when the business requires authenticated tenant-specific policies.

A fixed-window counter is a reasonable starting point for simple request allowances. Redis provides a way to share counters across application instances, but the implementation still needs atomic updates, trusted tenant identity, explicit error handling and appropriate monitoring.

As the product grows, the design should distinguish traffic protection, tenant fairness, concurrency control and usage metering rather than treating them as one problem.

The objective is not to block requests indiscriminately. It is to give each customer the capacity their subscription allows while preserving predictable performance for the rest of the platform.