One million requests per second

Published: 16 Aug 2026 22 min read

AWS architecture performance scalability EC2 PostgreSQL benchmarking

One million requests per second flowing through a compact server installation

The economics of AWS are tricky and require close attention. The situation gets even more nuanced when the final solution requirements are architecturally demanding. A static response cached at the edge can reach a million RPS while doing almost no application work. An authenticated write that must survive a regional failure is an entirely different system.

A good example of exploring AWS economics would be something in between: a web server receives an HTTP request, runs a little application logic, checks a cache, sometimes reads a database, and returns JSON. How close can we get to hardware economics while preserving the recognizable properties of such an architecture?

The obvious place to begin is with the services AWS built to remove infrastructure work. API Gateway, Lambda, DynamoDB, Fargate, ElastiCache, and CloudFront can all participate in a design for this endpoint. The interesting part is what happens when their pricing units are multiplied by one million, every second, for a month.

At that rate, the system receives 2.592 trillion requests over 30 days. Per-request pricing is wonderfully convenient at ordinary scale. Here, even a fraction of a dollar per million becomes a serious line item. API Gateway, for example, charges for the calls it receives as well as data transferred out.^[1]

Domain boundary #

Before choosing services or instances, the workload needs a domain boundary. The application exposes one public endpoint:

GET /products/{id}

It returns approximately 1 KB of JSON describing a product. The endpoint is public and read-only. There is no authentication or authorization. To keep the comparison meaningful, no architecture may satisfy the measured requests from a CDN or edge cache. Route 53 is also outside the experiment; generated service hostnames are sufficient for a benchmark.

At the target rate, the response bodies alone amount to roughly:

1,000,000 responses/second × 1 KB = 1 GB/second
1 GB/second × 8                    = 8 Gbit/second
1 GB/second × 30 days              = 2.592 PB/month

That is before HTTP headers, TLS records, TCP overhead, retransmissions, health checks, database traffic, and metrics. Request size matters too, but even this first calculation tells us something important: this is as much a networking challenge as a CPU challenge.

The capacity-test success condition is:

At least 1,000,000 completed responses per second for 30 consecutive minutes
A client-observed p99 latency below 50 ms
Fewer than 0.1% transport errors and non-successful HTTP responses
Full response bodies read and validated by the load generators
An open-loop generator that fails the run if it cannot offer the requested load

The final experiment measures one serving target. Removing a target and remaining above one million RPS is a separate resilience test that requires at least two targets and a load balancer. The load generators run on separate EC2 instances in the benchmark VPC and use private addresses. Their cost is reported separately because they are test equipment.

Four ways to serve #

There are many ways to draw this system on AWS. Four of them are credible starting points, but they optimize for different things.

Responses at the edge #

Client -> CloudFront -> S3 or an application origin

If product responses can remain public and cacheable for long enough, CloudFront is the natural answer. A cache hit is served from an edge location without sending a request to the origin.^[6] With a popular catalogue and a carefully chosen cache key, the origin might see only a small fraction of the headline traffic.

That would be a good production design and a bad answer to this experiment. It changes the work from "run an application request one million times per second" to "distribute cached objects one million times per second." CloudFront request and data-transfer costs would still matter, but the application architecture behind it would no longer be under meaningful load.

Therefore, CloudFront is out of the measured path.

Serverless #

Client
  -> API Gateway HTTP API or Lambda Function URL
  -> Lambda
  -> DynamoDB

This is the smallest operational footprint. API Gateway can own the public HTTPS endpoint, Lambda runs the application logic, and DynamoDB stores the products. For this one unauthenticated route, a Lambda Function URL is a leaner front door with no charge beyond the Lambda invocation and duration.^[15] There are no servers or database connections to manage, and every layer can scale.

"Can scale" does not mean "starts at one million requests per second." API Gateway's default regional throttle quota is 10,000 RPS in most regions and is adjustable.^[7] Lambda limits how quickly one function acquires new execution environments, and its account concurrency quota must be large enough for the request duration.^[8] DynamoDB on-demand is designed to reach millions of requests per second, but a new table starts with much lower throughput and its default table-level quota is 40,000 read request units.^[9]

A serious attempt would arrange quota increases with AWS, pre-warm the data path, ramp traffic gradually, choose a partition key that spreads the load, and provision enough Lambda concurrency. The economic problem is more fundamental. The hot path can meter one API Gateway call, one Lambda invocation plus its duration, and one DynamoDB read for every cache miss. At the monthly target, the request counts begin like this:

API Gateway calls       2.592 trillion
Lambda invocations      2.592 trillion
DynamoDB logical reads  2.592 trillion

For eventually consistent products smaller than 4 KB, those DynamoDB reads would consume half as many read request units. Adding DAX or ElastiCache could reduce them, but would put a remote cache back into the design.

The architecture replaces capacity planning with service and request metering. That is often an excellent trade. A steady million-RPS workload is the case least likely to benefit from paying a convenience premium on every invocation.

Managed containers #

Client
  -> load balancer
  -> ECS service on Fargate
  -> ElastiCache
  -> RDS or Aurora

Moving the application to ECS on Fargate changes the compute unit from invocations to requested vCPU, memory, and storage. ECS itself has no separate orchestration charge for Fargate; the tasks are billed for their configured resources while they run.^[10] For a high, stable baseline, that is easier to reason about than Lambda duration.

ElastiCache keeps most reads away from the relational database and gives every task the same cache contents. It can be serverless, where storage and processing units are metered, or node-based, where capacity is purchased by the node-hour.^[11] A sharded deployment can spread a million cache operations per second across nodes.

This is a credible production architecture, especially when cache consistency and managed task replacement matter more than the last unit of cost. It also means that almost every request crosses the network twice after reaching the application: once to the cache and once back. The cache becomes another high-throughput distributed system to size, shard, replicate, monitor, and pay for. Cross-AZ placement can add both cost and latency.

Instances with memory #

Client
  -> Network Load Balancer
  -> EC2 web servers with local caches
  -> RDS PostgreSQL on cache misses

The final option keeps the managed load balancer and durable relational database but removes request-metered compute and the remote cache from the hot path. Each EC2 instance terminates TLS, runs the application, and caches serialized responses in memory it already owns. The load balancer can route to those instances over private addresses.^[3]

This design accepts more responsibility. You have to choose instances, build images, roll deployments, tune the operating system, replace unhealthy capacity, and understand what happens when every process has a slightly different cache. In return, a cache hit becomes a memory lookup followed by a socket write, and the main compute bill is based on instance time rather than request count.

The trade-offs can already be summarized:

Design	Scaling unit	Cache location	Dominant concern
CloudFront and an origin	Edge requests and transfer	Edge locations	It avoids the application work being tested
API Gateway or Function URL, Lambda, DynamoDB	Requests, duration, and read units	Optional and service-dependent	Quotas, ramp rate, and a metered hot path
Fargate, ElastiCache, RDS	Task resources and cache capacity	Shared over the network	More network work and another distributed tier
NLB, EC2, local cache, RDS	Load-balancer and instance capacity	Web-server memory	More ownership and bounded cache inconsistency

The table identifies where the costs accumulate. We can now put public prices against the parts that do not depend on a benchmark.

Prices on the hot path #

These estimates use public on-demand prices for US East (N. Virginia) on 2026-08-24, in US dollars. A month is 720 hours, and the workload is exactly 2.592 trillion requests. Free tiers, negotiated discounts, taxes, support, logging, monitoring, and security services are excluded.

Internet data transfer is also excluded from every subtotal. All designs return the same 2.592 PB of response bodies, so including egress would add a very large common term without helping us compare the application paths. It will return in the final bill.

The calculations are estimates. AWS prices change, and several important quantities cannot be known until the application runs.

Serverless, one request at a time #

API Gateway HTTP APIs charge $1.00 per million for the first 300 million requests and $0.90 per million after that. The endpoint alone would therefore cost:

first 300 million calls    300 × $1.00       =         $300
remaining calls          2,591,700 × $0.90   =   $2,332,530
                                                  ----------
API Gateway calls                                $2,332,830

Lambda adds $0.20 per million invocations. Arm duration in the first pricing tier costs $0.0000133334 per GB-second, rounded up to the nearest millisecond.^[12] DynamoDB on-demand charges $0.125 per million read request units in this region; an eventually consistent read below 4 KB consumes half a unit.^[13]

For a Lambda configured with the minimum 128 MB of memory, the monthly calculation is:

Component	Calculation	Monthly cost
API Gateway HTTP API	Tiered price for `2.592 trillion` calls	`$2,332,830`
Lambda requests	`2,592,000` million-request units × `$0.20`	`$518,400`
DynamoDB reads	`1.296 trillion` RRUs × `$0.125` per million	`$162,000`
Lambda duration at `1 ms`	`324 million` GB-seconds	`$4,320`
Lambda duration at `5 ms`	`1.62 billion` GB-seconds	`$21,600`
Lambda duration at `10 ms`	`3.24 billion` GB-seconds	`$43,200`

At a still-aggressive 10 ms average duration, the API Gateway version totals approximately:

$2,332,830  API Gateway
   518,400  Lambda requests
    43,200  Lambda duration
   162,000  DynamoDB reads
----------
$3,056,430  per month before storage, observability, and egress

The Function URL variant removes the $2,332,830 API Gateway term, reducing the same estimate to $723,600 per month. It also gives up API Gateway features such as richer routing, built-in authorization options, fine-grained throttling, and request transformation. Either serverless front door still needs exceptional quota planning: synchronous Lambda invocation throughput is tied to concurrency, not merely to how quickly the handler returns.

The important result is that duration barely changes the order of magnitude. At this workload, Lambda invocation charges alone are more than half a million dollars per month.

Fargate and a remote cache #

Fargate removes the per-invocation compute charge. In this region, a Linux/Arm task costs $0.0000089944 per vCPU-second plus $0.0000009889 per GB-second of memory.^[14] A continuously running task with 1 vCPU and 2 GB of memory therefore costs about $28.44 for a 30-day month.

The application throughput of that task is unknown, so multiplying it by a convenient task count would manufacture a result. The honest compute table is a sensitivity analysis:

Fargate allocation	Monthly compute cost
`100 vCPU` and `200 GB`	`$2,844`
`500 vCPU` and `1,000 GB`	`$14,220`
`1,000 vCPU` and `2,000 GB`	`$28,440`

The benchmark must tell us which row, if any, can serve one million requests per second with failure headroom.

ElastiCache Serverless for Valkey meters a simple operation transferring up to 1 KB as at least one ECPU. At the published $0.0023 per million ECPUs, one cache lookup per request is approximately:

2.592 trillion ECPUs × $0.0023 per million = $5,962/month

Cache storage adds $0.084 per GB-hour, or $60.48 per GB-month, before object overhead. The aggregate service can accommodate the target, but the Zipf-like request distribution makes the per-slot limit relevant: a single slot supports 30,000 ECPUs/second, or 90,000 when reads use replicas.^[16] A few extremely popular products can become hot keys even when aggregate capacity looks comfortable.

A node-based Valkey cluster may be cheaper for a steady workload because it is billed by node-hour instead of ECPU. Its shard and replica count depend on measured throughput and hot-key behaviour, so pricing one now would hide another invented performance assumption.

The Network Load Balancer is common to the Fargate and EC2 designs. With persistent connections, the response bodies are likely to make processed bytes the largest NLCU dimension. One NLCU includes 1 GB/hour for TCP traffic and costs $0.006 per hour.^[5] Response bodies alone establish this floor:

1 GB/second × 3,600 seconds                  = 3,600 NLCUs
3,600 NLCUs × $0.006 × 720 hours             = $15,552
$0.0225 load-balancer hourly charge × 720     =     $16
                                                   -------
body-only Network Load Balancer estimate        $15,568/month

Headers, request bytes, retransmissions, and a connection pattern that makes another NLCU dimension larger will increase it.

The known part of the managed-container path is therefore about $21,530 per month for NLB body processing and one serverless cache lookup per request. To that we must add Fargate compute, cache storage, database capacity for misses, cross-AZ traffic, and the other shared operating costs.

This is already a useful separation. The fully serverless path is measurable in hundreds of thousands or millions of dollars before egress. The container path begins in tens of thousands, but its compute requirement remains unknown. The EC2 design shares the NLB floor while removing the remote-cache operation and buying compute in larger, potentially cheaper units.

Path	Monthly subtotal we can price now	Still unknown
API Gateway, Lambda at `10 ms`, DynamoDB	`$3,056,430`	Storage, observability, egress
Function URL, Lambda at `10 ms`, DynamoDB	`$723,600`	Storage, observability, egress
NLB, Fargate, ElastiCache	`$21,530` plus Fargate and cache storage	Required task capacity, database misses, cross-AZ traffic
NLB, EC2 local cache, RDS	`$15,568` plus EC2 and RDS	Required instance and database capacity, cross-AZ traffic

None of this proves the final EC2 system is cheaper. But it explains why that is the design worth measuring. For this fixed, enormous, read-heavy workload, it has the shortest metered hot path and the closest relationship between hardware capacity and serving cost.

The measured architecture #

We'll stick with the EC2 design. Comparing hypothetical prices is useful; pretending to have benchmarked four systems is not.

The first useful question is whether one server can do the application work at all. I therefore removed the load balancer from the measured path and built a deliberately compact capacity rig:

8 × c6in.2xlarge load generators
        |
        | 1,000,000 HTTPS requests/second
        | 8,192 persistent HTTP/1.1 connections
        v
+----------------------------------+
| 1 × c6i.16xlarge                 |
|                                  |
| Rust + Tokio                     |
| TLS 1.3 termination              |
| HTTP parsing and validation      |
| 16 GiB bounded local cache       |
+----------------+-----------------+
                 |
                 | cache misses only
                 v
+----------------------------------+
| 1 × db.r7i.2xlarge               |
| RDS PostgreSQL 17.10, Single-AZ  |
+----------------------------------+

Every resource was in us-east-1b, and the nine EC2 instances shared one cluster placement group. Benchmark traffic used private addresses and a Route 53 private name, server.benchmark.internal. The EC2 machines had public addresses for SSM, ECR, and package access, but there was no SSH ingress and RDS was private. The Python controller used SSM Run Command for deployment and execution.

The server was a c6i.16xlarge: 64 vCPUs, 128 GiB of memory, and up to 25 Gbit/s of network bandwidth.^[17] Each generator contributed 125,000 RPS over 1,024 connections. Eight generators prevented the client from becoming the benchmark and made each shard independently accountable for its share.

This is a slightly different production diagram from the one proposed above. It has one serving target, no load balancer, and a Single-AZ database. It measures the capacity and economics of the hot path. A production version still needs multiple serving targets, health-based routing, failure testing, and an appropriate RDS availability design. A TCP Network Load Balancer can preserve end-to-end TLS termination at the servers,^[2] while an RDS Multi-AZ primary and standby adds database failover.^[4]

There is deliberately no distributed cache. Adding one would turn one scaling problem into two. Every request would still cross the network to reach the cache, and the cache would need to absorb up to one million operations per second. Memory already attached to the server is cheaper and faster.

TLS without a domain #

The benchmark used a private Route 53 name:

server.benchmark.internal

For the non-public domain, the test used a private certificate authority:

Create a private test CA.
Issue a seven-day server certificate whose Subject Alternative Name is the private hostname.
Put the certificate and key on the server through encrypted SSM parameters.
Mount them read-only into the server container.
Add the private CA certificate to the load generators' trust store.

The resulting trust model is private, but the work done by TLS is real. Clients still validate a certificate chain. Servers still perform handshakes, negotiate ciphers, derive keys, and encrypt every response. Session resumption and connection reuse still matter.

The clients resolved the private hostname once before creating workers, then reused the resulting socket address for reconnects while retaining the hostname for TLS SNI and the HTTP Host header. That detail became important later.

The local cache #

The web process owns a fixed-size in-memory cache of serialized product responses. The hot path is therefore short:

parse request
    -> locate cached response by product ID
    -> write bytes to the existing TLS connection

On a miss, the application performs an indexed PostgreSQL lookup, serializes the result, and places the bytes in its local cache.

The cache has a few production-like properties:

A hard memory limit
An explicit eviction policy
A 3,600-second positive TTL with deterministic ±20% jitter
A 30-second negative TTL
Negative caching for unknown product IDs
Request coalescing so concurrent misses for one ID produce one database query
Metrics for hits, misses, evictions, load time, and current size

Local caching would duplicate data across a fleet and permits bounded staleness. Those are acceptable properties for this public product catalogue. They are also part of the economics: RAM already attached to the server removes a network service from the request path.

The cache-hit rate is configurable. Requests followed a repeatable, skewed distribution in which some products were much more popular than others. The measured hit rate emerged from cache size, TTL, and the access pattern.

Something to serve #

The database begins with ten million deterministic products. This was large enough to reveal whether the cache could retain the working set instead of turning PostgreSQL into an accidental hot-path service.

CREATE TABLE products (
    id           bigint PRIMARY KEY,
    sku          text NOT NULL,
    name         text NOT NULL,
    category_id  integer NOT NULL,
    price_cents  integer NOT NULL,
    description  text NOT NULL,
    updated_at   timestamptz NOT NULL
);

The records can be generated inside PostgreSQL without transferring a giant fixture across the network:

INSERT INTO products (
    id,
    sku,
    name,
    category_id,
    price_cents,
    description,
    updated_at
)
SELECT
    id,
    'SKU-' || lpad(id::text, 12, '0'),
    'Product ' || id,
    1 + (id % 1000),
    100 + ((id * 37) % 100000),
    repeat(md5(id::text), 28),
    timestamptz '2026-01-01 00:00:00+00'
        + ((id % 365) * interval '1 day')
FROM generate_series(1, 10000000) AS generated(id);

The repeated hash makes the description large enough to produce a response near the 1 KB target. The loader inserted bounded ranges rather than holding one enormous transaction open, then ran VACUUM (ANALYZE).

SELECT pg_size_pretty(
    pg_total_relation_size('products')
);

The AWS controller loaded bounded ranges, ran VACUUM (ANALYZE), and made initialization idempotent. Every run used the same deterministic records without quietly changing the database.

The request distribution #

Uniform random product IDs would make the cache look worse than most real catalogues. Repeating a few fixed IDs would make it look absurdly good. The generator therefore needs a stable, Zipf-like distribution:

a few products       -> requested constantly
a larger hot set     -> requested frequently
a long tail          -> requested occasionally
unknown product IDs  -> requested rarely

Every run used the same distribution parameters. The eight shards used deterministic, distinct seeds derived from 104729, preventing identical streams from moving in lockstep. A five-minute linear ramp warmed the system, a 120-second synchronization margin separated ramp from measurement, and the following 30-minute window produced the result. Cache-hit rate, database queries per second, and database latency were real results.

The benchmark #

A million requests per second is easy to print and surprisingly difficult to prove. The server and load generator were both written in Rust. The generator is open-loop: intended arrival times advance independently of response completion. Each of its 8,192 workers owns one persistent HTTP/1.1 connection and processes sequentially without pipelining.

Every response body is read and checked against the deterministic product record. The 0.1% requests for unknown IDs must return the expected 404; they are not counted as errors. Latency begins at the intended arrival time. Time spent waiting in a client-side queue is therefore visible in schedule-to-completion latency rather than disappearing through coordinated omission.

Each connection has a bounded queue of 64 requests. The queue can absorb a short permitted tail event, but it cannot rescue an overloaded system indefinitely. If any queue fills, if an arrival cannot be dispatched, if an offered request produces no observation, or if a shard misses its synchronized UTC start, the run is invalid. A non-zero load-generator exit status still leaves a machine-readable result explaining why it failed.

The controller records the source commit, immutable ECR image digest, instance types, Availability Zone, workload, CloudWatch series, PostgreSQL counters, server metrics, and every shard result. The contract was versioned whenever a supposedly small change altered comparability.

That discipline mattered because several early failures looked like server limits and were not.

Failures #

Capacity before traffic #

The successful rig required exactly 128 standard On-Demand EC2 vCPUs: 64 for the server and eight generators with eight vCPUs each. The account began with a 64-vCPU regional quota, so CloudFormation correctly refused to create the server. EC2 applies On-Demand quotas to instance-family buckets.^[19] AWS approved the increase to 128, but the new value took a few minutes to become effective after the request closed.

Quota did not guarantee physical capacity. c6i.16xlarge launches repeatedly failed in us-east-1a with an insufficient-capacity response. The same stack provisioned in us-east-1b. A reproducible benchmark must therefore record the Availability Zone and accept it as an explicit deployment parameter.

DNS - hidden packet limit #

One failure showed no bandwidth, packet-per-second, conntrack, or XDP drops on the server. Several generators, however, accumulated roughly 120,000 linklocal_allowance_exceeded events and eventually reported connection errors.

The reconnect path was resolving server.benchmark.internal for every new socket. On EC2, DNS and other link-local services share an allowance. A reconnect wave became a DNS wave, which caused more failed reconnects and then still more DNS queries. The fix was to resolve the target once before creating workers, cache the socket address, and retain the hostname only for TLS SNI and the HTTP Host header. After that change, reconnect attempts stopped consuming the link-local allowance.

TTL expiry #

The original positive cache TTL was 300 seconds. So was the warm-up ramp. Entries loaded early in the ramp began expiring before the measured window, and the 120-second synchronization margin made the overlap worse. What looked like an RDS capacity problem was a synchronized cache reload wave created by the test itself.

The positive TTL became 3,600 seconds with deterministic ±20% jitter and was added to every result manifest. Its minimum effective value is 2,880 seconds, longer than the complete 2,220-second ramp, transition, and measurement sequence. Expiration remains bounded, but warm-up entries no longer expire during this benchmark.

Eight GiB #

A 120-second measurement passed at one million RPS with an 8 GiB cache. The full 30-minute measurement did not:

Metric	8 GiB failed run	16 GiB successful run
Achieved valid RPS	`999,104`	`1,000,000`
Worst-shard p99	`313.898 ms`	`5.750 ms`
Measurement errors	`0`	`0`
Generator saturation events	`1,612,190`	`0`
Cache entries at finish	`7,288,006`	`9,984,266`
Cache weight at finish	`8.59 GB`	`11.77 GB`
Cache evictions	`17,810,974`	`0`
Database queries	`25,574,085`	`10,492,492`

The failed run was deceptive. Worker service p99 remained near 7.2 ms, the server produced no 5xx responses, RDS produced no errors, and neither server nor database CPU was exhausted. But continuous cache eviction sent long-tail keys back to PostgreSQL. Occasional slow responses blocked sequential connection workers, queued arrivals behind them, and raised schedule-to-completion p99 to roughly 300 ms. Eventually the bounded queues filled on all eight generators.

Increasing the number of connections could have hidden some head-of-line blocking. Instead, I changed only the cache from 8 GiB to 16 GiB and kept 8,192 connections. The following full run passed. That makes cache churn, rather than insufficient connection concurrency, the stronger explanation.

The final run #

The qualifying run used contract version 6.

Parameter	Value
Server	`c6i.16xlarge`
Load generators	`8 × c6in.2xlarge`
Database	`db.r7i.2xlarge`, PostgreSQL `17.10`, Single-AZ
Product rows	`10,000,000`
Access distribution	Zipf, exponent `1.1`
Positive cache TTL	`3,600 s`, deterministic `±20%` jitter
Cache capacity	`16 GiB`
PostgreSQL pool limit	`512` connections
Client connections	`8,192`
Per-connection queue	`64` requests
Warm-up	`300 s` linear ramp
Synchronization margin	`120 s`
Measurement	`1,800 s` at `1,000,000 RPS`

The aggregate result was exact:

offered requests       1,800,000,000
completed responses    1,800,000,000
valid responses        1,800,000,000
achieved rate          1,000,000 RPS
errors                 0
HTTP 200               1,798,200,000
expected HTTP 404          1,800,000
worst-shard p99                5.750 ms
worst-shard p99.9              7.984 ms

Every shard offered and completed exactly 225 million measured requests at 125,000 RPS. None saturated. Every connection attempt succeeded, and all response bodies were validated. The measured bodies totalled 1,872,228,717,767 bytes, approximately 1,040 bytes per response before protocol overhead.

Including warm-up, the server handled 1,950,150,000 requests. Its internal metrics reported:

Server observation	Result
Cache hits	`1,939,657,122`
Cache misses	`10,492,878`
Effective hit rate	`99.462%`
Coalesced cache loads	`386`
Cache evictions	`0`
Database queries	`10,492,492`
Database errors	`0`
Mean database query duration	`2.344 ms`
HTTP 5xx responses	`0`

PostgreSQL recorded 41,604,248 additional buffer hits and zero physical block reads during the run. The database was still real and on the miss path, but its hot data remained in the RDS buffer cache.

CloudWatch showed useful headroom rather than a machine balanced on a knife edge:

Resource	Average CPU	Maximum CPU	Maximum memory used
Server	`40.23%`	`48.74%`	`21.10%`
Load generators	`38.72–42.79%`	`46.89–52.70%`	`3.64%`
RDS PostgreSQL	`6.82%`	`29.27%`	—

CPU utilization alone does not prove another million RPS would fit; packet processing, memory bandwidth, locks, and network capacity can become limiting first. It does show that the successful result was not exhausting every resource continuously at 100%.

What it costs #

I queried the AWS Price List API for US East (N. Virginia) on August 31, 2026.^[18] The successful server cost $2.72/hour On-Demand, or $1,958.40 for a 720-hour month. The Single-AZ db.r7i.2xlarge cost $1.00/hour, or $720/month, and its 100 GB of gp3 storage cost $11.50/month.

That produces a measured-topology subtotal of approximately:

$1,958.40  c6i.16xlarge server
   720.00  db.r7i.2xlarge PostgreSQL
    11.50  100 GB RDS gp3 storage
---------
$2,689.90  per 720-hour month

This is only a capacity subtotal. It excludes the small EC2 root volume, public IPv4, Secrets Manager, monitoring, backups, support, taxes, and discounts. More importantly, it excludes a load balancer, redundant servers, Multi-AZ database capacity, cross-AZ traffic, and Internet egress.

The eight c6in.2xlarge generators cost $0.4536/hour each, or $3.6288/hour together. They are the test equipment. At on-demand rates, the server, database, and all generators cost about $7.35/hour; the 37-minute ramp, transition, and measurement sequence consumed about $4.53 of compute time. Provisioning, loading ten million rows, deployment, diagnostics, and idle time made the actual experiment cost higher.

The response bodies remain the elephant in the bill. At the measured average size and a continuous million RPS, they amount to about 2.70 PB per 30-day month before TLS and TCP overhead. Public egress pricing at that volume is not a footnote, and a production NLB would add processed-byte charges too. The result approaches hardware economics for application compute. Networking is not free.

Savings Plans or Reserved Instances would reduce the fixed compute subtotal for a permanent workload. It would be premature to buy them from this result alone because the production fleet topology has not yet been measured.

What one million means #

The claim is deliberately narrow: one c6i.16xlarge Rust server sustained one million validated HTTPS responses per second for 30 consecutive minutes, backed by a bounded 16 GiB local cache and RDS PostgreSQL on misses. Client-observed worst-shard p99 was 5.75 ms, and the measured window contained no errors or dropped arrivals.

For a production-ready system, you would need to address the following: There was no NLB in the path, one server was indispensable, RDS was Single-AZ, clients were colocated in the same Availability Zone, and the workload was one public read-only endpoint with a highly cacheable Zipf distribution. Authentication, writes, larger responses, WAN latency, multi-AZ transfer, deployments, and failure recovery would change the result.

The most valuable lesson was that long-duration, open-loop measurement changed the diagnosis. A short run said 8 GiB was enough. A 30-minute run exposed 17.8 million evictions, database tail latency, connection-level head-of-line blocking, and generator saturation. Doubling a cache inside memory already attached to the instance removed all evictions and restored exact throughput without adding connections or changing the server.

At this scale, the benchmark harness, DNS path, cache lifetime, AWS quota, Availability Zone capacity, and test duration are all part of the system. Measuring them honestly is what turns “one million requests per second” from a screenshot into a result.

Source code #

Reference implementation (opens in a new tab)