Most RDS outages that start with “the database fell over” are really “the database ran out of connection slots”. CPU sits at 30%, queries are fast, and then new requests fail with FATAL: remaining connection slots are reserved for non-replication superuser connections (PostgreSQL) or ERROR 1040 (HY000): Too many connections (MySQL). The problem is rarely the database’s capacity to do work. It’s the number of clients holding a seat at the table.
Where the connection limit comes from
On RDS, max_connections defaults to a formula based on the instance’s memory. For RDS for PostgreSQL it’s LEAST({DBInstanceClassMemory/9531392}, 5000); MySQL uses a different divisor. DBInstanceClassMemory is a bit less than the instance’s nominal RAM, since RDS reserves some for the OS and its own processes, but nominal memory gives the right order of magnitude:
db.t4g.medium 4 GiB -> max_connections ~ 450
db.r6g.large 16 GiB -> max_connections ~ 1802
db.r6g.xlarge 32 GiB -> max_connections ~ 3604(Computed with a short Python loop using nominal memory; actual values on a running instance are somewhat lower. Run SHOW max_connections; to see yours.)
You can raise the parameter in a parameter group, and people do when they hit the limit. That usually moves the problem rather than fixing it. In PostgreSQL every connection is a separate backend process with its own memory; idle ones cost a few megabytes each, and busy ones can use far more (work_mem is per sort or hash operation, per connection). A thousand mostly-idle connections eat into the memory the database would otherwise use for caching data pages. And a thousand active connections on a 4-vCPU instance means hundreds of queries fighting for CPU, so each one runs slower, holds locks longer, and throughput drops. Past a point, more connections means less work done.
Two numbers that rarely match
Ask a team how many connections they need and they’ll usually compute the number they hold, which is the number of app processes times the size of each process’s connection pool:
ECS: 40 tasks x pool 20 = 800 connections held openThe number they need is how many queries are in flight at once, which by Little’s law is the query rate times the average time a connection is busy per query:
needed: 3000 queries/s x 8 ms = 24 connections busy on averageThat’s 24 busy on average, against 800 open. Even allowing generous headroom for bursts and slow queries (say 3 to 4 times), you need under 100. The other 700 are idle, costing memory and counting against max_connections. A pool of 20 per task was probably picked as a default, not computed.
This gap is why “how big should my pool be?” has a better answer than “big”. A useful starting point is a small pool per process (5 to 10), then check pool metrics: if requests regularly wait for a connection while the database has spare CPU, grow it; if the database is CPU-bound, a bigger pool will make it worse, not better.
The gap gets dangerous when the fleet scales. An autoscaling group that goes from 40 to 120 tasks during a traffic spike triples held connections to 2,400 and crosses max_connections exactly when load is highest. New tasks fail their health checks because they can’t connect, get replaced, and the replacements try to connect too.
Lambda makes the multiplication explicit
Lambda is the extreme case. Each concurrent execution environment is its own process, and if each opens one connection at init, connections equal concurrency:
Lambda at 100 concurrent environments, 1 connection each = 100
Lambda at 400 concurrent environments, 1 connection each = 400
Lambda at 1000 concurrent environments, 1 connection each = 1000A Lambda function behind API Gateway can scale to the account’s concurrency limit in a burst. Worse, environments that are idle (frozen between invocations) keep their connections open until they’re retired, so the database sees connections from environments that aren’t doing anything. And frozen environments don’t respond to TCP keepalives, so the server may eventually close them, and the next invocation sees Connection terminated unexpectedly.
The controls, roughly in order of preference:
- Cap concurrency with reserved concurrency on the function, or maximum concurrency on an SQS event source mapping, so it can never exceed what the database can serve.
- Put RDS Proxy in between, so Lambda’s many short-lived connections are multiplexed onto a small, stable set.
- Reconsider the data store if the workload is truly spiky and simple key-value access; DynamoDB doesn’t have connections in this sense.
What RDS Proxy does, and when it doesn’t
RDS Proxy is a managed connection pooler. Clients connect to the proxy endpoint instead of the database. The proxy keeps a pool of database connections and, between transactions, hands a database connection to whichever client needs it next. A thousand client connections can share a few dozen database connections, as long as each client only needs one between BEGIN and COMMIT (or for a single statement in autocommit mode).
You set the ceiling with MaxConnectionsPercent, a percentage of the database’s max_connections the proxy may use. Other benefits come along: during a Multi-AZ failover the proxy keeps client connections open and reconnects to the new primary, which shortens the outage seen by apps, and it can enforce IAM authentication while fetching database credentials from Secrets Manager.
The catch is pinning. If a client does something that ties state to its database session, the proxy can’t safely hand that database connection to anyone else, so it pins it to the client until the client disconnects. Things that commonly cause pinning in PostgreSQL include SET statements that change session variables, creating temporary tables, using advisory locks, and some uses of prepared statements and DECLARE CURSOR outside a transaction. Some ORMs and drivers set session state on every new connection, which quietly pins everything. Then the proxy is just a pass-through with extra latency.
Watch the proxy’s DatabaseConnectionsCurrentlySessionPinned metric. If it tracks client connections, pinning is defeating the point; the proxy’s logs say which statement caused it. Moving session settings into the database parameter group or the connection string usually fixes it.
A capacity check you can do before an incident
Put real numbers next to each other:
max_connectionsfor the instance, minus a margin for admin and replication connections.- Maximum fleet size (autoscaling max, Lambda reserved concurrency) times connections per process.
- Peak queries per second times p95 query time, which is the concurrency you actually need.
- CloudWatch’s
DatabaseConnectionsmetric over the last month, and how close its peak came to the limit.
If the second number exceeds the first, you have an incident waiting for a traffic spike. The fixes are smaller pools, a fleet maximum that respects the database, a proxy, or read replicas for read traffic (each replica has its own connection limit, and reads that move there free capacity on the primary). Scaling the instance up raises the limit, but it’s the most expensive way to buy idle connections.
Interview practice: the database that fails when the app scales
A service on ECS with autoscaling (min 10, max 60 tasks, pool size 25) uses a db.r6g.large PostgreSQL instance. Normal traffic is fine. During a marketing campaign, the service scales to 60 tasks and starts failing with FATAL: remaining connection slots are reserved for non-replication superuser connections. Database CPU is at 45%. Someone proposes upgrading to db.r6g.4xlarge. What would you do instead?
Answer
The math explains it. At 60 tasks with a pool of 25, the fleet can hold 1,500 connections. A db.r6g.large has roughly 1,700 by default, minus reserved superuser slots, and some pools open extra connections during startup and health checks, so the fleet crosses the limit as it scales out. CPU at 45% says the database can do the work; it’s running out of slots, not capacity.
Upgrading would raise the limit, at several times the cost, while most of those connections still sit idle. I’d first estimate actual need: if the service does, say, 4,000 queries per second at 5 ms, that’s 20 busy connections on average. A pool of 5 per task gives 300 at full scale, far under the limit, with room for bursts. I’d load test that, watching pool wait time.
Then I’d make it safe structurally: set the autoscaling maximum with the database in mind, alarm on DatabaseConnections at around 80% of max_connections, and consider RDS Proxy if the fleet needs to grow further or failover time matters. If the campaign traffic is mostly reads, a read replica takes load off the primary. And I’d check pool settings like idle timeout and max lifetime so connections from scaled-in tasks don’t linger.