Servers in a private subnet don’t have public IP addresses and have no route to an internet gateway. That’s the point of a private subnet: nothing on the internet can reach them. But they still need to get out: to download OS patches, pull container images from a public registry, or call a payment provider’s API. A NAT gateway is the managed AWS component that allows exactly that, outbound only.
What NAT does to a packet
NAT is network address translation. An app server at 10.0.11.24 wants to reach api.stripe.com. Its subnet’s route table sends 0.0.0.0/0 to the NAT gateway, which sits in a public subnet and has an Elastic IP, say 52.18.40.7.
The flow goes like this:
- The server sends a packet from
10.0.11.24:40112to the API’s address on port 443. - The NAT gateway rewrites the source to
52.18.40.7:plus a port it picks, records the mapping, and forwards the packet through the public subnet’s route to the internet gateway. - The reply comes back to
52.18.40.7on that port. The NAT gateway looks up the mapping, rewrites the destination back to10.0.11.24:40112, and delivers it.
Because the mapping only exists after the inside host starts a connection, a packet arriving from the internet that doesn’t match a mapping has nowhere to go. That’s why NAT gives outbound access without inbound exposure. It’s a side effect of how translation works, not a firewall feature, so you still want security groups on the instances.
From the outside, every private instance behind that NAT gateway appears as 52.18.40.7. That’s useful when a partner wants a fixed IP to allowlist, and it’s one of the reasons teams keep NAT gateways even when they could avoid them.
The setup, and the mistake that breaks it
The pieces must line up exactly. Here’s the minimum for one Availability Zone:
Public subnet 10.0.1.0/24 route table rt-public
10.0.0.0/16 -> local
0.0.0.0/0 -> igw-0a12 NAT gateway nat-0b34 lives here, with an Elastic IP
Private subnet 10.0.11.0/24 route table rt-private-a
10.0.0.0/16 -> local
0.0.0.0/0 -> nat-0b34 app servers live hereThe classic mistake is creating the NAT gateway in the private subnet. It gets created without complaint, but its own subnet has no route to the internet gateway, so it can’t forward anything, and the private instances’ connections time out. The NAT gateway has to be in a subnet whose route table sends 0.0.0.0/0 to an internet gateway.
The other common mistake is the reverse loop: pointing the public subnet’s default route at the NAT gateway. Then the NAT gateway’s own outbound traffic is routed back to itself.
With the CLI, the two calls that matter look like this (illustrative IDs; not run against AWS):
aws ec2 create-nat-gateway --subnet-id subnet-public-a --allocation-id eipalloc-0c56
aws ec2 create-route --route-table-id rtb-private-a \
--destination-cidr-block 0.0.0.0/0 --nat-gateway-id nat-0b34One per Availability Zone
A NAT gateway is redundant within its Availability Zone, but it is a zonal resource. If you run app servers in three AZs and route all three private subnets through one NAT gateway in AZ a, two things follow.
First, if AZ a has a problem, servers in b and c lose outbound access even though their own zones are healthy. Your multi-AZ design now has a single-AZ dependency hiding in the route tables.
Second, traffic from b and c crosses AZs to reach the NAT gateway, and cross-AZ data transfer is billed in each direction.
So the standard design is one NAT gateway per AZ, each in that AZ’s public subnet, and each AZ’s private route table pointing at its local NAT gateway. It costs more in hourly charges. Teams with dev environments often accept a single NAT gateway there to save money, which is a reasonable trade-off as long as it’s deliberate.
Where the bill comes from
NAT gateways are charged by the hour and by the gigabyte of data processed, in both directions. Rates vary by Region; in us-east-1 at the time of writing both are about $0.045 (per hour and per GB). Check the VPC pricing page for your Region. With those numbers, a quick Python calculation:
1 NAT gateway(s), 2,000 GB/month: hours $32.85 + data $90.00 = $122.85
3 NAT gateway(s), 2,000 GB/month: hours $98.55 + data $90.00 = $188.55
if 1,500 GB of that is S3 moved to a gateway endpoint: saves $67.50/monthThe last line is where real savings usually are. Workloads that read and write a lot of S3 or DynamoDB data often send it all through the NAT gateway without anyone noticing, because it works. A gateway VPC endpoint for S3 or DynamoDB is free, adds a more specific route to your route tables, and takes that traffic off the NAT gateway. Container image pulls from ECR are another big one: ECR stores layers in S3, so an S3 gateway endpoint plus ECR interface endpoints keep image pulls off NAT. Interface endpoints have their own hourly and per-GB charges, so they’re worth it for heavy traffic, not by default.
To find what’s flowing through a NAT gateway, look at its BytesOutToDestination CloudWatch metric, and use VPC Flow Logs on the NAT gateway’s interface to see which destinations are largest.
Limits and errors you might see
A NAT gateway scales its bandwidth automatically (up to 100 Gbps), so throughput is rarely the problem. Connections are. It can support up to 55,000 simultaneous connections to each unique destination (same destination IP, port and protocol). A fleet that opens a fresh connection per request to one API endpoint can exhaust that, and the symptom is failed connections with the NAT gateway’s ErrorPortAllocation metric going above zero. The fixes are connection reuse (keep-alive, pooling), associating additional Elastic IPs with the NAT gateway (each adds another 55,000), or spreading traffic.
Idle connections are dropped after 350 seconds of inactivity. Long-lived connections that sit quiet, like some database or message-broker clients talking to an external service, need TCP keepalives shorter than that, or they’ll find their connection silently gone.
A few related facts that come up:
- IPv6 doesn’t use NAT. Private IPv6 subnets use an egress-only internet gateway for the same outbound-only behaviour.
- A private NAT gateway (no Elastic IP) exists for translating between private networks, for example when overlapping CIDRs need to talk via a transit gateway. It doesn’t give internet access on its own.
- A NAT instance (an EC2 instance doing NAT) is the old approach. It’s cheaper for tiny workloads but you manage patching, scaling and failover, and you must disable source/destination checks on it.
Interview practice: outbound access that fails in one zone
An application in three AZs has private subnets in each. Monitoring shows that only instances in AZ c can’t call an external API; a and b are fine. All three private route tables were created by the same template. What would you check, and what’s the likely fix?
Answer
Since only one zone fails, I’d compare that zone’s path with the others. Start with the route table associated with AZ c’s private subnet: does 0.0.0.0/0 point at a NAT gateway, and is the route active rather than blackhole? If the template creates one NAT gateway per AZ, AZ c’s NAT gateway might have failed to create, been deleted, or be in a subnet without an internet gateway route, which leaves that one route broken.
Then check the NAT gateway itself: its state should be Available, it should have an Elastic IP, and it must be in AZ c’s public subnet. If it’s healthy, look at its ErrorPortAllocation and PacketsDropCount metrics: if instances in c happen to make most of the calls to this one API, they could be exhausting connections to that single destination.
Finally, check whether the partner allowlists source IPs. Each NAT gateway has its own Elastic IP, so if AZ c’s NAT gateway was recreated with a new address that the partner hasn’t added, a and b would work and c would be rejected. That’s a very plausible cause, and the fix is to register the new IP or keep the Elastic IPs stable across rebuilds.