Idle NAT Gateways: $32 a Month for Nothing, and How to Remove One Safely

Imagine you call a taxi, right? It pulls into your driveway, the driver starts the meter, and then you just, you never get in. - You just leave it there. - Yeah, exactly. You leave it idling there for a month, which sounds completely absurd, but based on the stack of AWS billing teardowns and architectural notes we're looking at today, engineering teams do exactly this all the time.

- Oh, constantly. - So our mission in this deep dive is to hunt down this very specific, really expensive waste pattern, the idle NAT gateway. - Right, and for anyone who needs a quick refresher, a NAT gateway is basically the bouncer that lets your private servers talk to the public internet, you know, without exposing them to inbound threats. - Essential stuff. - Exactly, it's crucial infrastructure.

But because of that, people often assume it's simply necessary and they just look right past that meter taking in the background. - Well, let's actually look at that meter because the unit math in these sources is pretty eye-opening and we are skipping the marketing fluff here. What is the actual baseline cost for this taxi just sitting there doing nothing? - Yeah, let's break down the exact numbers. The hourly charge is $0.045 per hour.

- So four and a half cents. - Right, four and a half cents in US East one. You run that math and it comes out to about $32 a month and that is your baseline cost, whether or not a single byte of traffic actually flows through it. - Wow, okay. And if traffic actually does flow.

- Then the meter runs faster. The data processing charge is also $0.045 per gigabyte. - Wait, 45 cents? - No, no, never 45 cents, always four and a half cents per gigabyte. And I mean, it doesn't stop there.

The elastic IP attached to a public NAT gateway is billed as public IPv4 at $0.005 or half a cent per hour. - Half a cent per hour. Okay, so if I'm managing this infrastructure, my immediate thought is like, how do these resources pile up silently? If a server sits there racking up $32 a month doing absolutely nothing, you'd think an alarm would go off somewhere. - You'd think so.

But they survive because of a classic organizational trap. Nobody actually owns them. - Right, the shared responsibility black hole. - Exactly, like a developer spins one up for a specific project, right? The project architecture shifts over time, but the gateway just stays.

- Because it's not actually broken. - Precisely. A quiet gateway doesn't trigger operational alarms. And from a billing perspective, these costs are just incredibly easy to hide inside massive per-service AWS bills. - It just gets swallowed up in that huge monthly EC2 line item.

- Yeah, exactly. - Okay, so say I want to hunt these ghosts down right now, you know, without buying expensive software. I go into CloudWatch, I see zero traffic, and I just delete it. Problem solved, right? - Well, not quite.

I mean, you could look at the exact read-only signal today in CloudWatch. You want to check BytesOutToDestination and ActiveConnectionCount. - Okay, both of those. - Yeah, you need both to be at zero over at least a seven-day window. But, and this is a massive but here, you cannot just pull the plug based on that alone.

- Why not? - Because you must also check which route tables actually point at the gateway. - But why do the route tables matter if CloudWatch already proves no traffic went through for a whole week? - Well, because CloudWatch only tells you if traffic successfully went through historically. Your route table is like a map.

- Okay. - Even if no cars have driven on a road in a week, if the map still tells your private subnet servers to drive toward that gateway and you delete it. - Oh, I see. - Yeah, you instantly cut that subnet's outbound internet access. - So deleting the gateway is technically the correct remediation, but you're risking a total outage if the map is still pointing to a missing bridge.

- Exactly. You should never call a resource provably or guaranteed unused. - Because something might still be configured to rely on it. - Right. No check makes deletion completely safe.

And remember, deletion cannot be undone. You have to verify what dependencies like those private subnet route tables still rely on it. - And I'm guessing the billing doesn't entirely stop just because the gateway is gone, right? - Right again. Remember that elastic IP we mentioned?

- The half a cent one? - Yep, it's capped and it keeps billing you that half a cent per hour unless you explicitly release it. You really have to clean up both pieces. - Man. So if the risk of deleting these is so high, isn't there a way to just, you know, avoid needing them in the first place?

Like what if my private instances only need to talk to internal AWS services like S3 or DynamoDB? - Oh, in that case, use gateway endpoints. Gateway endpoints for S3 and DynamoDB are completely free and you avoid those data processing charges entirely. - So you basically stop paying for the public taxi if you're just walking over to your own backyard? - That's a great way to put it, yeah.

- That makes total sense. Now verifying all of those route tables and CloudWatch metrics manually sounds incredibly tedious. - It is tedious, yeah. - Well, if you're looking for this, there is a free source available scanner. You just run pip install cloudcostwise that runs read only on your own machine, flags this exact pattern and sends nothing to CloudWise.

- Nice. It's definitely better than clicking through the console for hours. - For sure. Which brings us back to that taxi idling in the driveway. It really makes you wonder what other invisible utilities are running in your infrastructure right now, taking up a monthly bill while every team just assumes someone else is using them.