Selected work · ~$1M Overcharge, Prevented
Why every account needs a budget alert before it needs anything else
Details identifying the cloud provider and the customer are withheld — this is about the mechanism, not the names. A billing gap around API metering nearly cost a customer roughly a million dollars. Here's what actually happened, and what it means for how you should be setting up alerting today.
A customer was load-testing an API integration. The billing dimension wasn't the number of API calls they'd budgeted for — it was the volume of data each call returned, a dimension the provider had no reliable way to show them in advance. The gap between what they expected to pay and what they were billed came out to roughly $1M.
01The root cause
- The customer had scoped their testing around call volume — a number they could see and control.
- The actual charge was driven by the size of the data returned per call — a number the provider's tooling didn't surface anywhere before or during the test.
- There was no way for the customer to have caught this themselves ahead of time — the billing-relevant metric simply wasn't visible.
02What I actually did
1. Caught it via monitoring, alerted the customer immediately
Usage monitoring flagged the anomaly early enough that the first and most obvious fix — the customer turning the API test off — happened right away, before the exposure grew further.
2. Opened a ticket with the provider and pushed past the support layer
I filed the overage dispute and stayed the single point of contact between my company, the customer, and the provider's technical team, until the root cause — the invisible per-call data-size billing — was fully understood on both sides.
3. Filed a feature request, and got the charge reversed
Once the provider's engineers confirmed there was no existing mechanism to preview data-size billing, I filed a formal feature request documenting the gap. That, combined with the case for the charge, was enough for the provider to reverse the ~$1M.
4. Built the missing visibility ourselves
The feature request didn't ship on the provider's side. So we built a custom monitoring solution for the customer instead — one that tracked the actual billing-relevant metric (response payload size) and alerted on it directly, rather than waiting on a platform fix that wasn't coming.
03The takeaway
Usage and cost in the cloud can both be genuinely unpredictable — and a billing dimension doesn't have to be exotic to be invisible until it's on an invoice. That's not a reason to under-monitor; it's the reason budgets and alerting need to exist before the first real workload runs, not after the first surprising bill.
- Track the metric that actually drives the charge, not just the metric that's easy to see (call count vs. response size, in this case).
- Set a hard cap, not just a soft alert, when you're testing something with an unfamiliar cost shape.
- Put a budget in place on every account before real usage starts — this is the same discipline covered in more depth on the FinOps & Cost Optimization page.
The charge wasn't fraud or a mistake — it was a metering gap nobody could see until it was too late. Catching it fast, escalating it correctly, and then building the missing visibility ourselves is what turned a ~$1M dispute into a resolved one, and a repeatable lesson: budgets and alerts go in before usage starts, on every service, every time.
I want to walk through how a roughly $1M overcharge got prevented — without naming the hyperscaler or the client involved, since this is really about the mechanism, not who was on either side of it.
The customer was testing out their APIs. They'd signed up expecting to be billed based on the number of API calls they invoked — a number they could plan around and control. But that's not actually what drove the charge. The provider didn't have a reliable way to show, ahead of time, how much data would come back from each call — and it was the volume of information returned, not the call count, that got billed. That gap between what the customer expected and what they were actually charged is what produced the roughly million-dollar number.
What actually happened next
The first thing was monitoring — we were able to share the alerts with the customer, and once they saw them, they did the first obvious thing anyone could do: they turned the API test down. That stopped the bleeding immediately.
From there, we alerted the hyperscaler and opened a ticket for the overage. It took some digging to get to the real root cause — it wasn't the number of calls to the API, it was the information returned from each call that was actually being billed, and that number wasn't something the customer had any way to see in advance.
I was the sole point of contact representing my company through that whole process — working directly with the customer and bringing in the technical team on the hyperscaler's side until we all understood exactly what had happened. I filed a feature request documenting the gap, and that, combined with the case we built, was enough for the hyperscaler to relieve the spend. The roughly $1M charge got reversed.
Since the fix never shipped
The feature request wasn't something the hyperscaler ended up implementing. So we didn't just leave it there — we worked with the customer to build a custom solution ourselves, one that could actually view these metrics and alert on the custom metric we built, since the platform wasn't going to give us that visibility on its own.