Posted: December 5, 2020

When I was a self-employed consultant, I spent hundreds of dollars a month on data center costs for my (non-production) R&D services – a legitimate business expense. These days, I still maintain server instances in the cloud. But now, they are a hobby cost that I pay out of pocket and I want to be frugal with that.

Most of my servers are leased in the Amazon AWS cloud. Anybody who has experience with AWS knows that the initial bite is cheap, but it’s really easy for costs to spiral out of control. When you use AWS, every bean is counted and you get charged for it.

With AWS, you aren’t paying just for a virtual computer instance. You also have to pay for the cost of data storage, the cost of data transfer to data storage, the cost of data transfer to the Internet, additional IP addresses … and on and on. As I said, each bean is counted.

My AWS bill for November 2020 was $52.88. I wanted to see if I could knock that down a bunch.

(I realize that those of you who use AWS professionally are now rolling your eyes, because you probably are facing thousands – if not tens of thousands – of dollars a month in AWS costs. However, the lessons here about good resource choices apply to you too, even more so.)

For the services I was running, I felt that $634/yr was way too much and I wanted to spend less.

The AWS complexity tax

I discovered that Amazon AWS has a complexity tax can be a significant part of your bill.

You face a lot of choices when setting up your cloud environment. Choose wisely and you can run one of the most massively complex data operations – say Netflix – in the cloud. Choose poorly, and you are overpaying for ill-selected components. The complexity presents a challenge to choosing the right components, making it easy to pay significantly more than you need.

By adjusting my resource selections, I was able to knock my daily charge down from $1.76 to $1.06 – a savings of nearly 40%. This involves no application tuning or server tweaking. I just made better choices for my AWS components.

Two mistakes in my setup incurred this 40% complexity tax.

Choose the right EC2 instance for right now

The virtual computers in Amazon AWS are called Amazon EC2. When you choose an EC2 instance, you are making a choice on a two-dimensional axis: instance type and instance size.

The instance type presents a virtual computer that’s optimized for a certain use case: general purpose, compute intensive, memory intensive, etc. You select an instance type, then you select an instance size: small, medium, large (to name but a few).

Put them together and they produce a cost matrix of your instance choices.

I’ve been very good about selecting the right instance size for my tasks. I start with a smaller size, then enlarge to get just the amount of performance and capacity that I need.

The problem was that I was using the wrong instance types for right now. They might have been good choices several years ago when I first set up my server cluster, but things change.

Back then, I picked the t2 instance type as a good choice for my application – low duty Internet servers. Technology advances, and several new instance types were introduced since then. Would one of them be a better choice? I think yes: the t3a instance type seemed well suited for my application – at a lower cost.

The current cost of the t2.micro instance type is $0.0116/hr. The current cost of the t3a.micro instance type is $0.0094/hr. That’s a savings of 19% right there.

Changing my instance types from t2 to t3a was pretty easy. I went through each EC2 instance, shut it down, reconfigured the instance type in the AWS console, and booted it back up. It took about 10 minutes to do this, and that one change saved me 19% off the compute cost of my AWS bill.

Compute cost was 51% of my overall bill, so this was a good start. But I wasn’t done.

The default EBS storage is hella expensive

The next cost component, at 38% of my overall bill, was the storage cost. A large part of that expense was due to poor choices that were easily correctable.

Every computer needs storage, typically a hard drive. So too do EC2 compute nodes. That’s what Amazon EBS does. Each EC2 node will (by default) have a small 8GB EBS volume that serves as its boot drive.

The problem is that the default EBS volume type – the gp3 type – is kind of expensive at $0.08/GB-month. The default 8GB volume created with a new EC2 instance costs $0.64/month.

There were two problems with my storage setup.

First, I had an unused EC2 node that I kept as a warm spare for testing. It had only a basic OS on it. I kept it shut down so it incurred no EC2 costs. Yet, it still incurred that EBS $0.64/month cost for the idle storage. That’s only 1.2% of the $52.88/month total, but that was an unnecessary cost. I decided to delete this node, and re-create it as needed. My setup is automated via Ansible, so this wouldn’t be a huge imposition.

Second – and this is the bigger problem – I had a low use 150GB data volume provisioned as gp3 storage. This could easily be handled by one of the less expensive storage types.

The cheapest EBS is the sc1 storage type, which is $0.015/GB-month, and would be perfectly fine for this application. By converting this data volume from gp3 to sc1 I could reduce this cost from $12/month to $2.25/month. That $9.75 saved represents 18% of my total monthly bill.

Once again, this was an easy conversion I did from the AWS administrative panel. Although that data volume was offline for a few minutes while the conversion was performed, I didn’t even have to reboot the EC2 node.

And then, there’s the laziness tax

I made the changes discussed in this article late in the evening on December 3, 2020. The daily cost results for the previous week are shown in the bar chart at the top of this article. My cost dropped from about $1.76/day earlier in the week to $1.06/day on December 4 – a cool 40% reduction.

I’m thinking about another round of cuts, to squeeze a bit more. After these changes, the EC2 charges are about 52% of my new monthly bill, so that would be the place to start.

One thing I could do is to consolidate to fewer EC2 nodes. I think I could save some money there.

The other is to prepay a year in advance. A t3a.micro instance costs $82/year when you pay by the hour. Pre-paying a year in advance makes that $55/year – a reduction of 33%.

This might get me another $5 off my expected $30/month bill – about another 5% reduction on my previous monthly bill. That’s a lot less return for a lot more work.

The 40% savings that I got just by making better choices was a big win. I’m delighted to stop paying a 40% complexity tax. I’m probably willing to continue paying a small 5% laziness tax.