This post is one in a series of posts that are sample chapters from my FREE eBook 20 Things You Can DoTo Save Money On Your Amazon Web Services Bill Today. Feel free to download the full eBook for free if you want the whole thing.

Section 1 – Always On Vs. Just In Time

When you boot up virtual infrastructure, you have a couple of options.

Option 1 - Always On (AKA Provisioned) Infrastructure:

You can launch an always-on infrastructure that you are paying for hourly. This is also known as provisioned infrastructure

Option 2 - On Demand Infrastructure:

The alternative is to boot up something that you only get charged for when it is in use. Oftentimes, this is described as “Serverless,” but the two terms are not perfectly synonymous.

Things to consider:

Speed: The “Always On” solution is most likely faster than using a serverless computing tool, but you will need to wait a little bit for it to boot up. We're typically talking milliseconds, but depending on the size of your codebase/docker image, it could be seconds.

For example, I have an ML tool for image generation with a worker process that runs on demand, and because of the size of that docker image, it takes 30 seconds or more to boot up. Now, that would never be acceptable for something needed to run in real-time, like a restful web server, but for a background worker? Why not?

Cost: The “Always On” servers typically have a fixed cost of $x/hr. This means that you always have at least a minimum baseline you will be paying even if there is no load on the compute resources.

Let’s contrast this to the “On Demand,” where the costs go up and down based on your needs.

You need to determine your baseline need and use that to decide which route is better for you.

Now, it might sound like I am saying “On Demand” is always more cost-efficient than “Always On,” and that is not completely true. At a certain scale, when you can accurately predict demand, the “Always On” solution combined with auto-scaling can be vastly more cost-effective than the “On Demand” service.

Look at it from the hosting provider's standpoint: If they have idle servers sitting around, that costs them money, so they build in an extra cost to the “On Demand” service to account for when no one is using it.

Scalability: In reality, the “Always On” and the “On Demand” solutions can be set up with auto-scaling. The difference is the “On Demand” solutions typically have autoscaling built in and can scale down to zero by design. A lot of the “Always On” solutions have auto-scaling, but you need to add those. For each of the entries in Section 1, I will discuss how to implement auto-scaling for the “Always On” resources.

Making The Right Decision: There are always trade-offs, and sometimes the “Always On” solution will be the right solution, but it shouldn’t be that by default. Use the information in this document to help guide you in making those decisions.