ECS Fargate "Tag Immutability" Is Breaking The Internet


Has your ECS Fargate Deployments behavior changed seemingly at random? Don't worry, you are not going crazy. AWS just decided to change how ECS deployments are handled. Here is the announcement:

https://aws.amazon.com/blogs/containers/announcing-software-version-consistency-for-amazon-ecs-services/

Today we are excited to announce a new feature for Amazon Elastic Container Service (ECS): software version consistency. Amazon ECS will now resolve a container image tag to its container image digest for every version (deployment) of an Amazon ECS Service. This ensures that the same container image is used throughout the lifecycle of the deployment, and increases both the security and consistency of your applications deployed as Amazon ECS services.

To give you some context that announcement was made in July of 2024 and my clients started noticing issues with their ECS Deployments in mid-August 2024.

I would have preferred they made it a setting or something you could turn off instead of forcing it on us.

Basically changing an image and cycling tasks no longer pulls the latest image matching the tag specified in the Task Definition. If you want to get the latest image tag you will need to do a full ECS deployment.

Unfortunately, they have NOT updated all their documentation yet so you will see many docs that still say the image will NOT cache.

There is a feature request on GitHub to make it a setting on the ECS Service that you can turn on and off which would be ideal: https://github.com/aws/containers-roadmap/issues/2393

If you need help fixing this issue feel free to reach out. Best of luck! ~Cheers.