Published on Wed, Feb 16, 2022 by Étienne Brouillard
Hopefully this won't be a controversial topic and an eye opener to some.
Here you go: You shouldn't name resources in AWS.
If you do, just don't put much effort into it. There is a better way which will give better results and won't cost as much in efficiency while improving on automation.
As programmer, we're told to always use significative variable names. It makes sense. Otherwise variables named as the likes of 'v092347absfoobar' would be cause for depression in teams really fast. Out of habit, teams have settled on naming conventions which eventually became some kind of industry standards.
In the past 4-5 years, programmer turned into DevOps and started writing Infrastructure as Code. It's the neatest invention since sliced bread, giving some kind of super powers to developers while improving on indempotency across all the software delivery process.
For developers, there's comfort in repeating practices that are well known, such as naming resources the same way they would with variables. For management and architect, there's also comfort in being able to control something like the names of resources because it's easy to grasp.
Let's start with this interesting naming convention:
{lambda|s3|sqs|etc.}-{domain}-{project}-{environment}-{stage}-{department}-{acl}-{zone}-{confidentiality}. (and I'm omitting some!)It contains all the information needed. From a glance, it gives right away what's necessary to understand the purpose of resource.
Speaking to the team, they obviously said they felt really comfortable with such a naming convention. The opposite would be strange. But in the end, something like sqs-cl-baboonherder-prod-green-rw-cac1a-c3 is just meaningful to the team only. For anyone else, this is just gibberish.
Resource names in AWS are just that: names. They do not help managing resources. Knowing that roughly 90% of the time names are optional, the following problems caused by enforcing AWS resource naming conventions are causes for madness.
The first issue with working with resource naming conventions is that it's a convulated solution for the simple objective of being able to identify what the resources are for. Is it efficient? No, not by any mean, figuring out what resources of a specific service are for a specific stage in one environment requires RegEx magic and complex API querying.
A second issue with this is that although names are optional for most resources in AWS, it's not because they're not important. In the case of an Elastic Container Service, some changes do require the replacement of the resource. In some situations, if the lifecycle is not managed properly, the infrastructure deployment will just fail. By simply foregoing the name the infrastructure pipeline will just run much smoother.
Finally, the teams building infrastructure end up asking themeselves a lot of questions, have committees, JIRA issue tickets issued as bug tickets. All this for a single thing that just isn't worth it. All this consumed time without any actual value...
AWS resources, in the strictest sense of Infrastructure as Code, are disposable. By being disposable that's where the true value lies, get them when you need them and destroy them when you don't. So many times I've seen architects turn into some kind of crazy cat lady which have an Excel spreadsheet of all the resources under their control. I kid you not. Don't be that crazy cat lady and treat your infrastructure as cattle, not pets.
Then what's the solution? Use tags and leave the naming to AWS (or if you do name them, the names should be based on tags and not independant from them). Tags are key-value pair labels applied to resources which describe facets of a resource instance. Tags can be applied to all resources which are billable.
At the organisation level you should invest in baselining the required tags for resources. Sysops will have means to query for resources which they're interested in monitoring. Billing managers will be able to apply chargeback efficiently. Data scientists will be able query for buckets containing the objects they're looking for. Developers will be able to create and dispose of environments at will while DevOps will able to setup monitoring through CloudWatch by tapping into EventBridge.
However, there's one caveat: there's no consistent API for Tag management across services. Each service has its own way of using tags. In my opinion EC2 is doing a great job at it by enabling querying and filtering straight out of the Describe-Instances action whereas S3 requires two step to get at the same point by using the List-Buckets and then the Get-Bucket-Tagging actions.
Out of a thousand of reasons for efficient resources tagging, being able to leverage these for resource querying is the most compelling one.
Each service has it's resource querying actions.
EC2 instances can be queried using tag filter. Filter queries do support wildcards and the following example queries all for all the instances in Canada Central region which have, for the project tag, the value triton-infserver-*:
aws ec2 describe-instances --filter 'Name=tag:project,Values=triton-infserver-*' --region ca-central-1It's also possible to get all instance having a tag, notwithstanding the value it contains. This query is great for inventory purposes:
aws ec2 describe-instances --filter 'Name=tag-key,Values=project' --region ca-central-1The following query is used to ensure all mandatory tags are configured correctly.It queries for the ID of instances which don't have a specific tag:
aws ec2 describe-instances --query 'Reservations[].Instances[?!not_null(Tags[?Key == "project"].Value)] | [].[InstanceId]' --region ca-central-1From a billing point of view, it actually makes a lot of sense to use tags. Tags categorizing projects, environments and stages can be used from the billing and cost management services to allocate costs.
For example, it's possible to query the Cost Explorer API using tags as a filter and group clauses:
aws ce get-cost-and-usage --filter '{"Tags": {"Key": "project", "Values":["abc"]}}' --time-period=Start=2022-02-01,End=2022-02-02 --granularity=DAILY --metrics=BlendedCost --group-by Type="TAG",Key="project" --region ca-central-1Scenarios for automation based on tags abound. To just list a few:
- EC2: Start or stop instances based on tags
- S3 Bucket Scanning
- Ensure infrastructure compliance AWS Config: Checking for Compliance with New Managed Rule Options
Names are just that, names. They have no value and shouldn't be worth any valuable time discussing name convention. If one is still required, it should be based on tags and name collision avoidance strategies.
While it's possible to peer review resource names, would it be considered a wise investment? Let's say that before acquire any kind of resource there's a naming validation process. So either during the review the team doesn't get its hands on the needed resources or worst, resources gets destroyed because they don't follow a naming convention. Quick reminder, a bucket can't be renamed. The time to value ratio of a process including a review of resource names is a mismanagement textbook example.
Luckily AWS Config can help. AWS Config is a fully managed service that provides you with an AWS resource inventory, configuration history, and configuration change notifications to enable security and governance. You should use AWS Config to enforce the application of tags on resources and act upon a non-compliancy using automation. The required-tags rule is a wonderful starting point. Several strategies can be put in place but the most common example is to apply default values to missing tags and use SNS to notify of deviations that require corrective actions.
It's also possible to run AWS Config custom rules deployed as Lambda functions which can apply custom business rules applicable to tagging conventions.
Invest time on tags and not resource naming conventions. It's just wasted time for everyone without actually providing the same value obtained by the use of tags. AWS provides numerous facilities and tools to leverage tags and it does so by not making tags stand in the way of teams by being infrastructure as code friendly.