Building AWS infrastructure via the AWS console is pretty easy using your mad pointy clicky skills. However, perhaps you wondered if there was a different way? A repeatable, “programmatic” way with less pointy clicky action.
Well, there is, and it’s called Cloud Formation. Amazon describes Cloud Formation as:
AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS.
You create a template that describes all the AWS resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and AWS CloudFormation takes care of provisioning and configuring those resources for you.
You don’t need to individually create and configure AWS resources and figure out what’s dependent on what; AWS CloudFormation handles all of that.
Check out the Cloud Formation documentation for more information.
I recently build my first Cloud Formation template by hand, and, this article discusses some of my observations whilst doing so.
First Up
Draw (design) what you need first
If you don’t have a good idea of what your infrastructure should look like, you’ve lost already. You’ll probably end up with a Frankenstein of a build with bits and pieces missing, or, additional components which will just cost you additional money.
Draw it out first using your favourite design tool such as Visio, Dia, Gliffy, OmniGraffle, PowerPoint, Paint, draw.io, cloudcraft.co, Pen and Paper or whatever you want to draw your infrastructure diagram with.
The first CF JSON file I was looking to build was based on the following diagram.
Diagram 1. High level diagram of the solution we are aiming to implement with CF.
Note: Check out the AWS architecture icon packs. Above was created with the Visio stencils.
Creation Options
Build via Console, dump configuration with Cloud Former
A manual option is to build your infrastructure as normal via the AWS console. Once complete, search for Cloud Formation and click on “Launch CloudFormer”. Cloud Former fires up some AWS instances and walks you through exporting your running AWS configuration into a JSON file, which, you can later import (Create a stack) back into Cloud Formation.
Using this option gives you an insight as to how a CF JSON file should look, the names it gives each element are not the names you gave the elements but the under the hood names AWS uses. Eg; vpc-4b7e932e
- The names are stored as Tags for each resource { ‘Name’: ‘MyVPC’ }
Whilst I preferred manually building the JSON file with the VS Code extensions mentioned below, the Cloud Former created JSON file helped identify some of the missing resources from the diagram. For example, AWS::EC2::SubnetRouteTableAssociation is not represented in the diagram.
Note: CloudFormer is a cloud formation stack of it’s own, and, you will be charged for the use of the resources it consumes. I suggest shutting it down as soon as possible after dumping your configuration.
Build with Cloud Formation Designer (drag and drop)
I really liked the idea of CF Designer. It’s a graphical tool to drag and drop elements onto a canvas and link them together. Unfortunately, I couldn’t get to grips with it and I felt clumsy working with it.
CF Designer allows you start from scratch, or, load a JSON file and it will display what it looks like as a diagram. For example, the export I took from CloudFormer (above) looks like this when loaded into CF Designer:
Diagram 2. CloudFormation Designer diagram
Build manually, “Create a stack”
Building out your JSON file by hand can be rewarding, but, time consuming. You’ll be wanting to refer back to the AWS Resource Types on a regular basis.
Whilst CF will be able to work out dependencies (if you’ve done your references correctly), the order of your resources in the CF stack JSON file won’t matter… however, it seemed easier to refer back to your design diagram and start from the outside working inwards.
eg. Outside to inside
- VPC
- Subnets
- Availability zone
- Security groups
- Elastic IP/Internet Gateway
- NAT gateways
- Load balancer nodes
- Compute nodes
As mentioned above, the diagram doesn’t capture everything you need in the CF JSON template to make the solution actually work. Below is a list of all the AWS Resource Types used in the final JSON template file.
-
AWS::EC2::VPC
-
AWS::EC2::Subnet
-
AWS::EC2::InternetGateway
-
AWS::EC2::VPCGatewayAttachment
-
AWS::EC2::RouteTable
-
AWS::EC2::SubnetRouteTableAssociation
- AWS::EC2::Route
- AWS::EC2::SecurityGroup
-
AWS::EC2::EIP
-
AWS::EC2::NatGateway
-
AWS::ElasticLoadBalancing::LoadBalancer
-
AWS::EC2::Instance
Check out some of the Visual Code Studio tools (below) to assist building the JSON file out.
Rinse and repeat (for free – mostly)
You will probably find yourself “Creating stack” and “Delete stack” on a regular basis until you get your JSON file just right.
Save yourself some money, and ensure you try to stick to the order above. There are a number of resources that are not chargeable, so, you can tinker with them in your JSON until your heart’s content.
Once you start spinning up Load Balancers, NAT Gateways, Elastic IP’s and Instances (and others) you will start getting charged.
Working with JSON
Handraulic
I started off without a decent IDE, and, without any extensions/plugins to make life easier. Whilst this was OK to start with, as more resources found their way into the CF JSON template it started becoming a mess.
IDE/Text Editor
Hopefully you’ve got a favourite text editor that will save you some time. Some considerations for a good IDE/text editor are intellisense and extensions/plugins.
A couple of decent editors include VIM, Atom, Notepad++, Visual Code studios. Obviously there are heaps of good editors other there.
Visual Code Studio
When working with Microsoft Visual Code Studio I found the following extensions super helpful for building out the JSON file.
Access the extensions area of visual studio code via (1) on the image.
Other useful extensions to load/install were 2 & 3.
2. CloudFormation by aws-scripting-guy
This is a great extension to help build out your CF JSON file. Install the extension, open a new file, set syntax to JSON, type start + tab and watch it populate a basic AWS CF template for you.
{“AWSTemplateFormatVersion”: “2010-09-09”,“Description”: “”,“Metadata”: {},“Parameters”: {},“Mappings”: {},“Conditions”: {},“Resources”: {},“Outputs”: {}}
Start typing the type of resource you wish to add (VPC for example), and, watch it populate a default VPC resource into your JSON. Then, you just work through filling in the blanks appropriately.
“name“: {“Type”: “AWS::EC2::VPC”,“Properties”: {“CidrBlock”: “—-/–“,“Tags”: [–]}}
3. JSON Tools by Erik Lynd
If your JSON is getting a bit unwieldy or ugly, JSON Tools helps fix up any alignment issues.
Ctrl(Cmd)+Alt+M for JSON pretty.
Alt+M for JSON minify.
Troubleshooting
Bastion Host
Sometimes, you’re not quite sure why a stack is not working as desired… ie. Why hasn’t my load balancers become active.
Well, you could look to get console access, or, I found it quicker and easier to setup another subnet, security gateway and micro EC2 instance to act as a bastion host. Make sure you allow it to be reachable from the internet with a Elastic IP and required security group.
Best Practices
Make sure you check out the AWS CF Best Practices.
Resulting Files
Will upload if any demand.