For the last year or so, the majority of our new projects at Made Tech have used Ansible as our go-to tool for provisioning and configuring our servers with the software that runs on them. We’ve paired Terraform with Ansible and Chef previously for creating our cloud resources, but have recently been experimenting with using Ansible to see if the one tool was capable of both of these stages in our infrastructure setups. We’ve not been disappointed with our experiences so far.
Ansible and Amazon Web Services
Ansible has support for a variety of cloud services out of the box with the more recent releases. To name a few, AWS, Azure, Google and Rackspace. Full support of each of services API varies, however it appears as though AWS has the most supported set of features by Ansible.
If you’re familiar with using Ansible for provisioning your servers, then creating resources in the cloud is almost identical. You use AWS specific modules that represent the resources in AWS and, once defined in YAML, you’re able to run an Ansible playbook to apply those resources to your AWS account.
AWS credentials
When using Ansible with AWS, when you run your playbook, Ansible will be running commands from your local machine using the AWS API. It uses a Python library behind the scenes called boto to do this. As with most AWS libraries, boto looks for a configuration file on your computer located at ~/.aws/credentials. In this file I find it useful to group credentials into separate profiles so that you can be explicit in which credentials to use when running a command. For example, your credentials file may look like:
I’d be able to use these credentials by specifying the profile playground in my commands.
Infrastructure playbook
When setting up new resources for an infrastructure project, the first thing I usually do is setup the SSH keys that we’ll be using in future to connect to any servers to provision them. I create an SSH on my machine, and then add it to AWS.
To generate the key I use:
The value for the -C flag is the comment that is included in the public key. This can be something descriptive related to the project you’re working on. The -f value is the name of the file that the keypair will be saved to. The value you specify will be the exact name for the private key. The public key will have a .pub appended onto the filename automatically.
Create another file called build-infra.yml and in this file include:
The first three options, hosts, connection and gather_facts are setting up the Ansible playbook to use your local machine to run the subsequent modules that we define in it. As mentioned above, that is because behind the scenes we use our local computer to do the communication with the AWS API.
We’re then defining a standard Ansible task using one of the AWS modules, in this case, ec2_key which is used for managing AWS security keys. We tell AWS to use the name web for the key, and to load the content from the local file web.pub as the key content in AWS.
To run this, we use the following command:
We prefix the standard ansible-playbook command with two environment variables that are used behind the scenes by the boto library to connect to the AWS library. We tell it to use the playground profile from our ~/.aws/credentials file to connect to the API, and to run the AWS operations within the eu-west-1 AWS region.
By building our AWS playbooks in this manner, it makes them far more reusable than if we were to hardcode these values directly into our playbooks.
Lack of state file
Running the above command, you should see some output with a [changed] value next to the Add web keypairtask. Run the command again, and the same task will have an [ok] flag next to it as nothing has changed.
I see this as one of the benefits to using Ansible compared to Terraform. Ansible will check for state directly with AWS rather than a local state file that is updated each time a command is run.
Using resources between tasks
Some of the AWS Ansible modules require you to reference other AWS resources that you have created. You can use Ansible variables to manage these. To better demonstrate this, lets build the following resources; a set of three EC2 instances with Elastic IP addresses associated to them, that are sitting behind an Elastic Load Balancer. We’ll setup a security group and assign the instances to these, and then attach each instance to the load balancer. In summary, the steps are:
- Create keypair for instances (already done above)
- Create a security group to assign instances to
- Create three instances
- Assign Elastic IP addresses to instances
- Create a security group for the load balancer
- Create a load balancer
- Assign instances to load balancer
For each of the above steps, we’ll add the following tasks to our YAML file:
This is a standard security group permitting all outgoing traffic from the instances, and only allowing SSH and HTTP access inbound.
The ec2 module creates, modifies and removes EC2 instances based on the count of instances that are present with a particular tag assigned to them. In this case, we will create the number of EC2 instances defined by the exact_count value that have the Name tag of web assigned to them.
We store the details of these instances in a variable called web_instances once the command has completed. We’ll then use this variable to assign Elastic IP addresses to them in the following command:
In the above Ansible module, we’re using the with_items option to loop over a variable and run the ec2_eip module on each value. The ec2_eip module will allocate an EIP and associate it to the instance specified by the device_idvalue.
Similar to our instance security group, we create another that will be associated to the load balancer. Again we permit all outgoing traffic, but restrict access to HTTP only. For the specific instances, we’ll connect directly to them over SSH, therefore this protocol doesn’t need to be enabled on this group.
The above creates our load balancer with our specified security group. We listen for traffic on HTTP port 80 only.
We need to assign each of the instances we created in the task above to the load balancer. We loop over the instances variable again, as we did with the EIP task, and one by one attach the instances to the load balancer.
Running the above playbook will in turn run each of the tasks within it, gradually building up our resources.
If you change the count in the ec2 task, and re-run the playbook, you’ll notice that instances are created or removed depending on the count that currently exist.
Provisioning our instances with Ansible
The best way to tie together this introduction is to provision our new EC2 instances with Ansible and set them up as web servers.
For this, we’ll create another Ansible playbook with the specifics for the software configuration to apply to the instances, as well as using the Ansible dynamic inventory script so that we don’t have to manually specify any of the instances we want to apply the configuration to. Ansible will be able to do this by us specifying tags that are assigned to instances.
In order for this to work, we need to download the dynamic inventory script to the same directoy as our playbook from here.
We then need to give the file execute permissions:
The dynamic inventory script depends on a configuration file, so we download the default to the same directory also:
Copy the below playbook file to provision-infra.yml:
And then provision the instances by running the following command:
Once completed, the instances should have PHP configured with a running test script. As we added a health check to our load balancer, once it detects the presence of the script, it will start serving traffic to that instance. In the output of the above command you should see a line beginning with TASK [debug]. Within this, it’ll have a msg key with a value that is the public DNS value of the new load balancer. So you can copy that and visit it in a browser. Each time you refresh the page, you should see the value of the hostname output change. This is traffic being routed to each of the different instances on each request.
About the Author
Senior Technology Adviser at Made Tech
Code, coffee, cake and Eurovision. In no particular order.
