Tag: ansible

Dynamic DNS in less than 25 lines of Ansible YAML and CloudFlare CDN

johniLeave a comment

Overview

The requirement for dynamic DNS has been around for decades and companies like DynDNS have been an enabler for just as long.  Script kiddies, IRC dudes, gamers & professionals often want to host services out of their homes for various reasons, but, may not have a static IP address for their internet connection.  Dynamic DNS services allows the user to update a hostname with a provider to point back to the dynamic IP address allocated to the users modem.  This allows people to reference the domain name record to return the IP address of the modem.

Note: I’m not talking about RFC2136 which includes a dynamic DNS mechanism in the DNS protocol.

I host a few services at home which I like to reach remotely from time to time, and, I’m too tight to pay for a static IP address.   A few years ago this was the task I decided to force myself to solve using python in an attempt to learn.  Whilst ugly, it served its purpose for quite some time until last night when I set myself a task to do this with Ansible in the evening.

The Players

Ansible  is a simple automation tool with use cases across a number of use cases such as Provisioning, Configuration Management, Application Deployment, Orchestration and others. Ansible has plugins and modules which extend it’s functionality.  In this case we are using the ipinfoio_facts and cloudflare_dns modules to query/communicate with…

Cloudflare I see as the Content Delivery Network (CDN) for the people.  Free basic plans,  API interfaces, proxying and DNS management.

ipinfo.io, a neat little site/service to give you geolocation information about where you are browsing from.  This site also returns the data in JSON format if requested, which, makes it nice an easy to query programatically.

A linux Ansible command host to run the Ansible playbooks from…. and setup a crontab to continually run the playbooks.

Some domain names that I have registered with various domain registrars.

The Process (TL;DR)

  1. Ensure you have a domain name to use.
  2. Ensure you have a Cloudflare account, with, the domain name associated.
    1. Take note of your cloudflare API token which is found under My Profile > API Key
  3. Ensure you have a linux box with Ansible installed on it (tested with 2.3.x)
  4. Clone https://github.com/Im0/CloudFlare_DyDNS_Playbook.git
  5. Update the following fields in the cf_dydns_update.yml file

    1. cf_api_token: ‘YOUR API KEY’

    2. cf_email: ‘YOUR CLOUDFLARE EMAIL’

    3. with_items: – The domain names you want to update
  6. Run ansible with:

    ansible-playbook cf_dydns_update.yml

Obviously, you’ll probably want different DNS records updated.  Change the ‘record: mail’ an A record of your choice.

More detail

 


1 ---
2 - hosts: localhost
3 gather_facts: no
4 vars:
5 cf_api_token: 'CF API token under My Profile.. API key'
6 cf_email: 'Cloud Flare email address'
7
8 tasks:
9 - name: get current IP geolocation data
10 ipinfoio_facts:
11 timeout: 5
12 register: ipdata
13
14 # - debug:
15 # var: ipdata.ansible_facts.ip
16
17 - name: Update mail A record
18 cloudflare_dns:
19 zone: '{{ item }}'
20 record: mail
21 type: A
22 value: '{{ ipdata.ansible_facts.ip }}'
23 account_email: '{{ cf_email }}'
24 account_api_token: '{{ cf_api_token }}'
25 register: record
26 with_items:
27 - domain1
28 - domain2
29
30 # - debug:
31 # var: record

 

Breaking down the YML file..

  1. Required at the top of oru YAML files
  2. As we are not configuring any nodes, we set localhost as the only node we want to call against.
  3. As we aren’t using any facts, we don’t need to collect them.
  4. Variables we’re going to need to talk to cloudflare
  5. The API token found under our profile
  6. Our sign up email address for cloudflare
  7. .
  8. The tasks section for all tasks we are going to execute in this playbook
  9. .
  10. Using the ipinfoio_facts module we query ipinfo.io for our externally visible IP address.  Note: If we are being a proxy of some sort this will likely break what we are trying to achieve.
  11. .
  12. This could probably be done a bit better and dropped.  We are registering the output of the module to the ipdata variable.  This could probably be removed as the returned data ends up in the gathered facts which we could use.
  13. .
  14. If we want to see what useful little nuggets of information that have come back, dump the variable contents.
  15. .
  16. .
  17. .
  18. Use the cloudflare_dns module to start talking to cloudflare
  19. Which domain (zone) are we talking about?  In this case we iterate over the domains listed starting line 26 ‘with_items’:
  20. record: is the record we wish to update.
  21. type, is the type of record we are working with.  A few other examples are on the cloudflare_dns module page.
  22. Use the data we received from ipinfoio.  We’ve stashed this away in the data structure: ipdata.ansible_facts.ip
  23. Our cloudflare email
  24. Our cloudflare API key
  25. Capture the output from the cloudflare_dns queries, if we want to dump it in debug later.
  26. With items is a list of items we iterate over… instead of hosts.

 

Ansible auditing callback plugin – Infrastructure testing “as code”

johniLeave a comment

I came across a project at work recently which was developing some code to audit the state of an environment.  At the same time I was starting to play around with Ansible at home to define my personal systems desired state as code.  Being new to Ansible,  I wondered what it would take to audit my home systems using Ansible.  I thought I’d have a go at writing the same thing out of hours and compare how the two solutions differed once complete.

From a beginners perspective, it seemed like it would be pretty easy.  The basic playbooks I had previously built returned OK’s, CHANGED and FAILED.  I figured an auditing script only needed to return OK’s and FAIL’s… at this point I hadn’t thought of the issue I was about to run into.

The default way Ansible operates is that it will stop processing the playbooks when an error is returned from a task.  This allows you to fix the problem and restart where you left off.  As it’s perfectly fine for an audit script to have failed tests, the default way Ansible worked was no good for my purpose here.

 

Work around

There was a way around this.  ‘ignore_errors: yes’ in a task allowed an Ansible playbook to continue processing if it found errors.  The only problem for me was that this resulted in any failed tasks being reported as OK in the end of playbook statistics.

The other thing that was throwing out the end of playbook statistics was that when using shell or command modules in the task, Ansible counts this as changed (system state was changed) and reports this in the statistics.  Using ‘changed_when: false’ helped on command/shell tasks, however, the ignore_errors issue still existed.

I really like Ansible… how could I work around the issue I was facing?

 

Develop a plugin

As Ansible was returning OK for tasks that FAILED when ignored_errors was set,  the statistics were skewed at the end of the playbook.  I decided to persevere with Ansible and wondered how to work around this.  I delved into the Ansible Architecture overview / developer guide.  It seemed like I’d need to store my own statistics to overcome the hurdle of ignore_failed statistics.

I came across Ansible Modules and Plugins.

Modules are essentially telling Ansible how to connecting to your node/device and configure it’s desired state.

Plugins allow you to alter how Ansible works.

From the Developer Guide:

The following types of plugins are available:

  • Action plugins are front ends to modules and can execute actions on the controller before calling the modules themselves.
  • Cache plugins are used to keep a cache of ‘facts’ to avoid costly fact-gathering operations.
  • Callback plugins enable you to hook into Ansible events for display or logging purposes.
  • Connection plugins define how to communicate with inventory hosts.
  • Filters plugins allow you to manipulate data inside Ansible plays and/or templates. This is a Jinja2 feature; Ansible ships extra filter plugins.
  • Lookup plugins are used to pull data from an external source. These are implemented using a custom Jinja2 function.
  • Strategy plugins control the flow of a play and execution logic.
  • Shell plugins deal with low-level commands and formatting for the different shells Ansible can encounter on remote hosts.
  • Test plugins allow you to validate data inside Ansible plays and/or templates. This is a Jinja2 feature; Ansible ships extra test plugins.
  • Vars plugins inject additional variable data into Ansible runs that did not come from an inventory, playbook, or the command line.

The Callback plugins allowed me to hook into Ansible events for the events I wanted to track.  For example:It looked like developing a Callback plugin was my best bet to capture OK/FAILED/SKIPPED/UNREACHABLE tasks and store statistics about them.


def runner_on_failed(self, host, data, ignore_errors=False):
"""Routine for handling runner (task) failures"""
...

def runner_on_ok(self, host, data):
"""Routine for handling runner (task) successes"""
...

def runner_on_skipped(self, host, item=None):
"""Routine for handling skipped tasks"""
...

def runner_on_unreachable(self, host, data):
"""Routine for handling unreachable hosts"""
...

These would allow me to keep track of the number of times any of these events fired and which host they were for.

 

 

Preparing to run the Ansible audit

To use the audit callback plugin, you’ll need:

  • Ansible 2.3.0.0 & 2.3.1.0 (tested)
    • Other versions may work
  • Python 2.7 (tested)

Then, download the callback plugin from: https://github.com/Im0/ansible-audit

Either write a sample playbook, or, download an example from here:  https://github.com/Im0/ansible-audit-playbook-example

 

Running the Ansible audit script

The easiest way to run an audit would be to clone the audit git repository and copy the audit.py and *.jinja files into the ansible plugins directory.  In my case the plugins directory resides here: /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/

The plugins directories may reside in other places as well.  For example, the default configuration file for Ansible (although commented out) refers to /usr/share/ansible/plugins/

git clone https://github.com/Im0/ansible-audit.git
sudo cp ansible-audit/audit* /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/
git clone https://github.com/Im0/ansible-audit-playbook-example.git
AUDIT_NAME='My Audit' CUSTOMER="Test Customer" ANSIBLE_CALLBACK_WHITELIST=audit ansible-playbook --ask-sudo-pass ansible-audit-playbook-example/site.yml -k

 

Checking the output

By default the plugin creates and outputs a zip file into: /var/log/ansible/audits/

The zip file contains a JSON file and two HTML files which contain the results.

 

Fruit salad output

 

More detail output

 

Todo

There are a few things I want to tidy up, such as:

  • Colour on UNREACHABLE hosts on the fruit salad output needs to be red.
  • Always output the zip file full path on exit
  • Add some error handling around the output directory creation
  • Update instructions to include output directory requirements
  • Tweak output filename
  • Consider sending output via email
  • Add options to select method of delivering output (ie. send via email, to zip file or just JSON output)
  • Windows examples in example playbook