In the last blog post I showed how to create and deploy a Serverless static Hugo site hosted on AWS. To do this I split the the AWS components that drive the Cloud architecure into multiple CloudFormation templates deployed across two regions; my native Australian region based in Sydney (ap-southeast-2) and USA based North Virginia (us-east-1).

This is a necessary step as North Virginia is often used as the default region to integrate a global service with a resource that is not. For example, A CloudFront Distribution (global) using a ACM certificate (regional) to provide SSL or a Lambda@Edge (global) function linked to a Lambda Function Version (regional) to customise routing of AWS S3 resources to CloudFront. North Virginia is the oldest region in AWS and due to it’s occasional outages (still extremely rare!) has playfully coined the phase “friends don’t let friends use us-east-1”, but sometimes you need to!

So how do I orchestrate the deployment of the four CloudFormation templates created in the last step over 2 different regions while maintaining the order dependencies? Well, using Ansible!

What is Ansible?

Ansible is often though of as a Configuration Management tool along the likes of Chef, Puppet and Salt. However, it is much more than that (as are the other tools mentioned!). It can be used to orchestrate the deployment of the AWS resources, building and uploading the Hugo site using a “play” defined in a “playbook” written in YAML. The playbook executes a series of ordered “tasks” using native or 3rd party “modules” on the “host”, which for this blog post is my local development machine, but could also be a CI/CD build server. There are many resources to learn Ansible, but if you want a detailed and well-rounded book to learn from I suggest “Ansible: Up and Running, 2nd Edition” published by O’Reilly®.

Creating an Ansible playbook to deploy CloudFormation

To get started I’ll assume you already have installed the AWS CLI and Hugo, which were required in the last post. You’ll also need to install Ansible, which like the AWS CLI can be installed using pip as it is a Python library. The steps bellow indicate how to create the Ansible playbook YAML file playbook.yml and assumes it is run at the same relative path as the CloudFormation templates. Anisble is compatible with both YAML file extensions .yaml and .yml, but Ansible® usually prefer the shorter of the two in their documentation.

1. Ansible Playbook settings and vars

To get started I need to define the initial values of my singular “play” in the playbook. The hosts value is my own localhost as I’m deploying the CloudFormation from my machine. This is different from say a Configuration Management playbook which would be running tasks on a remote host to change the state of it. As such, the gather_facts value is set to false, as I’m not interested in getting extra information from my own system. In the Configuration Management scenario you could use this information to determine different operations depending on the operating system, version, etc…

From there I define some vars related to the static site such as a domain name, which can be used globally in the playbook. Ansible variables can be referenced in playbooks by using the Jinja2 templating system.

# playbook.yml Part 1

# Note. Our play is an item in a list as an Ansible Playbook supports many
- hosts: localhost # It's running locally and not on a remote hosts
  gather_facts: false # We don't need to get details of the host it's running on

  vars: # These variables will be passed to the tasks via [Jinja2](https://jinja.palletsprojects.com/en/2.10.x/) `{{ my_var }}` references.
    stack_name: mysite
    domain_name: YOUR_SITE.com
    alternative_domain_names: "*.YOUR_SITE.com" 

2. Deploying CloudFormation using the official Ansible module

The playbook contains a series of ordered tasks to deploy my multi-region and modular CloudFormation templates. Ansible tasks make use of “modules”, which are maintained by Ansible® and 3rd parties. Ansible® maintains many AWS related modules to help orchestrate deployments, provision and/or interact with AWS resources. One such module is cloudformation, which as the name suggests helps create and update CloudFormation stacks in an AWS environment.

Previously the CloudFormation stacks were deployed individually using the AWS CLI and outputs of North Virginian based stacks retrieved and passed in a very “hacky” way by exporting the outputs to a bash variable. For example, previously the the ACM certificate ARN was captured by describing the CloudFormation stack:

  # Example: "Hacky" way to get the output of a CloudFormation Stack in Bash
  export WEBSITE_ACM_CERTIFCATE_ARN=$(aws cloudformation describe-stacks --stack-name mysite-certificate --region us-east-1 --query 'Stacks[0].Outputs[?OutputKey==`Certificate`].OutputValue' --output text)

In Ansible you are able to capture the output of a task to a variable using the register property. In the playbook below the ACM Certificate ARN and the Lambda Function Version are passed the the final CloudFormation template to enable SSL and custom routing on the Cloudfront Distribution respectively. For example, the outputs of a CloudFormation stack deployed via Ansible with the property register: certificate_stack can be accessed by other tasks by referencing certificate_stack.stack_outputs.Certificate.

Additionally variables relating to the domain and CloudFormation stack names are passed to the task using Jinja2 references directly '{{ stack_name }}' or via string concatenation '{{ stack_name }}-certificate'.

# playbook.yml Part 2
  ...
  tasks:
    - name: Hosted zones stack
      cloudformation:
        region: ap-southeast-2
        stack_name: '{{ stack_name }}-hosted-zones'
        state: present
        template: hosted-zones.yaml

    - name: Certificate stack
      cloudformation:
        region: us-east-1
        stack_name: '{{ stack_name }}-certificate'
        state: present
        template: certificate.yaml
        template_parameters:
          DomainName: '{{ domain_name }}'
          AlternativeDomainNames: '{{ alternative_domain_names }}'
      register: certificate_stack # Register a variable of that represents the CloudFormation deployment output object

    - name: Redirect index stack
      cloudformation:
        region: us-east-1
        stack_name: '{{ stack_name }}-redirect-index'
        state: present
        template: redirect-index.yaml
        template_parameters:
          DomainName: '{{ domain_name }}'
      register: redirect_index_stack

    - name: Site stack
      cloudformation:
        region: ap-southeast-2
        stack_name: '{{ stack_name }}'
        state: present
        template: static-site.yaml
        template_parameters:
          DomainName: '{{ domain_name }}'
          AcmCertificateArn: '{{ certificate_stack.stack_outputs.Certificate }}'  # Reference the certificate CloudFormation deployment
          OriginRequestLambdaVersionArn: '{{ redirect_index_stack.stack_outputs.RedirectionFunctionVersion }}' # Reference the custom routing CloudFormation deployment
      register: site_stack

3. Building the Hugo static site and upload to S3 using the official Ansible module

With the AWS infrastructure for the Serverless static site deployed using the previous tasks, only the build and upload of the Hugo generated site remain. For Hugo, there is no official or popular 3rd party Ansible module to use with a task. For this reason I’ve chosen to deploy it by using the command module which allows me to call hugo generation command /usr/local/bin/hugo -s src in bash.

I could have used the command or shell module to also deploy the CloudFormation tasks, but it would have been more raw and harder to get the output of the CloudFormation stacks, which was one of the main reasons to use Ansible. This mindset should be used for most Ansible tasks in general. If there is a native module to enable what you are trying to achieve, use that instead of command or shell.

Finally the generated static site content can be uploaded using the s3_sync module, which copies the content to the targeted bucket defined by the s3 bucket name output of my final template and delete any older files that are no longer in use with the delete: true property. This task is essential the equivalent of the AWS CLI command aws s3 sync src/public/ s3://mybucket --delete.

# playbook.yml Part 3
  ...
  tasks:
    ...
    - name: Hugo build
      command: /usr/local/bin/hugo -s src

    - name: S3 upload
      s3_sync:
        bucket: '{{ site_stack.stack_outputs.BucketName }}'
        file_root: src/public/
        delete: true

4. The Complete Ansible Playbook

Now that the Ansible playbook is complete it can be deployed using the ansible-playbook command!

# Deploy Command
ansible-playbook playbook.yml
# playbook.yml Complete
- hosts: localhost
  gather_facts: false

  vars:
    stack_name: mysite
    domain_name: YOUR_SITE.com
    alternative_domain_names: "*.YOUR_SITE.com" 

  tasks:
    - name: Hosted zones stack
      cloudformation:
        region: ap-southeast-2
        stack_name: '{{ stack_name }}-hosted-zones'
        state: present
        template: hosted-zones.yaml

    - name: Certificate stack
      cloudformation:
        region: us-east-1
        stack_name: '{{ stack_name }}-certificate'
        state: present
        template: certificate.yaml
        template_parameters:
          DomainName: '{{ domain_name }}'
          AlternativeDomainNames: '{{ alternative_domain_names }}'
      register: certificate_stack

    - name: Redirect index stack
      cloudformation:
        region: us-east-1
        stack_name: '{{ stack_name }}-redirect-index'
        state: present
        template: redirect-index.yaml
        template_parameters:
          DomainName: '{{ domain_name }}'
      register: redirect_index_stack

    - name: Site stack
      cloudformation:
        region: ap-southeast-2
        stack_name: '{{ stack_name }}'
        state: present
        template: static-site.yaml
        template_parameters:
          DomainName: '{{ domain_name }}'
          AcmCertificateArn: '{{ certificate_stack.stack_outputs.Certificate }}'
          OriginRequestLambdaVersionArn: '{{ redirect_index_stack.stack_outputs.RedirectionFunctionVersion }}'
      register: site_stack

    - name: Hugo build
      command: /usr/local/bin/hugo -s src

    - name: S3 upload
      s3_sync:
        bucket: '{{ site_stack.stack_outputs.BucketName }}'
        file_root: src/public/
        delete: true