Using Ansible to interact with web endpoints

I’m always looking for clever things to do with Ansible. With so many tools and services leveraging HTTP-based Application Programming Interfaces (APIs), it’s clear that interacting with API-driven services from Ansible programmatically is a valuable capability. This might sound like an advanced feature, but this article takes you through a use case that demonstrates how even simple environments can benefit from the power and simplicity of Ansible’s URI module.

Interacting with simple endpoints

First, I’ll walk you through a playbook that leverages Ansible’s HTTP abilities to make intelligent decisions during a webserver upgrade. The playbook below:

1. Runs a maintenance script.
2. Checks to ensure that a health check API endpoint returns an HTTP 503 Service Temporarily Unavailable message.
3. Runs a script to upgrade the application.
4. Runs a post-maintenance script to tell the webserver to begin responding normally again.
5. Rechecks the health check API to ensure that it’s responding with 200 OK.

Here is the playbook:

---

- hosts: all
  tasks:
    - name: Run maintenance start script
      command:
        cmd: /usr/local/sbin/start_maintenance.sh

    - name: Confirm that 503 Unavailable response is returned
      uri:
        url: "http://{{ ansible_host }}/api/v1/healthcheck"
        status_code: 503

    - name: Update application
      command:
        cmd: /usr/local/sbin/update_app.sh

    - name: Run maintenance end script
      command:
        cmd: /usr/local/sbin/end_maintenance.sh

    - name: Confirm that 200 OK response is returned
      uri:
        url: "http://{{ ansible_host }}/api/v1/healthcheck"
        status_code: 200

I’m using Ansible’s URI module to reach out to /api/v1/healthcheck on the server. The first URI call expects an HTTP 503 status code to be returned since the server should be in maintenance mode and not servicing requests. After the upgrade, the URI call expects an HTTP 200 status code, indicating that the webserver is healthy again.

This simple approach improves the safety of my playbook. If the server fails to enter maintenance mode, then Ansible won’t perform any patching:

fsh$ ansible-playbook -i inventory.ini playbook-healthcheck.yml

PLAY [all] ***********************************************************************************

TASK [Gathering Facts] ***********************************************************************
ok: [nyc1-apiserver-1.example.com]

TASK [Run maintenance start script] **********************************************************
changed: [nyc1-apiserver-1.example.com]

TASK [Confirm that 503 Unavailable response is returned] *************************************
fatal: [nyc1-apiserver-1.example.com]: FAILED! => changed=false 
  connection: close
  content: ''
  content_length: '0'
  content_type: application/octet-stream
  cookies: {}
  cookies_string: ''
  date: Fri, 11 Sep 2020 18:35:08 GMT
  elapsed: 0
  msg: 'Status code was 200 and not [503]: OK (0 bytes)'
  redirected: false
  server: nginx
  status: 200
  url: http://nyc1-apiserver-1.example.com/api/v1/healthcheck

PLAY RECAP ***********************************************************************************
nyc1-apiserver-1.example.com : ok=2    changed=1    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0  

If the server fails to come back up correctly after patching, then Ansible fails with an error:

fsh$ ansible-playbook -i inventory.ini playbook-healthcheck.yml

PLAY [all] ***********************************************************************************

TASK [Gathering Facts] ***********************************************************************
ok: [nyc1-apiserver-1.example.com]

TASK [Run maintenance start script] **********************************************************
changed: [nyc1-apiserver-1.example.com]

TASK [Confirm that 503 Unavailable response is returned] *************************************
ok: [nyc1-apiserver-1.example.com]

TASK [Update application] ********************************************************************
changed: [nyc1-apiserver-1.example.com]

TASK [Run maintenance end script] ************************************************************
changed: [nyc1-apiserver-1.example.com]

TASK [Confirm that 200 OK response is returned] **********************************************
fatal: [nyc1-apiserver-1.example.com]: FAILED! => changed=false 
  connection: close
  content: |-
    <html>
    <head><title>503 Service Temporarily Unavailable</title></head>
    <body>
    <center><h1>503 Service Temporarily Unavailable</h1></center>
    <hr><center>nginx</center>
    </body>
    </html>
  content_length: '190'
  content_type: text/html; charset=utf-8
  date: Fri, 11 Sep 2020 18:55:01 GMT
  elapsed: 0
  msg: 'Status code was 503 and not [200]: HTTP Error 503: Service Temporarily Unavailable'
  redirected: false
  server: nginx
  status: 503
  url: http://nyc1-apiserver-1.example.com/api/v1/healthcheck

PLAY RECAP ***********************************************************************************
nyc1-apiserver-1.example.com : ok=5    changed=3    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

These are simple checks that can be built into almost any playbook to add better safety guarantees before performing disruptive work or to ensure that disruptive work was successful before calling it a success.

Parsing returned JSON

The previous example works great for simple, HTTP-status based health checks. However, you’ll commonly want to retrieve some data from a web endpoint and then do something with the returned data. For example: What if I want to check the application version via an exposed endpoint and only perform updates if it isn't up to date?

My demo application has just such an endpoint. When queried, it returns the current version of the application:

fsh$ http nyc1-apiserver-1.example.com/api/v1/appVersion
HTTP/1.1 200 OK
Accept-Ranges: bytes
Connection: keep-alive
Content-Length: 24
Content-Type: application/json
Date: Fri, 11 Sep 2020 18:36:15 GMT
ETag: "5f5bc33b-18"
Last-Modified: Fri, 11 Sep 2020 18:34:35 GMT
Server: nginx

{
    "appVersion": "1.0.1"
}

Note: Curious about that HTTP command that I ran? Check out my fellow sudoer Jonathan Roemer’s article about HTTPie.

I can use the returned JSON from this endpoint to make decisions in my Ansible playbook. The previous version of this playbook would always run the application update script. However, I can improve on this by only updating the application when it doesn’t meet my desired version requirements:

---

- hosts: all
  vars:
    desired_app_version: "1.0.1"
  tasks:

    - name: Check API version
      uri:
        url: "http://{{ ansible_host }}/api/v1/appVersion"
      register: api_version_result

    - name: Perform maintenance tasks
      block:
        - name: Run maintenance start script
          command:
            cmd: /usr/local/sbin/start_maintenance.sh

        - name: Confirm that 503 Unavailable response is returned
          uri:
            url: "http://{{ ansible_host }}/api/v1/healthcheck"
            status_code: 503

        - name: Update application
          command:
            cmd: /usr/local/sbin/update_app.sh

        - name: Run maintenance end script
          command:
            cmd: /usr/local/sbin/end_maintenance.sh

        - name: Confirm that 200 OK response is returned
          uri:
            url: "http://{{ ansible_host }}/api/v1/healthcheck"
            status_code: 200

        - name: Check API version after updates
          uri:
            url: "http://{{ ansible_host }}/api/v1/appVersion"
          register: updated_api_version_result
          failed_when: updated_api_version_result['json']['appVersion'] != desired_app_version
      when: api_version_result['json']['appVersion'] != desired_app_version

This playbook introduces a few useful Ansible concepts. First, you can see that the URI module reaches out to the /api/v1/appVersion API endpoint and registers the output of this URI call to a variable. The update tasks have been moved into a block, which allows for the logical grouping of tasks. The addition of the when clause causes this block to only execute if the current app version is different from the desired app version, as returned by the /api/v1/appVersion endpoint. Finally, I’ve added an additional check to the update process. Once the updates have been run, another call to the /api/v1/appVersion endpoint ensures that the update succeeded and that the current app version matches the desired version. This uses the failed_when syntax, which allows you to define specific failure criteria for tasks.

Expressed in plain language, this Ansible block logic says: “Only run the application maintenance and installation scripts if the current version of the app doesn’t meet the desired version of the app. Once the update is done, make sure that the app has really been updated.”

Using only a few lines of Ansible code, I have written a powerful but simple way to use JSON returned from an API endpoint to make intelligent decisions in my playbooks.

Interacting with an authenticated endpoint

So far, I’ve covered interacting with API endpoints that don’t require authentication. However, you’re probably most used to interacting with APIs that require some type of authentication, such as an API token. The URI module supports this by setting the headers and body of an HTTP request.

[ You might also enjoy: 9 Ansible guides to help you ease into automation ]

I can take my maintenance playbook a step further by disabling and re-enabling alerting on each host in my monitoring system. This requires sending a POST request to an API endpoint on the monitoring server. The request must contain my API token and the host inside of the JSON-encoded body. Ansible makes this simple. Here is the final playbook:

---

- hosts: all
  vars:
    desired_app_version: "1.0.1"
    api_token: "8897e9a6-b10c-42c8-83a2-c83e9c8b6703"
  tasks:

    - name: Check API version
      uri:
        url: "http://{{ ansible_host }}/api/v1/appVersion"
      register: api_version_result

    - name: Perform maintenance tasks
      block:
        - name: Disable host in monitoring
          uri:
            url: "http://nyc1-monitoring-1.example.com/api/v1/startMaintenance"
            method: POST
            headers:
              X-API-KEY: "{{ api_token }}"
            body_format: json
            body:
              host: "{{ ansible_host }}"

        - name: Run maintenance start script
          command:
            cmd: /usr/local/sbin/start_maintenance.sh

        - name: Confirm that 503 Unavailable response is returned
          uri:
            url: "http://{{ ansible_host }}/api/v1/healthcheck"
            status_code: 503

        - name: Update application
          command:
            cmd: /usr/local/sbin/update_app.sh

        - name: Run maintenance end script
          command:
            cmd: /usr/local/sbin/end_maintenance.sh

        - name: Confirm that 200 OK response is returned
          uri:
            url: "http://{{ ansible_host }}/api/v1/healthcheck"
            status_code: 200

        - name: Check API version after updates
          uri:
            url: "http://{{ ansible_host }}/api/v1/appVersion"
          register: updated_api_version_result
          failed_when: updated_api_version_result['json']['appVersion'] != desired_app_version

        - name: Re-enable host in monitoring
          uri:
            url: "http://nyc1-monitoring-1.example.com/api/v1/stopMaintenance"
            method: POST
            headers:
              X-API-KEY: "{{ api_token }}"
            body_format: json
            body:
              host: "{{ ansible_host }}"

      when: api_version_result['json']['appVersion'] != desired_app_version

I’m now using the URI module to send HTTP POST requests (instead of the default GET requests) to the /api/v1/startMaintenance and /api/v1/stopMaintenance endpoints on nyc1-monitoring-1.example.com. These requests contain my API token for the monitoring server in the header, and the hostname of the server is included in the body. If either request fails with a non-200 status code, then the entire Ansible playbook fails.

Note: In practice, you’ll want to use something like Ansible Vault to store an API token, instead of placing it directly in the playbook.

This final set of tasks allows me to completely automate my upgrade workflow: Performing version checks, interacting with external monitoring to disable alerting for a system, and ensuring that the server returns the correct HTTP status codes before and after patching. I now have an end-to-end workflow that automates many of the common steps that I follow when performing upgrades on a system.

[ Need more on Ansible? Take a free technical overview course from Red Hat. Ansible Essentials: Simplicity in Automation Technical Overview. ] 

Wrapping Up

This article started with a simple playbook that performed basic web checks against unauthenticated API endpoints. I took you through parsing JSON responses and even interacting with authenticated API endpoints by setting custom headers and body contents on HTTP requests. Ansible makes interacting with web services easy, and I’m confident that you’ll find uses for this type of approach, even in simple environments.

If you’re looking to learn more and to challenge yourself, here are some ideas to implement on your own:

• Use the URI module to interact with your favorite API-based web service.
• See if you can figure out how to do authentication with client certificates.
• Learn how to submit a form to a website using Ansible.