Managing workflow tasks

Workflow tasks (also called stage definitions) describe a reusable task that can be added in to one or many workflows or can be executed directly.

There are several types of stage types:

  1. HTTPRequest

  2. AnsibleBundle

  3. WorkflowReference

  4. Copy file

In addition we provide several out of the box stages that allow the users to build fully custom provisioning workflows.

HTTP Request tasks

HTTP Requests stage definitions are very powerful and they provide a more or less universal mechanism that can be used to issue API calls, retrieve assets etc.

Any of the fields in stage_definition, such as the URL or ‘Authentication’ header field can be references to variables or secrets so that the stage can be parameterized for each situation in which it is used such as Authorization or Cookie fields.

$ metalcloud-cli stage-definition new --label "vcenter-login" --title "vcenter-login" --type "HTTPRequest" --http_request_method GET --http_request_url='{{vcenter}}/rest/com/vmware/cis/session' --vars "vcenter"

Ansible Bundle workflow tasks

Ansible is a very powerful tool to effect changes on one or more hosts via SSH. This stage can affect multiple hosts at the same time if the Workflow has been attached to an infrastructure.

A task entry holds one or more ansible playbook files in a .zip file that is stored in the Global controller’s database and sent to the Site controller for execution when needed. The root directory of the .zip file must have a main.yml file as the only execution entry point.

To create a new ansible stage using the cli:

$ metalcloud-cli stage-definition new --label "ansible1" --title "ansible1" --type "AnsibleBundle" --ansible_bundle_filename "/test/bundle1.zip"

A main.yaml file would look like this:

- hosts: all
  tasks:
  - name: Ansible create file with content example
    copy:
      dest: "~/ansible_bundle.txt"
      content: |
        default content

The projects in the .zip files must not contain the variables, private keys or inventory files. They are provided automatically when executing the Ansible project.

When executed the variables.json file is created automatically (no need to provide one in your Ansible project). Ansible is executed like this: ansible-playbook main.yml --tags all -i inventory.ini -e @variables.json

Ansible Bundle inventory file

The inventory file is created automatically with all active servers at the moment of execution (instance_array_label as group key and with instance_label as server key).

the private keys necessary for SSH provisioning for all the servers are added to the bundle as individual files

The variables file is populated with secrets() where the secret_label is used as variable name.

When the stage is executed “pre_deploy” then these variables are also available:

  • will_be_removed_instance_hostnames

  • will_be_added_instance_hostnames

  • will_be_active_instance_hostnames

When the stage is executed “post_deploy” then these variables are also available:

  • removed_instance_hostnames

  • added_instance_hostnames

The instances_hostnames variable is always populated with the currently active instances’ subdomains.

The generated inventory file is similar to this example:

[instance-array-48]
instance-36	ansible_ssh_user='root' ansible_ssh_host='192.168.138.130' ansible_ssh_private_key_file='instance-36.metalsoft.local.io.private.key' host_fqdn='instance-36.metalsoft.local.io' instance_ipv4='192.168.138.130' instance_ipv4_cidr='192.168.138.130/30' instance_ipv4_subnet_mask='255.255.255.252' instance_subdomain_permanent='instance-36.metalsoft.local.io'

[instance-array-49]
instance-39	ansible_ssh_user='root' ansible_ssh_host='192.168.138.131' ansible_ssh_private_key_file='instance-39.metalsoft.local.io.private.key' host_fqdn='instance-39.metalsoft.local.io' instance_ipv4='192.168.138.131' instance_ipv4_cidr='192.168.138.130/30' instance_ipv4_subnet_mask='255.255.255.252' instance_subdomain_permanent='instance-39.metalsoft.local.io'
instance-37	ansible_ssh_user='root' ansible_ssh_host='192.168.138.132' ansible_ssh_private_key_file='instance-37.metalsoft.local.io.private.key' host_fqdn='instance-37.metalsoft.local.io' instance_ipv4='192.168.138.132' instance_ipv4_cidr='192.168.138.130/30' instance_ipv4_subnet_mask='255.255.255.252' instance_subdomain_permanent='instance-37.metalsoft.local.io'

Additional variables

The Ansible playbook will also receive additional information in the form of variables (--extra-vars). The following variables are provided automatically:

{
"repo_url_root": "http://repo.metalsoft.io",
"datacenter_kms": "84.40.58.70:1688",
"datacenter_name": "us-chi-qts01-dc",
"dns_server_ip_0": "1.1.1.1",
"dns_server_ip_1": "172.18.38.6",
"ntp_server_ip_0": "45.55.58.103",
"infrastructure_id": 788,
"user_ssh_keys_yaml": "...",
"instances_hostnames": [
  "instance-2083.us01.metalsoft.io"
],
"monitoring_agent_ip": "176.223.248.11",
"infrastructure_label": "test-22",
"added_instance_hostnames": [],
"agents_intra_datacenter_ip": "172.18.38.6",
"removed_instance_hostnames": [],
"agents_externally_visible_ip": "176.223.248.11",
"infrastructure_subdomain_base": "us01.metalsoft.io",
"infrastructure_subdomain_permanent": "infrastructure-788.us01.metalsoft.io"
}

The user can add additional variables to the infrastructure, instance array or instance level via the infrastructure editor and they will be provided in this file. For more information consult Managing variables and secrets

How ansible gets executed

MetalSoft stores a separate private SSH key for each host in the database. The public key is deployed onto the host as part of the unattended installation file.

Ansible is invoked on the Site Controller like this:

ansible-playbook main.yml -c ssh -M _library --tags all -i inventory.ini -e @variables.json

WorkflowReference stages

This is an utility stage that allows an workflow to call another workflow allowing you to build a tree of workflows.

$ metalcloud-cli stage-definition new --type WorkflowReference --title "call test workflow" --label callwf1 --workflow test

Listing stages

To list stage definitions available to the current user:

$ metalcloud-cli stage-definitions list
Stage Definitions I have access to as user [email protected]:
+-------+-------------------+------------------+-------------+-------------+-----------------------------------------------------+----------------------------------------------------------------+----------------------+----------------------+
| ID    | LABEL             | TITLE            | DESCRIPTION | TYPE        | VARS_REQUIRED                                       | DEF.                                                           | CREATED              | UPDATED              |
+-------+-------------------+------------------+-------------+-------------+-----------------------------------------------------+----------------------------------------------------------------+----------------------+----------------------+
| 322   | vcenter-login     | vcenter-login    |             | HTTPRequest | vcenter                                             | HTTP Request URI: {{vcenter}}/rest/com/vmware/cis/session     | 2020-01-31T15:26:03Z | 2020-01-31T15:26:03Z |
| 328   | vcenter-host-add  | vcenter-host-add |             | HTTPRequest | vcenter,vcenter_creds,vmware_ipv4_0                 | HTTP Request URI: {{vcenter}}/rest/vcenter/host               | 2020-01-31T16:19:35Z | 2020-01-31T16:19:35Z |
| 332   | vcenter-login2    | vcenter-login2   |             | HTTPRequest | vcenterhost                                         | HTTP Request URI: {{vcenterhost}}/rest/com/vmware/cis/session | 2020-01-31T17:52:59Z | 2020-01-31T17:52:59Z |
| 336   | vcenter-host-add3 | vcenter-host-add |             | HTTPRequest | vcenterhost,vcenter_creds2,vmware_ipv4__address_0_0 | HTTP Request URI: {{vcenter}}/rest/vcenter/host               | 2020-01-31T18:14:01Z | 2020-01-31T18:14:01Z |
+-------+-------------------+------------------+-------------+-------------+-----------------------------------------------------+----------------------------------------------------------------+----------------------+----------------------+
Total: 4 Stage Definitions