Managing workflows

Note: Support for workflows is experimental and subject to change

Workflows are directed acyclic graphs (DAGs) of StageDefinitions (or tasks) that are executed in a context. Stage definitions can be of many types such as HTTP Requests, API Requests, arbitrary JavaScript as well as references to other workflows. See Managing stage definitions for more details on stage definitions.

An example install process that adds a host to a vcenter server: ../_images/managing_workflows1.pngan example install process

Workflow execution contexts

Workflows can be executed on their own or as part of an infrastructure’s deploy process. Stages can be parameterized with input variables or secrets.

The system will look for variables and secrets in the following order:

  1. The current user’s context
  2. The global (public) variables and secrets
  3. The infrastructure’s dynamic variables and secrets

Workflow usages

Workflows can have different “usages” which are used throughout the application as a way to filter which workflows can be used where. Possible workflow types:

  • infrastructure
  • network_equipment
  • server
  • free_standing
  • storage_pool
  • user
  • os_template

Creating a workflow

To create a ‘free standing’ (independent) workflow using the CLI:

metalcloud-cli workflow create --label test3 --title test3 --usage free_standing

Listing workflows

To list workflows of the current user:

metalcloud-cli  workflow list
Workflows I have access to as user alex.d@d.com:
+-------+-------+----------------+-------------+------+-------------------------+------------+----------------------+----------------------+
| ID    | LABEL | USAGE          | DESCRIPTION | TITLE| OWNER                   | DEPRECATED | CREATED              | UPDATED              |
+-------+-------+----------------+-------------+------+-------------------------+------------+----------------------+----------------------+
| 9999  | test  | infrastructure |             | test | alex.d@d.com | false      | 2020-03-04T17:17:43Z | 2020-03-04T17:17:43Z |
| 10000 | test2 | free_standing  |             | test3| alex.d@d.com | false      | 2020-03-05T14:54:33Z | 2020-03-05T14:54:33Z |
| 10002 | test3 | free_standing  |             | test3| alex.d@d.com | false      | 2020-03-05T14:56:00Z | 2020-03-05T14:56:00Z |
+-------+-------+----------------+-------------+------+-------------------------+------------+----------------------+----------------------+
Total: 3 Workflows

Adding stages to a workflow

To add a stage to a workflow into a runlevel using the CLI use:

$ metalcloud-cli stage-definition add-to-workflow --id vcenter-login --workflow test --runlevel 100

Add two stages on the same runlevel, after the first one:

$ metalcloud-cli stage-definition add-to-workflow --id vcenter-host-add --workflow test --runlevel 101

$ metalcloud-cli stage-definition add-to-workflow --id vcenter-host_add3 --workflow test --runlevel 101

Check the resulting workflow:

$ metalcloud-cli  workflow get --id test
Workflow test (9999) has the following stages:
+----------+---------------------------------------------------------------------------+
| RUNLEVEL | STAGES                                                                    |
+----------+---------------------------------------------------------------------------+
| 100      | vcenter-login(#322)-[WSI:# 44008]                                         |
| 101      | vcenter-host-add(#328)-[WSI:# 44009] vcenter-host-add(#336)-[WSI:# 44012] |
+----------+---------------------------------------------------------------------------+

Please note that a stage cannot be added more than one time on the same runlevel.

The behavior of WorkflowReference in instance array contexts

The InstanceArray context is special and will run the stage for each active instance at the moment of execution (the deploy planner anticipates which instances will be active at deploy time for the position where the stage was added).

../_images/managing_workflows2.png

First black square shows a WorkflowReference executed 3 times, once for each instance of the context InstanceArray (instance_array_id is defined on the InfrastructureDeployStageDefinitionReference).

Second black square its a simple HTTPRequest stage definition executed for each Instance of the context InstanceArray (instance_array_id is defined on the InfrastructureDeployStageDefinitionReference).

../_images/managing_workflows3.png

These are HTTPRequest stage definitions referenced in a Workflow which is referenced via WorkflowReference on the infrastructure.

The HTTPRequest stage definition reference is marked with an instance array label, not an ID. You can see it added 3 times, once per active instance.

Workflow Variables

Apart from the variables and secrets configured globally, or locally per stage in a workflow a series of contextual variables are added at runtime, such as:

 "extra_variables": {
            "cluster_id": 59,
            "instance_id": 58,
            "cluster_label": "vanilla",
            "instance_label": "instance-58",
            "infrastructure_id": 20,
            "instance_array_id": 71,
            "instances_hostnames": [
                "instance-58.metalsoft.local.io",
                "instance-65.metalsoft.local.io",
                "instance-66.metalsoft.local.io"
            ],
            "infrastructure_label": "aaaaaaaaaaa",
            "instance_array_label": "instance-array-71",
            "added_instance_hostnames": [],
            "removed_instance_hostnames": [],
            "cluster_subdomain_permanent": "cluster-59.metalsoft.local.io",
            "instance_subdomain_permanent": "instance-58.metalsoft.local.io",
            "instance_array_71_ipv4_cidr_0_0": "192.168.138.154/30",
            "instance_array_71_ipv4_cidr_1_0": "192.168.138.130/29",
            "instance_array_71_ipv4_cidr_2_0": "192.168.138.131/29",
            "instance_array_71_ipv4_mask_0_0": "255.255.255.252",
            "instance_array_71_ipv4_mask_1_0": "255.255.255.248",
            "instance_array_71_ipv4_mask_2_0": "255.255.255.248",
            "infrastructure_subdomain_permanent": "infrastructure-20.metalsoft.local.io",
            "instance_array_71_ipv4_address_0_0": "192.168.138.154",
            "instance_array_71_ipv4_address_1_0": "192.168.138.130",
            "instance_array_71_ipv4_address_2_0": "192.168.138.131",
            "instance_array_subdomain_permanent": "instance-array-71.metalsoft.local.io"
...

Variables defined on the stage reference (infrastructure or workflow) will be passed down into WorkflowReference stage definitions, thus allowing custom variables on a workflow execution for the entire workflow.