Deploying the MetalSoft Global Controller on Kubernetes

The MetalSoft Global Controller is a Kubernetes application and as such it runs as a collection of containers, services, configurations etc.

The MetalSoft Site Controller is still a separate VM, hosted outside of the Kubernetes cluster, one in each site.

Kubernetes cluster compute requirements

Any flavor of Kubernetes is supported, including, Vanilla Kubernetes, RedHat OpenShift, AWS EKS etc. You can use a pre-existing cluster, MetalSoft doesn’t need a dedicated cluster as long as the requirements are met.

Minimum Setup

A minimal cluster, with no redundancy requires the following:

• 3 x Master & Worker nodes

  CPU	RAM	Disk Space
  48 cores	64GB RAM	200GB Local Disk Space + 1TB shared space via CSI

Recommended Setup

A production cluster should have 3 master nodes and 2-3 worker nodes.

• 3 x Master nodes

  CPU	RAM	Disk Space
  4 cores	8GB RAM	200GB Disk Space

• 3 x Worker nodes

  CPU	RAM	Disk Space
  16 cores	32GB RAM	200GB Disk Space + 1TB shared space via CSI

Storage Requirements

Storage is the most important bottleneck for scaling MetalSoft. Both disk space and sufficient performance is required for a successful deployment.

Support for persistent volumes with a minimum of 1TB of disk space is required either via a CSI or with the following pre-configured PVs:

• image builder ISO storage 800GB RWM (read-write-many) both image builder and ms-tunnel server use this volume
• controller-pvc 5Gb RWX
• mysql-pvc 100Gb RWX
• redis-pvc 10Gb RWX
• repo-pvc 10Gb RWX
• other microservices 10Gb RWX

Storage performance requirements

Caution

For best performance we recommend that the storage system used by the Kubernetes cluster to be either external to the cluster (another cluster, appliance, single server, etc.) or inside the cluster but isolated on nodes specifically dedicated to storage. This will prevent from high I/O on the workload nodes affecting other MetalSoft components or Kubernetes system workloads.

The storage must also allow for ReadWriteMany access mode and have CSI driver/provisioner available for it (ex. NFS with NFS Subdir External Provisioner). This will allow the image builder volume to be mounted by multiple image-builder pods.

Reference performance numbers:

• 4k Random Read : 1000 IOPs, 4MB/s
• 4k Random Write: 500 IOPs, 2MB/s

Other requirements

1. An SSL certificate is required for the domain that will be mapped to the MetalSoft application in DNS.
2. A load balancer mechanism such as MetalLB. (1 or 2 IPAddresses)
   • Setup with 1 IP Address will use ports HTTP 80, 443 and TCP port 9091
   • Setup with 2 IP Addresses, will use HTTP 80, 443 on one IP and port 443 on the other

Installing the MetalSoft Global Controller

1. Copy the manifests to /opt/metalsoft/manifests/namespace && cd /opt/metalsoft/manifests/namespace

2. Apply the manifests to the cluster: kubectl -n namespace apply -f .

3. Check if the controller components are all running:

K8S ng-metal root@k8s01: /opt/metalsoft/manifests/ng-qa $ kubectl -n namespace get pods
NAME                                        READY   STATUS    RESTARTS        AGE
auth-66fd54f9d8-pftpn                       1/1     Running   1 (4h11m ago)   4h15m
couchdb-6c869cb756-rc2sz                    1/1     Running   2 (4h11m ago)   10h
eli-5bc878bc7-cj4l6                         1/1     Running   1 (4h11m ago)   10h
extensions-769b54d49b-vj2gv                 1/1     Running   1 (4h11m ago)   10h
gateway-api-86476dbb7c-ml6w6                1/1     Running   1 (4h11m ago)   4h15m
image-builder-5565dbd9c7-hk5gq              1/1     Running   1 (4h11m ago)   4h15m
inventory-5c44c99f89-hdwpc                  1/1     Running   1 (4h11m ago)   4h15m
ipam-6dc74bcf78-7plp9                       1/1     Running   1 (4h11m ago)   10h
kafka-74b77ddd5b-f9lzc                      1/1     Running   2 (4h11m ago)   10h
license-674bcc66b9-xgg78                    1/1     Running   1 (4h11m ago)   4h15m
metalsoft-docs-5d45b56479-rx4kk             1/1     Running   2 (4h11m ago)   10h
ms-agent-8c9c4cdbc-fxqv5                    1/1     Running   8 (4h11m ago)   10h
ms-tunnel-86ddbf8c86-6gjw5                  1/1     Running   3 (4h11m ago)   10h
mysql-697f56958d-58tt2                      1/1     Running   2 (4h11m ago)   10h
network-microservice-86cf6dc8fc-nqpb4       1/1     Running   1 (4h11m ago)   10h
notification-microservice-d685c974b-65zxc   1/1     Running   1 (4h11m ago)   10h
orchestrator-6cd869cb45-4st69               1/1     Running   2 (4h11m ago)   4h15m
pdns-6b658c4554-zt9km                       1/1     Running   2 (4h11m ago)   10h
redis-764bff6bd6-bvdkz                      1/1     Running   2 (4h11m ago)   10h
registry-local-5967b98b4c-6tz6n             1/1     Running   2 (4h11m ago)   10h
repo-7bdc745c68-7qcvr                       1/1     Running   2 (4h11m ago)   10h
sc-ansible-runner-7b94846fdd-wmhcx          1/1     Running   2 (4h11m ago)   10h
servers-54bc6dc79b-j299l                    1/1     Running   1 (4h11m ago)   10h
storage-6986748575-skk67                    1/1     Running   1 (4h11m ago)   10h
switch-64f87c8659-s6dxz                     1/1     Running   1 (4h11m ago)   10h
template-75cc88c59b-m2hg2                   1/1     Running   1 (4h11m ago)   4h15m
traefik-demo-metalsoft-7dc8574dc9-ddz5p     1/1     Running   1 (4h11m ago)   4h15m
ui-admin-75f9977d54-chfzx                   1/1     Running   1 (4h11m ago)   10h
ui-customer-68797dbf44-cjmv8                1/1     Running   1 (4h11m ago)   10h
vm-microservice-66d8d8b76-hlvjf             1/1     Running   1 (4h11m ago)   10h
zookeeper-b796dd65b-ffzw2                   1/1     Running   2 (4h11m ago)   10h

Deploying in an air-gapped environment

In an air-gapped environment, the Kubernetes cluster must be able to pull the MetalSoft images. This is usually done via:

• An external registry such as Artifactory that stores the MetalSoft images
• Local images from a tar.gz file provided by MetalSoft for both the GC and the SC

In addition, at least during the initial setup the administrator performing the installation should be able to retrieve and install OS templates from Github. See Managing OS templates for more details.

Firewall rules

Refer to the following for more details:

• Network Flows and Firewall Requirements