Deploying the MetalSoft Global Controller

The MetalSoft Global Controller is a Kubernetes application and as such it runs as a collection of docker containers, services, configurations etc. Any flavor of Kubernetes is supported: Vanilla Kubernetes, RedHat OpenShift, AWS EKS etc.

Kubernetes cluster compute requirements

A typical cluster should have the following resources available on at least 3 nodes:

  • 16 CPU cores

  • 32GiB RAM

  • 100GiB disk space

In addition, support for persistent volumes with a minimum of 240GB of disk space is required either via a CSI or with the following pre-configured PVs:

  • controller-pvc 5Gb RWX

  • mysql-pvc 100Gb RWX

  • redis-pvc 10Gb RWX

  • repo-pvc 10Gb RWX

  • image builder ISO storage 150GB RWM (read-write-many) both image builder and websocket tunnel server use this volume

  • other microservices 10Gb RWX

Important

For best performance we recommend that the storage system used by the Kubernetes cluster to be either external to the cluster (another cluster, appliance, single server, etc.) or inside the cluster but isolated on nodes specifically dedicated to storage. This will prevent from high I/O on the workload nodes affecting other MetalSoft components or Kubernetes system workloads.

The storage must also allow for ReadWriteMany access mode and have CSI driver/provisioner available for it (ex. NFS with NFS Subdir External Provisioner). This will allow the image builder volume to be mounted by multiple image-builder pods.

Other requirements

  1. An SSL certificate is required for the domain that will be mapped to the MetalSoft application in DNS.

  2. A load balancer mechanism such as MetalLB.

Installing the MetalSoft controller kubernetes application

  1. Install helm

helm repo add traefik https://helm.traefik.io/traefik
helm install traefik-{{ env }} traefik/traefik --values traefik-helm-chart-configvalues.yaml -n {{ namespace }}

  1. Run kubectl apply on all the manifest files on the provided directory

kubectl apply -f *

  1. Check if the controller components are all running:

administrator@dtsy1lvpmsc001:~$ kubectl get pods -n metalsoft
NAME                                       READY   STATUS    RESTARTS       AGE
auth-microservice-5d848c9789-z48cv         1/1     Running   40 (26d ago)   44d
config-microservice-76565b74d8-flgfp       1/1     Running   10 (26d ago)   66d
controller-64c79f9dcc-4d5zj                1/1     Running   0              37h
couchdb-8475576f55-mgc59                   1/1     Running   3              66d
dell-redfish-6857f69cbd-8kvsf              1/1     Running   0              32h
event-microservice-794d797847-wnjt4        1/1     Running   20 (26d ago)   66d
gateway-api-84b5f69d4d-rphm7               1/1     Running   0              26d
image-builder-59fd97b95f-t5pl9             1/1     Running   0              22h
kafka-74fb7b999c-4gkpv                     1/1     Running   2              42d
metal-cloud-ui-7db8657b75-4r4js            1/1     Running   0              37h
mysql-9594dfb87-sb4wc                      1/1     Running   0              26d
pdns-8f747c64c-fszgn                       1/1     Running   0              26d
redis-84864d55d7-9cz9b                     1/1     Running   3              66d
repo-678df55b7b-6sf9l                      1/1     Running   3              66d
servers-5f45ff6955-6qqbg                   1/1     Running   0              31h
traefik-metalsoft-prod-6f5bcb7c65-nnbwr    1/1     Running   0              37h
websocket-tunnel-server-57887d758d-n77r2   1/1     Running   0              32h
zookeeper-d587fc894-6nmc6                  1/1     Running   3              66d

Required controller firewall configuration

Ports open for inbound from agents

The following ports open on the controller, on the MetalLB IP (the ip that moves between hosts), need to be accessible by agents:

  • web: TCP port 80

  • websecure: TCP port 443

  • eventservice: TCP port 9003

  • gateway-api: TCP port 9009

  • ws-tunnel-9090: TCP port 9090

  • ws-tunnel-9091: TCP port 9091

  • ws-tunnel-9011: TCP port 9011

  • ws-tunnel-9010: TCP port 9010

  • powerdns: UDP port 53

Ports open for inbound from clients

The following ports, on the MetalLB IP (the ip that moves between hosts), need to be accessed by end-clients (or admins).

  • web: TCP port 80

  • websecure: TCP port 443

  • graphql: TCP port 9009

  • dns: TCP/UDP port 53

Outbound traffic

The controller generates traffic towards the following destinations:

  • downloads.dell.com TCP port 443

  • downloads.linux.hpe.com TCP port 80

  • repo.metalsoft.io TCP ports 80,443

  • apt.kubernetes.io TCP ports 80,443

  • registry.metalsoft.dev TCP port 443

  • quay.io TCP ports 80,443

  • cdn.quay.io TCP ports 80,443

  • cdn01.quay.io TCP ports 80,443

  • cdn02.quay.io TCP ports 80,443

  • cdn03.quay.io TCP ports 80,443

  • k8s.io TCP port 443

  • registry.k8s.io TCP port 80,443

  • git.k8s.io TCP port 443

  • k8s.gcr.io TCP port 80,443

  • gcr.io TCP port 80,443

  • cloud.google.com TCP port 80,443

  • helm.traefik.io TCP port 443

  • smtp.office365.com TCP port 587 -> this is only if office365 is used for email alerts

  • archive.ubuntu.com TCP port 80 -> for base OS package updates

  • security.ubuntu.com TCP port 80 -> for base OS package updates

  • docker.io TCP ports 80,443 -> for pulling standard images

  • hub.docker.com TCP ports 80,443 -> for pulling standard images

  • registry.hub.docker.com TCP ports 80,443 -> for pulling standard images

  • registry-1.docker.io TCP ports 80,443 -> for pulling standard images

  • 1.1.1.1 ICMP -> for testing purposes

  • 1.1.1.1 TCP ports 80,443 -> for testing purposes

For installing kubernetes

  • download.opensuse.org TCP port 80,443

  • packages.cloud.google.com TCP port 443

  • github.com TCP port 80,443

  • raw.githubusercontent.com TCP port 80,443

  • metallb.universe.tf TCP port 80,443

  • helm.traefik.io TCP port 443

Inter-clustrer traffic

  • kubernetes api TCP 6443

  • storage traffic (depends on the storage solution used)