Deploying the MetalSoft controller

The MetalSoft controller runs as a collection of docker containers running in a Kubernetes cluster.

Using an existing Kubernetes cluster

Kubernetes cluster should have:

  • minimum three data plane nodes with the following resources available on each:
    • 16 CPU cores
    • 32GiB RAM
    • 100GiB disk space
  • support for persistent volumes with a minimum of 240GB of disk space
  • SSL certificate for the domain

Contact Metalsoft for a zip file with the Kubernetes manifest files.

Using a new Kubernetes cluster

MetalSoft can provide an Ansible playbook along with the needed Ansible roles for setting up Kubernetes if none is available.

Compute requirements

Control plane:

  • minimum 1 node, recommended 3 nodes, with the following resources on each:
    • 4 CPU cores
    • 8GiB RAM
    • 50GiB disk space

Data plane:

  • 3x nodes with the following resources on each:
    • 16 CPU cores
    • 32GiB RAM
    • 100GiB disk space

Storage requirements

  • controller-pvc 5Gb RWX
  • mysql-pvc 100Gb RWX
  • redis-pvc 10Gb RWX
  • repo-pvc 10Gb RWX
  • image builder ISO storage 50gb RWM (read-write-many) both image builder and websocket tunnel server use this volume
  • other microservices 10Gb RWX

Installing the MetalSoft controller kubernetes application

  1. Install helm
helm repo add traefik https://helm.traefik.io/traefik
helm install traefik-{{ env }} traefik/traefik --values traefik-helm-chart-configvalues.yaml -n {{ namespace }}
  1. Run kubectl apply on all the manifest files on the provided directory
kubectl apply -f *
  1. Check if the controller components are all running:
administrator@dtsy1lvpmsc001:~$ kubectl get pods -n metalsoft
NAME                                       READY   STATUS    RESTARTS       AGE
auth-microservice-5d848c9789-z48cv         1/1     Running   40 (26d ago)   44d
config-microservice-76565b74d8-flgfp       1/1     Running   10 (26d ago)   66d
controller-64c79f9dcc-4d5zj                1/1     Running   0              37h
couchdb-8475576f55-mgc59                   1/1     Running   3              66d
dell-redfish-6857f69cbd-8kvsf              1/1     Running   0              32h
event-microservice-794d797847-wnjt4        1/1     Running   20 (26d ago)   66d
gateway-api-84b5f69d4d-rphm7               1/1     Running   0              26d
image-builder-59fd97b95f-t5pl9             1/1     Running   0              22h
kafka-74fb7b999c-4gkpv                     1/1     Running   2              42d
metal-cloud-ui-7db8657b75-4r4js            1/1     Running   0              37h
mysql-9594dfb87-sb4wc                      1/1     Running   0              26d
pdns-8f747c64c-fszgn                       1/1     Running   0              26d
redis-84864d55d7-9cz9b                     1/1     Running   3              66d
repo-678df55b7b-6sf9l                      1/1     Running   3              66d
servers-5f45ff6955-6qqbg                   1/1     Running   0              31h
traefik-metalsoft-prod-6f5bcb7c65-nnbwr    1/1     Running   0              37h
websocket-tunnel-server-57887d758d-n77r2   1/1     Running   0              32h
zookeeper-d587fc894-6nmc6                  1/1     Running   3              66d

Required controller firewall configuration

Ports open for inbound from agents

The following ports open on the controller, on the MetalLB IP (the ip that moves between hosts), need to be accessible by agents:

  • web: TCP port 80
  • websecure: TCP port 443
  • eventservice: TCP port 9003
  • gateway-api: TCP port 9009
  • ws-tunnel-9090: TCP port 9090
  • ws-tunnel-9091: TCP port 9091
  • ws-tunnel-9011: TCP port 9011
  • ws-tunnel-9010: TCP port 9010
  • powerdns: UDP port 53

Ports open for inbound from clients

The following ports, on the MetalLB IP (the ip that moves between hosts), need to be accessed by end-clients (or admins).

  • web: TCP port 80
  • websecure: TCP port 443
  • graphql: TCP port 9009
  • dns: TCP/UDP port 53

Outbound traffic

The controller generates traffic towards the following destinations:

  • 1.1.1.1 ICMP
  • 1.1.1.1 TCP ports 80,443
  • downloads.dell.com TCP port 443
  • downloads.linux.hpe.com TCP port 80
  • repo.metalsoft.io TCP ports 80,443
  • apt.kubernetes.io TCP ports 80,443
  • k8s.io TCP port 443
  • registry.metalsoft.dev TCP port 443
  • quay.io, cdn.quay.io, cdn01.quay.io, cdn02.quay.io, cdn03.quay.io TCP ports 80,443
  • gcr.io TCP port 443
  • k8s.io TCP port 443
  • git.k8s.io TCP port 443
  • k8s.gcr.io and gcr.io TCP port 80,443
  • cloud.google.com TCP port 80,443
  • helm.traefik.io TCP port 443
  • smtp.office365.com TCP port 587 -> this is only if office365 is used for email alerts
  • archive.ubuntu.com, security.ubuntu.com TCP port 80 -> for base OS package updates
  • docker.io, hub.docker.com, registry.hub.docker.com registry-1.docker.io TCP ports 80,443 -> for pulling standard images

Inter-clustrer traffic

  • kubernetes api TCP 6443
  • storage traffic (depends on the storage solution used)