Deploying the MetalSoft Global Controller¶
The MetalSoft Global Controller is a Kubernetes application and as such it runs as a collection of containers, services, configurations etc. Any flavor of Kubernetes is supported: Vanilla Kubernetes, RedHat OpenShift, AWS EKS etc.
Kubernetes cluster compute requirements¶
A typical cluster should have the following resources available on at least 3 nodes:
16 CPU cores
32GiB RAM
100GiB disk space
In addition, support for persistent volumes with a minimum of 240GB of disk space is required either via a CSI or with the following pre-configured PVs:
controller-pvc 5Gb RWX
mysql-pvc 100Gb RWX
redis-pvc 10Gb RWX
repo-pvc 10Gb RWX
image builder ISO storage 150GB RWM (read-write-many) both image builder and ms-tunnel server use this volume
other microservices 10Gb RWX
Important
For best performance we recommend that the storage system used by the Kubernetes cluster to be either external to the cluster (another cluster, appliance, single server, etc.) or inside the cluster but isolated on nodes specifically dedicated to storage. This will prevent from high I/O on the workload nodes affecting other MetalSoft components or Kubernetes system workloads.
The storage must also allow for ReadWriteMany access mode and have CSI driver/provisioner available for it (ex. NFS with NFS Subdir External Provisioner). This will allow the image builder volume to be mounted by multiple image-builder pods.
Other requirements¶
An SSL certificate is required for the domain that will be mapped to the MetalSoft application in DNS.
A load balancer mechanism such as MetalLB. (1 or 2 IPAddresses)
Setup with 1 IP Address will use ports HTTP 80, 443 and TCP port 9091
Setup with 2 IP Addresses, will use HTTP 80, 443 on one IP and port 443 on the other
Installing the MetalSoft controller kubernetes application¶
Install helm
helm repo add traefik https://helm.traefik.io/traefik
helm install traefik-{{ env }} traefik/traefik --values traefik-helm-chart-configvalues.yaml -n {{ namespace }}
Run kubectl apply on all the manifest files on the provided directory
kubectl apply -f *
Check if the controller components are all running:
administrator@dtsy1lvpmsc001:~$ kubectl get pods -n metalsoft
NAME READY STATUS RESTARTS AGE
auth-microservice-5d848c9789-z48cv 1/1 Running 40 (26d ago) 44d
config-microservice-76565b74d8-flgfp 1/1 Running 10 (26d ago) 66d
controller-64c79f9dcc-4d5zj 1/1 Running 0 37h
couchdb-8475576f55-mgc59 1/1 Running 3 66d
dell-redfish-6857f69cbd-8kvsf 1/1 Running 0 32h
event-microservice-794d797847-wnjt4 1/1 Running 20 (26d ago) 66d
gateway-api-84b5f69d4d-rphm7 1/1 Running 0 26d
image-builder-59fd97b95f-t5pl9 1/1 Running 0 22h
kafka-74fb7b999c-4gkpv 1/1 Running 2 42d
metal-cloud-ui-7db8657b75-4r4js 1/1 Running 0 37h
mysql-9594dfb87-sb4wc 1/1 Running 0 26d
pdns-8f747c64c-fszgn 1/1 Running 0 26d
redis-84864d55d7-9cz9b 1/1 Running 3 66d
repo-678df55b7b-6sf9l 1/1 Running 3 66d
servers-5f45ff6955-6qqbg 1/1 Running 0 31h
traefik-metalsoft-prod-6f5bcb7c65-nnbwr 1/1 Running 0 37h
websocket-tunnel-server-57887d758d-n77r2 1/1 Running 0 32h
zookeeper-d587fc894-6nmc6 1/1 Running 3 66d
Required controller firewall configuration¶
Ports open for inbound from agents¶
The following ports open on the controller, on the MetalLB IP (the ip that moves between hosts), need to be accessible by agents:
web: TCP port 80
websecure: TCP port 443
ms-tunnel-9091: TCP port 9091
powerdns: UDP port 53
Ports open for inbound from clients¶
The following ports, on the MetalLB IP (the ip that moves between hosts), need to be accessed by end-clients (or admins).
web: TCP port 80
websecure: TCP port 443
dns: TCP/UDP port 53
Outbound traffic¶
The controller generates traffic towards the following destinations:
For Firmware Upgrades:
downloads.dell.com TCP port 443
downloads.linux.hpe.com TCP port 80
For pulling ISO files, can be hosted on customers own http storage:
repo.metalsoft.io TCP ports 80,443
For installing/upgrading Kubernetes:
apt.kubernetes.io TCP ports 80,443
k8s.io TCP port 443
registry.k8s.io TCP port 80,443
git.k8s.io TCP port 443
k8s.gcr.io TCP port 80,443
gcr.io TCP port 80,443
cloud.google.com TCP port 80,443
For pulling MetalSoft images at installation/upgrade time:
registry.metalsoft.dev TCP port 443 or registry-qts.metalsoft.dev TCP port 443 => Based on country Global Controller will be installed in
For pulling standard images at installation/upgrade time:
quay.io TCP ports 80,443 => MetalLB and Ceph images
cdn.quay.io TCP ports 80,443 => MetalLB and Ceph images
cdn01.quay.io TCP ports 80,443 => MetalLB and Ceph images
cdn02.quay.io TCP ports 80,443 => MetalLB and Ceph images
cdn03.quay.io TCP ports 80,443 => MetalLB and Ceph images
helm.traefik.io TCP port 443 => Helm chart for Traefik
docker.io TCP ports 80,443 => traefik, busybox and Rancher images
hub.docker.com TCP ports 80,443 => traefik, busybox and Rancher images
registry.hub.docker.com TCP ports 80,443 => traefik, busybox and Rancher images
registry-1.docker.io TCP ports 80,443 => traefik, busybox and Rancher images
Mail Server requirements If using Office365 for email alerts:
smtp.office365.com TCP port 587
For base OS package updates:
archive.ubuntu.com TCP port 80
security.ubuntu.com TCP port 80
For testing connectivity:
1.1.1.1 ICMP
1.1.1.1 TCP ports 80,443
For installing kubernetes¶
download.opensuse.org TCP port 80,443
packages.cloud.google.com TCP port 443
github.com TCP port 80,443
raw.githubusercontent.com TCP port 80,443
metallb.universe.tf TCP port 80,443
helm.traefik.io TCP port 443
Inter-clustrer traffic¶
kubernetes api TCP 6443
storage traffic (depends on the storage solution used)