Setup of MetalSoft Kubernetes Cluster on K3S¶

Base setup¶

To setup MetalSoft Kubernetes Cluster on K3S, we usually use a clean Ubuntu 22.04 installation on which we install k3s:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=servicelb --disable-cloud-controller --write-kubeconfig-mode=600 --disable=traefik" sh -
test -f /etc/rancher/k3s/k3s.yaml && chmod 600 /etc/rancher/k3s/k3s.yaml && export KUBECONFIG=/etc/rancher/k3s/k3s.yaml && echo "export KUBECONFIG=/etc/rancher/k3s/k3s.yaml" |tee -a ${HOME}/.bashrc >> /etc/bash.bashrc

Install Helm:¶

curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
helm repo add traefik https://helm.traefik.io/traefik
helm repo update

Install MetalLB:¶

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/main/config/manifests/metallb-native.yaml

MetalSoft Setup¶

Once k3s is running, navigate to the MetalSoft manifests folder (provided by MetalSoft)

cd manifests

If storage is set to NFS, you will need to pre-create the folders which NFS will use and access from the Nodes.
Usually these folders can be seen with:

grep path: volumes.yaml |awk '{print $2}'

To configure MetalLB with the IP to access the UI:

export ip=<YOUR_IPv4>

then run:

cat <<EOF | kubectl -n metallb-system apply -f -
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: pool
  namespace: metallb-system
spec:
  addresses:
  - ${ip}-${ip}
EOF

Add SSL certificate to Kubernetes cluster¶

Generate your SSL certificate and key. If using a self signed certificate, add the CA certificate to the ConfigMap of the Global Controller.
To put the CA certificate in the ConfigMap on the Global Controller, edit configmaps.yaml and put the CA certificate in the ca-pemstore ConfigMap resource definition. For example:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ca-pemstore
  namespace: eveng-qa02-metalcloud
data:
  metalsoft_CA.crt: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----

Copy the SSL certificate and key to the Kubernetes cluster and run these

To add the certificate to secrets.yaml¶

cat crt.pem <crt_chain.pem> |base64 -w0
Copy the output into the tls.crt line of secrets.yaml

To add the key to secrets.yaml¶

cat key.pem |base64 -w0
Copy the output into the tls.key line of secrets.yaml

Generate encryption keys in the Kubernetes cluster¶

Important

Please ensure you backup this encryption_keys value to a safe location outside of the Global Controller setup. If lost, will cause loss of data and access to the platform

cd manifests
./scripts/encryption_keys_gen

Edit secrets.yaml which is in the manifests folder and add the above output to the encryption_keys line

Initiate the MetalSoft setup:¶

When in manifests folder, create the Namespace:

kubectl apply -f namespace.yaml

Export the name of the created Namespace, so it can be used in the next few steps:

export ns="YOUR_NAMESPACE_FROM_namespace.yaml"

Apply the manifests:

find . -maxdepth 1 -type f \( -name "*.yaml" ! -name traefik-helm-chart-configvalues.yaml \) -size +1|while read z;do
echo "applying manifest: $z ..."
kubectl -n $ns apply -f $z >/dev/null
done

Install traefik:

test -d traefik-helmchart && helm upgrade --install traefik-${ns} ./traefik-helmchart --values traefik-helm-chart-configvalues.yaml -n $ns

Install Debug utilities:¶

touch /usr/local/bin/diagnostics-collector
curl -fsSLk https://raw.githubusercontent.com/metalsoft-io/scripts/main/env-scripts/metalsoft-updates|bash
curl -fsSLo /usr/local/bin/ms-prerequisite-check https://repo.metalsoft.io/extra/ms-prerequisite-check && chmod +x /usr/local/bin/ms-prerequisite-check

Check the state of the setup¶

When all of the above is complete, you can check the state of the pods, and services:

kubectl -n $ns get pods

kubectl -n $ns get svc | grep traefik
# this command should show the IP you've asignedd for the UI, and not show '<pending>'

As this point you should be able to access the UI via the hostname you’ve provided. (which hostname should point to the IP for the UI)