# Setup of MetalSoft Kubernetes Cluster on K3S

## Base setup

To setup MetalSoft Kubernetes Cluster on K3S, we usually use a clean Ubuntu 22.04 installation on which we install k3s:
```
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=servicelb --disable-cloud-controller --write-kubeconfig-mode=600 --disable=traefik" sh -
test -f /etc/rancher/k3s/k3s.yaml && chmod 600 /etc/rancher/k3s/k3s.yaml && export KUBECONFIG=/etc/rancher/k3s/k3s.yaml && echo "export KUBECONFIG=/etc/rancher/k3s/k3s.yaml" |tee -a ${HOME}/.bashrc >> /etc/bash.bashrc
```

### Install Helm:
```
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
helm repo add traefik https://helm.traefik.io/traefik
helm repo update
```

### Install MetalLB:
```
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/main/config/manifests/metallb-native.yaml
```

## MetalSoft Setup

Once k3s is running, navigate to the MetalSoft manifests folder (provided by MetalSoft)
```
cd manifests
```

If storage is set to NFS, you will need to pre-create the folders which NFS will use and access from the Nodes.  
Usually these folders can be seen with:
```
grep path: volumes.yaml |awk '{print $2}'
```

To configure MetalLB with the IP to access the UI:
```
export ip=<YOUR_IPv4>
```

then run:
```
cat <<EOF | kubectl -n metallb-system apply -f -
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: pool
  namespace: metallb-system
spec:
  addresses:
  - ${ip}-${ip}
EOF
```
### Add SSL certificate to Kubernetes cluster
- Generate your SSL certificate and key.  If using a self signed certificate, add the CA certificate to the ConfigMap of the Global Controller.  
To put the CA certificate in the ConfigMap on the Global Controller, edit configmaps.yaml and put the CA certificate in the ca-pemstore ConfigMap resource definition.  For example:
```
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ca-pemstore
  namespace: eveng-qa02-metalcloud
data:
  metalsoft_CA.crt: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
```

- Copy the SSL certificate and key to the Kubernetes cluster and run these

#### To add the certificate to secrets.yaml
``` cat crt.pem <crt_chain.pem> |base64 -w0 ```  
Copy the output into the `tls.crt` line of `secrets.yaml`

#### To add the key to secrets.yaml
``` cat key.pem |base64 -w0 ```  
Copy the output into the `tls.key` line of `secrets.yaml`

### Generate encryption keys in the Kubernetes cluster
:::{important}
Please ensure you backup this encryption_keys value to a safe location outside of the Global Controller setup. If lost, will cause loss of data and access to the platform
:::
```
cd manifests
./scripts/encryption_keys_gen
```
Edit `secrets.yaml` which is in the `manifests` folder and add the above output to the `encryption_keys` line


### Initiate the MetalSoft setup:

When in `manifests` folder, create the Namespace:

```
kubectl apply -f namespace.yaml
```

Export the name of the created Namespace, so it can be used in the next few steps:

```
export ns="YOUR_NAMESPACE_FROM_namespace.yaml"
```

Apply the manifests:
```
find . -maxdepth 1 -type f \( -name "*.yaml" ! -name traefik-helm-chart-configvalues.yaml \) -size +1|while read z;do
echo "applying manifest: $z ..."
kubectl -n $ns apply -f $z >/dev/null
done
```

Install traefik:
```
test -d traefik-helmchart && helm upgrade --install traefik-${ns} ./traefik-helmchart --values traefik-helm-chart-configvalues.yaml -n $ns
```

### Install Debug utilities:
```
touch /usr/local/bin/diagnostics-collector
curl -fsSLk https://raw.githubusercontent.com/metalsoft-io/scripts/main/env-scripts/metalsoft-updates|bash
curl -fsSLo /usr/local/bin/ms-prerequisite-check https://repo.metalsoft.io/extra/ms-prerequisite-check && chmod +x /usr/local/bin/ms-prerequisite-check
```

## Check the state of the setup

When all of the above is complete, you can check the state of the pods, and services:
```
kubectl -n $ns get pods

kubectl -n $ns get svc | grep traefik
# this command should show the IP you've asignedd for the UI, and not show '<pending>'
```

As this point you should be able to access the UI via the hostname you've provided. (which hostname should point to the IP for the UI)