[[TOC(Documentation/f*, depth=3)]]

== Tutorial: Kubernetes Multi Node Setup==
Requirements:

1. [https://www.orbit-lab.org/wiki/Documentation/CGettingStarted Setup/Practice] of Orbit Nodes.

2. Any available Orbit Sandbox Node

3. Ubuntu 20.04.5 LTS 

== Kubernetes Multi Node lab setup using the kubeadm utility ==
Let’s take a look at this Kubernetes Multi-Node lab setup step-by-step and see how it is put together.

In this tutorial, we will be using one of the sb9 nodes for setting up Kubernetes. Any sb(1-10) should also be capable of Kubernetes setup. 

Make sure you have Ubuntu 20.04.5 (**baseline20.04.ndz**) or 22.04.2 (**ubuntu2204-beta.ndz**) LTS. To get the Ubuntu Version, follow these steps after ssh into your console. Substitute for the image and node you want.

The below steps illustrate loading Ubuntu 20.04.5 LTS for node1-5 of sb1. For the sb1 console,

{{{#!shell 
username@console:~$ omf tell -t node1-5 -a offh 
}}}

{{{#!shell
username@console:~$ omf load -t node1-5 -i baseline20.04.ndz
}}}

{{{#!shell
username@console:~$ omf tell -t node1-5 -a on
}}}

To verify if you have loaded your Ubuntu version properly, use this command after ssh into the node1-5

{{{#!shell
root@node1-5:~# lsb_release -a
}}} 

This will list the current version of Ubuntu loaded.

=== Set up a Kubernetes Cluster ===
After loading your version of Ubuntu, you can follow the below steps to install Kubernetes

1. Update and Upgrade Packages:
{{{#!shell 
root@node1-5:~# sudo apt-get update && sudo apt-get upgrade
}}}

2. Install Docker:
{{{#!shell
root@node1-5:~# sudo apt install docker.io
}}}

3. Configure Docker to use systemd for the management of the container’s cgroups:
{{{#!shell
root@node1-5:~# cat <<EOF | sudo tee /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
}}}

4. Enable Docker:
{{{#!shell
root@node1-5:~# sudo systemctl enable docker
}}}

5. Reload the systemd manager configuration:
{{{#!shell
root@node1-5:~# sudo systemctl daemon-reload
}}}

6. Restart Docker:
{{{#!shell
root@node1-5:~# sudo systemctl restart docker
}}}

7. Update and Upgrade Packages again:
{{{#!shell 
root@node1-5:~# apt update && apt upgrade -y
}}}

8. Add Kubernetes to the repository list:
{{{#!shell
root@node1-5:~# curl -sSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/kubernetes-archive-keyring.gpg >/dev/null
}}}
{{{#!shell
root@node1-5:~# echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
}}}

9. Install Kubernetes:
{{{#!shell
root@node1-5:~# sudo apt-get update
}}}
{{{#!shell
root@node1-5:~# sudo apt-get install -y kubelet kubeadm kubectl
}}}
{{{#!shell
root@node1-5:~# sudo apt-mark hold kubelet kubeadm kubectl
}}}

10. Initialize the Kubernetes Cluster:
This command outputs kubeadm join command; make sure you copy this as this will be helpful for joining worker nodes to the master node
{{{#!shell
root@node1-5:~# kubeadm init --pod-network-cidr=10.19.0.0/16
}}}

You should see an output which includes a `kubeadm join` command, save this command for use on the worker nodes.

11. Set KUBECONFIG environment variable:
{{{#!shell
root@node1-5:~# export KUBECONFIG=/etc/kubernetes/admin.conf
}}}

12. Apply Flannel network overlay:
{{{#!shell
root@node1-5:~# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
}}}

=== Joining Worker Nodes to the Cluster ===
Make sure you follow till step 9[including] on the worker nodes, and then
To join worker nodes to the Kubernetes cluster, run the `kubeadm join` command on the worker nodes that was outputted by the above `kubeadm init` command.


Example of kubeadm join command generated by kubeadm init, lets say we want to add node1-6 as the worker node:
{{{#!shell
root@node1-6:~# kubeadm join 10.19.1.5:6443 --token o1cttx.z9al8w8ljqcmqb4y --discovery-token-ca-cert-hash sha256:6ecd74d7eca0299b80499cf2e2e1c87c4079c3d234282be5822761880998853e
}}}

To get the current status of nodes
{{{#!shell
root@node1-5:~# kubectl get nodes
}}}