Go to file
Sambo Chea 5eaefccdb7
Updated drone ci
2022-04-01 11:25:29 +07:00
drone-runner Updated drone ci 2022-04-01 11:25:29 +07:00
k8s Add cert manager 2021-12-29 10:33:22 +07:00
kube-token Updated install for kubectl windows and kube-token 2021-10-11 09:24:47 +07:00
metallb Add bgp routing with metallb 2021-12-22 18:12:46 +07:00
mysql Add mysql cluster operator and rbac in sample 2022-01-14 10:33:35 +07:00
nfs-provisioner Add sc and pv, pvc not clear implements 2021-10-25 10:36:23 +07:00
pv-pvc Add pv and pvc with nfs storage 2021-10-24 20:37:12 +07:00
rbac Add mysql cluster operator and rbac in sample 2022-01-14 10:33:35 +07:00
sample Updated the sample for pod and svc 2021-10-22 10:52:40 +07:00
sc-provider Create local-path.md 2022-03-09 09:07:55 +07:00
skaffold Add skaffold install and guide 2021-10-20 08:29:50 +07:00
traefik Updated traefik script 2021-12-29 14:35:13 +07:00
verdaccio Add sc and pv, pvc not clear implements 2021-10-25 10:36:23 +07:00
.gitignore Add drone runner deployment 2021-10-11 20:12:50 +07:00
DOCKER-PRIVREGISTRY.md Updated the sample for pod and svc 2021-10-22 10:52:40 +07:00
INGRESS-NGINX.md Add install kube cluster script and add ingress nginx guide 2021-10-12 18:06:25 +07:00
KASTENIO.md Updated kasten ref 2021-10-20 20:17:58 +07:00
KIND.md Add kind install guide 2021-10-20 15:29:19 +07:00
KOMPOSE.md Rename KOMPOS.md to KOMPOSE.md 2021-12-09 19:55:05 +07:00
KUBEAPPS.md Add kubeapps install for kubernetes 2021-10-14 10:32:01 +07:00
LB-K8S-HA.md Updated the ha support for kubernetesx 2021-10-13 09:16:31 +07:00
METALLB.md Add metallb installation for kubernetes 2021-10-14 10:37:56 +07:00
NGINX_ING.md Updated nginx 2022-03-02 15:37:52 +07:00
OLM.md Add install guide for operator olm 2021-10-19 08:02:46 +07:00
README.md Update README.md 2022-01-14 10:07:11 +07:00
kube-cluster-worker-install.sh Add lb k8s ha guide and fixed install script 2021-10-12 20:25:26 +07:00
portainer.md Create portainer.md 2022-03-09 09:24:48 +07:00


Installation Guide for Kubernetes Cluster

Quick install for kubernetes cluster for worker node

curl -s -L https://raw.githubusercontent.com/sombochea/kubernetes-installation/main/kube-cluster-worker-install.sh?v=121020215 | bash

1. Download kubectl

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

2. Validate kubectl

curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check

3. Install kubectl

sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
If none-root access (for local user)
chmod +x kubectl
mkdir -p ~/.local/bin/kubectl
mv ./kubectl ~/.local/bin/kubectl
# and then add ~/.local/bin/kubectl to $PATH

4. Verify kubectl installed

kubectl version --client

Install Helm 3

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Setup network

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

sudo sysctl --system

Install kubernetes tools

1. Install CNI plugins (required for most pod network)

sudo mkdir -p /opt/cni/bin
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-${ARCH}-${CNI_VERSION}.tgz" | sudo tar -C /opt/cni/bin -xz
sudo mkdir -p $DOWNLOAD_DIR

2. Install crictl (required for kubeadm / Kubelet Container Runtime Interface (CRI))

curl -L "https://github.com/kubernetes-sigs/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-${ARCH}.tar.gz" | sudo tar -C $DOWNLOAD_DIR -xz

3. Install kubeadm, kubelet and add a kubelet systemd service

RELEASE="$(curl -sSL https://dl.k8s.io/release/stable.txt)"
sudo curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/${ARCH}/{kubeadm,kubelet}
sudo chmod +x {kubeadm,kubelet}

curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service
sudo mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

sudo systemctl enable --now kubelet

4. Verify installation for kubernetes tools

kubeadm version

5. Disable swap and install docker.io

sudo swapoff -a
wget https://sh.osa.cubetiqs.com/docker-setup.sh
bash docker-setup.sh
sudo systemctl start docker
sudo systemctl enable docker

cat <<EOF | sudo tee /etc/docker/daemon.json
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  "storage-driver": "overlay2"

sudo systemctl daemon-reload
sudo systemctl restart docker

Install some required tools

sudo apt-get -y install socat conntrack

6. Configure containerd

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf

sudo modprobe overlay
sudo modprobe br_netfilter

# Setup required sysctl params, these persist across reboots. (If using crio)
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1

# Apply sysctl params without reboot
sudo sysctl --system
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml

sudo systemctl restart containerd

If using crio

  • Update config
sudo nano /etc/containerd/config.toml
  • Change SystemdCgroup from false to true
    SystemdCgroup = true
  • Restart containerd service
sudo systemctl restart containerd

7. Cluster on Master node

sudo kubeadm init --pod-network-cidr
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
OR Join Cluster
sudo kubeadm join ip-api-server:6443 --token $TOKEN --discovery-token-ca-cert-hash $DISCOVERY_HASH

Cluster Netowrk with Flannel

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/k8s-manifests/kube-flannel-rbac.yml

For Kube 1.17+

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Cluster Network with Calico

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

Kubernetes Dashboard

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml
  • Proxy for kubernetes dashboard
kubectl proxy --namespace kubernetes-dashboard service/kubernetes-dashboard
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')

Deploy Storage Class with External NFS server

  • Install nfs client for all nodes
sudo apt install nfs-common -y
  • Install NFS External Provider
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner
helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
    --set nfs.server=nfs-server-ip \
    --set nfs.path=/exported-path

Create Service Account for Kubernetes Dashboard Token

  • Create file: dashboard-adminuser.yml
apiVersion: v1
kind: ServiceAccount
  name: admin-user
  namespace: kube-system
 kubectl apply -f dashboard-adminuser.yml
  • Create file: admin-role-binding.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
  name: admin-user
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system
 kubectl apply -f admin-role-binding.yml
  • Get Token
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')

Get Kubernetes PKI Hash for Kubeadm

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \
   openssl dgst -sha256 -hex | sed 's/^.* //'

Fix Helm Kube Config

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: ~/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: ~/.kube/config
chmod o-r ~/.kube/config
chmod g-r ~/.kube/config

Install kubectl and helm for Windows