[TEST LAB] kubeadm 으로 PC 에서 실습환경 구성하기

0. Master / Worker Node 아키텍처 구성
 - 2 Master, 1 Worker Node 로 Test Lab 구성
 - 각각의 Worker Node 는 2 Core, 4GB Mem 으로 구성 (VirtualBox)
 - OS 는 CentOS 7, Ubuntu 16.04, Ubuntu 18.04, CentOS 8 버전으로 각각 테스트 Lab 실행 예정

 

1. OS 기본 세팅 - OS 초기화 스크립트 확인 필요(만들기)
 - CentOS 초기 설치 후 필요한 작업
ㄴ Network interface ONBOOT=no -> ONBOOT=yes 수정 후 서버 reboot
ㄴ network 툴인 net-tools 설치 "yum install -y net-tools"
ㄴ 도메인 질의를 위한 bind-utils 설치 "yum install -y bind-utils"

 

 

2. Docker 설치 (docs.docker.com/engine/install/centos/)

 

 - centos extra yum repo 활설화 여부 체크 (check shell 구성)
 - overlay2 storage driver 는 추천사항 (check shell 구성)

 - Old 버전의 Docker 및 Docker-engine 제거
sudo yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine

 

 - yum-utils 설치
sudo yum install -y yum-utils

 -  yum-utils 를 통해 docker repo 추가
sudo yum-config-manager \

    --add-repo \

    https://download.docker.com/linux/centos/docker-ce.repo 

 

 - Docker 시작
sudo systemctl start docker

 

 - Docker 잘 동작하는지 테스트
sudo docker run hello-world

 - Docker Service 등록
sudo systemctl enable docker

 

PS. Docker 삭제시

 - Docker 삭제
sudo yum remove docker-ce docker-ce-cli containerd.io

sudo rm -rf /var/lib/docker

 

3. Kubeadm 설치 (kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/)

 

 - One or more machines running one of:
Ubuntu 16.04+
Debian 9+
CentOS 7
Red Hat Enterprise Linux (RHEL) 7
Fedora 25+
HypriotOS v1.0.1+
Flatcar Container Linux (tested with 2512.3.0)

 - 2 GB or more of RAM per machine (any less will leave little room for your apps)
 - 2 CPUs or more
 - Full network connectivity between all machines in the cluster (public or private network is fine)
 - Unique hostname, MAC address, and product_uuid for every node. See here for more details.
 - Certain ports are open on your machines. See here for more details.
 - Swap disabled. You MUST disable swap in order for the kubelet to work properly.

 

 - Hostname 수정

sudo hostnamectl set-hostname master-node01

sudo vi /etc/hosts
192.168.1.11 master-node01
192.168.1.16 master-node02
192.168.1.27 node1 worker-node01

 

 - Control Plane 에서 필요한 방화벽 정책
TCP Inbound 
6443* - Kubernetes API Server, ALL
2379~2380 - etcd server client API / kube-apiserver, etcd
10250 - kubelet API / Self, Control plane
10251 - kube-scheduler / Self
10252 - kube-controller-manager / Self

### Master Node
sudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --add-port=2379-2380/tcp
sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=10251/tcp
sudo firewall-cmd --permanent --add-port=10252/tcp
sudo firewall-cmd --permanent --add-port=10255/tcp
sudo firewall-cmd --reload

 

 - Worker Node 에서 필요한 방화벽 정책
TCP Inbound
10250 - Kubelet API / Self, Control Plane
30000-32767 - NodePort Servicest / All

### Worker Node
sudo firewall-cmd --permanent --add-port=10251/tcp
sudo firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --reload

 

 - Update Iptables Settings

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

EOF
sudo sysctl --system

 

 - Disable SELinux
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

 - Disable SWAP
sudo sed -i '/swap/d' /etc/fstab
sudo swapoff -a

 

 - kubeadm, kubectl, kubelet 설치

### kubernetes yum repo 등록
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo

[kubernetes]
name=Kubernetes

baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch

enabled=1
gpgcheck=1
repo_gpgcheck=1

gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg 

exclude=kubelet kubeadm kubectl

EOF

 

### 설치
sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

 

### systemctl 에 등록
sudo systemctl enable --now kubelet

 

 

4. Kubeadm init
### kubeadm 으로 init 할시 사용할 cidr 값은 임의로 지정가능 (k8s Cluster 가 사용할 Cidr 값 지정)
sudo kubeadm init --pod-network-cidr=10.0.0.0/16

 

 - kubenetes config 
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

 

 - Pod Network 구성 (flannel)
sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

sudo firewall-cmd --permanent --add-port=8285/tcp

 

Flannel 을 사용하려면 8285 트레픽에 대한 방화벽 허용규칙이 있어야 한다. 

 

# kubectl get pod -A
NAMESPACE     NAME                                            READY   STATUS    RESTARTS   AGE
kube-system   coredns-f9fd979d6-dqfvf                         1/1     Running   0          54m
kube-system   coredns-f9fd979d6-flwqc                         1/1     Running   0          54m
kube-system   etcd-localhost.localdomain                      1/1     Running   0          54m
kube-system   kube-apiserver-localhost.localdomain            1/1     Running   0          54m
kube-system   kube-controller-manager-localhost.localdomain   1/1     Running   0          54m
kube-system   kube-flannel-ds-lqstk                           1/1     Running   0          47m
kube-system   kube-proxy-4kg4r                                1/1     Running   0          54m
kube-system   kube-scheduler-localhost.localdomain            1/1     Running   0          54m

 

참고 reference

phoenixnap.com/kb/how-to-install-kubernetes-on-centos

How to Install Kubernetes on CentOS 7 (Step by Step)

kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

Installing kubeadm

linuxer.name/2020/10/k8s-centos7-install/