Kubernetes with cri-containerd and kata containers

The days of Docker Engine as a native runtime on Kubernetes are coming to an end. Kubernetes is pushing the CRI plugin system, this plugin is enabled Kubernetes to change the container runtime without a change in kubelet.

Kata containers run with difference mechanism, With Kata Containers, each container starts up in its own virtual machine (hypervisor) and runs one or more containers inside it. You get the benefit of full security isolation and the trade-off on the start time and memory consumption isn’t all that high. Kata containers using qemu-lite which includes several optimizations to reduce start time and memory footprint.

Specification:

  • Ubuntu 16.04
  • cri-containerd 1.2.6
  • latest kata containers
  • Kubernetes v1.14.1 with kubeadm

The first thing is to install kata containers in all node, run this bash script:

bash -c "$(curl -fsSL https://raw.githubusercontent.com/kata-containers/tests/master/cmd/kata-manager/kata-manager.sh) install-packages"

Install cri-containerd

sudo apt-get update
sudo apt-get install libseccomp2

VERSION=1.2.6
wget https://storage.googleapis.com/cri-containerd-release/cri-containerd-${VERSION}.linux-amd64.tar.gz
sudo tar --no-overwrite-dir -C / -xzf cri-containerd-${VERSION}.linux-amd64.tar.gz
sudo systemctl start containerd

command -v containerd

Configure containerd to use kata containers

mkdir /etc/containerd
containerd config default > /etc/containerd/config.toml
nano /etc/containerd/config.toml
[plugins]
  [plugins.cri]
    [plugins.cri.containerd]
      no_pivot = false
    [plugins.cri.containerd.runtimes]
      [plugins.cri.containerd.runtimes.runc]
         runtime_type = "io.containerd.runc.v1"
         [plugins.cri.containerd.runtimes.runc.options]
           NoPivotRoot = false
           NoNewKeyring = false
           ShimCgroup = ""
           IoUid = 0
           IoGid = 0
           BinaryName = "runc"
           Root = ""
           CriuPath = ""
           SystemdCgroup = false
      [plugins.cri.containerd.runtimes.kata]
         runtime_type = "io.containerd.kata.v2"
      [plugins.cri.containerd.runtimes.katacli]
         runtime_type = "io.containerd.runc.v1"
         [plugins.cri.containerd.runtimes.katacli.options]
           NoPivotRoot = false
           NoNewKeyring = false
           ShimCgroup = ""
           IoUid = 0
           IoGid = 0
           BinaryName = "/usr/bin/kata-runtime"
           Root = ""
           CriuPath = ""
           SystemdCgroup = false

    [plugins.cri.cni]
      # conf_dir is the directory in which the admin places a CNI conf.
      conf_dir = "/etc/cni/net.d"

change containerd service

nano /etc/systemd/system/containerd.service
ExecStart=/usr/local/bin/containerd --config /etc/containerd/config.toml
systemctl daemon-reload
systemctl restart containerd

configure Linux network to support containerd

modprobe br_netfilter
cat <<EOF >>  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

nano /etc/sysctl.conf
net.ipv4.ip_forward = 1
sudo sysctl -p

Testing kata containers

ctr image pull docker.io/library/busybox:latest
sudo ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest hello sh

Install kubelet, kubeadm, and kubectl

apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

configure Kubernetes to use containerd

cat << EOF | sudo tee  /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]                                                 
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
EOF
sudo systemctl daemon-reload

make sure containerd running

sudo systemctl restart containerd
sudo systemctl status containerd

Init Kubernetes cluster and join worker node

swapon -s
sudo swapoff -a
sudo kubeadm init --cri-socket /run/containerd/containerd.sock --pod-network-cidr=10.244.0.0/16

Create RuntimeClass for kata containers

apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
  name: kata
handler: kata

Create Pod with spesific RuntimeClass

apiVersion: v1
kind: Pod
metadata:
  name: nginx-untrusted
spec:
  runtimeClassName: kata
  containers:
  - name: nginx
    image: nginx

 

 

 

Comments are closed.