CNI Working

Understanding the CNI Workflow in Kubernetes

The Container Network Interface (CNI) is a standardized specification that enables Kubernetes to configure networking for pods, ensuring they can communicate within and across nodes. The workflow outlines how Kubernetes’ kubelet, container runtime, and CNI plugin collaborate to set up pod networking.

Unified CNI Workflow

The CNI workflow involves Kubernetes components and the CNI plugin working together to assign IPs, configure network interfaces, and enable pod communication. Here’s a detailed, step-by-step explanation:

  1. Kubelet Initiates Pod Creation:
  2. What Happens: When you deploy a pod (e.g., kubectl run nginx --image=nginx), the Kubernetes API server schedules it to a node, and the node’s kubelet (the agent running on each node) is responsible for creating the pod.
  3. Action: Kubelet instructs the container runtime (e.g., containerd, CRI-O) to create the pod’s sandbox, a network namespace that isolates the pod’s networking environment.
  4. Details:
    • The sandbox is a Linux network namespace (netns) that acts as the pod’s isolated networking stack.
    • Kubelet passes pod metadata (e.g., name, namespace, container ID) to the runtime.

  5. In Your Setup: Using containerd on Ubuntu 22.04, kubelet communicates via the CRI socket (/var/run/containerd/containerd.sock), as configured in your kubeadm init --cri-socket.

  6. Container Runtime Requests Network Setup:

  7. What Happens: The container runtime, before starting the pod’s containers, needs to configure the pod’s network (e.g., assign an IP, set up interfaces). It delegates this to the CNI interface.
  8. Action: The runtime invokes the CNI plugin by executing scripts or binaries specified in the CNI configuration files located at /etc/cni/net.d/ (e.g., 10-calico.conflist for Calico).
  9. Details:
    • The runtime passes pod details (e.g., pod name, namespace, container ID, network namespace path) to the CNI plugin via environment variables or JSON.
    • The CNI interface is a standardized API (part of the CNI spec) that ensures compatibility between runtimes and plugins.

  10. In Your Setup: Your kind-config-file.yaml sets podSubnet: "10.244.0.0/16" and disableDefaultCNI: true, meaning containerd will call your manually installed Calico plugin.

  11. CNI Interface Triggers the CNI Plugin:

  12. What Happens: The CNI interface identifies the configured plugin (e.g., Calico, Flannel) from /etc/cni/net.d/ and triggers it to handle networking setup.
  13. Action: The plugin’s binary (e.g., Calico’s calico or calico-ipam) is executed with the pod’s details.
  14. Details:
    • The plugin reads its configuration from /etc/cni/net.d/ (e.g., Calico’s IP pool settings).
    • It communicates with the cluster’s networking infrastructure (e.g., Calico’s Felix agent or etcd for IP allocation).

  15. In Your Setup: Calico is configured with CALICO_IPV4POOL_CIDR: "10.244.0.0/16", ensuring the plugin uses the correct IP range.

  16. CNI Plugin Configures Networking:

  17. What Happens: The CNI plugin performs the necessary networking tasks to connect the pod to the cluster network.
  18. Action:
    • Assigns an IP: Allocates an IP address from the configured range (e.g., 10.244.0.5 from 10.244.0.0/16).
    • Sets Up Interfaces: Creates a virtual ethernet (veth) pair, linking the pod’s network namespace to the host’s network (e.g., a bridge or routing table).
    • Configures Routes: Adds routing rules to enable communication between pods, nodes, and external networks.
    • Applies Firewall Rules: Implements network policies or NAT rules (e.g., Calico’s iptables for policies).
  19. Details:
    • For Calico (Layer 3), IPs are assigned via IPAM (IP Address Management), and routes are managed using BGP or IP-in-IP encapsulation for inter-node communication.
    • Example: A pod gets 10.244.0.5, connected via a veth pair to the host’s cali0 interface, with BGP routes to other nodes.

  20. In Your Setup:

    • Calico assigns IPs from 10.244.0.0/16, matching your podSubnet and --pod-network-cidr.
    • Your past issue (errors with 192.168.0.0/16) occurred because a commented-out CALICO_IPV4POOL_CIDR prevented IP pool creation. Using 10.244.0.0/16 with an uncommented CIDR resolves this.

  21. Pod is Connected to the Cluster Network:

  22. What Happens: Once the CNI plugin completes setup, the pod is fully networked and can communicate with other pods, services, and external resources.
  23. Action:
    • The runtime starts the pod’s containers within the configured network namespace.
    • Kubelet marks the pod as Running (if other conditions, like image pulling, are met).
  24. Details:
    • The pod’s IP (e.g., 10.244.0.5) is visible via kubectl get pod -o wide.
    • Communication relies on the CNI plugin’s routing (e.g., Calico’s BGP for cross-node traffic).
  25. In Your Setup:
    • Pods receive IPs like 10.244.0.5, enabling communication across your Kind cluster (k8s-master-1, k8s-worker-1).
    • Verified via: 
      kubectl get pods -o wide
kubectl get ippool -o yaml
      Expected: spec.cidr: 10.244.0.0/16.

How CNI Works (Summary)

The Container Network Interface (CNI) enables Kubernetes to configure pod networking, assigning IPs and enabling communication. Here’s how it works:

  1. Kubelet Initiates Pod Creation:

  2. The kubelet, Kubernetes’ node agent, schedules a pod (e.g., nginx) and instructs the container runtime (e.g., containerd) to create a pod sandbox—a network namespace isolating the pod’s networking.

  3. Container Runtime Requests Networking:

  4. The runtime calls the CNI interface, passing pod details (e.g., name, namespace, container ID) via /etc/cni/net.d/ configuration files (e.g., 10-calico.conflist).

  5. CNI Interface Triggers the Plugin:

  6. The CNI interface invokes the configured plugin (e.g., Calico, Flannel) to set up networking.

  7. CNI Plugin Configures Networking:

  8. The plugin:

    • Assigns an IP (e.g., 10.244.0.5) from the configured range (e.g., 10.244.0.0/16).
    • Creates a virtual ethernet (veth) pair, linking the pod’s namespace to the host’s network.
    • Configures routes (e.g., via BGP for Calico) for pod-to-pod and node communication.
    • Applies firewall rules or network policies.

  9. Pod Joins the Cluster Network:

  10. The pod receives its IP and is fully connected, enabling communication with other pods, services, and external resources.

Example: Calico in Your Setup

  • Context: You run: 
    curl -s https://raw.githubusercontent.com/ibtisam-iq/SilverKube/main/kind-calico-config-file.yaml | kind create cluster --config -
    Then apply Calico: 
    curl -O https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml
sed -i 's/# - name: CALICO_IPV4POOL_CIDR/- name: CALICO_IPV4POOL_CIDR/' calico.yaml
sed -i 's/#   value: "192.168.0.0\/16"/  value: "10.244.0.0\/16"/' calico.yaml
kubectl apply -f calico.yaml
  • Workflow:
  • Deploy a pod: 
    kubectl run nginx --image=nginx --port=80
  • Step 1: Kubelet on k8s-worker-1 tells containerd to create the nginx pod’s sandbox.
  • Step 2: Containerd calls the CNI interface (/etc/cni/net.d/10-calico.conflist).
  • Step 3: The Calico plugin is triggered, reading CALICO_IPV4POOL_CIDR: "10.244.0.0/16".
  • Step 4: Calico:
    • Assigns 10.244.0.5 to the pod.
    • Creates a veth pair (e.g., cali123 in the pod’s netns, linked to cali0 on the host).
    • Sets BGP routes via Felix to reach other nodes (e.g., k8s-master-1).
  • Step 5: The pod is Running with IP 10.244.0.5, reachable cluster-wide.
  • Verification: 
    kubectl get pod nginx -o wide
    Output: 
    NAME    READY   STATUS    RESTARTS   AGE   IP           NODE
nginx   1/1     Running   0          5m    10.244.0.5   k8s-worker-1