Kube API Server

🧠 1. Deep Dive into the `command` section of `kube-apiserver.yaml`¶

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.102.134:6443
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=192.168.102.134
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction
    - --enable-bootstrap-token-auth=true
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
    - --requestheader-allowed-names=front-proxy-client
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --requestheader-group-headers=X-Remote-Group
    - --requestheader-username-headers=X-Remote-User
    - --secure-port=6443
    - --service-account-issuer=https://kubernetes.default.svc.cluster.local
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --service-account-signing-key-file=/etc/kubernetes/pki/sa.key
    - --service-cluster-ip-range=172.20.0.0/16
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    image: registry.k8s.io/kube-apiserver:v1.33.0
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: 192.168.102.134
        path: /livez
        port: 6443
        scheme: HTTPS
      initialDelaySeconds: 10
      periodSeconds: 10
      timeoutSeconds: 15
    name: kube-apiserver
    readinessProbe:
      failureThreshold: 3
      httpGet:
        host: 192.168.102.134
        path: /readyz
        port: 6443
        scheme: HTTPS
      periodSeconds: 1
      timeoutSeconds: 15
    resources:
      requests:
        cpu: 250m
    startupProbe:
      failureThreshold: 24
      httpGet:
        host: 192.168.102.134
        path: /livez
        port: 6443
        scheme: HTTPS
      initialDelaySeconds: 10
      periodSeconds: 10
      timeoutSeconds: 15
    volumeMounts:
    - mountPath: /etc/ssl/certs
      name: ca-certs
      readOnly: true
    - mountPath: /etc/ca-certificates
      name: etc-ca-certificates
      readOnly: true
    - mountPath: /etc/kubernetes/pki
      name: k8s-certs
      readOnly: true
    - mountPath: /usr/local/share/ca-certificates
      name: usr-local-share-ca-certificates
      readOnly: true
    - mountPath: /usr/share/ca-certificates
      name: usr-share-ca-certificates
      readOnly: true
  hostNetwork: true
  priority: 2000001000
  priorityClassName: system-node-critical
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  volumes:
  - hostPath:
      path: /etc/ssl/certs
      type: DirectoryOrCreate
    name: ca-certs
  - hostPath:
      path: /etc/ca-certificates
      type: DirectoryOrCreate
    name: etc-ca-certificates
  - hostPath:
      path: /etc/kubernetes/pki
      type: DirectoryOrCreate
    name: k8s-certs
  - hostPath:
      path: /usr/local/share/ca-certificates
      type: DirectoryOrCreate
    name: usr-local-share-ca-certificates
  - hostPath:
      path: /usr/share/ca-certificates
      type: DirectoryOrCreate
    name: usr-share-ca-certificates
status: {}

This tells the container what arguments the API server binary should run with.

🔸 Basic Info¶

- kube-apiserver

This simply starts the Kubernetes API server binary. Everything else are flags modifying its behavior.

🧩 Pairwise Breakdown of Each Flag with Explanation¶

🔹 `--advertise-address=192.168.102.134`¶

📌 Purpose: IP for other control-plane components to reach this API server.

✅ Must be reachable from other nodes.

🔀 In Multi-Control Plane:

Each control-plane node will have its own static pod manifest with its own IP here.
A load balancer front-end is used to unify access.

🔹 `--allow-privileged=true`¶

📌 Enables privileged pods, like DaemonSets requiring host access.

🧠 Generally always true for clusters to work with CNI, etc.

🔹 `--authorization-mode=Node,RBAC`¶

📌 This is how access control is enforced:

Node: Allows kubelet to do node-specific things.
RBAC: Role-Based Access Control for fine-grained security.

🔹 `--client-ca-file=/etc/kubernetes/pki/ca.crt`¶

📌 Used to authenticate incoming client certs (e.g., kubectl client).

This CA must have signed all trusted client certs (like kubelet or users).

🔹 `--enable-admission-plugins=NodeRestriction`¶

📌 Enables admission controllers, which intercept API requests before they persist in etcd.

NodeRestriction blocks a kubelet from modifying other nodes/pods.

🔹 `--enable-bootstrap-token-auth=true`¶

📌 Lets the API server authenticate kubelets using a bootstrap token (during kubeadm join).

These configure how the API server talks to etcd, its backend database.

- --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
- --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
- --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
- --etcd-servers=https://127.0.0.1:2379

✅ Authenticated, encrypted communication with etcd.

🔀 In Multi-Control Plane:

--etcd-servers will have multiple entries, like:

--etcd-servers=https://10.0.0.1:2379,https://10.0.0.2:2379,https://10.0.0.3:2379

* These will point to a clustered etcd deployment.

🧩 Kubelet Communication¶

- --kubelet-client-certificate
- --kubelet-client-key
- --kubelet-preferred-address-types

📌 The API server talks to kubelets on nodes to validate pod status, exec, logs, etc.

It needs a client cert/key to authenticate itself to kubelet.
Address preference defines how it connects to nodes.

🔀 No difference in multi-control-plane setup.

🧩 Aggregation Layer (Extension APIs like Metrics Server)¶

- --proxy-client-cert-file
- --proxy-client-key-file
- --requestheader-allowed-names
- --requestheader-client-ca-file
- --requestheader-extra-headers-prefix
- --requestheader-group-headers
- --requestheader-username-headers

📌 This is for API aggregation – allowing 3^rd-party APIs (like metrics.k8s.io) to integrate.

🔀 No difference in multi-node.

🔹 `--secure-port=6443`¶

📌 This is the main API server port.

🔀 In multi-control-plane, all nodes expose 6443 and are load-balanced.

🧩 Service Account Signing¶

- --service-account-issuer=https://kubernetes.default.svc.cluster.local
- --service-account-key-file
- --service-account-signing-key-file

📌 Used for token-based authentication in pods (e.g., communicating with API server).

🔀 All nodes share the same SA key pair (sa.key / sa.pub) to validate JWTs.

🔹 `--service-cluster-ip-range=172.20.0.0/16`¶

📌 IP range used for ClusterIP services (virtual service IPs).

❗ Must be identical on all control-plane nodes!

🧩 TLS for Serving¶

- --tls-cert-file
- --tls-private-key-file

📌 TLS cert/key for encrypting traffic to the API server.

🔀 Each control-plane node can have unique certs, signed by a cluster CA, for their own hostname/IP.

🧠 2. What `kubeadm` command generated this file?¶

The manifest is auto-generated by kubeadm during kubeadm init.

Based on the IP and configs, the command was likely:

kubeadm init \
  --apiserver-advertise-address=192.168.102.134 \
  --pod-network-cidr=192.168.0.0/16 \
  --service-cidr=172.20.0.0/16

🧪 Optional additional flags:

--control-plane-endpoint=<LB IP>:6443 → If multi-control-plane planned.
--upload-certs → To share certs across nodes for HA setup.

controlplane ~ ➜  cat /opt/kubeadm-config.yaml 
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: "192.168.182.22"
  bindPort: 6443
nodeRegistration:
  ignorePreflightErrors:
    - SystemVerification
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: "v1.34.0"
controlPlaneEndpoint: "controlplane"
networking:
  podSubnet: "172.17.0.0/16"
  serviceSubnet: "172.20.0.0/16"
apiServer:
  certSANs:
    - "controlplane"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: cgroupfs

🧠 3. What would change in Multi-Control-Plane Setup?¶

Here’s what would differ:

Component	Single-Node	Multi-Control Plane
`--advertise-address`	Node IP	Each node uses its own IP
`--etcd-servers`	127.0.0.1:2379	List of all etcd peers
Certificates	Local	Shared via `kubeadm init --upload-certs`
Load Balancer	Not needed	Needed for unified API access
Static Pod Files	On one node	On all control-plane nodes

📌 Bonus: How Static Pod is Generated¶

Kubeadm uses the kubelet’s static pod path:

/etc/kubernetes/manifests/

Kubelet auto-runs anything placed here — no scheduler involved.

💡 These YAML files are built by kubeadm and stored statically, not managed by Kubernetes.

✅ Summary Cheatsheet¶

🧠 --etcd-* = talks to etcd
🔐 --tls-*, --service-account-*, --client-ca-* = authentication
🔀 Multi-control-plane needs load balancer, shared certs, etcd cluster
🛠️ Generated during kubeadm init

📘 Grouped Breakdown of kube-apiserver Flags¶

Group	Flag(s)	Purpose	Multi-Control Plane Notes
🧭 Basic API Server Setup	`kube-apiserver`	Starts the API server binary.	Same across all nodes.
	`--advertise-address=192.168.102.134`	IP to advertise for this API server.	Each control-plane uses its own IP here. Load balancer required in front.
	`--secure-port=6443`	Port where the API server listens securely.	Same on all nodes. Load balancer forwards traffic here.
	`--allow-privileged=true`	Allows privileged pods (e.g. CNI plugins).	Always true. No change.

Group	Authentication & Authorization	Purpose	Multi-Control Plane Notes
🔐 Client Auth	`--client-ca-file=/etc/kubernetes/pki/ca.crt`	Validates client certs (e.g. `kubectl`, kubelet).	Shared CA across all nodes.
🛡 Authorization	`--authorization-mode=Node,RBAC`	Enables Node authz + RBAC.	Consistent on all nodes.
🪪 Bootstrap Token	`--enable-bootstrap-token-auth=true`	Allows token-based kubelet join.	Needed on all nodes accepting joins.

Group	etcd Configuration	Purpose	Multi-Control Plane Notes
📦 etcd	`--etcd-servers=https://127.0.0.1:2379`	Address of etcd server(s).	List of all etcd peers: `--etcd-servers=https://10.0.0.1:2379,https://10.0.0.2:2379,...`
	`--etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt` `--etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt` `--etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key`	TLS-based secure etcd access.	Shared CA and certs. Generated by kubeadm or manually managed.

Group	Kubelet Communication	Purpose	Multi-Control Plane Notes
🧬	`--kubelet-client-certificate` `--kubelet-client-key`	Used by API server to auth itself to kubelets.	Same certs across control-plane nodes.
🌐	`--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname`	Chooses how to connect to nodes.	No change. Controls preference order.

Group	API Aggregation Layer (for extension APIs)	Purpose	Multi-Control Plane Notes
🧩	`--proxy-client-cert-file` `--proxy-client-key-file` `--requestheader-allowed-names=front-proxy-client` `--requestheader-client-ca-file` `--requestheader-extra-headers-prefix=X-Remote-Extra-` `--requestheader-group-headers=X-Remote-Group` `--requestheader-username-headers=X-Remote-User`	Allows trusted external services (like metrics-server) to proxy requests through API server.	Same setup on all nodes.

Group	Service Account Tokens (JWT-based auth)	Purpose	Multi-Control Plane Notes
🔑	`--service-account-issuer=https://kubernetes.default.svc.cluster.local` `--service-account-key-file=/etc/kubernetes/pki/sa.pub` `--service-account-signing-key-file=/etc/kubernetes/pki/sa.key`	Signs tokens used by pods to authenticate with API server.	Must be the same across all control-plane nodes for token verification. Use `--upload-certs` with `kubeadm`.

Group	Networking Config	Purpose	Multi-Control Plane Notes
🕸️	`--service-cluster-ip-range=172.20.0.0/16`	IP range for ClusterIP services.	Must be same across all nodes.

Group	TLS for HTTPS Server	Purpose	Multi-Control Plane Notes
🔐	`--tls-cert-file=/etc/kubernetes/pki/apiserver.crt` `--tls-private-key-file=/etc/kubernetes/pki/apiserver.key`	API server HTTPS encryption.	Can be different for each node (cert per node signed by CA). Or use wildcard cert.

✅ Recap of What's Different in Multi-Control Plane¶

Component	Single Node	Multi-Control Plane
`--advertise-address`	Local IP	Unique per node
`--etcd-servers`	`127.0.0.1`	List of all peer IPs
Certificates	Local	Shared via `--upload-certs`
Service Cluster IP	Same	Same
Load Balancer	❌ Not needed	✅ Mandatory
Static pod YAML	On one node	On every control-plane node

Kube API Server

🧠 1. Deep Dive into the command section of kube-apiserver.yaml¶

🔸 Basic Info¶

🧩 Pairwise Breakdown of Each Flag with Explanation¶

🔹 --advertise-address=192.168.102.134¶

🔹 --allow-privileged=true¶

🔹 --authorization-mode=Node,RBAC¶

🔹 --client-ca-file=/etc/kubernetes/pki/ca.crt¶

🔹 --enable-admission-plugins=NodeRestriction¶

🔹 --enable-bootstrap-token-auth=true¶

🧩 ETCD-RELATED FLAGS¶