Using the untaint controller

Prevent pods from starting before Istio CNI is ready by using Kubernetes node taints managed by the Istio untaint controller.

When Kubernetes starts pods on a node before the istio-cni agent configures node networking, those pods do not have traffic redirection configured. The missing configuration creates a window where traffic is not controlled by Istio and can bypass any configured policy.

To avoid this race condition, use node taints. New pods cannot be scheduled until Istio’s untaint controller removes the taint.

Configure Istio

To enable the untaint controller, set pilot.taint.enabled=true when installing or upgrading the istiod Helm chart.

helm upgrade istiod istio/istiod -n istio-system \
  --set pilot.taint.enabled=true \
  --wait

Some environments require istio-cni in a different namespace than istiod. If needed, specify the namespace to watch with the pilot.taint.namespace setting.

helm upgrade istiod istio/istiod -n istio-system \
  --set pilot.taint.enabled=true \
  --set pilot.taint.namespace=kube-system \
  --wait

Create your nodes

Configure your node deployment, such as a node pool, auto-scaling group, or CI template, to add the cni.istio.io/not-ready startup taint to nodes when they are created. The following example uses a Karpenter NodePool.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  template:
    metadata:
      labels:
        billing-team: my-team
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      startupTaints:
        - key: cni.istio.io/not-ready
          effect: NoSchedule

In Google Kubernetes Engine, you can specify node taints with the --node-taints flag when you create the cluster or node pool.

The cni.istio.io/not-ready taint prevents pods from being scheduled onto the node unless they tolerate it. System add-ons, such as the istio-cni agent, are usually configured to tolerate all taints. When the istio-cni agent starts, the untaint controller removes the taint and pods can be scheduled.

Gate the taint on ztunnel readiness

By default, the untaint controller considers only the istio-cni agent. Because ztunnel starts independently of the agent, a node can lose its taint while ztunnel is still unavailable, and pods scheduled onto that node accumulate without a functioning data plane. To close this gap, extend the readiness check to ztunnel.

Set the PILOT_NODE_UNTAINT_CHECK_ZTUNNEL environment variable on istiod. The untaint controller then keeps the taint until both the istio-cni agent and a ztunnel pod report ready on the node.
```
helm upgrade istiod istio/istiod -n istio-system \
  --set pilot.taint.enabled=true \
  --set pilot.env.PILOT_NODE_UNTAINT_CHECK_ZTUNNEL=true \
  --wait
```
The check applies to ambient mode only. In sidecar mode, no ztunnel exists, and the setting is ignored.
If ztunnel runs outside the istiod namespace, confirm that trustedZtunnelNamespace names that namespace. The controller searches the istiod namespace and the namespace that trustedZtunnelNamespace declares; the platform-openshift profile declares kube-system. When ztunnel runs in a namespace that neither covers, the controller finds no ready ztunnel and the nodes keep the taint indefinitely.
```
helm upgrade istiod istio/istiod -n istio-system \
  --set pilot.taint.enabled=true \
  --set pilot.env.PILOT_NODE_UNTAINT_CHECK_ZTUNNEL=true \
  --set trustedZtunnelNamespace=kube-system \
  --wait
```
The pilot.taint.namespace setting locates the istio-cni agent and does not affect where the controller looks for ztunnel.

Debug the untaint controller

You can use the following steps to verify that the untaint controller is running, or to diagnose issues with node taint removal.

Connect to the debug page on the istiod instance, as the untaint controller runs as part of istiod.
```
kubectl port-forward deployment/istiod -n istio-system 8080:8080
```
Navigate to http://localhost:8080/debug/krtz?pretty. The page shows whether the untaint controller is running and its current state. The node-untaint/nodes field shows the status of nodes, and node-untaint/ready-cni-nodes shows the CNI agents that are ready. When PILOT_NODE_UNTAINT_CHECK_ZTUNNEL is set to true, the node-untaint/ztunnel-pods field shows the ztunnel pods that are ready.

View the untaint controller logs at the default info level.

kubectl logs -f deployment/istiod -n istio-system

Example output:

2025-07-18T01:23:00.952072Z	info	krt	node-untaint/nodes synced	owner=node-untaint/nodes
2025-07-18T01:23:00.952085Z	info	krt	node-untaint/pods synced	owner=node-untaint/pods
2025-07-18T01:23:00.954215Z	info	krt	node-untaint/cni-pods synced	owner=node-untaint/cni-pods
2025-07-18T01:23:00.956745Z	info	controllers	starting	controller=untaint nodes
2025-07-18T01:23:00.956785Z	info	krt	node-untaint/ready-cni-nodes synced	owner=node-untaint/ready-cni-nodes

To see events as nodes are created, set the untaint controller log level to debug.
```
istioctl admin log --level untaint:debug
```

Add a taint to a node to confirm that the untaint controller detects and removes it.

kubectl taint nodes ambient-worker2 cni.istio.io/not-ready:NoSchedule

Example output:

2025-07-18T01:35:11.698525Z	debug	untaint	adding node to queue event: ambient-worker2
2025-07-18T01:35:11.698838Z	debug	untaint	reconciling node ambient-worker2
2025-07-18T01:35:11.698855Z	debug	untaint	removing readiness taint from node ambient-worker2
2025-07-18T01:35:11.705994Z	debug	untaint	removed readiness taint from node ambient-worker2

If you enabled PILOT_NODE_UNTAINT_CHECK_ZTUNNEL and a node keeps its taint, check the same debug output for the ztunnel gate. The controller names each node that it holds.

Example output:
```
2025-07-18T01:35:11.699120Z	debug	untaint	node ambient-worker2 has a ready cni pod but no ready ztunnel, keeping it tainted
```
To confirm which namespaces the controller searches, start istiod with the untaint scope already at debug, because the controller resolves the namespaces once at startup and logs them only then. The istioctl admin log command raises the level on a running instance and therefore comes too late for this line.
```
helm upgrade istiod istio/istiod -n istio-system \
  --set pilot.taint.enabled=true \
  --set pilot.env.PILOT_NODE_UNTAINT_CHECK_ZTUNNEL=true \
  --set global.logging.level="default:info\,untaint:debug" \
  --wait
```
Example output:
```
2025-07-18T01:35:09.112043Z	debug	untaint	node untainter looking for ztunnel in namespaces [istio-system]
```
When the list omits the namespace that ztunnel runs in, set trustedZtunnelNamespace to that namespace.
Restore the log level to the default info setting.
```
istioctl admin log --level untaint:info
```

Uninstall Using Solo.io builds of Istio