Securing Pod-to-Pod Encryption with mTLS Using Istio and PeerAuthentication

Problem Statement

Implement mutual TLS (mTLS) encryption for pod-to-pod communication within a specific namespace using Istio and the PeerAuthentication resource.

Solution

Mutual TLS (mTLS) provides strong security for service-to-service communication by ensuring both the client and server verify each other’s identity. Istio’s PeerAuthentication resource makes it easy to enable and enforce mTLS across a namespace. This solution demonstrates how to implement mTLS for all services within a specific namespace.

Prerequisites

1. A Kubernetes cluster with Istio installed
2. kubectl configured to interact with your cluster
3. Administrative access to the cluster
4. Istio’s control plane (istiod) running

Step 1: Verify Istio Installation

First, let’s verify that Istio is properly installed in your cluster:

# Check if Istio components are installed and running
kubectl get pods -n istio-system

# Sample output:
# NAME                                    READY   STATUS    RESTARTS   AGE
# istio-ingressgateway-6b9c847cf-m4xb7    1/1     Running   0          24h
# istiod-5d49996999-8rvdl                 1/1     Running   0          24h

Step 2: Create or Identify the Target Namespace

For this example, let’s create a namespace called secure-apps where we’ll enforce mTLS:

# Create the namespace
kubectl create namespace secure-apps

# Label the namespace for Istio injection
kubectl label namespace secure-apps istio-injection=enabled

The istio-injection=enabled label ensures that the Istio sidecar proxy is automatically injected into all pods created in this namespace, which is required for mTLS to function.

Step 3: Apply PeerAuthentication Policy for mTLS

Now, let’s create a PeerAuthentication resource that enforces STRICT mTLS for all services in the secure-apps namespace:

# Save this as mtls-policy.yaml
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: secure-apps
spec:
  mtls:
    mode: STRICT

Apply this configuration:

kubectl apply -f mtls-policy.yaml

Let’s understand the important parts of this configuration:

• kind: PeerAuthentication: Specifies that this is an Istio PeerAuthentication resource
• metadata.namespace: secure-apps: The policy applies to the secure-apps namespace
• mtls.mode: STRICT: Enforces strict mTLS, meaning:
• All traffic to services in this namespace must use mTLS
• Plaintext connections will be rejected
• Communications are encrypted and authenticated

Step 4: Verify mTLS Enforcement

Let’s verify that mTLS is properly configured and enforced:

# Check the PeerAuthentication policy status
kubectl get peerauthentication -n secure-apps

# Sample output:
# NAME      MODE     AGE
# default   STRICT   30s

For a more detailed verification, deploy test applications in the secure-apps namespace and verify mTLS is working:

# Deploy a simple application in the secure-apps namespace
kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
  namespace: secure-apps
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      containers:
      - name: sleep
        image: curlimages/curl
        command: ["/bin/sleep", "3650d"]
        imagePullPolicy: IfNotPresent
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  namespace: secure-apps
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin
  namespace: secure-apps
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
  template:
    metadata:
      labels:
        app: httpbin
    spec:
      containers:
      - image: kennethreitz/httpbin
        imagePullPolicy: IfNotPresent
        name: httpbin
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin
  namespace: secure-apps
spec:
  ports:
  - name: http
    port: 8000
    targetPort: 80
  selector:
    app: httpbin
EOF

Now you can verify mTLS is working by checking the connection from sleep to httpbin:

# Get the sleep pod name
SLEEP_POD=$(kubectl get pod -l app=sleep -n secure-apps -o jsonpath={.items..metadata.name})

# Check the TLS stats to verify mTLS is active
kubectl exec -n secure-apps $SLEEP_POD -c istio-proxy -- curl -s http://localhost:15000/stats | grep tls_inspector

# Try a connection and check if it's using mTLS
kubectl exec -n secure-apps $SLEEP_POD -c istio-proxy -- curl http://httpbin.secure-apps:8000/headers -s | grep X-Forwarded-Client-Cert

If the connection is encrypted with mTLS, you’ll see the X-Forwarded-Client-Cert header in the response, which contains the client certificate information.

Step 5: Visualize mTLS with Kiali (Optional)

If you have Kiali installed as part of your Istio setup, you can visualize the mTLS status:

# If Kiali is not exposed, you can port-forward to access it
kubectl port-forward svc/kiali 20001:20001 -n istio-system

Then open your browser to http://localhost:20001, log in, and navigate to the Graph view. You should see secure (padlock) icons on the connections between services in the secure-apps namespace.

Understanding mTLS Modes

Istio provides different mTLS modes in PeerAuthentication:

1. STRICT: All connections must use mTLS. Non-mTLS connections are rejected.
2. PERMISSIVE: Both mTLS and plaintext connections are allowed (useful during migration).
3. DISABLE: mTLS is disabled, only plaintext connections are allowed.

For our example, we’ve used STRICT to enforce full encryption.

Fine-tuning mTLS (Optional)

If you need more granular control, you can apply mTLS settings at the service level:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: service-specific
  namespace: secure-apps
spec:
  selector:
    matchLabels:
      app: httpbin  # This policy applies only to the httpbin service
  mtls:
    mode: STRICT

Conclusion

We’ve successfully configured pod-to-pod encryption using mTLS in the secure-apps namespace with Istio’s PeerAuthentication resource. The configuration ensures that:

1. All communication between services is encrypted and authenticated
2. Only services with valid certificates can communicate with each other
3. The entire namespace is protected with a single policy

This approach provides strong security for microservices communication without requiring changes to the application code, as Istio’s service mesh handles the TLS negotiation, certificate management, and encryption transparently.

Additional Security Considerations

• Rotation: Istio automatically rotates certificates (default validity is 24 hours)
• Monitoring: Implement monitoring for failed authentication attempts using Istio telemetry
• Gradual Rollout: For production environments, consider starting with PERMISSIVE mode, then monitoring, before switching to STRICT mode
• API Gateway: Consider configuring an AuthorizationPolicy alongside PeerAuthentication for additional security

By implementing mTLS across your namespace, you’ve significantly improved your security posture by ensuring all service-to-service communication is encrypted and authenticated.