Skip to content

GKE Cluster Types

Description

GKE offers two distinct cluster modes of operation: Standard and Autopilot. Each provides different levels of control, management responsibility, and pricing models. Understanding the differences is crucial for choosing the right cluster type for your workload.

Standard GKE Clusters

Description

Standard GKE clusters give you complete control over cluster configuration and node management. You’re responsible for configuring node pools, managing scaling, security, and updates while Google manages the control plane.

Model: You manage nodes, Google manages control plane.

Key Features

Full Node Control

  • Custom Machine Types: Choose any Compute Engine machine type
  • Node Customization: Configure boot disk, local SSDs, GPUs, taints, labels
  • SSH Access: Direct SSH access to nodes for debugging
  • Custom Images: Use custom node images if needed
  • DaemonSets: Run privileged pods on all nodes

Flexible Node Pools

  • Multiple Node Pools: Create pools with different machine types
  • Node Taints and Labels: Control pod scheduling
  • Spot VMs: Use preemptible/spot VMs for cost savings
  • Node Pool Management: Manual or automated scaling per pool

Networking Options

  • Routes-Based: Traditional Kubernetes networking
  • VPC-Native: Alias IP ranges (recommended)
  • Network Policies: Calico or GKE Dataplane V2
  • Private Clusters: No public IPs on nodes

Advanced Features

  • Windows Node Pools: Run Windows containers
  • Multi-Cluster Ingress: Share load balancers across clusters
  • Config Connector: Manage GCP resources as Kubernetes objects
  • Istio/ASM: Full service mesh capabilities

Important Limits

Limit Value Notes
Nodes per cluster 15,000 Across all node pools
Node pools per cluster 1,000 Different configurations
Pods per node 110 (default), 256 (max) Via –max-pods-per-node
Pods per cluster 200,000 Theoretical maximum
Services per cluster 10,000 LoadBalancer type limited

When to Use Standard GKE

✅ Use Standard When:

  1. Custom Infrastructure Requirements

  2. Need specific machine types or custom configurations

  3. Require GPU or TPU nodes
  4. Need local SSDs for high-performance storage

  5. Custom kernel modules or system-level modifications

  6. Full Control Over Nodes

  7. Need SSH access to nodes for debugging

  8. Want to run privileged pods or DaemonSets
  9. Require custom node images

  10. Need to configure node-level security

  11. Windows Workloads

  12. Running Windows containers

  13. Mixed Linux/Windows workloads

  14. .NET Framework applications

  15. Cost Optimization with Spot VMs

  16. Fault-tolerant batch workloads

  17. CI/CD pipelines
  18. Development/testing environments

  19. Up to 91% cost savings acceptable with interruptions

  20. Complex Networking Requirements

  21. Multiple network interfaces

  22. Custom CNI plugins
  23. Advanced network policies
  24. Specific IP address management

❌ Don’t Use Standard When:

  1. Want Minimal Management

  2. Team lacks Kubernetes operations expertise

  3. Prefer hands-off infrastructure management

  4. Don’t want to manage node scaling/updates

  5. Unpredictable Workloads

  6. Highly variable traffic patterns

  7. Sporadic batch jobs

  8. Cost efficiency more important than control

  9. Simplicity is Priority

  10. Small team without dedicated platform engineers

  11. Rapid prototyping and development
  12. Quick time-to-market needed

Pricing

Standard GKE Costs:

  • Control Plane:
  • Zonal clusters: Free

  • Regional clusters: $0.10/hour

  • Nodes: Standard Compute Engine pricing

  • e2-medium: ~$0.03/hour

  • Spot VMs: ~$0.008/hour (up to 91% discount)

  • Networking: Egress charges apply

Cost Optimization:

# Use Spot VMs (preemptible) for cost savings
gcloud container node-pools create spot-pool \
  --cluster=my-cluster \
  --machine-type=e2-medium \
  --spot \
  --num-nodes=3

# Enable cluster autoscaling
gcloud container clusters update my-cluster \
  --enable-autoscaling \
  --min-nodes=1 \
  --max-nodes=10

Autopilot GKE Clusters

Description

Autopilot is a fully managed GKE mode where Google manages the entire cluster infrastructure including nodes, node pools, scaling, security, and networking. You only configure and deploy your pods.

Model: Google manages everything, you manage workloads.

Key Features

Fully Managed Infrastructure

  • No Node Management: Google provisions and scales nodes automatically
  • Automatic Scaling: Nodes scale based on pod resource requests
  • Hardened Security: Security best practices enforced by default
  • Automatic Updates: Both control plane and nodes updated automatically
  • Optimized Configuration: Google-optimized settings for performance and cost

Simplified Operations

  • No Node Pools: Infrastructure abstracted away
  • Per-Pod Billing: Pay only for CPU and memory requested by pods
  • Resource-Based Scaling: Nodes added/removed based on pod requests
  • Hands-Off Upgrades: No maintenance windows or disruption management

Built-In Security

  • Workload Identity: Enabled by default
  • Shielded Nodes: All nodes use shielded GKE nodes
  • Secure by Default: Security best practices enforced
  • No SSH Access: Nodes are not directly accessible (improved security)

Restrictions (for Security and Optimization)

  • No Privileged Pods: Cannot run privileged containers
  • No Host Network: Pods can’t use host networking
  • No DaemonSets: With node selectors (some exceptions apply)
  • Predefined Pod Resources: Must specify CPU/memory requests
  • No Windows Nodes: Linux only

Important Limits

Limit Value Notes
Pods per cluster Auto-managed Scales based on demand
Pod CPU request 0.25 to 32 vCPU In 0.25 vCPU increments
Pod memory request 0.5 to 128 GB Specific ratios to CPU
Ephemeral storage Up to 10 GB included Per pod
Persistent volumes Unlimited (quota-based) Standard limits apply
Services (LoadBalancer) Auto-managed Google handles capacity

Pod Resource Classes

Autopilot uses predefined CPU-to-memory ratios:

Class CPU:Memory Ratio Example
General Purpose 1:4 GB 1 vCPU : 4 GB RAM
Scale-Out 1:1 GB 1 vCPU : 1 GB RAM
Balanced 1:2 GB 1 vCPU : 2 GB RAM
Memory-Optimized 1:6.5 GB 1 vCPU : 6.5 GB RAM

When to Use Autopilot GKE

✅ Use Autopilot When:

  1. Minimal Operational Overhead

  2. Small team or no dedicated platform engineers

  3. Want Google to handle all infrastructure decisions
  4. Prefer hands-off cluster management

  5. Focus on application deployment, not infrastructure

  6. Unpredictable or Variable Workloads

  7. Traffic patterns vary significantly

  8. Batch jobs with sporadic execution
  9. Development and testing environments

  10. Cost efficiency through automatic scaling

  11. Security is Critical

  12. Want hardened defaults

  13. Need compliance with security baselines
  14. Prefer least-privilege by default

  15. Don’t need privileged containers

  16. Optimal Cost Management

  17. Pay only for what you use (per-pod resources)

  18. No over-provisioning nodes
  19. Automatic right-sizing

  20. Scale-to-zero capability

  21. Standard Kubernetes Workloads

  22. Stateless applications

  23. Microservices
  24. HTTP services and APIs
  25. Standard containerized applications

❌ Don’t Use Autopilot When:

  1. Need Privileged Access

  2. Running privileged containers

  3. DaemonSets with node selectors
  4. Host networking required

  5. SSH access to nodes needed

  6. Custom Node Configuration

  7. Specific machine types required

  8. GPUs or TPUs needed
  9. Local SSDs for storage

  10. Custom node images

  11. Windows Workloads

  12. Running Windows containers

  13. .NET Framework applications

  14. Windows-specific requirements

  15. Special Network Requirements

  16. Custom CNI plugins

  17. Multiple network interfaces

  18. Non-standard networking configurations

  19. Very Stable, Predictable Workloads

  20. 24/7 steady-state traffic

  21. Reserved capacity might be cheaper
  22. Committed use discounts apply (Standard with CUDs may be cheaper)

Pricing

Autopilot GKE Costs:

  • vCPU: $0.0445/hour per vCPU requested
  • Memory: $0.00488/hour per GB requested
  • Control Plane: Included in pod pricing
  • Networking: Egress charges apply

Example Cost Calculation:

Pod Request: 1 vCPU, 4 GB RAM
Hourly Cost: (1 × $0.0445) + (4 × $0.00488) = $0.064/hour
Monthly Cost: $0.064 × 730 hours = ~$46.72/month per pod

Cost Optimization:

# Right-size pod resources - you pay for requests
apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:

  - name: app
    image: my-app:latest
    resources:
      requests:
        cpu: "250m"      # 0.25 vCPU
        memory: "512Mi"  # 0.5 GB
      limits:
        cpu: "1000m"
        memory: "2Gi"

Comparison Matrix

Feature Standard GKE Autopilot GKE
Node Management Manual configuration Fully automated
Pricing Per node-hour Per pod resource request
Scaling Configure autoscaling Automatic
Node Pools Manual creation/config Not applicable
Machine Types Full choice Google-optimized
SSH to Nodes Yes No
Privileged Pods Yes No
DaemonSets Yes (unrestricted) Limited
GPU/TPU Yes Limited GPU support
Windows Nodes Yes No
Local SSDs Yes No
Control Plane Cost $0.10/hr (regional) Included
Security Baseline Manual config Hardened by default
Best For Custom requirements Simplicity, cost efficiency

Choosing Between Standard and Autopilot

Decision Framework

Start Here: Do you need Windows nodes, GPUs, or privileged containers?
    │
    ├─ Yes → Standard GKE
    │
    └─ No
        │
        Do you have dedicated platform/ops team?
        │
        ├─ Yes → Do you need custom node configuration?
        │   │
        │   ├─ Yes → Standard GKE
        │   └─ No → Autopilot GKE (less operational overhead)
        │
        └─ No → Autopilot GKE (hands-off management)

When to Choose Standard

  • Full control over infrastructure
  • Custom hardware requirements (GPU, TPU, local SSD)
  • Windows workloads
  • Privileged containers or DaemonSets
  • Spot VMs for cost optimization
  • Experienced Kubernetes operations team

When to Choose Autopilot

  • Minimal operational overhead
  • Pay-per-pod cost model
  • Unpredictable or variable workloads
  • Security hardening by default
  • Small team or no dedicated ops
  • Standard containerized applications

Migration Considerations

Standard to Autopilot

Potential Issues:

  • Privileged pods will be rejected
  • DaemonSets with node selectors may not work
  • Need to define resource requests/limits
  • Custom node configurations lost

Migration Path:

  1. Audit existing workloads for incompatibilities
  2. Add resource requests/limits to all pods
  3. Remove privileged security contexts
  4. Test in new Autopilot cluster
  5. Migrate workloads gradually

Autopilot to Standard

Reasons to Switch:

  • Need GPU/TPU support
  • Require Windows nodes
  • Want Spot VM cost savings
  • Need privileged containers

Migration Path:

  1. Create Standard cluster with similar configuration
  2. Configure node pools and autoscaling
  3. Migrate workloads
  4. Optimize node pool configuration

Regional vs Zonal Clusters

Both Standard and Autopilot support regional and zonal deployments:

Zonal Clusters

Characteristics:

  • Control plane in single zone
  • Nodes in single zone (Standard) or multi-zone (Standard with manual pools)
  • Lower cost (free control plane for Standard)
  • Lower SLA (no SLA for zonal)

Use Cases:

  • Development and testing
  • Non-critical workloads
  • Cost-sensitive applications

Regional Clusters

Characteristics:

  • Control plane replicated across 3 zones
  • Nodes distributed across zones
  • Higher availability (99.95% SLA)
  • Higher cost ($0.10/hour for Standard, included for Autopilot)

Use Cases:

  • Production workloads
  • High-availability requirements
  • Mission-critical applications

Best Practices

For Standard Clusters

  1. Use Multiple Node Pools

  2. Separate pools for different workload types

  3. Production vs. batch workloads

  4. Different machine types for different needs

  5. Enable Autoscaling

  6. Configure cluster autoscaler

  7. Set appropriate min/max nodes

  8. Use node affinity for workload placement

  9. Use Spot VMs Wisely

  10. Only for fault-tolerant workloads

  11. Not for critical services

  12. Implement pod disruption budgets

  13. Right-Size Nodes

  14. Don’t over-provision nodes

  15. Use smaller nodes for better bin packing
  16. Monitor utilization and adjust

For Autopilot Clusters

  1. Define Resource Requests

  2. Required for all pods

  3. Directly impacts cost

  4. Use VPA for recommendations

  5. Optimize Pod Resources

  6. Right-size requests to actual usage

  7. Avoid over-requesting resources

  8. Use Vertical Pod Autoscaler

  9. Understand Restrictions

  10. No privileged pods

  11. Limited DaemonSet capabilities

  12. Plan accordingly

  13. Leverage Auto-Scaling

  14. HPA for pod replicas

  15. Let Autopilot handle nodes
  16. No need to configure cluster autoscaler