GKE Storage

GKE supports multiple storage options for persistent data: Persistent Disk for block storage, Filestore for shared file storage, and Cloud Storage buckets for object storage. Understanding when to use each storage type and how to configure them is essential for running stateful applications on GKE.

Storage Overview

Kubernetes abstracts storage through PersistentVolumes (PVs) and PersistentVolumeClaims (PVCs). On GKE, these map to Google Cloud storage services:

Storage Classes:

• Define storage types and parameters
• Enable dynamic provisioning
• Specify volume binding modes
• Configure default storage class

PersistentVolumeClaims:

• Request storage from a storage class
• Specify size and access modes
• Automatically provision volumes

PersistentVolumes:

• Actual storage resources
• Created by storage provisioner
• Bound to PVCs

Persistent Disk (PD) CSI Driver

The Persistent Disk Container Storage Interface (CSI) driver provides block storage using Google Cloud Persistent Disk. Persistent Disks are network-attached block storage devices that persist independently of Compute Engine VM lifecycles.

Architecture

Storage Classes

Define storage classes for different Persistent Disk types:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: pd-standard
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-standard
  replication-type: regional-pd
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
reclaimPolicy: Delete

Persistent Disk Types:

Storage Class Parameters:

• type - Persistent Disk type (pd-standard, pd-balanced, pd-ssd, pd-extreme)
• replication-type - Regional or zonal (regional-pd or none)
• fsType - Filesystem type (ext4, xfs)
• volumeBindingMode - WaitForFirstConsumer or Immediate

Creating Persistent Volumes

Dynamic Provisioning (Recommended):

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: pd-ssd
  resources:
    requests:
      storage: 100Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:8.0
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: secretpassword
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: mysql-pvc

Access Modes:

• ReadWriteOnce - Single node read/write (PD default)
• ReadOnlyMany - Multiple nodes read-only (not supported by PD)
• ReadWriteMany - Multiple nodes read/write (not supported by PD, use Filestore)

Regional Persistent Disks

Use regional Persistent Disks for high availability:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: pd-ssd-regional
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-ssd
  replication-type: regional-pd
volumeBindingMode: WaitForFirstConsumer

Benefits:

• Synchronous replication across two zones
• Automatic failover
• Better availability for stateful workloads

Volume Expansion

Persistent Disks can be expanded:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: pd-ssd-expandable
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-ssd
allowVolumeExpansion: true  # Enable expansion

Expand a PVC:

# Edit PVC to increase size
kubectl patch pvc mysql-pvc -p '{"spec":{"resources":{"requests":{"storage":"200Gi"}}}}'

# Verify expansion
kubectl get pvc mysql-pvc

Volume Snapshots

Create snapshots for backups:

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: pd-snapshot-class
driver: pd.csi.storage.gke.io
deletionPolicy: Retain
---
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: mysql-snapshot
spec:
  volumeSnapshotClassName: pd-snapshot-class
  source:
    persistentVolumeClaimName: mysql-pvc

Restore from snapshot:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc-restored
spec:
  dataSource:
    name: mysql-snapshot
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
  accessModes:
    - ReadWriteOnce
  storageClassName: pd-ssd
  resources:
    requests:
      storage: 100Gi

Filestore

Google Cloud Filestore provides shared file storage that can be accessed by multiple pods simultaneously. Filestore is network-attached storage that supports NFS protocol.

Architecture

Creating Filestore Instance

# Create Filestore instance
gcloud filestore instances create my-filestore \
  --zone us-central1-a \
  --tier STANDARD \
  --file-share name=vol1,capacity=1TB \
  --network name=my-vpc

Using Filestore

apiVersion: v1
kind: PersistentVolume
metadata:
  name: filestore-pv
spec:
  capacity:
    storage: 1Ti
  accessModes:
    - ReadWriteMany
  nfs:
    server: 10.0.0.2
    path: /vol1
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared-storage
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Ti
  volumeName: filestore-pv
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: app
        image: nginx:latest
        volumeMounts:
        - name: shared
          mountPath: /shared
      volumes:
      - name: shared
        persistentVolumeClaim:
          claimName: shared-storage

Filestore Use Cases:

• Shared configuration files
• Content management systems
• Shared application data
• Log aggregation
• Multi-pod read/write access

Filestore Tiers:

• STANDARD - General purpose, up to 63.9 TB
• PREMIUM - High performance, up to 63.9 TB
• ENTERPRISE - Enterprise grade, up to 100 TB

Cloud Storage FUSE

Mount Cloud Storage buckets as file systems:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-with-gcs
spec:
  template:
    spec:
      containers:
      - name: app
        image: gcr.io/cloud-builders/gsutil
        volumeMounts:
        - name: gcs-bucket
          mountPath: /mnt/gcs
      volumes:
      - name: gcs-bucket
        csi:
          driver: gcsfuse.csi.storage.gke.io
          volumeAttributes:
            bucketName: my-bucket
            mountOptions: "implicit-dirs"

Cloud Storage Use Cases:

• Object storage access
• Large file storage
• Data lake integration
• Backup storage

Multi-Zone Storage Considerations

Zonal Persistent Disks

Zonal Persistent Disks are tied to a specific zone:

Considerations:

• Persistent Disks are tied to a specific zone
• Pods must be scheduled in the same zone as the disk
• Use WaitForFirstConsumer binding mode
• Replicate data across zones for high availability

Regional Persistent Disks

Regional Persistent Disks replicate across two zones:

Benefits:

• Synchronous replication across two zones
• Automatic failover
• Better availability for stateful workloads
• No data loss during zone failures

Filestore Multi-Zone

Filestore provides multi-zone availability:

• Accessible from all zones in the region
• Automatic replication
• High availability
• Perfect for shared storage

Best Practices

1. Use pd-ssd for Most Workloads - Best price/performance for general use

2. Use Regional Persistent Disks - For high availability requirements

3. Use WaitForFirstConsumer - For proper zone placement

4. Plan for Multi-Zone - Use regional disks or Filestore for critical data

5. Enable Volume Expansion - Start small, expand as needed

6. Use Snapshots - Regular snapshots for backups

7. Right-Size Volumes - Match storage size to requirements

8. Use Filestore for Shared Data - When multiple pods need access

9. Monitor Storage Usage - Set up Cloud Monitoring alerts

10. Tag Resources - Tag volumes for cost allocation

Common Issues

Volume Not Attaching

Problem: Pod stuck in Pending, volume not attaching

Solutions:

• Check PD CSI driver status
• Verify service account permissions
• Check firewall rules
• Verify disk is in same zone as node
• Review Cloud Logging for errors

Filestore Mount Timeout

Problem: Filestore mount fails or times out

Solutions:

• Verify Filestore instance is running
• Check firewall rules (port 2049)
• Verify network connectivity
• Check Filestore instance network configuration
• Review Cloud Logging for errors

Volume Expansion Fails

Problem: Volume expansion doesn’t work

Solutions:

• Verify storage class allows expansion
• Check Persistent Disk type supports expansion
• Ensure pod is running (expansion requires mounted volume)
• Check PD CSI driver version
• Review Cloud Logging for errors

See Also

• Cluster Setup - Initial cluster configuration
• Add-ons - Storage driver configuration
• Troubleshooting - Storage issues