Kubernetes 1.28: Planternetes — Stability, Sustainability, and Scalability

Introduction

On August 15, 2023, the Kubernetes project released version 1.28, codenamed “Planternetes.”
This release focused on platform stability, API refinement, and long-term sustainability, with 45 enhancements — 19 graduating to stable (GA), 13 to beta, and 13 newly introduced as alpha.

Official Highlights

1. Node Lifecycle Management Improvements

Kubernetes 1.28 introduced Node expansion features and improved the Node readiness lifecycle, enhancing cluster resilience during node maintenance and making node transitions (e.g., drain, reboot) smoother and more predictable.

Benefits:

• Reduced downtime: Smoother node maintenance with better pod eviction strategies
• Predictable transitions: Improved node readiness state management
• Automated scaling: Better reliability for cluster autoscalers
• Maintenance windows: Enhanced support for planned node maintenance

Node Readiness Improvements:

• Better handling of node conditions during maintenance
• Improved pod scheduling during node transitions
• Enhanced node taint and untaint operations
• Better integration with cluster autoscalers

Example - Node Maintenance:

# Mark node as unschedulable
kubectl cordon node-worker-1

# Drain node with grace period
kubectl drain node-worker-1 --ignore-daemonsets --delete-emptydir-data --grace-period=300

# Perform maintenance...

# Make node schedulable again
kubectl uncordon node-worker-1

Verification:

# Check node readiness
kubectl get nodes -o wide

# Monitor node conditions
kubectl describe node node-worker-1

“Kubernetes 1.28 plants the seeds for long-term reliability and sustainable growth.”
— Kubernetes 1.28 Release Team

2. Storage & CSI Enhancements

VolumeGroupSnapshots (Alpha) - Kubernetes 1.28 introduces group-level snapshotting for stateful workloads, enabling consistent snapshots across multiple volumes simultaneously.

Benefits:

• Data consistency: Snapshot multiple volumes atomically for stateful applications
• Backup efficiency: Create consistent backups of distributed applications
• Disaster recovery: Faster recovery with group-level snapshots
• Simplified management: Single operation for multi-volume snapshots

Example:

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeGroupSnapshot
metadata:
  name: my-app-snapshot
spec:
  volumeGroupSnapshotClassName: default
  source:
    volumeGroupSnapshotContentName: vgsc-12345
---
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeGroupSnapshotClass
metadata:
  name: default
driver: csi-driver.example.com
deletionPolicy: Retain

CSI Migration Completion: All major in-tree storage drivers (AWS EBS, GCE PD, Azure Disk) have completed migration to CSI, providing:

• Better extensibility and maintainability
• Consistent storage interface across cloud providers
• Improved performance and reliability
• Future-proof storage architecture

StorageCapacity API Improvements: The StorageCapacity API enhancements provide better visibility into available storage, enabling:

• Smarter pod scheduling based on storage availability
• Reduced scheduling failures due to storage constraints
• Better multi-zone storage provisioning
• Improved cluster resource utilization

Verification:

# Check CSI drivers
kubectl get csidrivers

# Verify storage capacity
kubectl get csistoragecapacity

# Check volume group snapshots (if supported)
kubectl get volumegroupsnapshots

3. Networking & Pod Lifecycle

• Pod Readiness Gates and Pod Disruption Budget logic improved for large-scale workloads.
• Dual-Stack Networking continued refinement for hybrid IPv4/IPv6 clusters.
• CNI plugin lifecycle and validation became more robust with better error handling.

4. Security & Policy Enhancements

KMS v2 API (GA) - Kubernetes 1.28 graduates the KMS v2 API to General Availability, providing enhanced secret encryption at rest using external key management providers.

Benefits:

• Enhanced security: Improved encryption for secrets at rest
• Provider flexibility: Support for multiple KMS providers (AWS KMS, Azure Key Vault, GCP KMS, HashiCorp Vault)
• Performance: Better performance compared to v1 API
• Rotation support: Enhanced key rotation capabilities
• Audit logging: Better tracking of encryption operations

Configuration Example:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - kms:
      name: kms-provider
      endpoint: unix:///tmp/kms-provider.sock
      cachesize: 100
      timeout: 3s
      apiVersion: v2

Supported Providers:

• AWS Key Management Service (KMS)
• Azure Key Vault
• Google Cloud KMS
• HashiCorp Vault
• Other providers with KMS v2 support

Verification:

# Check encryption configuration
kubectl get encryptionconfig -o yaml

# Verify secrets are encrypted
kubectl get secret my-secret -o yaml | grep encrypted

Additional Improvements:

• PodSecurity Admission received usability and audit mode improvements
• Seccomp and AppArmor defaults better aligned across platforms

5. API Stability & Deprecations

Kubernetes 1.28 continued the cleanup of legacy APIs:

• Deprecated beta APIs for PodSecurityPolicy, ServiceAccountIssuerDiscovery, and NodeAuthorizer;
• Promoted stable APIs for CSIStorageCapacity, CRD Validation, and ServiceAccount Token Projection.

This release marked a steady evolution toward a leaner, more reliable API surface.

Milestones Timeline

Patch Releases for 1.28

Patch releases (1.28.x) focused on networking, CSI, and security fixes.

Legacy and Impact

Kubernetes 1.28 “Planternetes” symbolized a maturing ecosystem — improving sustainability, stability, and scale.
With GA features like KMS v2 and refined node management, this release established Kubernetes as a stable foundation for modern cloud-native operations.

Getting Started

Upgrade Path

Prerequisites:

• Kubernetes 1.27+ cluster
• Backup etcd and cluster state
• Review deprecation notices

Upgrade Steps:

# For kubeadm clusters
kubeadm upgrade plan
kubeadm upgrade apply v1.28.0

# Verify upgrade
kubectl get nodes
kubectl version

# Check deprecated APIs
kubectl get --raw /api/v1 | grep -i deprecated

Feature Gates:

# Enable alpha features (if needed)
--feature-gates=VolumeGroupSnapshots=true

# Check current feature gates
kubectl get --raw /metrics | grep feature_gate

Migration Guide:

• Review 1.28 CHANGELOG
• Test in non-production environment first
• Update custom controllers and operators
• Review deprecated API usage (PodSecurityPolicy, ServiceAccountIssuerDiscovery, NodeAuthorizer)

Compatibility:

• Supported upgrade path: 1.27.x → 1.28.x
• kubectl version: 1.28+ recommended
• Minimum node versions: 1.26+ for 1.28 control plane

Summary

Next in the Series

Next up: Kubernetes 1.29 (December 2023) — bringing enhanced API consistency, security posture improvements, and continued CRI performance upgrades.