Add-ons via Helm/Kustomize

Add-ons are software components that extend Kubernetes cluster functionality beyond the core platform. Examples include monitoring systems (Prometheus), ingress controllers (NGINX, Traefik), DNS servers (CoreDNS), network policies (Calico), and logging aggregators (Fluentd). While Kubernetes provides the orchestration platform, add-ons provide the operational capabilities needed for production workloads.

Installing and managing add-ons manually with raw YAML files works for simple cases, but becomes challenging as you manage multiple environments, versions, and configurations. Package managers like Helm and configuration management tools like Kustomize help install, configure, and maintain add-ons consistently and efficiently.

Think of add-ons like apps on your phone, and Helm/Kustomize like app stores that help you find, install, and update them. Helm is like a full-featured app store with packages, while Kustomize is like a customization tool that lets you modify existing configurations.

What Are Add-ons?

Add-ons are cluster-level software components that run on top of Kubernetes:

Monitoring & Observability - Prometheus, Grafana, Jaeger
Networking - Ingress controllers, service meshes, network policies
Security - Policy engines (OPA, Kyverno), secret management
Storage - CSI drivers, storage provisioners
CI/CD - ArgoCD, Tekton, Jenkins
Development Tools - Skaffold, Tilt, DevSpace

Add-ons typically consist of multiple Kubernetes resources (Deployments, Services, ConfigMaps, etc.) that work together to provide functionality.

graph TB A[Kubernetes Cluster] --> B[Core Components] A --> C[Add-ons] B --> D[API Server] B --> E[Scheduler] B --> F[Controller Manager] B --> G[etcd] C --> H[Monitoring Prometheus, Grafana] C --> I[Networking Ingress, CNI] C --> J[Security OPA, Kyverno] C --> K[Storage CSI Drivers] C --> L[CI/CD ArgoCD, Tekton] style A fill:#e1f5ff style B fill:#fff4e1 style C fill:#f3e5f5 style H fill:#e8f5e9 style I fill:#e8f5e9 style J fill:#e8f5e9 style K fill:#e8f5e9 style L fill:#e8f5e9

Add-on Management Challenges

Managing add-ons involves several challenges:

Complexity - Add-ons often include many interconnected resources
Configuration - Different environments need different configurations
Versioning - Tracking which versions are installed and compatible
Dependencies - Some add-ons depend on others
Updates - Upgrading add-ons safely without breaking existing deployments
Consistency - Ensuring the same add-ons are installed across environments

Helm: The Package Manager

Helm is Kubernetes’ package manager, similar to apt for Linux or npm for Node.js. It packages Kubernetes applications into charts—collections of templates and default values that can be installed, upgraded, and uninstalled.

How Helm Works

Helm uses charts (packages) that contain:

Templates - Kubernetes resource definitions with variables
Values - Default configuration values
Metadata - Chart name, version, dependencies
Documentation - README and usage instructions

graph LR A[Helm Chart] --> B[Templates] A --> C[Values] A --> D[Metadata] B --> E[Deployment.yaml] B --> F[Service.yaml] B --> G[ConfigMap.yaml] C --> H[values.yaml Default Config] I[helm install] --> A A --> J[Rendered Kubernetes Resources] J --> K[Applied to Cluster] style A fill:#e1f5ff style B fill:#fff4e1 style C fill:#f3e5f5 style J fill:#e8f5e9

Helm Benefits

Reusability - Charts can be used across environments with different values
Versioning - Track and upgrade chart versions
Dependency Management - Charts can depend on other charts
Rollback - Easily rollback to previous versions
Sharing - Share charts through repositories (Helm Hub, Artifact Hub)
Templating - Parameterize configurations with values

Helm Workflow

Find a chart - Search Helm repositories for available charts
Customize values - Override default values for your environment
Install - helm install renders templates and applies resources
Upgrade - helm upgrade updates to new versions
Rollback - helm rollback reverts to previous versions if needed

Kustomize: Configuration Management

Kustomize is a configuration management tool built into kubectl. Instead of templating (like Helm), Kustomize uses a patch-based approach where you define a base configuration and apply overlays to customize it for different environments.

How Kustomize Works

Kustomize builds configurations through:

Base - Base set of Kubernetes resources
Overlays - Environment-specific customizations (dev, staging, prod)
Patches - Modify, add, or remove resources
Resource generation - Generate resources (ConfigMaps, Secrets) from files

graph TB A[Base Configuration] --> B[kustomization.yaml] B --> C[Dev Overlay] B --> D[Staging Overlay] B --> E[Prod Overlay] C --> F[Dev Values] D --> G[Staging Values] E --> H[Prod Values] F --> I[Dev Resources] G --> J[Staging Resources] H --> K[Prod Resources] style A fill:#e1f5ff style B fill:#fff4e1 style C fill:#f3e5f5 style D fill:#f3e5f5 style E fill:#f3e5f5 style I fill:#e8f5e9 style J fill:#e8f5e9 style K fill:#e8f5e9

Kustomize Benefits

Native kubectl - Built into kubectl, no separate tool needed
Git-friendly - Plain YAML files, works well with Git workflows
Declarative - Declarative approach aligns with Kubernetes philosophy
Composable - Combine multiple bases and overlays
No templating - Avoids template complexity, uses patches instead
Validation - Can validate resources before applying

Kustomize Workflow

Define base - Create base Kubernetes resources
Create overlays - Create environment-specific overlays
Build - kubectl kustomize generates final YAML
Apply - kubectl apply -k applies the kustomization

Helm vs. Kustomize

Both tools solve add-on management but with different approaches:

Use Helm when:

Installing third-party charts from repositories
You need versioning and dependency management
Working with packages created by others
You want templating capabilities
Managing complex applications with many dependencies

Use Kustomize when:

Managing your own applications or configurations
You prefer declarative, Git-native workflows
Working with plain YAML without templating
You want simplicity and native kubectl integration
Managing environment-specific configurations

Use both when:

Using Helm charts as bases for Kustomize overlays
Installing charts with Helm, then customizing with Kustomize
Combining package management (Helm) with configuration management (Kustomize)

Add-on Installation Flow

The typical flow for installing add-ons:

sequenceDiagram participant Admin participant Helm/Kustomize participant Cluster Admin->>Helm/Kustomize: Prepare configuration Helm/Kustomize->>Helm/Kustomize: Render resources Helm/Kustomize->>Cluster: Apply resources Cluster->>Cluster: Create resources Cluster->>Admin: Add-on running Note over Admin,Cluster: Repeat for updates and upgrades

Choose tool - Decide on Helm or Kustomize (or both)
Prepare configuration - Get chart or create kustomization
Customize - Override values or create overlays
Review - Preview rendered resources before applying
Install - Apply resources to cluster
Verify - Confirm add-on is running correctly
Document - Document installation and configuration

Best Practices

Use version control - Store Helm values and Kustomize configs in Git
Document configurations - Document why you chose specific values
Test in non-production - Test add-on installations in dev/staging first
Review changes - Review rendered resources before applying
Monitor after installation - Watch add-on health after installation
Plan upgrades - Have a plan for upgrading add-ons
Backup before changes - Backup cluster state before major add-on changes
Use namespaces - Install add-ons in dedicated namespaces when possible
Understand dependencies - Know what other add-ons or components are needed
Keep charts updated - Stay current with chart versions for security patches

Topics

Helm - Detailed guide to using Helm for add-on management
Kustomize - Detailed guide to using Kustomize for configuration management