Submariner 0.1: Multi-Cluster Networking for Kubernetes

Introduction

On December 10, 2019, the Submariner project released version 0.1, the first release of a multi-cluster networking solution for Kubernetes. Submariner enables secure connectivity between Kubernetes clusters across different environments—whether they’re in different clouds, regions, or on-premises data centers.

Multi-cluster architectures were becoming common, but connecting clusters was still a challenge. Teams resorted to VPNs, complex routing rules, or simply accepting that clusters couldn’t communicate. Submariner 0.1 provided a Kubernetes-native way to connect clusters, making multi-cluster networking as simple as deploying a few components.

Why Submariner?

• Kubernetes-Native: Uses Kubernetes APIs and resources, not external VPN appliances.
• Cross-Cloud: Works across different cloud providers and on-premises environments.
• Secure by Default: Built-in encryption and security features.
• Service Discovery: Enables service discovery across cluster boundaries.

Core Architecture

• Submariner Gateway: Runs on each cluster, establishing VPN tunnels to other clusters.
• Service Discovery: Enables DNS-based service discovery across clusters.
• Network Policies: Supports network policies across cluster boundaries.
• Broker: Central component that coordinates cluster registration and connectivity.

Key Features (0.1)

1. VPN-Based Connectivity: Establishes secure VPN tunnels between clusters.
2. Service Discovery: DNS-based service discovery across clusters.
3. Cross-Cluster Services: Access services in other clusters using standard Kubernetes Service DNS.
4. Network Policies: Basic support for network policies across clusters.

Getting Started

Install Submariner broker:

subctl deploy-broker

Join clusters to the broker:

subctl join broker-info.subm --clusterid cluster1 --clustercidr 10.0.0.0/16
subctl join broker-info.subm --clusterid cluster2 --clustercidr 10.1.0.0/16

Verify connectivity:

subctl show networks

VPN Connectivity

Submariner 0.1 uses VPN tunnels to connect clusters:

• IPsec Tunnels: Secure IPsec tunnels between cluster gateways.
• Automatic Routing: Automatic route configuration for cross-cluster traffic.
• Failover: Automatic failover when gateway nodes fail.
• Encryption: All cross-cluster traffic is encrypted by default.

Service Discovery

Enable service discovery across clusters:

apiVersion: v1
kind: Service
metadata:
  name: my-service
  namespace: default
  annotations:
    submariner.io/globalService: "true"
spec:
  selector:
    app: my-app
  ports:
  - port: 80

Services annotated with submariner.io/globalService: "true" are accessible from other clusters.

Use Cases

• Multi-Cloud: Connect clusters across different cloud providers.
• Hybrid Cloud: Connect on-premises clusters with cloud clusters.
• Disaster Recovery: Connect primary and secondary clusters for failover.
• Geographic Distribution: Connect clusters in different regions.

Comparison with Alternatives

Operational Considerations

• Network Requirements: Requires network connectivity between clusters (internet or VPN).
• Gateway Nodes: Gateway nodes need public IPs or VPN connectivity.
• CIDR Planning: Ensure cluster CIDRs don’t overlap.
• Performance: VPN tunnels add latency; consider geographic proximity.

Common Patterns

• Active-Passive: Connect primary and secondary clusters for disaster recovery.
• Active-Active: Distribute workloads across multiple clusters.
• Hybrid Cloud: Connect on-premises and cloud clusters.
• Multi-Region: Connect clusters in different geographic regions.

Limitations (0.1)

• Early Software: 0.1 is an initial release; expect API changes and feature additions.
• VPN Overhead: VPN tunnels add latency compared to direct connectivity.
• Gateway Requirements: Requires gateway nodes with network connectivity.
• CIDR Constraints: Cluster CIDRs must not overlap.

Looking Ahead

Submariner 0.1 established the foundation for:

• Enhanced Connectivity: Improved VPN and connectivity options.
• Service Discovery: Better service discovery capabilities.
• Network Policies: Enhanced network policy support across clusters.
• Performance Improvements: Optimizations for cross-cluster communication.

Summary

Submariner 0.1 demonstrated that multi-cluster networking could be Kubernetes-native and straightforward. By providing VPN-based connectivity with built-in encryption and service discovery, it made connecting clusters across different environments practical without requiring complex external infrastructure.