Hubble 0.1: eBPF-Based Network Observability for Cilium

Introduction

On February 10, 2020, the Cilium project released Hubble 0.1, the first version of its network observability platform. Built on eBPF, Hubble provides real-time visibility into network flows, service dependencies, and security events within Cilium-powered Kubernetes clusters.

Traditional network monitoring relies on packet capture, flow logs, or sidecar instrumentation—all with performance overhead or blind spots. Hubble takes a different approach: it taps into Cilium’s eBPF data plane to observe network events at the kernel level, providing visibility without adding latency or requiring application changes.

Why Hubble Matters

Zero-Overhead Observability: eBPF-based observation happens in the kernel, not in user space.
Real-Time Visibility: See network flows as they happen, not after the fact.
Service Dependency Mapping: Automatically discover which services communicate with which.
Security Event Detection: Identify policy violations, DNS queries, and connection attempts.

Core Components

Hubble Agent: Runs as a DaemonSet, collecting flow data from Cilium’s eBPF programs.
Hubble Relay: Aggregates flow data from all agents and provides a gRPC API.
Hubble UI: Web interface for visualizing service topology and flow details.
Hubble CLI: Command-line tool for querying flows and service dependencies.

Key Features

Flow Visibility: See every network connection with source, destination, protocol, and bytes transferred.
Service Map: Visualize service dependencies and traffic flows between services.
DNS Observability: Track DNS queries and responses for troubleshooting service discovery.
Policy Verification: See which network policies are applied to specific flows.
Prometheus Integration: Export metrics for integration with existing monitoring stacks.

Getting Started

Install Hubble with Cilium:

helm install cilium cilium/cilium --version 1.8.0 \
  --namespace kube-system \
  --set hubble.enabled=true \
  --set hubble.ui.enabled=true

Access the Hubble UI:

kubectl port-forward -n kube-system svc/hubble-ui 12000:80

Use the CLI to query flows:

hubble observe --follow

Use Cases

Troubleshooting Connectivity: Quickly identify why services can’t communicate.
Service Dependency Discovery: Map out which services depend on which without reading code.
Security Auditing: Verify that network policies are working as expected.
Performance Analysis: Identify slow or high-volume connections between services.

Architecture

eBPF Hooks: Hubble taps into Cilium’s eBPF programs that handle packet processing.
Flow Generation: Network events are captured and converted into flow records.
Aggregation: Hubble Relay collects flows from all nodes and provides a unified view.
Storage: Flows can be stored in external systems (e.g., ClickHouse) for historical analysis.

Comparison with Alternatives

Tool	Approach	Overhead	Real-Time
Hubble	eBPF kernel hooks	Minimal	Yes
tcpdump	Packet capture	High	Yes
Flow Logs	Network device logs	Medium	Delayed
Sidecar Proxies	Application-level	Medium	Yes

Operational Considerations

Resource Usage: Hubble agents add minimal overhead but monitor resource usage.
Flow Volume: High-traffic clusters generate many flows; consider flow filtering or sampling.
Retention: By default, flows are not persisted; integrate with external storage for history.
Security: Flow data may contain sensitive information; restrict access to Hubble UI/API.

Common Patterns

Service Topology: Use hubble service map to visualize service dependencies.
Flow Filtering: Filter flows by namespace, service, or protocol to focus on specific traffic.
Policy Verification: Use hubble observe --verdict to see policy allow/deny decisions.
DNS Troubleshooting: Filter flows by DNS to identify service discovery issues.

Limitations (0.1)

Cilium-Only: Hubble only works with Cilium CNI; not compatible with other CNI plugins.
No Historical Storage: Flows are not persisted by default; need external integration.
Early Software: 0.1 is an initial release; expect API changes and feature additions.
Resource Requirements: High-traffic clusters may need flow sampling to reduce overhead.

Integration with Observability Stack

Prometheus: Export metrics for integration with Prometheus/Grafana.
Grafana: Use Hubble’s Prometheus metrics in Grafana dashboards.
Jaeger: Integrate with distributed tracing for end-to-end request visibility.
ClickHouse: Store flows in ClickHouse for historical analysis and alerting.

Looking Ahead

Hubble 0.1 established the foundation for:

Hubble UI Enhancements: Improved service map visualization and flow filtering.
Historical Storage: Built-in support for flow persistence and querying.
Advanced Filtering: More sophisticated flow query capabilities.
Multi-Cluster Support: Observability across multiple Kubernetes clusters.

Summary

Aspect	Details
Release Date	February 10, 2020
Key Innovations	eBPF-based observability, zero-overhead flow visibility, service dependency mapping
Significance	Demonstrated that network observability could be achieved without performance penalties

Hubble 0.1 proved that network observability didn’t have to come at the cost of performance. By leveraging eBPF to observe network events at the kernel level, it provided real-time visibility into service communication without adding latency or requiring application instrumentation. This approach would become a model for observability tools in the eBPF ecosystem.

Table of Contents