Sigstore: Software Supply Chain Security for Kubernetes

Introduction

By mid-2022, Sigstore had reached general availability, providing a comprehensive solution for software supply chain security in Kubernetes. With tools like Cosign for image signing, Fulcio for certificate authority, and Rekor for transparency logs, Sigstore enabled teams to verify the authenticity and integrity of container images before deployment.

This mattered because supply chain attacks had become a major security concern. Compromised container images, malicious dependencies, and tampered artifacts could introduce vulnerabilities into production clusters. Sigstore provided cryptographic verification that images came from trusted sources and hadn’t been tampered with, addressing a critical gap in container security.

Historical note: Sigstore was initiated in 2020 as an open-source project to improve software supply chain security. By 2022, it had reached GA and gained significant adoption in the Kubernetes ecosystem.

Sigstore Components

Cosign: Image Signing and Verification

• Image Signing: Sign container images with cryptographic signatures.
• Signature Verification: Verify image signatures before deployment.
• Key Management: Support for keyless signing and key-based signing.
• Kubernetes Integration: Admission controllers for automatic verification.

Fulcio: Certificate Authority

• OIDC Integration: Issues certificates based on OpenID Connect authentication.
• Short-Lived Certificates: Issues short-lived certificates for signing.
• Keyless Signing: Enables signing without managing private keys.
• Identity Verification: Verifies signer identity through OIDC providers.

Rekor: Transparency Log

• Immutable Log: Maintains an immutable log of all signatures.
• Signature Verification: Enables verification of signature authenticity.
• Audit Trail: Provides audit trail for compliance and security investigations.
• Public API: Public API for querying signature logs.

Supply Chain Security Workflow

Image Signing

# Sign image with Cosign
cosign sign --key cosign.key my-registry.com/my-app:v1.0.0

# Keyless signing (using OIDC)
cosign sign my-registry.com/my-app:v1.0.0

# Sign with annotations
cosign sign -a environment=production my-registry.com/my-app:v1.0.0

Image Verification

# Verify image signature
cosign verify --key cosign.pub my-registry.com/my-app:v1.0.0

# Verify with keyless
cosign verify my-registry.com/my-app:v1.0.0

# Verify with policy
cosign verify --policy policy.json my-registry.com/my-app:v1.0.0

Kubernetes Integration

Admission Controller

# Policy Controller for automatic verification
apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
  name: require-signature
spec:
  images:
    glob: "my-registry.com/**"
  authorities:
  - key:
      data: |
        -----BEGIN PUBLIC KEY-----
        ...
        -----END PUBLIC KEY-----

Policy Enforcement

# Require signatures for all images
apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
  name: require-all-signatures
spec:
  images:
    glob: "**"
  authorities:
  - keyless:
      identities:
      - issuer: https://github.com/login/oauth
        subject: "https://github.com/my-org/*"

Comparison: Sigstore vs Manual Signing

Supply Chain Security Best Practices

Sign All Images

• CI/CD Integration: Sign images automatically in CI/CD pipelines.
• Keyless Signing: Use OIDC-based keyless signing for simplicity.
• Policy Enforcement: Enforce signature verification via admission controllers.

Verify Before Deploy

• Admission Controllers: Use Policy Controller to verify signatures automatically.
• Manual Verification: Verify signatures manually for critical deployments.
• Policy Compliance: Ensure all images meet signature requirements.

Monitor and Audit

• Transparency Logs: Monitor Rekor logs for signature activity.
• Audit Trails: Maintain audit trails for compliance and security investigations.
• Alerting: Alert on unsigned images or signature verification failures.

Getting Started

# Install Cosign
wget -O cosign https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64
chmod +x cosign
sudo mv cosign /usr/local/bin/

# Generate key pair
cosign generate-key-pair

# Sign image
cosign sign --key cosign.key my-registry.com/my-app:v1.0.0

# Verify image
cosign verify --key cosign.pub my-registry.com/my-app:v1.0.0

Practical Considerations

Key Management

• Keyless Signing: Use keyless signing for simplicity (OIDC-based).
• Key-Based Signing: Use key-based signing for offline or air-gapped environments.
• Key Rotation: Plan for key rotation and certificate renewal.

Policy Enforcement

• Gradual Rollout: Enable signature verification gradually, starting with audit mode.
• Exception Handling: Plan for exceptions (legacy images, emergency deployments).
• Policy Updates: Update policies as signing requirements evolve.

Caveats & Lessons Learned

• Key Management: Keyless signing requires OIDC provider; plan for identity management.
• Policy Complexity: Complex policies can impact deployment speed; optimize policies.
• Legacy Images: Existing images may not be signed; plan for migration.

Common Failure Modes

• “Unsigned images blocked”: Policies block unsigned images; sign images or create exceptions.
• “Signature verification fails”: Signatures don’t match; verify signing keys and policies.
• “OIDC authentication fails”: Keyless signing fails; check OIDC provider configuration.

Conclusion

Sigstore’s GA in 2022 marked a turning point in software supply chain security for Kubernetes. It provided teams with the tools and infrastructure needed to sign and verify container images, ensuring their authenticity and integrity. While supply chain security had been a concern for years, Sigstore made it accessible and practical for teams to implement.

For organizations deploying containerized applications, Sigstore became an essential supply chain security tool. It demonstrated that supply chain security didn’t have to be complex or expensive—it could be simple, automated, and integrated into standard CI/CD workflows. Sigstore proved that cryptographic verification could be both powerful and accessible, enabling teams to secure their software supply chains from source to deployment.

The patterns and practices established with Sigstore would influence the development of advanced supply chain security tools and set the foundation for comprehensive supply chain security in Kubernetes. Sigstore demonstrated that software supply chain security could be both comprehensive and practical, enabling teams to verify the authenticity of every component in their containerized applications.