J47h.putty PDocsCloud Computing
Related
Understanding the iCloud Lock-In Lawsuit: A Detailed Guide to the UK Tribunal's Decision and the 'Forgone Consumer Surplus' TheoryAWS Announces Instant Aurora PostgreSQL Serverless Deployment with Express Configuration at re:Invent 2025How to Accelerate AI Development with Runpod Flash: A No-Container GuideTwo Decades of AWS S3: How a Simple Storage Service Transformed Cloud Computing8 Essential CSS Features and Tools You Need to Know Now10 Essential Steps to Deploy a Serverless Spam Classifier on AWSAutomated Cost Optimization for Azure Blob and Data Lake Storage: Smart Tier Now Generally AvailableCSS & Web Platform Q&A: Clip-Path Puzzles, View Transitions, Scoping, and More

The Silent Upgrade: How Kubernetes Image Promotion Got a Modern Makeover

Last updated: 2026-05-01 14:36:04 · Cloud Computing

Every container image you pull from registry.k8s.io arrives thanks to kpromo, the Kubernetes image promoter. This tool copies images from staging registries to production, signs them with cosign, replicates signatures across over 20 regional mirrors, and generates SLSA provenance attestations. If kpromo breaks, no Kubernetes release ships. Recently, the team rewrote its core from scratch, deleted 20% of the codebase, made it dramatically faster, and nobody noticed. That was the goal.

A Brief History

The image promoter began in late 2018 as an internal Google project by Linus Arver. The aim was to replace manual, Googler-gated image copying into k8s.gcr.io with a community-owned GitOps workflow. Push to a staging registry, open a PR with a YAML manifest, get reviewed and merged, and automation handles the rest. KEP-1734 formalized this.

The Silent Upgrade: How Kubernetes Image Promotion Got a Modern Makeover

In early 2019, the code moved to kubernetes-sigs/k8s-container-image-promoter and grew quickly. Stephen Augustus consolidated multiple tools (cip, gh2gcs, krel promote-images, promobot-files) into a single CLI called kpromo. The repo was renamed to promo-tools. Adolfo García Veytia (Puerco) added cosign signing and SBOM support. Tyler Ferrara built vulnerability scanning. Carlos Panato maintained the project. 42 contributors made ~3,500 commits across 60+ releases. It worked, but by 2025 the codebase carried seven years of incremental additions. The README noted duplication, multiple techniques, and many TODOs.

Problems to Solve

Production promotion jobs for Kubernetes core images regularly took over 30 minutes and often failed with rate limit errors. The core promotion logic became a monolith hard to extend and test, making new features painful. On the SIG Release roadmap, two work items lingered: "Rewrite artifact promoter" and "Make artifact validation more robust." Discussions at meetings and KubeCons led to open research spikes on project board #171 capturing eight questions.

The Phased Rewrite

In February 2026, the team opened issue #1701 to rewrite the artifact promoter pipeline. All eight spikes were answered in a single tracking issue. The rewrite was phased so each step could be reviewed, merged, and validated independently.

Phase 1: Rate Limiting (#1702)

Rewrote rate limiting to properly throttle all registry operations with adaptive backoff.

Phase 2: Interfaces (#1704)

Put registry and auth operations behind clean interfaces for independent swapping and testing.

Phase 3: Pipeline Restructuring

The pipeline was restructured into composable stages, allowing easier extension for provenance and vulnerability scanning. This modular approach reduced coupling and simplified testing.

Conclusion: Invisible Improvements

The rewrite made kpromo faster, more reliable, and easier to maintain. The 20% code deletion and new architecture ensure future features can be added without breaking existing workflows. The team succeeded in making the improvements invisible to users — exactly as intended.