Kubernetes engineering blog

Kubernetes SIG Apps chairs explain how workload controllers such as Deployments, StatefulSets, Jobs, and DaemonSets manage application lifecycles and resilience. The discussion covers node failures, coordinated restarts for distributed AI workloads, backward-compatible API evolution, and the proposed KEP-4443 enhancement for distinguishing Job failure causes.

Kubernetes v1.37 promotes PersistentVolumeClaimUnusedSinceTime to Beta and enables it by default. The PVC protection controller now reports whether non-terminal pods reference each claim, while lastTransitionTime enables administrators to identify and automate cleanup of long-unused storage.

The post explains how Kubernetes v1.37 hardens container storage with alpha bind mount options and configurable emptyDir permissions. It shows how noexec, nosuid, nodev, and sticky-bit modes reduce execution, privilege-escalation, and cross-container file-deletion risks, including manifests, verification steps, and runtime limitations.

Kubernetes v1.37 graduates Pod-Level Resource Managers to beta, enabling the Kubelet’s CPU, memory, and topology managers to use pod-level resource declarations for NUMA-aware placement. The update supports hybrid allocation for primary containers and sidecars and adds pod-level exclusive assignment reporting to the PodResources API.

Kubernetes v1.37 promotes Memory QoS to Beta and enables its feature gate by default while preserving existing workload behavior through a null memory throttling default. The post explains cgroup v2 configuration for throttling and tiered reservation, upgrade considerations, and the node-wide limitations of memory protection.

Kubernetes’ Changed Block Tracking API has moved from Alpha to Beta with the v1.0.0 release of external-snapshot-metadata. The post details the v1beta1 CRD migration, compatibility requirements, required upgrade steps, and how CSI drivers and backup applications can try the API.

Kubernetes v1.37 enables Prometheus native histograms by default, providing higher-resolution quantiles with fewer time series and lower storage overhead than classic bucketed histograms. The post explains dual exposition, exponential bucket configuration, PromQL queries, Prometheus scraping options, dashboard migration, and rollback strategies.

The post explains how Kubernetes v1.37 enables scheduler preemption for deferred in-place Pod resizes, allowing higher-priority workloads to reclaim capacity from lower-priority Pods without restarting. It covers the scheduling architecture, node-level controls, race handling, feature-gate requirements, and a kind-based demonstration.

Kubernetes v1.37 introduces five well-known Node lifecycle conditions for reporting drains, maintenance, and graceful shutdowns through a shared status channel. The post explains their semantics, current alpha limitations, operational usage, and how they could enable more lifecycle-aware controller behavior.

Kubernetes v1.37 advances Workload-Aware Scheduling with Beta Workload and PodGroup APIs, gang scheduling, workload-aware preemption, and shared DRA ResourceClaims. It introduces hierarchical CompositePodGroups, multi-level topology constraints, controller integration APIs, and explicit Job scheduling configuration for complex distributed workloads.

Kubernetes v1.37 promotes KubeletInUserNamespace, enabling kubelet and other node components to run as non-root users within a Linux user namespace. The post explains its security benefits, compatibility caveats, changes from alpha to beta, and setup options using kind, minikube, Usernetes, and k3s.

Kashish Verma details the Kubernetes 1.37 Dynamic Resource Allocation updates, including GA support for extended resources, device taints, standardized attributes, and workload ResourceClaims. The post also covers new alpha features such as derived attributes, compatibility groups, fractional capacity, and scheduler performance improvements.

Kubernetes v1.37 makes HorizontalPodAutoscaler scale workloads to zero and back using object or external metrics, with the feature enabled by default. The post explains Prometheus Adapter configuration, HPA conditions, cold-start trade-offs, and upgrade considerations for queue-driven workloads.

The post explains how Kubernetes v1.37 uses etcd v3.7’s RangeStream RPC to stream large list reads in adaptive, byte-bounded chunks, reducing peak memory use in etcd and the API server. It covers feature-gate requirements, fallback behavior, and the metrics query for confirming that streaming is active.

Kubernetes v1.37 makes the StorageVersionMigration API and control-plane migrator generally available and enabled by default. The post explains how declarative migrations rewrite existing resources to the current storage version, supporting CRD version upgrades, encryption key rotation, status verification, and manifest-based workflows.

Kubernetes v1.37 makes Pod Certificates and Cluster Trust Bundles generally available, bringing X.509-based TLS and mTLS identity to workloads. The post explains the Kubelet, signer, and trust-bundle architecture, automatic rotation, security controls, and how to experiment with a third-party signer.

Kubernetes v1.37 graduates the metrics.k8s.io API to stable v1 without changing its resources or fields. The post explains compatibility with v1beta1, kubectl top and HPA support, API aggregation requirements, and commands for verifying served versions.

Kubernetes v1.37, codenamed Garhwal, delivers 67 enhancements across Stable, Beta, and Alpha stages. Highlights include resilient watch-cache initialization, HPA scale-to-zero, gang scheduling, native histograms, memory QoS, pod-level resource management, improved CSI autoscaling, and new admission-control and checkpoint/restore capabilities.

Kashish Verma explains KYAML, a stricter YAML dialect for Kubernetes manifests that makes structure and string types explicit while remaining compatible with existing YAML tooling. The post shows how to generate and format KYAML with kubectl and multiple yamlfmt implementations, and discusses its adoption trade-offs.

Gateway API v1.6 promotes TCPRoute and UDPRoute to the stable v1 API, enabling portable Layer 4 TCP and UDP routing in Kubernetes. It also introduces the experimental XBackend resource and separates experimental APIs into a distinct group, with examples and migration considerations for Gateway users.