[CCE Update] k8s 1.21 version & migration tool 1.19 to 1.21

Flexible Engine
Release Notes

April, 2022

Support for Kubernetes 1.21

As of Kubernetes 1.21, CentOS’ Docker storage mode is changing from Device Mapper to OverlayFS.

Kubelet

The bug that was causing runtime probe delays to not take effect is fixed. Before this bug was fixed, the runtime probe did not take into account the timeoutSeconds field. The probe will run indefinitely, even beyond its configured timeout, until the result is returned. With this bug fix, if no value is specified, the default value, 1 second, is used. If the polling time exceeds 1 second, the application health check may fail. Update the timeoutSeconds field for applications that use this feature when upgrading. The fix provided by the newly introduced ExecProbeTimeout feature gate allows the cluster operator to restore the previous behavior, but this behavior will be locked and removed in future releases.

API Change

Dual-stack services are supported. The dual-stack API changes from ipFamily to three fields: ipFamilypolicy (SingleStack, PreferDualStack, RequireDualStack), ipFamilies and clusterIPs. If the user does not specify the dual stack, the single stack services are used by default.

The RuntimeClass function is changed to GA, and the API groups in node.k8s.io are updated to v1. v1beta1 APIs will be dropped.

New serving and terminating states are added to the EndpointSlice API. The serving node traces ready endpoints whether or not the pod is in the terminating state. The ready node does not cover the pod in the terminating state. terminating indicates that a pod has been deleted.

The allocateLoadBalancerNodePorts field is added to services, indicating whether to allocate node ports to LoadBalancer services. The default behavior is the same as the current behavior.

When the service type is updated, some unnecessary fields are automatically removed. For example, when a Service changes from LoadBalancer to ClusterIP, the node ports are automatically cleared.

The maxSurge can be an integer or a percentage in DaemonSet continuous update policies, which start updated pods on the node and wait for them to be ready before marking old pods as deleted. This allows workloads to avoid downtime during upgrades when deployed as DaemonSets. This feature is in alpha phase and can be enabled via the DaemonSetUpdateSurge feature gate.

The controller.kubernetes.io/pod-deletion-cost annotation is supported. It represents the cost of deleting a pod relative to other pods belonging to the same ReplicaSet. Pods with the lowest deletion cost are preferred to be deleted first. This feature is currently in alpha phase.

Deprecation

Kubelet

  • Dockershim is officially discarded and will be removed in later versions.
  • metrics/resource replaces metrics/resource/v1alpha1.

Kube-proxy

  • –healthz-bind-address and –metrics-bind-address replace –healthz-port and –metrics-port.
  • EndpointSliceProxying replaces EndpointSlices to control whether kube-proxy enables the endpoint slice feature. The default value is false.

Kube-scheduler

  • Metrics binding_duration_seconds and binding_duration_seconds are discarded and replaced by framework_extension_point_duration_seconds.
  • The topologyKeys field is deprecated for Services and will be replaced by the topology awareness subset and service-internal traffic policy.
  • The obsolete flag –cleanup-ipvs is deleted from kube-proxy.
  • Metrics scheduling_algorithm_preemption_evaluation_seconds and binding_duration_seconds are replaced by scheduler_framework_extension_point_duration_seconds for kube-scheduler.
  • etcd_object_counts is changed to apiserver_storage_object_counts for kube-apiserver.
  • CronJobs have been discarded in batch/v2alpha1.

EndpointSlices have been deprecated in discovery.k8s.io/v1beta1 and will not serve since v1.25. Use discovery.k8s.io/v1.

Two-way authentication for accessing domain names

When a cluster is created, the access domain name and kube-apiserver EIP can be configured by adding the transaction processing certificate.

The Access Domain Name and kube-apiserver EIP of an existing cluster can be changed. A transaction re-signing certificate must be added, the domain name and EIP information must be added, and the kube-apiserver component must be restarted.

Master overload optimization

As of version 1.21:

CCE control plane nodes reserve resources for kubelet to ensure that system processes and kubelet components can run stably even under overload.

The oom_score component of the control plane is properly adjusted. The priority of the stateless oom_score component is not lower than that of a system process.

A new namespace has default resource quotas determined by the cluster specification and can be modified by users.

CCE cluster version management

Starting from CCE v1.21, the cluster version format is changed to v1.21. The original minor version number is not displayed.

OverlayFS for CentOS 7 nodes

The Docker storage mode of nodes running CentOS 7 in CCE clusters is changed from Device Mapper to OverlayFS.