New Cloud Container Engine (CCE) upgrade

Flexible Engine
Release Notes

February, 2023

What’s new?

Cloud Container Engine now supports Kubernetes version 1.23 and you can easily migrate from version 1.21 to 1.23.

Cloud Container Engine

What is it?

Based on the Kubernetes orchestrator to deploy and manage Docker applications, the CCE service provides a visual orchestration tool to easily create and deploy applications.

Support for Kubernetes 1.23

  • FlexVolume is deprecated. Use CSI.
  • HorizontalPodAutoscaler v2 is promoted to GA, and HorizontalPodAutoscaler API v2 is gradually stable in version 1.23. The HorizontalPodAutoscaler v2beta2 API is not recommended. Use the v2 API.
  • PodSecurity moves to beta, replacing the deprecated PodSecurityPolicy. PodSecurity is an admission controller that enforces pod security standards on pods in the namespace based on specific namespace labels that set the enforcement level. PodSecurity is enabled by default in version 1.23. 
  • Ingresses no longer support networking.k8s.io/v1beta1 and extensions/v1beta1 APIs. If you use the API of an earlier version to manage ingresses, an application cannot be exposed to external services. Use networking.k8s.io/v1.
  •  CustomResourceDefinitions no longer support the apiextensions.k8s.io/v1beta1 API. If you use the API of an earlier version to create a CRD, the creation will fail, which affects the controller that reconciles this CRD. Use apiextensions.k8s.io/v1.
  • ClusterRoles, ClusterRoleBindings, Roles, and RoleBindings no longer support the rbac.authorization.k8s.io/v1beta1 API. If you use the API of an earlier version to manage RBAC resources, application permissions control is affected and even cannot work in the cluster. Use rbac.authorization.k8s.io/v1.
  • The Kubernetes release cycle is changed from four releases a year to three releases a year.
  • StatefulSets support minReadySeconds.
  • During scale-in, pods are randomly selected and deleted based on the pod UID by default (LogarithmicScaleDown). This feature enhances the randomness of the pods to be deleted and alleviates the problems caused by pod topology spread constraints. For more information, seeKEP-2185 and issue 96748.

Removing a Node

  • Removing a node from a cluster will re-install the node OS and clear CCE components on the node. Removing a node will not delete the server corresponding to the node. You are advised to remove nodes at off-peak hours to avoid impacts on your services. After a node is removed from the cluster, the node is still running and incurs fees.
  • Nodes can be removed only when the cluster is in the Available or Unavailable state.
  • A CCE node can be removed only when it is in the Active, Abnormal, or Error state.
  • A CCE node in the Active state can have its OS re-installed and CCE components cleared after it is removed.
  • If the OS fails to be re-installed after the node is removed, manually re-install the OS. After the re-installation, log in to the node and run the clearance script to clear CCE components.
  • Removing a node will lead to pod migration, which may affect services. Perform this operation during off-peak hours.
  • Unexpected risks may occur during the operation. Back up data in advance.
  • While the node is being deleted, the backend will set the node to the unschedulable state.
  • After you remove the node and re-install the OS, the original LVM partitions will be cleared and the data managed by LVM will be cleared. Therefore, back up data in advance.

For more information on CCE please visit the Help Center.