Kubernetes management: Difference between revisions

From MediaWiki
Jump to navigation Jump to search
Line 241: Line 241:
</tabber>
</tabber>


After running <code>openstack coe cluster upgrade</code>, you can inspect the cluster's state using <code>openstack coe cluster list</code>:
After running <code>openstack coe cluster upgrade</code>, you can inspect the cluster's state using <code>openstack coe cluster list</code> (it will take some time for the cluster upgrade to be completed):
<tabber>
<tabber>
|-| Command =
|-| Command =

Revision as of 16:19, 7 August 2024

Overview

This page describes the creation and management of Kubernetes clusters in our OpenStack-based stoney cloud using the Magnum container orchestration engine (COE).

Note: This page is a work-in-progress, so if you face any challenges, please contact us with the specific issue.

Images

To list available Kubernetes images, use openstack image list with the following yq/jq filter: 

openstack image list -f yaml | yq '[ .[] | select(.Name | . != null and match("(?i)kubernetes")) ]'

Example output: 

- ID: 4c6f82a1-1ebc-451d-b989-947cffe72f0e
  Name: 'Ubuntu 22.04 (20240702): Kubernetes v1.28.11 (raw)'
  Status: active
- ID: c2ac4670-1422-4de6-864c-a5b813cddf61
  Name: 'Ubuntu 22.04 (20240702): Kubernetes v1.29.6 (raw)'
  Status: active
- ID: 21ad55f9-21fa-4586-932f-c9ecd20b403d
  Name: 'Ubuntu 22.04 (20240702): Kubernetes v1.30.2 (raw)'
  Status: active


openstack image list -f json | jq '[ .[] | select(.Name | . != null and match("(?i)kubernetes")) ]'

Example output: 

[
  {
    "ID": "4c6f82a1-1ebc-451d-b989-947cffe72f0e",
    "Name": "Ubuntu 22.04 (20240702): Kubernetes v1.28.11 (raw)",
    "Status": "active"
  },
  {
    "ID": "c2ac4670-1422-4de6-864c-a5b813cddf61",
    "Name": "Ubuntu 22.04 (20240702): Kubernetes v1.29.6 (raw)",
    "Status": "active"
  },
  {
    "ID": "21ad55f9-21fa-4586-932f-c9ecd20b403d",
    "Name": "Ubuntu 22.04 (20240702): Kubernetes v1.30.2 (raw)",
    "Status": "active"
  }
]

Creation

Creation - Cluster template

Use openstack coe cluster template create to create a cluster template: 

# ID or name of the OpenStack Kubernetes image to use:
image=c539d525-d912-4acb-a7c3-bfcaf5f533c5 # Ubuntu 22.04 (20240605): Kubernetes v1.30.1

# Name of the cluster template to create:
cluster_template=''

openstack coe cluster template create "$cluster_template" \
  --coe kubernetes \
  --image "$image" \
  --external-network public \
  --master-flavor 'Standard Düdingen c002m0004' \
  --flavor 'Standard Düdingen c002m0004' \
  --public \
  --master-lb-enabled


Request to create cluster template kubernetes-1.30.2 accepted
+-----------------------+--------------------------------------+
| Field                 | Value                                |
+-----------------------+--------------------------------------+
| insecure_registry     | -                                    |
| labels                | {}                                   |
| updated_at            | -                                    |
| floating_ip_enabled   | True                                 |
| fixed_subnet          | -                                    |
| master_flavor_id      | Standard Düdingen c002m0004          |
| uuid                  | d69969e7-1239-4810-b092-c758d3e9ef2e |
| no_proxy              | -                                    |
| https_proxy           | -                                    |
| tls_disabled          | False                                |
| keypair_id            | -                                    |
| public                | True                                 |
| http_proxy            | -                                    |
| docker_volume_size    | -                                    |
| server_type           | vm                                   |
| external_network_id   | public                               |
| cluster_distro        | ubuntu                               |
| image_id              | db68e8e8-d4b4-4c4f-af41-4166eb33973d |
| volume_driver         | -                                    |
| registry_enabled      | False                                |
| docker_storage_driver | overlay2                             |
| apiserver_port        | -                                    |
| name                  | kubernetes-1.30.2                    |
| created_at            | 2024-08-07T14:39:52.314174+00:00     |
| network_driver        | flannel                              |
| fixed_network         | -                                    |
| coe                   | kubernetes                           |
| flavor_id             | Standard Düdingen c002m0004          |
| master_lb_enabled     | True                                 |
| dns_nameserver        | 8.8.8.8                              |
| hidden                | False                                |
| tags                  | -                                    |
+-----------------------+--------------------------------------+

For a list of all labels see Magnum User Guide - Labels.

Creation - Cluster

Use openstack coe cluster create to create a cluster: 

# ID or name of the cluster template to use:
cluster_template=''

# Name of the cluster to create:
cluster=''

openstack coe cluster create "$cluster" \
  --cluster-template "$cluster_template" \
  --master-count 1 \
  --node-count 2


Request to create cluster 9bb54949-2469-4b7e-99c8-4cb2730d4e8f accepted

Options:

  • --master-count: number of control plane nodes.
  • --node-count: number of worker nodes.

After running openstack coe cluster create, you can inspect the cluster's state using openstack coe cluster list (it will take some time for the cluster to be created): 

openstack coe cluster list

Status during cluster creation: 

+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| uuid                                 | name           | keypair    | node_count | master_count | status             | health_status |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| e41cc55c-9f63-4880-9c18-0c021545efa4 | sst-yde-test-1 | None       |          1 |            1 | CREATE_IN_PROGRESS | None          |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+

Status after that the cluster has been successfully created: 

+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| uuid                                 | name           | keypair    | node_count | master_count | status             | health_status |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| e41cc55c-9f63-4880-9c18-0c021545efa4 | sst-yde-test-1 | None       |          1 |            1 | CREATE_COMPLETE    | HEALTHY       |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+

Creation - Cluster - Access

To access a created cluster with kubectl, helm etc., use openstack coe cluster config to retrieve the cluster's configuration (essentially the "kubeconfig" file): 

# ID or name of the cluster:
cluster=''

# Path to the directory where the Kubeconfig file will be stored:
directory="$HOME"/.kube

# Make sure the directory exists:
mkdir --parent "$directory"

# Download the kubeconfig.
# This will create the kubeconfig file at $directory/config
openstack coe cluster config --dir "$directory" "$cluster"

Depending on where the kubeconfig file has been store, you will need to set the environment variable KUBECONFIG accordingly (if you used the directory .kube in your home directory, this step is not required as ~/.kube/config is the default kubeconfig path): 

export KUBECONFIG=/path/to/kube/config

You will now be able to access the cluster using kubectl: 

kubectl get node -o wide


NAME                                                     STATUS   ROLES           AGE   VERSION    INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
sst-yde-test-1-fj6ladv5sne3-control-plane-jl86d          Ready    control-plane   45m   v1.28.11   10.0.0.149    <none>        Ubuntu 22.04.4 LTS   5.15.0-113-generic   containerd://1.7.13
sst-yde-test-1-fj6ladv5sne3-default-worker-b2c7j-5bh2w   Ready    <none>          44m   v1.28.11   10.0.0.150    <none>        Ubuntu 22.04.4 LTS   5.15.0-113-generic   containerd://1.7.13

Management

Management - Control plane

TBD

Management - Worker nodes

Use openstack coe cluster resize to change the number of worker nodes in a cluster: 

# ID or name of the cluster:
cluster=''

# Number of worker nodes:
nodes=''

openstack coe cluster resize "$cluster" "$nodes"

Upgrade

To upgrade a cluster (upgrading the version of Kubernetes), a new cluster template needs to be created first. This cluster template must use a newer Kubernetes version than the one used in your current cluster template. Downgrades are also possible, but haven't been tested in detail.

After creating the cluster template, use openstack coe cluster upgrade to assign the new cluster template to your cluster: 

# ID or name of the cluster to upgrade:
cluster=''

# ID or name of the new cluster template to use:
cluster_template=''

openstack coe cluster upgrade "$cluster" "$cluster_template"


Request to upgrade cluster sst-yde-test-1 has been accepted.

After running openstack coe cluster upgrade, you can inspect the cluster's state using openstack coe cluster list (it will take some time for the cluster upgrade to be completed): 

openstack coe cluster list

During the upgrade: 

+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| uuid                                 | name           | keypair    | node_count | master_count | status             | health_status |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| e41cc55c-9f63-4880-9c18-0c021545efa4 | sst-yde-test-1 | None       |          1 |            1 | UPDATE_IN_PROGRESS | UNHEALTHY     |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+

After the upgrade: 

+--------------------------------------+----------------+------------+------------+--------------+-----------------+---------------+
| uuid                                 | name           | keypair    | node_count | master_count | status          | health_status |
+--------------------------------------+----------------+------------+------------+--------------+-----------------+---------------+
| e41cc55c-9f63-4880-9c18-0c021545efa4 | sst-yde-test-1 | None       |          1 |            1 | UPDATE_COMPLETE | HEALTHY       |
+--------------------------------------+----------------+------------+------------+--------------+-----------------+---------------+

Deletion

Deletion - Cluster template

Use openstack coe cluster template delete to delete a cluster template: 

# ID or name of the cluster template to delete:
cluster_template=''

openstack coe cluster template delete "$cluster_template"

Deletion - Cluster

Use openstack coe cluster delete to delete a cluster: 

# ID or name of the cluster to delete:
cluster=''

openstack coe cluster delete "$cluster"

Troubleshooting

Troubleshooting - Adjusting the Kubernetes image label kube_version

When creating a cluster using openstack coe cluster create, you might encounter the following error: 

Image c539d525-d912-4acb-a7c3-bfcaf5f533c5 does not have a kube_version property.

To fix this problem, add the kube_version property to the image: 

# ID or name of the OpenStack Kubernetes image to adjust:
image_id=''

# The Kubernetes version of the image:
k8s_image_version=v1.30.1

# Adjust the image and add the "kube_version" property:
openstack image set "$image_id" --property kube_version="$k8s_image_version"

After adjust the image property you will have to delete the existing cluster and re-create it using openstack coe cluster create.

Retrieved from "https://wiki.stoney-cloud.com/w/index.php?title=Kubernetes_management&oldid=609"