Kubernetes management: Difference between revisions

From MediaWiki
Jump to navigation Jump to search
Line 116: Line 116:


=== Creation - Cluster - Access ===
=== Creation - Cluster - Access ===
To access a created cluster with <code>kubectl</code>, <code>helm</code> etc., use <code>openstack coe cluster config</code> to retrieve the cluster's configuration:
To access a created cluster with <code>kubectl</code>, <code>helm</code> etc., use <code>openstack coe cluster config</code> to retrieve the cluster's configuration (essentially the "kubeconfig" file):
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
# ID or name of the cluster:
# ID or name of the cluster:
Line 127: Line 127:
mkdir --parent "$directory"
mkdir --parent "$directory"


# Download the Kubeconfig.
# Download the kubeconfig.
# This will create the Kubeconfig file at $directory/config
# This will create the kubeconfig file at $directory/config
openstack coe cluster config --dir "$directory" "$cluster"
openstack coe cluster config --dir "$directory" "$cluster"
</syntaxhighlight>
</syntaxhighlight>


Depending on where the Kubeconfig has been store, you will need to set the environment variable <code>KUBECONFIG</code> accordingly (if you used the directory <code>.kube</code> in your home directory, this step is not required as it is the default path used by <code>kubectl</code>):
Depending on where the kubeconfig file has been store, you will need to set the environment variable <code>KUBECONFIG</code> accordingly (if you used the directory <code>.kube</code> in your home directory, this step is not required as <code>~/.kube/config</code> is the default kubeconfig path):
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
export KUBECONFIG=/path/to/config
export KUBECONFIG=/path/to/kube/config
</syntaxhighlight>
</syntaxhighlight>


Revision as of 15:35, 7 August 2024

Overview

This page describes the creation and management of Kubernetes clusters in our OpenStack-based stoney cloud using the Magnum container orchestration engine (COE).

Note: This page is a work-in-progress, so if you face any challenges, please contact us with the specific issue.

Images

To list available Kubernetes images, use openstack image list with the following yq/jq filter: 

openstack image list -f yaml | yq '[ .[] | select(.Name | match("(?i)kubernetes.*raw")) ]'

Example output: 

- ID: 4c6f82a1-1ebc-451d-b989-947cffe72f0e
  Name: 'Ubuntu 22.04 (20240702): Kubernetes v1.28.11 (raw)'
  Status: active
- ID: c2ac4670-1422-4de6-864c-a5b813cddf61
  Name: 'Ubuntu 22.04 (20240702): Kubernetes v1.29.6 (raw)'
  Status: active
- ID: 21ad55f9-21fa-4586-932f-c9ecd20b403d
  Name: 'Ubuntu 22.04 (20240702): Kubernetes v1.30.2 (raw)'
  Status: active


openstack image list -f json | jq '[ .[] | select(.Name | match("(?i)kubernetes.*raw")) ]'

Example output: 

[
  {
    "ID": "4c6f82a1-1ebc-451d-b989-947cffe72f0e",
    "Name": "Ubuntu 22.04 (20240702): Kubernetes v1.28.11 (raw)",
    "Status": "active"
  },
  {
    "ID": "c2ac4670-1422-4de6-864c-a5b813cddf61",
    "Name": "Ubuntu 22.04 (20240702): Kubernetes v1.29.6 (raw)",
    "Status": "active"
  },
  {
    "ID": "21ad55f9-21fa-4586-932f-c9ecd20b403d",
    "Name": "Ubuntu 22.04 (20240702): Kubernetes v1.30.2 (raw)",
    "Status": "active"
  }
]

Creation

Creation - Cluster template

Use openstack coe cluster template create to create a cluster template: 

# ID or name of the OpenStack Kubernetes image to use:
image=c539d525-d912-4acb-a7c3-bfcaf5f533c5 # Ubuntu 22.04 (20240605): Kubernetes v1.30.1

# Name of the cluster template to create:
cluster_template=''

openstack coe cluster template create "$cluster_template" \
  --coe kubernetes \
  --image "$image" \
  --external-network public \
  --master-flavor 'Standard Düdingen c002m0004' \
  --flavor 'Standard Düdingen c002m0004' \
  --public \
  --master-lb-enabled

For a list of all labels see Magnum User Guide - Labels.

Creation - Cluster

Use openstack coe cluster create to create a cluster: 

# ID or name of the cluster template to use:
cluster_template=''

openstack coe cluster create my_k8s_cluster \
  --cluster-template "$cluster_template" \
  --master-count 1 \
  --node-count 2

Options:

  • --master-count: number of control plane nodes.
  • --node-count: number of worker nodes.

After running openstack coe cluster create, you can inspect the cluster's state using openstack coe cluster list: 

openstack coe cluster list

Status during cluster creation: 

+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| uuid                                 | name           | keypair    | node_count | master_count | status             | health_status |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| e41cc55c-9f63-4880-9c18-0c021545efa4 | sst-yde-test-1 | None       |          1 |            1 | CREATE_IN_PROGRESS | None          |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+

Status after that the cluster has been successfully created: 

+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| uuid                                 | name           | keypair    | node_count | master_count | status             | health_status |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| e41cc55c-9f63-4880-9c18-0c021545efa4 | sst-yde-test-1 | None       |          1 |            1 | CREATE_COMPLETE    | HEALTHY       |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+

Creation - Cluster - Access

To access a created cluster with kubectl, helm etc., use openstack coe cluster config to retrieve the cluster's configuration (essentially the "kubeconfig" file): 

# ID or name of the cluster:
cluster=''

# Path to the directory where the Kubeconfig file will be stored:
directory="$HOME"/.kube

# Make sure the directory exists:
mkdir --parent "$directory"

# Download the kubeconfig.
# This will create the kubeconfig file at $directory/config
openstack coe cluster config --dir "$directory" "$cluster"

Depending on where the kubeconfig file has been store, you will need to set the environment variable KUBECONFIG accordingly (if you used the directory .kube in your home directory, this step is not required as ~/.kube/config is the default kubeconfig path): 

export KUBECONFIG=/path/to/kube/config

You will now be able to access the cluster using kubectl: 

kubectl get node -o wide


NAME                                                     STATUS   ROLES           AGE   VERSION    INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
sst-yde-test-1-fj6ladv5sne3-control-plane-jl86d          Ready    control-plane   45m   v1.28.11   10.0.0.149    <none>        Ubuntu 22.04.4 LTS   5.15.0-113-generic   containerd://1.7.13
sst-yde-test-1-fj6ladv5sne3-default-worker-b2c7j-5bh2w   Ready    <none>          44m   v1.28.11   10.0.0.150    <none>        Ubuntu 22.04.4 LTS   5.15.0-113-generic   containerd://1.7.13

Management

Management - Control plane

TBD

Management - Worker nodes

# ID or name of the cluster:
cluster=''

# Number of worker nodes:
nodes=''

openstack coe cluster resize "$cluster" "$nodes"

Upgrade

To upgrade a cluster (upgrading the version of Kubernetes), a new cluster template needs to be created first. This cluster template must use a newer Kubernetes version than the one used in your current cluster template. Downgrades are also possible, but haven't been tested in detail.

After creating the cluster template, use openstack coe cluster upgrade to assign the new cluster template to your cluster: 

# ID or name of the cluster to upgrade:
cluster=''

# ID or name of the new cluster template to use:
cluster_template=''

openstack coe cluster upgrade "$cluster" "$cluster_template"


Request to upgrade cluster sst-yde-test-1 has been accepted.

After running openstack coe cluster upgrade, you can inspect the cluster's state using openstack coe cluster list: 

openstack coe cluster list

During the upgrade: 

+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| uuid                                 | name           | keypair    | node_count | master_count | status             | health_status |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+
| e41cc55c-9f63-4880-9c18-0c021545efa4 | sst-yde-test-1 | None       |          1 |            1 | UPDATE_IN_PROGRESS | UNHEALTHY     |
+--------------------------------------+----------------+------------+------------+--------------+--------------------+---------------+

After the upgrade: 

+--------------------------------------+----------------+------------+------------+--------------+-----------------+---------------+
| uuid                                 | name           | keypair    | node_count | master_count | status          | health_status |
+--------------------------------------+----------------+------------+------------+--------------+-----------------+---------------+
| e41cc55c-9f63-4880-9c18-0c021545efa4 | sst-yde-test-1 | None       |          1 |            1 | UPDATE_COMPLETE | HEALTHY       |
+--------------------------------------+----------------+------------+------------+--------------+-----------------+---------------+

Deletion

Deletion - Cluster template

Use openstack coe cluster template delete to delete a cluster template: 

# ID or name of the cluster template to delete:
cluster_template=''

openstack coe cluster template delete "$cluster_template"

Deletion - Cluster

Use openstack coe cluster delete to delete a cluster: 

# ID or name of the cluster to delete:
cluster=''

openstack coe cluster delete "$cluster"

Troubleshooting

Troubleshooting - Adjusting the Kubernetes image label kube_version

When creating a cluster using openstack coe cluster create, you might encounter the following error: 

Image c539d525-d912-4acb-a7c3-bfcaf5f533c5 does not have a kube_version property.

To fix this problem, add the kube_version property to the image: 

# ID or name of the OpenStack Kubernetes image to adjust:
image_id=''

# The Kubernetes version of the image:
k8s_image_version=v1.30.1

# Adjust the image and add the "kube_version" property:
openstack image set "$image_id" --property kube_version="$k8s_image_version"

After adjust the image property you will have to delete the existing cluster and re-create it using openstack coe cluster create.

Retrieved from "https://wiki.stoney-cloud.com/w/index.php?title=Kubernetes_management&oldid=602"