VM creation (Linux)

Cluster Operations

Cluster Operations - Creating

<tabber> |-|VM creation CLI=

OpenStack CLI Installation

To use OpenStack CLI, you need to install the OpenStack CLI Client for your OS:

# Fedora
dnf install python3-openstackclient
# Debian / Ubuntu
apt install python3-openstackclient

The official installaton guide can be found on docs.openstack.org.

OpenStack client environment script (OpenRC file)

To increase efficiency of client operations, OpenStack supports simple client environment scripts also known as OpenRC files. These scripts typically contain common options for all clients, but also support unique options.

You may download such an OpenRC file from the dashboard.

After logging into the dashboard, click on your user name in the upper right corner and choose the menu option «OpenStack RC File v3» and save it the desired location on your local machine. As you can have multiple OpenRC files, we recommend to use a sub directory. For example openrc in your home directory.

Source the OpenRC file specific to this project (use single quotation marks to avoid problems with spaces in the file name):

source ${HOME}/openrc/'Project Name-openrc.sh'

After you sourced the file, you need to enter a password:

Please enter your OpenStack Password for project Project Name as user user:

Check if you have access to the OpenStack API by listing the API endpoints:

openstack catalog list

Expected output:

+------------+-----------+--------------------------------------------------------------------------------------------------+
| Name       | Type      | Endpoints                                                                                        |
+------------+-----------+--------------------------------------------------------------------------------------------------+
| glance     | image     | duedingen-production                                                                             |
|            |           |   internal: https://glance.ctrl-int.os.stoney-cloud.com:9292                                     |
|            |           | duedingen-production                                                                             |
|            |           |   public: https://api.os.stoney-cloud.com:9292                                                   |
|            |           | duedingen-production                                                                             |
|            |           |   admin: https://glance.ctrl-int.os.stoney-cloud.com:9292                                        |
[...]
| cinderv2   | volumev2  | duedingen-production                                                                             |
|            |           |   internal: https://cinder.ctrl-int.os.stoney-cloud.com:8776/v2/616812eda14e44de89138f3377841187 |
|            |           | duedingen-production                                                                             |
|            |           |   admin: https://cinder.ctrl-int.os.stoney-cloud.com:8776/v2/616812eda14e44de89138f3377841187    |
|            |           | duedingen-production                                                                             |
|            |           |   public: https://api.os.stoney-cloud.com:8776/v2/616812eda14e44de89138f3377841187               |
|            |           |                                                                                                  |
+------------+-----------+--------------------------------------------------------------------------------------------------+

VM Creation - One-Disk Setup

VM Creation - One-Disk Setup - Variables

We define bash variables so that in every command the same value is being used.

Another advantage of variables is, that the documentation is significantly easier, as another person could replicate the server if they know the variables that were set.

Set the following variables that we will use in later commands:

# Host name of the server.                 Example: hostname="sst-int-tmp-041"
hostname=""

# Display name of the server in OpenStack. Example: vmname="sst-int-tmp-041: debian test cli"
vmname=""

# Domain name.                             Example: domain="os.stoney-cloud.com"
domain=""

We need to set the project_id variable to the project our previously sourced openrc file belongs to:

# List the projects of your OpenStack domain.
openstack project list

# Project ID of the project in which the VM will be created.
#   Example (stepping stone AG - Internal Systems Temporary): project_id="6fd0ccd8b5ae44d292c67f0d3e75ca20"
project_id=""

A flavour defines the CPU and RAM resources of the VM.

The flavour is in the following format: cXXmYYYY where XX is the amount of CPUs and YYYY the about of RAM:

# List all available flavors
openstack flavor list --column Name --column ID | tail -n+3 | head -n-1 | sort -k3 -t'|'

# Set the flavor ID, default: Standard Düdingen c001m0004 (719c82d4-df94-47fc-a7df-f18d5c6d3727).  
#    Example: flavor_id="719c82d4-df94-47fc-a7df-f18d5c6d3727" 
flavor_id=""

We will add our VM to the internal network of our project. This is the default network:

# List the networks of the current project
openstack network list --project ${project_id}

# Network ID, usually the ID for the network "internal".  
#    Example: network_id="919c2dde-6996-494a-86de-fc3b08248418"
network_id=""

For this example, we use the "SSH" and "default" security groups (firewall-rules), so that we can access our server via ssh:

# List the security groups of the current project and search for ssh and default
openstack security group list --project ${project_id} | egrep -i '(ssh|default)'

# We set the default security group ID "default" ("default" is required for outgoing traffic!):
#    Example: default_security_group_id="3f576bd2-11fe-47f3-806e-aaa219cff589" # default
default_security_group_id=""

# We set the "SSH World" security group ID: 
#    Example: ssh_security_group_id="8083a9f3-e6c0-4061-b4bb-eb0dd24a86ef"     # SSH World
ssh_security_group_id=""

VM Creation - One-Disk Setup - Disk creation

In OpenStack every Image has an ID.

To create a new VM, we need to create a disk first.

We can list the available images using the openstack-cli - we search for Ubuntu:

openstack image list | grep "Ubuntu"

Expected output:

| 0ebe72db-55cf-4caa-ac47-a08994e7f163 | Ubuntu 20.04 LTS (Focal Fossa) Daily Build [20210210]     | active |
| 18d38120-873e-4902-850d-ce7be3a62d93 | Ubuntu 20.04 LTS (Focal Fossa) Daily Build [20210323]     | active |
| 50735f95-4963-4994-9c65-dac44773977b | Ubuntu 22.04 LTS (Jammy Jellyfish) Daily Build [20220616] | active |

We set the ID of the image and the size of the disk as variables for later use.

# Set the Image ID used for the first volume (Ubuntu 22.04 LTS (Jammy Jellyfish) Daily Build [20220616])
vda_image_id="50735f95-4963-4994-9c65-dac44773977b"

# Size for the first volume in GiB
volume_size_vda="10"

Now we can create the volume using the variables we just set.

We can set volume_vda_id as the command returns the ID of the newly created disk:

volume_vda_id=$( openstack volume create \ --property os-vol-tenant-attr:tenant_id=${project_id} \ --bootable \ --size ${volume_size_vda} \ --image ${vda_image_id} \ --description "OS disk (/dev/vda) for ${hostname}." \ --column id \ --format value \ "${hostname}: OS" )

Set the variable ${volume_vda_id}. Creates a new disk. The disk belongs to the current project. The disk can be bootable. The size of the disk is ${volume_size_vda}. The image of the disk is ${vda_image_id. We set the description of the disk. The id column will be printed out as output. Only the value will be printed out as output. Name of the disk. -

We ask OpenStack for the status of the current disk.

If the disk has been successfully created, we can move on and create the VM:

openstack volume show \
    "${volume_vda_id}" \
    --column status \
    --format value

Expected output:

available

VM Creation - One-Disk Setup - VM creation

We create a configuration file. In it, we can define the hostname of the server as well as commands that will be ran on startup. We also add our ssh public keys to this file.

# Create a folder for the cloud-init files.
mkdir -p ${HOME}/openstack

The following command creates a cloud-config file in ${HOME}/openstack/${hostname}.cloud-init. Replace the ssh public keys with your own!

IMPORTANT: The newest Ubuntu Version does not accept ssh-rsa and ssh-dss keys by default. We must use a ed25519 key!

cat <<EOF > ${HOME}/openstack/${hostname}.cloud-init
#cloud-config
hostname: ${hostname}
fqdn: ${hostname}.${domain}
runcmd:
 - [ ln, "--symbolic", "--force", "../usr/share/zoneinfo/Europe/Zurich", "/etc/localtime" ]
ssh_authorized_keys:
 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAQpBvIVaGryODNUG0MzqSIUy6TM8fasewZUEMVyFpmN alain.sinzig@stepping-stone.ch"
 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGusGXrY+UgTHH66YKS/o0vzUxnHVjoMzp0GnbatBzFb wahid.amiry@stepping-stone.ch"
 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMIHnEB/9uqcJ4P+i6kMrfjt11HBnmgbDodHh5zqEXAa fabian.zoffel@stepping-stone.ch"
 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICK+4kHiDYdb2CzvY9CTLiWpJ33c0cyDP+sZJUlvc3+N niklaus.hofer@stepping-stone.ch"
 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKb6iBxDn/IPnuHU81HPxaw9Gp/coE27C2Y9n12K8K0p yannick.denzer@stepping-stone.ch"
 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHSTJu9OiQLi+kwfg77mVuzs7V5ildZL0b/u5PquQ01w michael.eichenberger@stepping-stone.ch"
EOF

The script links the correct time zone to /etc/localtime and adds the public keys.

Review the cloud-init file:

${EDITOR} ${HOME}/openstack/${hostname}.cloud-init

Finally, we can create the server using the openstack server create command:

server_id=$( openstack server create \ --property project_id=${project_id} \ --flavor "${flavor_id}" \ --nic "net-id=${network_id}" \ --volume "${volume_vda_id}" \ --security-group "${default_security_group_id}" \ --security-group "${ssh_security_group_id}" \ --user-data ${HOME}/openstack/${hostname}.cloud-init \ --column id \ --format value \ "${vmname}" )

We set the server_id variable to the OpenStack id of the server. We create a new VM. We set the project to ${project_id}. We set the flavor to ${flavor_id}. We set the nic to ${network_id}. We set the volume to ${volume_vda_id}. Use the security-group (firwall rule) ${default_security_group_id}. Use the security-group (firwall rule) ${ssh_security_group_id}. We use our previously generated clopud init script as user-data. The id column will be printed out as output. Only the value will be printed out as output. We set the name of the VM. -

Check the status of the newly created instance (should be ACTIVE)

openstack server show ${server_id} --column status

Expected Output:

+--------+--------+
| Field  | Value  |
+--------+--------+
| status | ACTIVE |
+--------+--------+

VM Creation - One-Disk Setup - Floating IP

Create a floating IP:

floating_ip_id=$(
    openstack floating ip create \
        --column id \
        --format value \
        public
)

Get the IP-Adress of the newly created VM and output it:

floating_ip=$(
    openstack floating ip show \
        --column floating_ip_address \
        --format value \
        "${floating_ip_id}"
)
echo ${floating_ip}

Assign that floating IP to the newly created VM:

openstack server add floating ip \
    "${server_id}" \
    "${floating_ip}"

VM Creation - One-Disk Setup - VM Login

Finally, you can connect to your newly created VM:

ssh ubuntu@${floating_ip}

Expected output:

The authenticity of host '185.85.126.34 (185.85.126.34)' can't be established.
ED25519 key fingerprint is SHA256:lviEBYSl+ij7KJmKxmsDzkkPjgUCA9K4hB+3ES0LSn8.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '185.85.126.34' (ED25519) to the list of known hosts.
Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.15.0-39-generic x86_64)

[...]

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@hostname:~$

VM Creation - One-Disk Setup - Checks

Become root:

sudo -i

Check if the correct image was selected:

# Check the OS
cat /etc/os-release

PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Check if the disks are mounted correctly:

# Check the block devices
lsblk

NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
loop0     7:0    0 61.9M  1 loop /snap/core20/1518
loop1     7:1    0 79.9M  1 loop /snap/lxd/22923
loop2     7:2    0   47M  1 loop /snap/snapd/16010
vda     252:0    0   10G  0 disk 
├─vda1  252:1    0  9.9G  0 part /
├─vda14 252:14   0    4M  0 part 
└─vda15 252:15   0  106M  0 part /boot/efi

# Check the disks
df -h

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           393M 1016K  392M   1% /run
/dev/vda1       9.6G  1.4G  8.2G  15% /
tmpfs           2.0G     0  2.0G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/vda15      105M  5.3M  100M   5% /boot/efi
tmpfs           393M  4.0K  393M   1% /run/user/1000

Check if you selected the correct flavour:

# Check the numbers cpus
nproc

1

# Check the amount of RAM
free -h

               total        used        free      shared  buff/cache   available
Mem:           3.8Gi       193Mi       3.3Gi       0.0Ki       331Mi       3.4Gi
Swap:             0B          0B          0B

|-|VM creation Dashboard=

Overview

This page describes the steps for creating a Ubuntu 22.04 LTS (Jammy Jellyfish) VM via horizon Dashboard.

VM Creation

VM Creation - Login via Dashboard

Log in to the stoney cloud dashboard with the credentials you have received from us.

VM Creation - Launch Instance - Preperations

1. Select the appropriate project from the drop down menu at the top left.

3. On the Project tab, open the Compute tab and click Instances category.

4. Click Launch Instance.

5. A dialog box opens. Specify the values as follows:

• Instance Name: Assign a name to the virtual machine.
• Click Source tab and chose Image in Select Boot Source.
• Type Ubuntu in the search bar below and allocate the image 'Ubuntu 22.04 LTS (Jammy Jellyfish) Daily Build [20220616]' from the image list by clicking the arrow pointing up.

• Click Flavour tab and allocate the desired Flavour (For example: Standard Düdingen c001m0002) from the list below by clicking the arrow pointing up.

• Click Security Groups and allocate the security group SSH Client from the list below by clicking the arrow pointing up.

VM Creation - Launch Instance - Add Public Key

The newest Ubuntu Version does not accept ssh-rsa and ssh-dss keys by default. You must use a ed25519 key!

Click Configuration tab and paste the following into the Customization Script (Modified) field. Replace the ssh public key with your own.

#!/usr/bin/env sh
cat << EOF >> /home/ubuntu/.ssh/authorized_keys
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGusGXrY+UgTHH66YKS/o0vzUxnHVjoMzp0GnbatBzFb wahid.amiry@stepping-stone.ch
EOF

ClickLaunch Instance to create the VM. After no more than a minute the VM is created and listed in the Instances page.

VM Creation - Floating IP

In theInstances page go to the Actions column on the top right and select Associate Floating IP.

A dialog box appears. Select an available IP address and click on Associate to assign a Floating IP to the VM

If there are not available IP addresses, create a new one by clicking on the '+' sign and click on Allocate IP.

VM Creation - VM Login

Finally, you can connect to your newly created VM:

ssh ubuntu@185.85.126.29

Expected output:

The authenticity of host '185.85.126.29 (185.85.126.29)' can't be established.
ED25519 key fingerprint is SHA256:UhyGZCGExlSN8aldY+PWtnYyxiG5XGh1YRpH1IeWsdU.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '185.85.126.29' (ED25519) to the list of known hosts.
Enter passphrase for key '/home/sst-wam/.ssh/id_ed25519': 
Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.15.0-39-generic x86_64)

[...]

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@hsst-int-tmp-038:~$