Using OpenStack Providers
OpenStack providers of the EGI Cloud Compute service offer native OpenStack features via native APIs integrated with EGI Check-in accounts.
The extensive OpenStack user documentation includes details on every OpenStack project most providers offer access to:
- Keystone, for identity
- Nova, for VM management
- Glance, for VM image management
- Cinder, for block storage
- Swift, for object storage
- Neutron, for network management
- Horizon, as a web dashboard
The Horizon Web-dashboard of the OpenStack providers can be accessed using your EGI Check-in credentials directly. See the Getting Started guide for more information. The rest of this guide will focus on CLI/API access.
Installation
The OpenStack client is a command-line client for OpenStack that brings the command set for Compute, Identity, Image, Object Storage and Block Storage APIs together in a single shell with a uniform command structure.
Installation of the OpenStack client can be done using:
pip install openstackclient
As there are non-pure Python packages needed for installation, the Microsoft C++ Build Tools is a prerequisite, please make sure it’s installed with the following options selected:
- C++ CMake tools for Windows
- C++ ATL for latest v142 build tools (x86 & x64)
- Testing tools core features - Build Tools
- Windows 10 SDK (
<latest>)
In case you prefer to use non-Microsoft alternatives for building non-pure packages, please see here.
Installation of the OpenStack client can be done using:
pip install openstackclient
Add IGTF CA to python's CA store:
cat /etc/grid-security/certificates/*.pem >> $(python -m requests.certs)
Authentication
Check the documentation at the authentication and authorisation section on how to get the right credentials for accessing the providers.
OpenStack token for other clients
Most OpenStack clients allow authentication with tokens, so you can easily use them with EGI Cloud providers just doing a first step for obtaining the token. With the OpenStack client you can use the following command to set the OS_TOKEN variable with the needed token:
$ OS_TOKEN=$(openstack --os-auth-type v3oidcaccesstoken \
--os-protocol openid --os-identity-provider egi.eu \
--os-auth-url <keystone url> \
--os-access-token <your access token> \
--os-project-id <your project id> token issue -c id -f value)
You can easily obtain an OpenStack token with the FedCloud client:
fedcloud openstack --site <NAME_OF_THE_SITE> --vo <NAME_OF_VO> token issue -c id -f value
Useful commands with OpenStack CLI
Usage of the OpenStack client is described in detail here.
Please refer to the nova documentation for a complete guide on the VM management features of OpenStack. We list in the sections below some useful commands for the EGI Cloud.
Registering an existing ssh key
It’s possible to register an ssh key that can later be used as the default ssh
key for the default user of the VM (via the --key-name argument to
openstack server create):
openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey
Creating a VM
openstack flavor list
FLAVOR=<FLAVOR_NAME>
openstack image list
IMAGE_ID=<IMAGE_ID>
openstack network list
# Pick FedCloud network
NETWORK_ID=<NETOWRK_ID>
openstack security group list
openstack server create --flavor $FLAVOR --image $IMAGE_ID \
--nic net-id=$NETWORK_ID --security-group default \
--key-name mykey oneprovider
# Creating a floating IP
openstack floating ip create <NETOWRK_NAME>
# Assigning floating IP to server
openstack server add floating ip <SERVER_ID> <IP>
# Removing floating IP from server
openstack server show <SERVER_ID>
# Deleting server
openstack server remove floating ip <SERVER_ID> <IP>
openstack server delete <SERVER_ID>
# Deleting floating IP
openstack floating ip delete <IP>
Using cloud-init
openstack server create --flavor m3.medium \
--image d0a89aa8-9644-408d-a023-4dcc1148ca01 \
--user-data userdata.txt --key-name My_Key server01.example.com
Shell script data as user data
#!/bin/sh
adduser --disabled-password --gecos "" clouduser
cloud-config data as user data
#cloud-config
hostname: mynode
fqdn: mynode.example.com
manage_etc_hosts: true
Creating a snapshot image from running VM
You can create a new image from a snapshot of an existing VM that will allow you to easily recover a previous version of your VM or to use it as a template to clone a given VM.
openstack server image create <your VM> --name <name of the snapshot>
Once the snapshot is ready openstack image show <name of the snapshot> will
give your the details you can use it as any other image at the provider:
openstack server create --flavor <flavor> \
--image <name of the snapshot> \
<name of the new VM>
You can override files in the snapshot if needed, e.g. changing the SSH keys:
openstack server create --flavor <flavor> \
--image <name of the snapshot> \
--file /home/ubuntu/.ssh/authorized_keys=my_new_keys \
<name of the new VM>
Terraform
Terraform supports EGI Cloud OpenStack providers by using valid access tokens for Keystone. For using this, just configure your provider as usual in Terraform, but do not include user/password information. Instead, use the FedCloud client client to configure environment variables as follows:
# export OS_AUTH_URL and OS_PROJECT_ID with
$ eval "$(fedcloud site show-project-id --site <NAME_OF_SITE> --vo <NAME_OF_VO>)"
# now get a valid token
$ export OS_TOKEN=$(fedcloud openstack --site <NAME_OF_SITE> --vo <NAME_OF_VO> \
token issue -c id -f value)
Here is a sample main.tf configuration file for Terraform:
terraform {
required_providers {
openstack = {
source = "terraform-provider-openstack/openstack"
}
}
}
# Create a server
resource "openstack_compute_instance_v2" "vm" {
name = "testvm"
image_id = "..."
flavor_id = "..."
security_groups = ["default"]
}
Initialize Terraform with:
$ terraform init
Now check the deployment plan:
$ terraform plan
If you are happy with the plan, perform the deployment with:
$ terraform apply
For more information on how to use Terraform with OpenStack please check the OpenStack provider documentation.
libcloud
Apache libcloud supports OpenStack and
EGI authentication mechanisms by setting the ex_force_auth_version to
3.x_oidc_access_token or 2.0_voms respectively. Check the
libcloud docs on connecting to OpenStack
for details. See below two code samples for using them
OpenID Connect
import requests
from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver
refresh_data = {
'client_id': '<your client_id>',
'client_secret': '<your client_secret>',
'grant_type': 'refresh_token',
'refresh_token': '<your refresh_token>',
'scope': 'openid email profile',
}
r = requests.post("https://aai.egi.eu/auth/realms/egi/protocol/openid-connect/token",
auth=(client_id, client_secret),
data=refresh_data)
access_token = r.json()['access_token']
OpenStack = get_driver(Provider.OPENSTACK)
# first parameter is the identity provider: "egi.eu"
# Second parameter is the access_token
# The protocol 'openid' is specified in ex_tenant_name
# and tenant/project cannot be selected :(
driver = OpenStack('egi.eu', access_token, ex_tenant_name='openid',
ex_force_auth_url='https://keystone_url:5000',
ex_force_auth_version='3.x_oidc_access_token')
VOMS
from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver
OpenStack = get_driver(Provider.OPENSTACK)
# assume your proxy is available at /tmp/x509up_u1000
# you can obtain a proxy with the voms-proxy-init command
# no need for username
driver = OpenStack(None, '/tmp/x509up_u1000', ex_tenant_name='EGI_FCTF',
ex_force_auth_url='https://sbgcloud.in2p3.fr:5000',
ex_force_auth_version='2.0_voms')