Command Line Interface

Command-line tools

The various public EGI services can be managed and used/accessed with a wide variety of command-line interface (CLI) tools. The documentation of each service contains a summary of the CLIs that can be used with that service, together with recommendations on which one to use in what context.

The FedCloud client

The FedCloud client is a high-level Python package for a command-line client designed for interaction with the OpenStack services in the EGI infrastructure.

FedCloud client has the following modules (features):

• Check-in allows checking validity of access tokens and listing Virtual Organisations (VOs) of a token
• Endpoint can search endpoints in the Configuration Database and extract site-specific information from unscoped/scoped tokens
• Sites allows management of site configurations
• OpenStack can perform commands on OpenStack services deployed to sites
• EC3 allows deploying elastic cloud compute clusters

Installation

The FedCloud client can be installed with the pip3 Python package manager (without root or administrator privileges).

• Linux / Mac
• Windows

$ pip3 install fedcloudclient

> pip3 install fedcloudclient

Check if the installation is correct by executing the client:

$ fedcloud --version

Installing EGI Core Trust Anchor certificates

Some sites in the EGI infrastructure use certificates issued by Certificate Authorities (CAs) that are not included in the default OS distribution. If you receive error message “SSL exception connecting to…”, install the EGI Core Trust Anchor Certificates by running the following commands:

$ wget https://raw.githubusercontent.com/tdviet/python-requests-bundle-certs/main/scripts/install_certs.sh
$ bash install_certs.sh

Using via Docker container

The FedCloud client can also be used without installation, by running it in a Docker container. In this case, the EGI Core Trust Anchor certificates are pre-installed.

• Linux
• Mac
• Windows

$ docker pull tdviet/fedcloudclient
$ docker run -it tdviet/fedcloudclient bash

$ docker pull tdviet/fedcloudclient
$ docker run -it tdviet/fedcloudclient bash

> docker pull tdviet/fedcloudclient
> docker run -it tdviet/fedcloudclient bash

Once you have a shell running in the container with the FedCloud client, usage is the same as from the command-line.

Using from EGI Notebooks

EGI Notebooks are integrated with access tokens so it simplifies using the FedCloud client. First make sure that you follow the installation steps above. Then, below are the commands that you need to run inside a terminal in JupyterLab:

export OIDC_ACCESS_TOKEN=`cat /var/run/secrets/egi.eu/access_token`
fedcloud token check

Please follow instructions below to learn how to use the fedcloud command.

Using from the command-line

The FedCloud client has these subcommands:

• fedcloud token for checking access tokens (see token subcommands)
• fedcloud endpoint for querying the Configuration Database (see endpoint subcommands)
• fedcloud site for manipulating site configurations (see site subcommands)
• fedcloud openstack or fedcloud openstack-int for performing OpenStack commands on sites (see openstack subcommands)
• fedcloud ec3 for provisioning elastic cloud compute clusters (see cluster subcommands)

Performing any OpenStack command on any site requires only three options: the site, the VO and the command. For example, to list virtual machine (VM) images available to members of VO fedcloud.egi.eu on the site CYFRONET-CLOUD, run the following command:

$ fedcloud openstack image list --vo fedcloud.egi.eu --site CYFRONET-CLOUD

Authentication

Many of the FedCloud client commands need access tokens for authentication. Users can choose whether to provide access tokens directly (via option --oidc-access-token), or generate them on the fly with oidc-agent (via option --oidc-agent-account) or from refresh tokens (via option --oidc-refresh-token, which must be provided together with option --oidc-client-id and option --oidc-client-secret).

If multiple methods of getting access tokens are given at the same time, the FedCloud client will try to get an access token from the oidc-agent first, then obtain one using the refresh token.

The default authentication protocol is openid. Users can change the default protocol via the option --openstack-auth-protocol. However, sites may have the protocol fixed in the site configuration (e.g. oidc for the site INFN-CLOUD-BARI).

The default OIDC identity provider is EGI Check-in (https://aai.egi.eu/auth/realms/egi). Users can set another OIDC identity provider via option --oidc-url.

Environment variables

Most of the FedCloud client options can be set via environment variables:

Getting help

The FedCloud client can display help for the commands and subcommands it supports. Try running the following command to see the commands supported by the FedCloud client:

$ fedcloud --help
Usage: fedcloud [OPTIONS] COMMAND [ARGS]...

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  ec3            EC3 related commands
  endpoint       endpoint command group for interaction with GOCDB and...
  openstack      Executing OpenStack commands on site and VO
  openstack-int  Interactive OpenStack client on site and VO
  site           Site command group for manipulation with site...
  token          Token command group for manipulation with tokens

Similarly, you can see help for e.g. the openstack subcommand by running the command below:

$ fedcloud openstack --help
Usage: fedcloud openstack [OPTIONS] OPENSTACK_COMMAND...

  Executing OpenStack commands on site and VO

Options:
  --oidc-client-id TEXT           OIDC client id
  --oidc-client-secret TEXT       OIDC client secret
  --oidc-refresh-token TEXT       OIDC refresh token
  --oidc-access-token TEXT        OIDC access token
  --oidc-url TEXT                 OIDC URL  [default: <https://aai.egi.eu/auth/realms/egi>]
  --oidc-agent-account TEXT       short account name in oidc-agent
  --openstack-auth-protocol TEXT  Check-in protocol  [default: openid]
  --openstack-auth-type TEXT      Check-in authentication type  [default:
                                  v3oidcaccesstoken]
  --openstack-auth-provider TEXT  Check-in identity provider  [default:
                                  egi.eu]
  --vo TEXT                       Name of the VO  [required]
  -i, --ignore-missing-vo         Ignore sites that do not support the VO
  -j, --json-output               Print output as a big JSON object
  --help                          Show this message and exit.

Using from Python

The FedCloud client can be used as a library for developing other services and tools for EGI services. Most of the functionalities can be called directly from Python code without side effects.

An usage example is available on GitHub. Just copy/download the code, add your access token and execute python demo.py to see how it works.

Using in scripts

The FedCloud client can also be used in scripts for simple automation, either for setting environment variables for other tools, or to process outputs from OpenStack commands.

Setting environment variables for external tools

Some FedCloud commands generate output that contains shell commands to set environment variables with the returned result, as exemplified below.

• Linux / Mac
• Windows
• PowerShell

$ export EGI_SITE=IISAS-FedCloud
$ export EGI_VO=eosc-synergy.eu
$ fedcloud site show-project-id --site $EGI_SITE
export OS_AUTH_URL="https://cloud.ui.savba.sk:5000/v3/";
export OS_PROJECT_ID="51f736d36ce34b9ebdf196cfcabd24ee";

$ eval $(fedcloud site show-project-id)

$ echo $OS_AUTH_URL
https://cloud.ui.savba.sk:5000/v3/

$ echo $OS_PROJECT_ID
51f736d36ce34b9ebdf196cfcabd24ee

> set EGI_SITE=IISAS-FedCloud
> set EGI_VO=eosc-synergy.eu
> fedcloud site show-project-id --site %EGI_SITE%
set OS_AUTH_URL=https://cloud.ui.savba.sk:5000/v3/
set OS_PROJECT_ID=51f736d36ce34b9ebdf196cfcabd24ee

> set OS_AUTH_URL=https://cloud.ui.savba.sk:5000/v3/
> set OS_PROJECT_ID=51f736d36ce34b9ebdf196cfcabd24ee

> set OS_AUTH_URL
OS_AUTH_URL=https://cloud.ui.savba.sk:5000/v3/

> set OS_PROJECT_ID
OS_PROJECT_ID=51f736d36ce34b9ebdf196cfcabd24ee

> $Env:EGI_SITE="IISAS-FedCloud"
> $Env:EGI_VO="eosc-synergy.eu"
> fedcloud site show-project-id --site $Env:EGI_SITE
$Env:OS_AUTH_URL="https://cloud.ui.savba.sk:5000/v3/";
$Env:OS_PROJECT_ID="51f736d36ce34b9ebdf196cfcabd24ee";

> fedcloud site show-project-id --site $Env:EGI_SITE `
                | Out-String | Invoke-Expression

> $Env:OS_AUTH_URL
https://cloud.ui.savba.sk:5000/v3/

> $Env:OS_PROJECT_ID
51f736d36ce34b9ebdf196cfcabd24ee

Processing output from OpenStack commands

The fedcloud openstack subcommand’s output can be converted to JavaScript Object Notation (JSON) format by using the --json-output option. This is useful for further machine processing of the command output.

$ export EGI_SITE=IISAS-FedCloud
$ export EGI_VO=eosc-synergy.eu
$ fedcloud openstack flavor list --site $EGI_SITE --json-output
[
{
  "Site": "IISAS-FedCloud",
  "VO": "eosc-synergy.eu",
  "command": "flavor list",
  "Exception": null,
  "Error code": 0,
  "Result": [
    {
      "ID": "0",
      "Name": "m1.nano",
      "RAM": 64,
      "Disk": 1,
      "Ephemeral": 0,
      "VCPUs": 1,
      "Is Public": true
    },
    {
      "ID": "2e562a51-8861-40d5-8fc9-2638bab4662c",
      "Name": "m1.xlarge",
      "RAM": 16384,
      "Disk": 40,
      "Ephemeral": 0,
      "VCPUs": 8,
      "Is Public": true
    },
    ...
  ]
}
]

# The following jq command selects flavors with VCPUs=2 and prints their names
$ fedcloud openstack flavor list --site IISAS-FedCloud --json-output | \
    jq -r '.[].Result[] | select(.VCPUs == 2) | .Name'
m1.medium