Documentation – Getting Started

Users: EGI Architecture

Mon, 01 Jan 0001 00:00:00 +0000

The EGI Federated Cloud (FedCloud) is a multi-national cloud system that integrates community, private and/or public clouds into a scalable computing platform for research. The Federation pools resources from a heterogeneous set of cloud providers using a single authentication and authorisation framework that allows the portability of workloads across multiple providers, and enables bringing computing to data. The current implementation is focused on Infrastructure-as-a-Service (IaaS) services, but can be easily applied to Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS) layers.

Each resource centre of the federated infrastructure operates a Cloud Management Framework (CMF) according to its own preferences and constraints and joins the federation by integrating this CMF with components of the EGI service portfolio. CMFs must at least be integrated with EGI Authentication and Authorization Infrastructure (AAI) so users can access services with a single identity, integration with other components and APIs to be provided are agreed by the community the resource centre provides services to.

EGI follows a Service Integration and Management (SIAM) approach to manage the federation with processes that cover the different aspects of the IT Service Management. Providers in the federation keep complete control of their services and resources. EGI creates Virtual Organizations (VOs) for each research community, and EGI VO Operation Level Agreements (OLAs) establish a reliable, trust-based communication channel between the community and the providers, by agreeing on the services, their levels and the types of support.

Note

EGI VO OLAs are not legal contracts but, as agreements, they outline the clear intentions to collaborate and support research.

Federated IaaS

The EGI FedCloud IaaS resource centres deploy a Cloud Management Framework (CMF) that provide users with an API-based service for management of Virtual Machines and associated Block Storage to enable persistence and Networks to enable connectivity of the Virtual Machines (VMs) among themselves and third party resources.

The IaaS federation is a thin layer that brings the providers together with:

Federated authentication
Resource discovery
Central VM image catalogue
Usage accounting
Monitoring

The IaaS capabilities (VM, block storage, network management, etc.) must be provided via community agreed APIs (OpenStack is supported at the moment) that allow integration with EGI Check-in for authentication and authorisation of users.

Note

Those providers that limit the interaction to web dashboards and do not expose APIs to direct consumption for users cannot be considered part of the EGI IaaS Cloud services.

Users and Community platforms built on top of the EGI IaaS can interact with the cloud providers at three different layers:

Directly using the IaaS APIs or CLIs to manage individual resources. This option is recommended for pre-existing use cases with requirements on specific APIs.
Using Provisioning systems allow users to define infrastructure as code, then manage and combine resources from different providers, thus enabling the portability of application deployments between them (e.g. Infrastructure Manager or Terraform).

EGI provides ready-to-use software components to enable the federation for OpenStack. These components rely on public APIs of the IaaS system and use Check-in accounts for authenticating into the provider.

Implementation

Authentication and authorization

Federated identity ensures that users of the federation can use a single account for accessing the resources.

OpenID Connect

Providers of the EGI Cloud support authentication with OAuth2 tokens provided by Check-in OpenID Connect Identity provider. Support builds on the AAI guide for SPs with detailed configuration provided at the EGI IaaS Service providers documentation.

The integration relies on the OpenStack Keystone OS-FEDERATION API.

Information discovery

The Configuration Database contains the list of resource centres and their endpoints, while the cloud information system collects this information in a central service for discovery, providing a real-time view of the actual capabilities of federation participants (can be used by both human users and machine services).

Configuration Database

The EGI Configuration Database is used to catalogue the static information of the production infrastructure topology (e.g. the list of resource centres and their endpoints).

To allow resource providers to expose IaaS federation endpoints, the following service types are available:

org.openstack.horizon
org.openstack.nova
org.openstack.swift
eu.egi.cloud.accounting
eu.egi.cloud.vm-metadata.marketplace

All providers must enter cloud service endpoints into the Configuration Database to enable integration with EGI.

The Cloud Info Provider extracts information from the resource centres using their native APIs and formats it following Glue, an OGC recommended standard. This information is consumed to provide a central information discovery service that aggregates several other sources of information about the infrastructure.

Virtual Machine Image management

In a distributed, federated IaaS service, users need solutions for efficiently managing and distributing their VM images across multiple resource providers. EGI provides a catalogue of VM images (VMIs) that allows any user to share their VMI, and communities to select those VMIs relevant for distribution across providers. These images are automatically replicated at the providers supporting the community and converted as needed to ensure the correct instantiation when used.

The Artefact Registry stores VM images, which are clean and lean images designed to run on a virtualisation platform, that provide a software solution out-of-the-box, ready to be used with minimal or no set-up.

The Registry allows research communities (VOs) to upload their own images, that that resource centres subscribe to. The subscription enables the periodic download, conversion and storage of those images to the image repository of the indicated resource centres.

Accounting

Federated Accounting provides an integrated view about resource/service usage: it pulls together usage information from the federated sites and services, integrates the data and presents them in such a way that both individual users as well as whole communities can monitor their own resource/service usage across the whole federation.

Usage of resources is gathered centrally using EGI Accounting repository and available for visualisation at EGI Accounting portal.

Cloud Usage Record

The federated cloud task force has agreed on a Cloud Usage Record, which inherits from the OGF Usage Record (GFD.204). This record defines the data that resource providers must send to EGI’s central Accounting repository.

Version 0.4 of the Cloud Accounting Usage Record was agreed at the FedCloud Face to Face in Amsterdam in January 2015. A summary table of the format is shown below:

Cloud Usage Record Property	Type	Null	Definition
VMUUID	`varchar(255)`	No	Virtual Machine's Universally Unique Identifier concatenation of CurrentTime, SiteName and MachineName
SiteName	`varchar(255)`	No	GOCDB SiteName - GOCDB now has cloud service types and a cloud-only site is allowed.
CloudComputeService (NEW)	`varchar(255)`		Name identifying cloud resource within the site. Allows multiple cloud resources within a site, i.e. a level of granularity.
MachineName	`varchar(255)`	No	VM ID - the site name for the VM
LocalUserId	`varchar(255)`		Local username
LocalGroupId	`varchar(255)`		Local group name
GlobalUserName	`varchar(255)`		Global identity of user (certificate DN)
FQAN	`varchar(255)`		Use if VOs part of authorization mechanism
Status	`varchar(255)`		Completion status - completed, started or suspended
StartTime	`datetime`		Must be set when Status = started
EndTime	`datetime`		Set to NULL until Status = completed
SuspendDuration	`datetime`		Set when Status = suspended (Timestamp)
WallDuration	`int`		WallClock time - actual time used
CpuDuration	`int`		CPU time consumed (Duration)
CpuCount	`int`		Number of CPUs allocated
NetworkType	`varchar(255)`		Needs clarifying
NetworkInbound	`int`		GB received
NetworkOutbound	`int`		GB sent
PublicIPCount (NEW)	`int`		Number of public IP addresses assigned to VM Not used.
Memory	`int`		Memory allocated to the VM
Disk	`int`		Size in GB allocated to the VM
BenchmarkType (NEW)	`varchar(255)`		Name of benchmark used for normalization of times (eg HEPSPEC06)
Benchmark (NEW)	`Decimal`		Value of benchmark of VM using ServiceLevelType benchmark’
StorageRecordId	`varchar(255)`		Link to other associated storage record Need to check feasibility
ImageId	`varchar(255)`		Every image has a unique ID associated with it. For images from the Artecfact Registry this should be `VMCATCHER_EVENT_AD_MPURI`; for images from other repositories it should be an equivalent; for local images - local identifier of the image.
CloudType	`varchar(255)`		Type of cloud infrastructure: OpenNebula; OpenStack; Synnefo; etc.

Public IP Usage Record

The fedcloud task force has agreed on an IP Usage Record. The format uses many of the same fields as the Cloud Usage Record. The Usage Record should be a "snapshot" of the number of IPs currently assigned to a user. A table defining v0.2 of the format is shown below:

Cloud Usage Record Property	Type	Null	Definition	Notes
MeasurementTime	`datetime`	No	The time the usage was recorded.	In the message format, must be a UNIX timestamp, i.e. the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970)
SiteName	`varchar(255)`	No	The GOCDB site assigning the IP
CloudComputeService	`varchar(255)`	Yes	See Cloud Usage Record
CloudType	`varchar(255)`	No	See Cloud Usage Record
LocalUser	`varchar(255)`	No	See Cloud Usage Record
LocalGroup	`varchar(255)`	No	See Cloud Usage Record
GlobalUserName	`varchar(255)`	No	See Cloud Usage Record
FQAN	`varchar(255)`	No	See Cloud Usage Record
IPVersion	`byte`	No	4 or 6
IPCount	`int(11)`	No	The number of IP addresses of IPVersion this user currently assigned to them

A JSON schema defining a valid Public IP Usage message can be found at: https://github.com/apel/apel/blob/9476bd86424f6162c3b87b6daf6b4270ceb8fea6/apel/db/__init__.py

GPU Usage Record

The fedcloud task force has agreed on an GPU Usage Record. The format uses many of the same fields as the Cloud Usage Record. A table defining Draft 4 – 24/02/2021 is shown below:

GPU Usage Record Property	Type	Null	Definition
MeasurementMonth	`int`	No	The month/year the reported usage should be assigned to. If the month/year is the current month/year, the usage should be up to the point of reporting.
MeasurementYear	`int`	No
AssociatedRecordType	`varchar(255)`	No	The context in which the reported usage was used. I.e. “cloud” for an accelerator attached to a VM.
AssociatedRecord	`varchar(255)`	No	VMUUID if AssociatedRecordType is “cloud”
GlobalUserName	`varchar(255)`	Yes	See the definition of your AssociatedRecordType
FQAN	`varchar(255)`	No	See the definition of your AssociatedRecordType
SiteName	`varchar(255)`	No	See the definition of your AssociatedRecordType
Count	`decimal`	No	A count of the Accelerators attached to the VM. At the moment Accelerators are not shared among VMs but it will change when Accelerator virtualization is applied, so we should have the field at decimal type instead of integer (e.g. Count = 0.5 when it is shared between two VMs).
Cores	`int(11)`	Yes	Total number of cores. i.e. So if an Accelerator has 64 cores and a VM has 2 like that attached then we would report: Count=2 and Processors=128
ActiveDuration	`int(11)`	Yes	Actual usage duration of the Accelerator in seconds for the given month/year (in case some systems could report actual usage). At the moment, ActiveDuration will be the same as the AvailableDuration due to the limitation of currently used technologies (impossible to get ACCELERATOR utilization from outside of the VM, no ACCELERATOR hot-plug into running VM) but it may change in near future so it is good to have the fields separately. Set to AvailableDuration if ActiveDuration is omitted from the record
AvailableDuration	`int(11)`	No	Time accelerator was available in seconds for the given month/year (Wall)Time that a GPU was attached to a VM.
BenchmarkType	`varchar(255)`	Yes	Name of benchmark used for normalization of times
Benchmark	`decimal`	Yes	Value of benchmark of Accelerator
Type	`varchar(255)`	No	High level description of accelerator, i.e. GPU, FPGA, Other
Model	`varchar(255)`	Yes	model number, spec, some other concept that 2 ACCELERATORs with the same number of cores might be different etc

APEL and accounting portal

Once generated, records are delivered to the central accounting repository using APEL SSM (Secure STOMP Messenger). SSM client packages can be obtained at https://apel.github.io. A Cloud Accounting Summary Usage Record has also been defined and summaries created on a daily basis from all the accounting records received from the Resource Providers are sent to the EGI Accounting Portal. The Accounting portal also runs SSM to receive these summaries and provides a web view of the accounting data received from the Resource Providers.

cASO delivers an implementation of the extractor probes for OpenStack.

Monitoring

The endpoints published in the Configuration Database are monitored via ARGO. Specific probes to check functionality and availability of services must be provided by service developers.

The current set of probes used for monitoring IaaS resources consists of:

Accounting probe (eu.egi.cloud.APEL-Pub): Checks if the cloud resource is publishing data to the Accounting repository
TCP checks (org.nagios.Broker-TCP, org.nagios.CDMI-TCP, and org.nagios.CloudBDII-Check): Basic TCP checks for services.

Roadmap

The TCB-Cloud board defines the roadmap for the technical evolution of the EGI Cloud. All the components are continuously maintained to:

Improve their programmability, providing complete APIs specification in adequate format for facilitating the generation clients (e.g. following the OpenAPI initiative and Swagger).
Lower the barriers to integrate and operate resource centres in the federation by a) minimizing the number of components used; b) contributing code to upstream distributions; and c) use only public APIs of the Cloud Management Frameworks.

Currently, the EGI FedCloud TaskForce is focused on moving to a central operations model, where providers only need to integrate their system with EGI Check-in but do not need to deploy and configure the different tools (accounting, discovery, VMI management, etc.) locally but delegate this to a central EGI team.

Users: Using OpenStack Providers

Mon, 01 Jan 0001 00:00:00 +0000

OpenStack providers in the EGI infrastructure offer services and features via OpenStack APIs, and the command-line interface (CLI), both integrated with EGI Check-in accounts.

The extensive OpenStack user documentation includes details on every OpenStack project, but most providers offer:

Keystone, for identity
Nova, for VM management
Glance, for VM image management
Cinder, for block storage
Swift, for object storage
Neutron, for network management
Horizon, as a web dashboard

The Horizon web-dashboard of the OpenStack providers can be used to manage and use services. It can be accessed using EGI Check-in credentials directly. Select EGI Check-In:

Or OpenID Connect:

Or egi.eu:

In the Authenticate using drop-down menu of the login screen.

Additionally you may need to select aai.egi.eu/auth/realms/egi as well:

Tip

You can quickly find the dashboards of all providers in the EGI infrastructure that are accessible to you (use the correct VO) with the FedCloud Client:

$ fedcloud endpoint list --service-type org.openstack.horizon --site ALL_SITES

The same way you can also discover other types of resources, just use the correct resource type:

org.openstack.horizon for dashboards
org.openstack.nova for virtual machines
org.openstack.swift for object storage

Note

For more advanced information discovery, including resources not based on OpenStack deployments, check out the EGI architecture summary.

Users: Command Line Interface

Mon, 01 Jan 0001 00:00:00 +0000

Command-line tools

The various public EGI services can be managed and used/accessed with a wide variety of command-line interface (CLI) tools. The documentation of each service contains a summary of the CLIs that can be used with that service, together with recommendations on which one to use in what context.

The FedCloud client

The FedCloud client is a high-level Python package for a command-line client designed for interaction with the OpenStack services in the EGI infrastructure.

Tip

The FedCloud client is the recommended command-line interface to use with most EGI services.

FedCloud client has the following modules (features):

Check-in allows checking validity of access tokens and listing Virtual Organisations (VOs) of a token
Endpoint can search endpoints in the Configuration Database and extract site-specific information from unscoped/scoped tokens
Sites allows management of site configurations
OpenStack can perform commands on OpenStack services deployed to sites
EC3 allows deploying elastic cloud compute clusters

Installation

The FedCloud client can be installed with the pip3 Python package manager (without root or administrator privileges).

Linux / Mac
Windows

To install the FedCloud client:

$ pip3 install fedcloudclient

This installs the latest version of the FedCloud client, together with its required packages (like openstackclient). It will also create executables fedcloud and openstack, adding them to the bin folder corresponding to your current Python execution environment ($VIRTUAL_ENV/bin for executing pip3 in a Python virtual environment, ~/.local/bin for executing pip3 as user (with --user option), and /usr/local/bin when executing pip3 as root).

As there are non-pure Python packages needed for installation, the Microsoft C++ Build Tools is a prerequisite, make sure it’s installed with the following options selected:

C++ CMake tools for Windows
C++ ATL for latest v142 build tools (x86 & x64)
Testing tools core features - Build Tools
Windows 10 SDK (<latest>)

In case you prefer to use non-Microsoft alternatives for building non-pure packages, please see Python Windows Compilers.

To install the FedCloud client:

> pip3 install fedcloudclient

Check if the installation is correct by executing the client:

$ fedcloud --version

Installing EGI Core Trust Anchor certificates

Some sites in the EGI infrastructure use certificates issued by Certificate Authorities (CAs) that are not included in the default OS distribution. If you receive error message “SSL exception connecting to…”, install the EGI Core Trust Anchor Certificates by running the following commands:

$ wget https://raw.githubusercontent.com/tdviet/python-requests-bundle-certs/main/scripts/install_certs.sh
$ bash install_certs.sh

Note

The above script does not work on all Linux distributions. Change python to python3 in the script if needed, see the README for more details, or follow the official instructions for installing EGI Core Trust Anchor certificates in production environments.

Using via Docker container

The FedCloud client can also be used without installation, by running it in a Docker container. In this case, the EGI Core Trust Anchor certificates are pre-installed.

To run the FedCloud client in a container, make sure Docker is installed, then run the following commands:

$ docker pull tdviet/fedcloudclient
$ docker run -it tdviet/fedcloudclient bash

To run the FedCloud client in a container, make sure Docker is installed, then run the following commands:

$ docker pull tdviet/fedcloudclient
$ docker run -it tdviet/fedcloudclient bash

To run the FedCloud client in a container, make sure Docker is installed, then run the following commands:

> docker pull tdviet/fedcloudclient
> docker run -it tdviet/fedcloudclient bash

Once you have a shell running in the container with the FedCloud client, usage is the same as from the command-line.

Using from EGI Notebooks

EGI Notebooks are integrated with access tokens so it simplifies using the FedCloud client. First make sure that you follow the installation steps above. Then, below are the commands that you need to run inside a terminal in JupyterLab:

export OIDC_ACCESS_TOKEN=`cat /var/run/secrets/egi.eu/access_token`
fedcloud token check

Please follow instructions below to learn how to use the fedcloud command.

Using from the command-line

The FedCloud client has these subcommands:

fedcloud token for checking access tokens (see token subcommands)
fedcloud endpoint for querying the Configuration Database (see endpoint subcommands)
fedcloud site for manipulating site configurations (see site subcommands)
fedcloud openstack or fedcloud openstack-int for performing OpenStack commands on sites (see openstack subcommands)
fedcloud ec3 for provisioning elastic cloud compute clusters (see cluster subcommands)

Note

See also the complete documentation or read and contribute to the source code.

Performing any OpenStack command on any site requires only three options: the site, the VO and the command. For example, to list virtual machine (VM) images available to members of VO fedcloud.egi.eu on the site CYFRONET-CLOUD, run the following command:

$ fedcloud openstack image list --vo fedcloud.egi.eu --site CYFRONET-CLOUD

Authentication

Many of the FedCloud client commands need access tokens for authentication. Users can choose whether to provide access tokens directly (via option --oidc-access-token), or generate them on the fly with oidc-agent (via option --oidc-agent-account) or from refresh tokens (via option --oidc-refresh-token, which must be provided together with option --oidc-client-id and option --oidc-client-secret).

Tip

Users of EGI Check-in can get a Check-in client ID and refresh token, as well as all the information needed to obtain access tokens for their FedCloud client, by visiting EGI Check-in Token Portal.

Tip

To provide access tokens automatically via oidc-agent, follow these instructions to register a client, then pass the client name (account name used during client registration) to the FedCloud client via option --oidc-agent-account.

Important

Refresh tokens have long lifetime (one year in EGI Check-in), so they must be properly protected. Exposing refresh tokens via environment variables or command-line options is considered insecure and will be disabled in the near future in favour of using oidc-agent.

If multiple methods of getting access tokens are given at the same time, the FedCloud client will try to get an access token from the oidc-agent first, then obtain one using the refresh token.

The default authentication protocol is openid. Users can change the default protocol via the option --openstack-auth-protocol. However, sites may have the protocol fixed in the site configuration (e.g. oidc for the site INFN-CLOUD-BARI).

The default OIDC identity provider is EGI Check-in (https://aai.egi.eu/auth/realms/egi). Users can set another OIDC identity provider via option --oidc-url.

Note

Remember to also set the identity provider’s name accordingly for OpenStack commands, by using the option --openstack-auth-provider.

Environment variables

Most of the FedCloud client options can be set via environment variables:

Tip

To save a lot of time, set the frequently used options like access token, VO, etc. using environment variables.

Tip

When you want commands to work on all sites in the EGI infrastructure, use ALL_SITES for the --site parameter.

Environment variable	Command-line option	Default value
OIDC_AGENT_ACCOUNT	`--oidc-agent-account`
OIDC_ACCESS_TOKEN	`--oidc-access-token`
OIDC_REFRESH_TOKEN	`--oidc-refresh-token`
OIDC_CLIENT_ID	`--oidc-client-id`
OIDC_CLIENT_SECRET	`--oidc-client-secret`
OIDC_URL	`--oidc-url`	https://aai.egi.eu/auth/realms/egi
OPENSTACK_AUTH_PROTOCOL	`--openstack-auth-protocol`	openid
OPENSTACK_AUTH_PROVIDER	`--openstack-auth-provider`	egi.eu
OPENSTACK_AUTH_TYPE	`--openstack-auth-type`	v3oidcaccesstoken
EGI_VO	`--vo`

Getting help

The FedCloud client can display help for the commands and subcommands it supports. Try running the following command to see the commands supported by the FedCloud client:

$ fedcloud --help
Usage: fedcloud [OPTIONS] COMMAND [ARGS]...

Options:
 --version Show the version and exit.
 --help Show this message and exit.

Commands:
 ec3 EC3 related commands
 endpoint endpoint command group for interaction with GOCDB and...
 openstack Executing OpenStack commands on site and VO
 openstack-int Interactive OpenStack client on site and VO
 site Site command group for manipulation with site...
 token Token command group for manipulation with tokens

Similarly, you can see help for e.g. the openstack subcommand by running the command below:

$ fedcloud openstack --help
Usage: fedcloud openstack [OPTIONS] OPENSTACK_COMMAND...

 Executing OpenStack commands on site and VO

Options:
 --oidc-client-id TEXT OIDC client id
 --oidc-client-secret TEXT OIDC client secret
 --oidc-refresh-token TEXT OIDC refresh token
 --oidc-access-token TEXT OIDC access token
 --oidc-url TEXT OIDC URL [default: <https://aai.egi.eu/auth/realms/egi>]
 --oidc-agent-account TEXT short account name in oidc-agent
 --openstack-auth-protocol TEXT Check-in protocol [default: openid]
 --openstack-auth-type TEXT Check-in authentication type [default:
 v3oidcaccesstoken]
 --openstack-auth-provider TEXT Check-in identity provider [default:
 egi.eu]
 --vo TEXT Name of the VO [required]
 -i, --ignore-missing-vo Ignore sites that do not support the VO
 -j, --json-output Print output as a big JSON object
 --help Show this message and exit.

Note

Most commands support multiple levels of subcommands, you can get help for all of them using the same principle as above.

Using from Python

The FedCloud client can be used as a library for developing other services and tools for EGI services. Most of the functionalities can be called directly from Python code without side effects.

An usage example is available on GitHub. Just copy/download the code, add your access token and execute python demo.py to see how it works.

Using in scripts

The FedCloud client can also be used in scripts for simple automation, either for setting environment variables for other tools, or to process outputs from OpenStack commands.

Setting environment variables for external tools

Some FedCloud commands generate output that contains shell commands to set environment variables with the returned result, as exemplified below.

Run a command to get details of a project:

$ export EGI_SITE=IISAS-FedCloud
$ export EGI_VO=eosc-synergy.eu
$ fedcloud site show-project-id --site $EGI_SITE
export OS_AUTH_URL="https://cloud.ui.savba.sk:5000/v3/";
export OS_PROJECT_ID="51f736d36ce34b9ebdf196cfcabd24ee";

Run the same command but set environment variables with the returned values:

$ eval $(fedcloud site show-project-id)

The environment variables will have their values set to what the command returned:

$ echo $OS_AUTH_URL
https://cloud.ui.savba.sk:5000/v3/

$ echo $OS_PROJECT_ID
51f736d36ce34b9ebdf196cfcabd24ee

Run a command to get details of a project:

> set EGI_SITE=IISAS-FedCloud
> set EGI_VO=eosc-synergy.eu
> fedcloud site show-project-id --site %EGI_SITE%
set OS_AUTH_URL=https://cloud.ui.savba.sk:5000/v3/
set OS_PROJECT_ID=51f736d36ce34b9ebdf196cfcabd24ee

If you copy the returned output and execute it as commands in a command prompt:

> set OS_AUTH_URL=https://cloud.ui.savba.sk:5000/v3/
> set OS_PROJECT_ID=51f736d36ce34b9ebdf196cfcabd24ee

The environment variables will have their values set to what the command returned:

> set OS_AUTH_URL
OS_AUTH_URL=https://cloud.ui.savba.sk:5000/v3/

> set OS_PROJECT_ID
OS_PROJECT_ID=51f736d36ce34b9ebdf196cfcabd24ee

Run a command to get details of a project:

> $Env:EGI_SITE="IISAS-FedCloud"
> $Env:EGI_VO="eosc-synergy.eu"
> fedcloud site show-project-id --site $Env:EGI_SITE
$Env:OS_AUTH_URL="https://cloud.ui.savba.sk:5000/v3/";
$Env:OS_PROJECT_ID="51f736d36ce34b9ebdf196cfcabd24ee";

Run the same command but set environment variables with the returned values:

> fedcloud site show-project-id --site $Env:EGI_SITE `
 | Out-String | Invoke-Expression

The environment variables will have their values set to what the command returned:

> $Env:OS_AUTH_URL
https://cloud.ui.savba.sk:5000/v3/

> $Env:OS_PROJECT_ID
51f736d36ce34b9ebdf196cfcabd24ee

Processing output from OpenStack commands

The fedcloud openstack subcommand’s output can be converted to JavaScript Object Notation (JSON) format by using the --json-output option. This is useful for further machine processing of the command output.

Tip

JSON output can be processed with a tool like jq, which can slice, filter, map, and transform structured data. It acts as a filter: it takes an input and produces an output. Check out the tutorial for using it to extract data from JSON sources.

$ export EGI_SITE=IISAS-FedCloud
$ export EGI_VO=eosc-synergy.eu
$ fedcloud openstack flavor list --site $EGI_SITE --json-output
[
{
 "Site": "IISAS-FedCloud",
 "VO": "eosc-synergy.eu",
 "command": "flavor list",
 "Exception": null,
 "Error code": 0,
 "Result": [
 {
 "ID": "0",
 "Name": "m1.nano",
 "RAM": 64,
 "Disk": 1,
 "Ephemeral": 0,
 "VCPUs": 1,
 "Is Public": true
 },
 {
 "ID": "2e562a51-8861-40d5-8fc9-2638bab4662c",
 "Name": "m1.xlarge",
 "RAM": 16384,
 "Disk": 40,
 "Ephemeral": 0,
 "VCPUs": 8,
 "Is Public": true
 },
 ...
 ]
}
]

# The following jq command selects flavors with VCPUs=2 and prints their names
$ fedcloud openstack flavor list --site IISAS-FedCloud --json-output | \
 jq -r '.[].Result[] | select(.VCPUs == 2) | .Name'
m1.medium

Note

Note that --json-output option can be used only with those OpenStack commands that have outputs. Using this parameter with commands with no output (e.g. setting properties) will generate an unsupported parameter error.

Users: Community Specific Documentation

Mon, 01 Jan 0001 00:00:00 +0000

In the following sections you can find documentation for communities that are using EGI services. These will typically cover an introduction of the scientific or research community, the procedure to gain access to the resources of the community, and guidelines about how to participate in the community.