Advanced level

A deep dive on the advanced services to conduct world-class research and innovation.
Target Audience: Scientific communities, developers, integrators and end users."Webinar: Introduction to the new generation EGI container execution platform" (January, 2024)

Agenda, slides and recording: https://www.egi.eu/event/webinar-introduction-to-the-container-platform/



About: The new generation EGI container execution platform is a system that delivers fully managed Kubernetes clusters for end users. The webinar covers how to build container images, how to store and share them through the Harbor repository, and how to run them on fully managed Kubernetes clusters with the Rancher service.
Target Audience: e-Infrastructure platform and technology providers and those researching or applying (distributed) deep learning at any level."Webinar: Distributed Deep Learning with Horovod" (March, 2023)

Agenda, slides and recording: https://www.egi.eu/event/webinar-distributed-deep-learning-with-horovod/



About: This webinar is focused on the Horovod distributed deep learning framework and the reference architecture and service(s) built for supporting it. The aim of the presented works is to enable the efficient utilisation of cloud resources in the heavily resource intensive task of distributed deep learning. The concept of reference architectures is also briefly presented, along with the experiences gained from their continuous development.
Target Audience: Scientific communities, developers, integrators and end users."FAIR EVA (Evaluator, Validator & Advisor)" (April, 2022)”

Agenda, slides and recording: https://indico.egi.eu/event/5876/



About: FAIR EVA (Evaluator, Validator and Advisor) has been developed to check the FAIRness level of digital objects from different repositories or data portals. It requires the object identifier and the repository to check and it can be adapted to different contexts and environments. Developed within the EOSC-Synergy project it aims at helping data producers and data managers to evaluate the adoption of the FAIR principles based on the RDA indicators. This webinar will present the tool as well as show how it can be deployed, how the different tests work and how it can be adapted to different data systems.
Target Audience: Scientific communities, developers, integrators and end users."OpenRDM - FAIR research data management as a service to the scientific community" (January, 2022)

Agenda, slides and recording: https://indico.egi.eu/event/5753/



About: openRDM.eu provides FAIR research data management. It offers research data management as a service to the scientific community, based on the powerful openBIS platform. The service is available as a preview version containing an openBIS instance. Preview is intended for end-users to learn service & eventually plan on-premise and/or, own cloud based deployment. Alternatively, self-hosting using local IT infrastructure at the respective institution can also be agreed. Consulting & support for on-premise and/or own cloud based deployment of openBIS is also offered along with user support including data model generation, to be able to import data into openBIS & training for the use of openBIS as a data management platform.

OpenBIS is designed to facilitate robust data management for a wide variety of experiment types and research subjects. It allows tracking, annotating, and sharing of data throughout distributed research projects in different quantitative sciences.
Target Audience: User communities that want to use GPUs in Clouds."How to train your AI models in EOSC", (Dec. 2021)

Agenda, slides and recording: https://indico.egi.eu/event/5742/



About: Learn how you can train and develop AI models in EOSC using the distributed and federated computing infrastructure of EGI and the services developed during the Deep Hybrid DataCloud project. The webinar, sponsored by the EGI-ACE project, covers:
  1. How to prototype, build and train AI applications exploiting resources from EU e-infrastructures.
  2. Use and share AI trained models developed by other researchers (from and outside your communities) with the DEEP Marketplace.
Target Audience: Scientific communities, developers, integrators and end users."How to orchestrate services in the EOSC Compute Platform with the INDIGO PaaS" (Oct. 2021)

Agenda, slides and recording: https://indico.egi.eu/event/5720/



About: The INDIGO PaaS implements an abstraction and federation layer on top of heterogeneous distributed computing environments: it allow to orchestrate and coordinate the provisioning of virtualized compute and storage resources on Cloud Management Frameworks, both private and public (like OpenStack, OpenNebula, AWS, etc.), and the deployment of dockerized long-running services and batch jobs on Container Orchestration Platforms like Apache Mesos and Kubernetes.

In this webinar, we will describe the architecture of the INDIGO Platform and its main features. The demo will show how users can easily interact with this orchestration system using both the command line interface and the web dashboard.
Target Audience: Scientific communities, developers, integrators and end users."Analyze your data using DODAS generated cluster" (September, 2021)

Agenda, slides and recording: https://indico.egi.eu/event/5695/



About: DODAS enables the execution of user analysis code both in batch mode and interactively via the Jupyter interface. DODAS is highly customizable and offers several building blocks that can be combined together in order to create the best service composition for a given use case. The currently available blocks allow to combine Jupyter and HTCondor as well as Jupyter and Spark or simply a jupyter interface. In addition, they allow the management of data via caches to optimise the processing of remote data. This can be done either via XCache or MinIO S3 object storage capabilities. DODAS is based on docker containers and the related orchestration relies on Kubernetes that enables the possibility to compose the building blocks via a web-based user interface thanks to Kubeapps.

In this webinar we will explain the DODAS fundamentals and we will provide a user oriented demo.
Target Audience: Scientific communities, developers, integrators and end users."Running containers in your user space with udocker" (June, 2021)

Agenda, slides and recording: https://indico.egi.eu/event/5541/



About: udocker enables the execution of docker containers in user space without requiring root privileges for installation or use. Udocker implements the pull, load, import and execution of containers by non-privileged users in Linux systems where docker is not available. It can be used in Linux batch systems and interactive clusters that are managed by other entities, such as grid infrastructures or externally managed batch or interactive systems. udocker does not require any type of privileges nor the deployment of services by system administrators. It can be downloaded and executed entirely by the end user.

udocker offers several execution modes exploiting system call interception, library call interception and namespaces. udocker integrates several tools to provide a subset of the docker capabilities aimed at container execution.

In this webinar we will explain the udocker fundamental, how to use udocker to execute Linux containers and how to best exploit the several execution engines.
Target Audience: Scientific communities, for programmers and IT-service providers who support research and education."Deploying virtual infrastructures with Infrastructure Manager (IM)" (May, 2021)

Agenda, slides and recording: https://indico.egi.eu/event/5495/



About: The Infrastructure Manager (IM) is a framework that eases the access and the usability of IaaS clouds by automating the VMI selection, deployment, configuration, software installation, monitoring and update of Virtual Appliances. It supports APIs from a large number of virtual platforms, making user applications cloud-agnostic. In addition it integrates a contextualization system to enable the installation and configuration of all the user required applications providing the user with a fully functional infrastructure. It is a service that features a web-based GUI, a XML-RPC API, a REST API and a command-line application.

In this webinar we will focus the usage of the IM Dashboard an easy to use web interface designed to enable not advanced users to deploy a set of predefined and well tested customizable virtual infrastructures (Kubernetes, SLURM, Mesos, Galaxy, …) in top of a wide range of cloud providers (EGI Cloud Compute and also commercial and open clouds – AWS, Google Cloud, Azure, OpenStack, OpenNebula, …) with a single set of mouse clicks.

IM has been developed by the Grid and High Performance Computing Group (GRyCAP) at the Instituto de Instrumentacion para Imagen Molecular (I3M) from the Universitat Politecnica de Valencia (UPV).
Target Audience: Site administrators already familiar with an earlier version of ARC CE or planning a migration to ARC from the CREAM platform."Rolling out ARC6 CE" (July 2020)

Agenda, slides and recording: https://indico.egi.eu/event/5157/



About: The Advanced Resource Connector (ARC) middleware integrates computing resources (usually, computing clusters managed by a batch system), making them available via a secure common layer. Conceptually, ARC provides an edge service to batch systems. Through this service, called ARC Compute Element (ARC-CE), scientific communities can launch and manage computational tasks in a uniform manner.

A bit more than a year ago a non backward compatible major version, the ARC6.0 was released bringing new functionality, enhanced manageability and increased stability to the community. With the availability of the ARC6 release the support for the previous ARC5 deployments are to be discontinued with the end of June 2020.

This interactive webinar will introduce the major new features of the ARC6 release and cover the deployment steps of an ARC6 CE both as a new installation and as an upgrade from a previous ARC5-based deployment. Special attention will be given to the accounting system related changes and the new one-shop-stop sysadmin toolbox built around arcctl.
Target Audience: Scientific communities, developers, integrators and end users."High performance software - Easy gains with simple CUDA" (April 2023)



About: This tutorial provides an introduction to CUDA in high performance software, covering roughly these topics:
  • Best practices for high performance software engineering, such as avoiding premature optimization, ensuring cache alignment, etc.
  • A broad introduction to GPUs, including their hardware and which categories of problems they are/aren't best suited for.
  • Installing and working with GPU frameworks
  • An overview of profiler tools and how to use them
  • A live coding session to implement and diagnose a basic CUDA program, with the level of detail dependent on available time
  • Q&A and stories from the trenches
Please note that the training will not cover multi-GPU setups or provide a detailed dive into GPU hardware and CUDA specifics. Participants should have basic knowledge of Python and matrix computation libraries like NumPy.

Slides and code
Last modified April 23, 2024 by Sebastian Luna-Valero : Add latest EGI Webinars (#652)