Documentation – Joining EGI as a provider

Providers: Joining as a Federated Resource Centre

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

A Resource Centre (RC) is the smallest resource administration domain in the EGI Federation. It can be either localised or geographically distributed and provides a minimum set of local or remote IT Services compliant with well-defined IT Capabilities (HTC, Cloud, Storage, etc.) necessary to make resources accessible to Users. EGI is a Resource Infrastructure federating RCs to constitute a homogeneous operational domain.

Registration and certification

Note

In order to join the EGI Infrastructure, a RC needs to present the request to the Research Infrastructure Provider (RP) existing in its country. A RP is a legal organisation, part of the EGI Resource Infrastructure, responsible for managing and operating a number of operational services at national level, supporting EGI RCs and user communities. Please have a look at the Operations Start Guide to get familiar with the terms mentioned above, and to have a complete picture of the several actors participating in our landscape.

The RP operators are going to guide and support the RC during the registration and certification procedures.

Firstly, the RC will be asked to read, understand, and accept:

the RCs Operational Level Agreement (OLA), an agreement made between the RC and its RP that defines the minimum set of operational services and the respective quality parameters that a Resource Centre is required to provide in EGI;
the Security Policies defined in EGI to guarantee that all the security aspects with the service delivery are fulfilled and enforced.

Secondly, the RC should be registered in the EGI Configuration Database: the provided information, from the generic contacts and roles of people to the service endpoints details, is needed to trigger the daily operations of other services and activities provided by the EGI Infrastructure such as the Monitoring of the resources, the Accounting, the Support, and the Security activities.

Once the entry in the Configuration Database is complete, the RP changes the RC status from “Candidate” to “Uncertified”, and the certification procedure can start: it comprises a series of technical controls to verify that the provided services work according to the expectations defined in the RC OLA. Any identified issue is notified by the RP operators to the RC and investigated until its solution.

When all the certification controls are successfully passed, the RC status is changed to “Certified” meaning that the RC is included in the EGI production infrastructure and its resources can be consumed by the users of the infrastructure.

Resource Centres responsibilities

Incidents and service requests

Providing support is a fundamental part of the daily activity of a provider participating in a large research infrastructure such as EGI. The support is not only for the users accessing the resources but also for those who are involved in the management and oversight of the infrastructure.

As defined in the RC OLA, the RC will handle incidents and service requests registered as tickets in the EGI Helpdesk service, with the expectation to acknowledge and process any notified issue, within the agreed response time associated with the priority of the ticket.

The response time is defined by the Quality of Support levels, and for the RCs the level will be Medium, meaning that there will 4 priorities for the incidents (requiring for example up to 5 working days for the “less urgent” tickets and up to 1 working day for the “top priority” ones), while any service request will be processed as “less urgent” ticket.

Security topics

The security posture of the infrastructure is framed by the set of policies constituting the Security Policies. Those policies cover different complementary activities including the operation of services, the processing of personal data and the management of security incidents and vulnerabilities.

Dealing with security incidents

The Security Incident Response Policy aims at coordinating the incident response across the infrastructure, ensuring that that incidents are promptly reported, and that all incidents are investigated as fully as possible.

Security incidents are to be treated as serious matters and their investigation must be resourced appropriately.

Resources Centres must report suspected security incidents to their RP Security Officer and to EGI Computer Security Incident Response Team (CSIRT) within 4 hours of discovery. This initial step will start the coordination of the incident response as documented in the procedure SEC01 EGI CSIRT Security Incident Handling Procedure.

This procedure has been implemented according to the Security Incident Response Policy, to minimise the impact of security incidents affecting the Resource Centres part of the infrastructure
It covers guidance on how the incident response should be coordinated, describing the responsibilities of the various parties, and encourages post-mortem analysis and promotes cooperation between Resource Centres.

Handling of vulnerabilities

The handling of vulnerabilities is a very formal process involving many different entities.

Anyone can report a software vulnerability via a form or by email contacting the Software Vulnerability Group (SVG).

The report will trigger an assessment, following the SEC02 Software Vulnerability Issue Handling procedure, by the Software Vulnerability Group, of the risk level associated with this vulnerability in the context of the activities of the EGI Infrastructure.

Once a vulnerability has been identified as presenting a risk to the infrastructure, it will be decided if an advisory should be prepared and circulated to the security contacts of the sites. After an agreed period of time, and depending on their confidentiality, advisories are made public.

Vulnerabilities identified as critical are handled according to the procedure SEC03 EGI-CSIRT Critical Vulnerability Handling. When applicable, this usually involves developing a custom security monitoring probe created to identify on High Throughput Compute RCs if their resources are vulnerable to the vulnerability. The status is closely monitored by the security team and accessible to the affected RCs.

Using this information correlated with the one from Pakiti, the patch management service collecting information about the patches deployed at the various High Throughput Compute RCs, the Incident Response Task Force (IRTF) on duty Security Officer will open tickets against the impacted sites according to the WI07 Security Vulnerability Handling procedure.

The EGI Service Delivery and Information Security (SDIS) team of the EGI Foundation, formerly known as the EGI Operations team, will follow up with the resource provider to work on resolving the ticket. The first duty of the resource provider is to acknowledge the vulnerability and then work on a prompt resolution as suggested in the ticket. In case a satisfactory resolution is not reached in due time, or a sign of active progress on addressing the vulnerability is not visible, the specific Resource Centre may be suspended.

Serving user communities

Once part of the production infrastructure, the RC is ready to deliver its resources to any of the users’ communities consuming the infrastructure.

The RC can continue serving local user communities, and at the same time deliver capacity for international user communities that approach EGI and therefore reach the federated RCs.

International communities reach EGI through the following channels:

The EGI site where they can request access to services.
The EGI-ACE Call for Use Cases.

Service and Operation Level Agreements (SLAs, OLAs)

The User Community Support team of the EGI Foundation receives these requests and negotiates the details of access, with the involvement of relevant and ‘fit-for-purpose’ RCs. This Service Level Management (SLM) process intervenes as a matchmaker between service expectations and needs of the user communities, acting as ‘customers’, and the capabilities of the RCs. Customers are enabled access to the resources in the form of Virtual Organisations (VOs).

In order to select providers for provisioning services to a given customer, technical requirements are collected from the customer then transferred to relevant providers. The Expression of Interests for support (EoIs) are collected from the interested providers during the negotiation phase, resulting in the best match with customer’s requirements and expectations (both technical and financial). Several aspects are considered during the negotiation phase, including the geographical location of the customer, national roadmap and priority of the providers, and costs of the service provisioning in case of a pay-for-use model.

The result of the negotiation is a ‘Service Level Agreement’ (SLA), and several ‘Operation Level Agreements’ (OLA), one with each contributing provider. (See SLAs-OLAs examples.)

SLAs and OLAs are typically signed for at least 1 year, and are automatically renewed, as long as the provider(s) or the customer do not express a decision to terminate the Agreement at least a month before the expiration date.

Performance reports: enforcing OLAs

As defined in the RCs OLA, the performance of the delivered services should meet the Service Level Targets: the monthly performance of the RCs is monitored, and when the targets are not achieved for three consecutive months, the affected RCs are notified through a ticket about the OLA violation and requested to provide within 10 working days an explanation for the low performance and a plan for improvement.

The RCs not providing a satisfactory explanation or not replying at all are eligible for suspension.

In order to re-join the EGI Infrastructure, any suspended RC should undergo a new certification procedure. The “Suspended” status cannot last for more than 4 months, after which a RCs is either in production again or definitely closed.

Besides the Targets defined in the RCs OLA, which are enforced to guarantee the permanence of the RC in the infrastructure, the targets promised to the users in the VO SLAs should also be met on a monthly basis: when a violation occurs, the RC is requested to provide a justification and a plan for improving the quality of the provided services.

If repeated violations occur, the SLA can be renegotiated with the customer, either by changing the Service Level Targets, or by choosing a different RCs as a provider.

Providers: Joining as a Technology Provider

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Technology Providers (TP) develop or deliver technology and software for specific user communities or customisation for specific requirements. In EGI case, they maintain the middleware which the RCs install and allowing the users to exploit the compute, storage, data, and cloud resources.

Integration of middleware stack

To maintain an appropriate quality of service of the EGI production infrastructure, every middleware stack (Compute, Storage, etc.) installed on and delivered by the RCs must fulfil a number of requirements.

Note

PROC19 was defined to ensure that any single aspect of the integration of the new piece of middleware with the infrastructure is reviewed and assessed.

After the creation of the request in the EGI Helpdesk, with details about the technology, the contacts, the expected customers, and the motivation, the integration steps cover the following areas (where possible, steps can be done in parallel):

Underpinning Agreement (UA) between EGI Foundation and the technology provider
- it could be either the Corporate-level Technology Provider Underpinning Agreement or a customised version.
Configuration Management: mapping of the new technology in the Configuration Management Database (CMDB)
Information System: evaluating if the new technology should publish information in the Information System according to the GLUE Schema.
Monitoring: the new technology should allow external monitoring. If particular aspects of the technology need to be monitored, specific monitoring probes should be provided by the TPs and deployed on the EGI Monitoring service.
Support: the Support Unit where incidents and service requests will be addressed needs to be defined in the EGI Helpdesk, associated to the Quality of Support defined in the UA.
Accounting: the need to gather usage data, which depends on the technology itself and on the infrastructure requirements and will be published in the EGI Accounting Portal.
Integration in UMD or CMD: the Unified Middleware Distribution (UMD) and Cloud Middleware Distribution (CMD) are the integrated sets of software components contributed by Technology Providers and packaged for deployment as production quality services in EGI.
Documentation: exhaustive documentation for RC administrators and users should be provided and may be added to the EGI Documentation.
Security: a security assessment of the software is required according to a number of guidelines defined by the EGI Security team.

Providers: Joining as a provider for existing EGI services

Mon, 01 Jan 0001 00:00:00 +0000

To become a provider of one of the Services for Research already included in the EGI Portfolio, you first have to become provider of a federated resource centre. This first step typically requires you to become either an EGI High Throughput Compute (HTC) provider, by deploying Compute and Storage capabilities, or a Cloud service provider, by deploying OpenStack and federating it into the EGI Cloud Compute service.

Once such a foundational role is fulfilled you can configure/deploy the additional service within your resource centre.

Providers: Joining as a provider of a Core Service

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

With the term “Central” or “Core” Service we refer to a category of services in the EGI Infrastructure providing capabilities that support and complement the other services of the infrastructure and the related activities. A list of those services is available in the section Services for Federation of our site.
The EGI Core Services are co-funded by EGI Foundation and the providers are selected through a bidding process: the technical details of the services that should be delivered are advertised to the EGI Council, and only the providers with the EGI Participant role in the Council can apply to the bid.
- How to join the EGI Federation as a member of the Council.
- Current members of the EGI Council
Differently from services with a distributed nature such as HTC, Cloud, and Storage, they cannot be ordered through the Marketplace, but they become available as soon as a user joins the infrastructure (e.g., the access to the EGI Helpdesk service).

Selection of the providers and registration

With the selected providers, EGI Foundation negotiates and signs an OLA defining the terms and conditions for the delivery of the services; at the same time, the process to add the services to the EGI Service portfolio is started, if the service is not already included.

At this point, steps, similar to the ones for Resource Centers and Technology Providers, follow in order to guarantee the regular day-to-day operation of the service, such as:

registration in the Configuration Database and certification;
definition of the Support Unit in the Helpdesk system to handle incidents and service requests;
enabling of the monitoring;
periodic performance reports as defined in the given OLA to verify that the Service Level Targets are achieved.

In addition to this, the providers are also requested to create a capacity plan, an availability and continuity plan, and to interact with Change Management (CHM) and Release and Deployment Management (RDM) processes for managing changes and new releases of their services.

Capacity plan

The capacity plan is important to assess if the capacity of the service is sufficient to respond to the current and to the future demand of the service.

Any capacity aspect of the service delivery is analysed (human, technical, and financial) with the definition of quantitative parameters to measure the usage and the load of the service. The approach to adjust the capacity of the service in relation to a change in the demand is defined, and recommendations on capacity requirements for the next reporting period are provided as well.

EGI Foundation defined a template for the Capacity plans and contacts and supports the service providers for the creation of a new Capacity plan or for their regular reviews (the reviews are usually performed at least twice per year).

Availability and continuity plan

In the Availability and Continuity plan, a number of risks affecting the availability and continuity of the service is identified and assessed: each risk is rated in terms of likelihood and impact with the definition of countermeasures to implement that should avoid the occurrence of the given risk. Any remaining vulnerability is identified as well, and in case the rating of a risk is considered to be too high in relation to the risk acceptance criteria, a plan either to improve the existing countermeasures or to implement new ones is created, with the aim either to reduce the likelihood of a risk or to mitigate the impact in case a risk occurs.

The plan is completed by a continuity and recovery test, where the continuity of the service and its recovery capacity are tested against a simulated disruption scenario: the performance of this test is useful to spot any issue in the recovery procedures of the service.

Also in this case, the discussion of the Availability and Continuity plan is started and overseen by EGI Foundation who shares with the providers a template that will be filled in with the details of the given service. Availability and Continuity plans are reviewed on a yearly basis.

Managing changes and new releases

All changes to the services should follow the EGI Change Management (CHM) process according to the Change Policy, in order to evaluate the potential impact that a change can have on the service and on the infrastructure as a whole.

When registering a Request for Change, besides a general description of the change, the providers are expected to provide:

the risk level as a result of assessing the impact and the likelihood of things going wrong,
the type of change,
the eventual list of other services potentially affected by the change,
if it is possible to revert the change,
the proposed date for the implementation of the change,
if it is needed to schedule a downtime of the service.

Requests for Changes are assessed by the Change Advisory Board (CAB), deciding whether the Change is going to be approved or rejected.

For recurrent changes with a relatively low risk level, the providers can request to classify them as Standard Changes, so that invoking the CAB won’t be necessary and they can undergo the release process after the release plan is agreed with the EGI CHM staff.

The Emergency Changes are created when there is an urgent need to fix either a newly discovered security vulnerability or a critical issue: in this case the formal approval of the CAB is not required, and it is enough that the release plan is accepted by CHM staff.

In all of the other situations the changes are classified as Normal and depending on their risk level they might need to be assessed by the CAB.

Before the release in the production environment, the providers may invite the users or other impacted service providers, to test the new changes on a pre-production instance in order to gather feedback and to find out possible issues that might be overlooked during the preparation of the new release. If no problems are found, the release can go live, and a Post Implementation Review is conducted after a few days to close the case.

Security aspects

The providers of central services are subject to the same policies, procedures and requirements applying to the federated service providers, as documented at this page.

Nevertheless, they are also subject to the following requirement that is documented in the service OLA that is being agreed with them:

They should immediately report suspected security incidents to the EGI Foundation. This is not exempting them to follow the SEC01 EGI CSIRT Security Incident Handling Procedure and inform EGI CSIRT within 4 hours.

It’s also important to understand that when processing of personal data is taking place, EGI Foundation holds the role of Data Controller and the provider is a Data Processor, as defined in the Global Data Protection Regulation (GDPR).

Data Processing Agreements, regulating the conditions and constraints of the data processing activities conducted by the Data Processor on behalf of the Data controller, will be put in place on the initiative of the EGI Foundation staff.
EGI Foundation will also prepare, together with the service provider, adequate Privacy Notice and Acceptable Use policy that have to be presented and made available to the users of the service.
EGI Foundation is also complying with all the principles set out by the REFEDS Data Protection Code of Conduct in its most recent version, implying that the central service provider should also comply with them.

The central service provider should also follow requirements relating to the software and covering the usage of a proper licence, the access to and management of the source code, implementation of best practices; as well as requirements relating to the IT Service management and covering the need for having key staff properly trained about IT Service Management, committing to continual improvement and having their Service Management System (SMS) interfacing with the EGI SMS, especially for important processes like the Change Management process.

Providers: Introducing a new EGI service

Mon, 01 Jan 0001 00:00:00 +0000

The EGI services are contributed by the EGI Council members, who can propose new services through the EGI Service Strategy, that defines the overall direction to EGI services for a few years time window. The services are designed, validated with the EGI Community, and with the help of the EGI Services and Solutions Board (SSB).

The evolution of the services is driven by 5 Strategic Objectives (SO) which are based on the knowledge of the user community needs that are being gathered through research and innovation projects, customer interviews and analysis of trends. Each defines an area for possible future work, and if approved, can be developed into more concrete goals and intended results.

SO1: Federated compute continuum
SO2: Federated Data Lakes and repositories
SO3: Data analytics and scientific tools including AI
SO4: Professional support and consultancy
SO5: Investigation of a trusted compute platform for sensitive data processing

If you want to propose a new service, you should approach your national EGI Council member, or fallback to the EGI SSB.