Users: FedLLM

Mon, 01 Jan 0001 00:00:00 +0000

Overview

FedLLM is an EGI service that provides access to Large Language Models (LLMs) through:

A web-based chat interface: chat.ai.egi.eu
An OpenAI-compatible API: llm.ai.egi.eu

The service enables users to generate and analyse text, run retrieval-augmented generation (RAG) workflows, and integrate AI capabilities into scientific applications.

Access and Authentication

Access is managed through EGI Check-in:

Create an EGI account
Enrol in a supported Virtual Organisation (VO) such as vo.fedllm.egi.eu
Access the service via chat.ai.egi.eu
To use the API, generate an API key from the web interface (Account settings → Create a new secret key)
Use the API key to access the llm.ai.egi.eu API endpoints

Using AI Models

Web Interface

After logging into the web interface:

Select a model from the dropdown menu
Enter your prompt in the input field
Submit the request to receive a response

API Usage

You can retrieve a list of available models:

curl https://llm.ai.egi.eu/v1/models \
 -H "Authorization: Bearer $API_KEY" | jq .

You can then send a request using one of the available models:

curl https://llm.ai.egi.eu/v1/chat/completions \
 -H "Authorization: Bearer $API_KEY" \
 -H "Content-Type: application/json" \
 -d '{
 "model": "gpt-oss-120b",
 "messages": [
 { "role": "user", "content": "Hello!" }
 ]
 }' | jq .

For detailed API specifications, refer to the API reference. Note that not all endpoints are supported.

For client access and development, refer to the LiteLLM SDK or the OpenAI Client Libraries.

High-level Service architecture

The FedLLM service is composed of the following components:

Web interface (Open WebUI): user interaction and API key management
API gateway (LiteLLM): OpenAI-compatible interface and routing
Inference backend (e.g. vLLM): execution of LLM workloads on GPU resources

User data is processed only for service delivery and is not retained or reused for training.

Usage considerations

Available models may vary depending on your VO
Usage may be subject to quotas and rate limits

Documentation – Artificial Intelligence