Data Transfer API

The Application Programming Interface of EGI Data Transfer


EGI Data Transfer offers Application Programming Interfaces (APIs) for both regular users and administrator. This page focuses on the user APIs, available in two favors:

  • Python Easy Bindings

Authentication & Authorisation

Users have to authenticate before they can call the API.

During the authentication phase, credentials are delegated to the FTS service, which will contact the storages to steer the data transfers on behalf of the users.

The FTS service supports both plain X.509 proxies and X.509 proxies extended with VO information (VOMS) for authentication and authorisation. You can learn more about VOMS configuration and proxy creation.


The User RESTFul APIs can be used to submit transfers jobs (collections of single transfers), monitor and cancel existing transfers. Please check the CERN documentation for the full API details. Here we will provide some examples usage using the Curl client.

Checking how the server sees the identity of the user

curl --capath /etc/grid-security/certificates -E $X509_USER_PROXY \
  --cacert $X509_USER_PROXY
  "dn": [
    "/DC=org/DC=terena/DC=tcs/C=NL/O=Stichting EGI/CN=Andrea Manzi",
    "/DC=org/DC=terena/DC=tcs/C=NL/O=Stichting EGI/CN=Andrea Manzi/CN=proxy"
  "vos_id": [
  "roles": [],
  "delegation_id": "9ab8068853808c6b",
  "user_dn": "/DC=org/DC=terena/DC=tcs/C=NL/O=Stichting EGI/CN=Andrea Manzi",
  "level": {
    "transfer": "vo"
  "is_root": false,
  "base_id": "01874efb-4735-4595-bc9c-591aef8240c9",
  "vos": [
  "voms_cred": [
  "method": "certificate"

Getting a list of jobs running

Filtered by VO

curl --capath /etc/grid-security/certificates -E $X509_USER_PROXY \
  --cacert $X509_USER_PROXY

    "cred_id": "1426115d1660de4d",
    "user_dn": "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ftssuite/CN=737188/CN=Robot:
    fts3 testsuite",
    "job_type": "N",
    "retry": -1,
    "job_id": "94560e74-7ca3-11e9-97dd-02163e00d613",
    "cancel_job": false,
    "job_state": "FINISHED",
    "submit_host": "",
    "priority": 3,
    "source_space_token": "",
    "max_time_in_queue": null,
    "job_metadata": {
      "test": "test_bring_online",
      "label": "fts3-tests"
    "source_se": "mock://",
    "bring_online": 120,
    "reason": null,
    "space_token": "",
    "submit_time": "2019-05-22T15:09:22",
    "retry_delay": 0,
    "dest_se": "mock://",
    "internal_job_params": "nostreams:1",
    "overwrite_flag": false,
    "copy_pin_lifetime": -1,
    "verify_checksum": "n",
    "job_finished": null,
    "vo_name": "dteam"

Cancelling a job

curl --capath /etc/grid-security/certificates -E $X509_USER_PROXY \
  --cacert $X509_USER_PROXY \
   -X DELETE \

Getting expiration time of delegated credentials

curl --capath /etc/grid-security/certificates -E $X509_USER_PROXY \
  --cacert $X509_USER_PROXY \
  "voms_attrs": [
  "termination_time": "2020-07-31T22:50:28"

Python Bindings

The Python bindings for FTS can be installed from the EPEL package repository (EL6 and EL7 packages are available) with Python 2.7 being supported.

yum install python-fts -y

For using the bindings, you need to import, although for convenience it can be renamed as something else:

import as fts3

In the following code snippets, an import as above is assumed.

In order to be able to do any operation, information about the state of the user credentials and remote endpoint needs to be kept. That’s the purpose of a Context.

context = fts3.Context(endpoint, ucert, ukey, verify=True)

The endpoint to use corresponds to the FTS instance REST server and it must have the following format:


for instance

If you are using a proxy certificate, you can either specify only user_certificate, or point both parameters to the proxy. user_certificate and user_key can be safely omitted, and the program will use the values defined in the environment variables X509_USER_PROXY or X509_USER_CERT + X509_USER_KEY.

If verify is False, the server certificate will not be verified.

Here are some examples about creating a context, submitting a job with a single transfer and getting the job status:

# pretty print the json outputs
>>> import pprint
>>> pp = pprint.PrettyPrinter(indent=4)

# creating the context
>>> context = fts3.Context("")
# printing the whoami info
>>>  pp.pprint (fts3.whoami(context))

{   u'base_id': u'01874efb-4735-4595-bc9c-591aef8240c9',
    u'delegation_id': u'9ab8068853808c6b',
    u'dn': [   u'/DC=org/DC=terena/DC=tcs/C=NL/O=Stichting EGI/CN=Andrea Manzi',
               u'/DC=org/DC=terena/DC=tcs/C=NL/O=Stichting EGI/CN=Andrea Manzi/CN=proxy'],
    u'is_root': False,
    u'level': {   u'transfer': u'vo'},
    u'method': u'certificate',
    u'roles': [],
    u'user_dn': u'/DC=org/DC=terena/DC=tcs/C=NL/O=Stichting EGI/CN=Andrea Manzi',
    u'voms_cred': [u'/dteam/Role=NULL/Capability=NULL'],
    u'vos': [u'dteam'],
    u'vos_id': [u'6b10f4e4-8fdc-5555-baa2-7d4850d4f406']}

# creating a new transfer and submiting a job
>>> transfer = fts3.new_transfer(
...   'gsiftp://source/path', 'gsiftp://destination/path',
...   checksum='ADLER32:1234', filesize=1024,
...   metadata='Submission example'
... )
>>> job = fts3.new_job([transfer])
>>> job_id = fts3.submit(context, job)
>>> print job_id

# get the job status
>>> pp.pprint(fts3.get_job_status(context, job_id))
{   u'bring_online': -1,
    u'cancel_job': False,
    u'copy_pin_lifetime': -1,
    u'cred_id': u'9ab8068853808c6b',
    u'dest_se': u'gsiftp://destination',
    u'http_status': u'200 Ok',
    u'internal_job_params': u'nostreams:1',
    u'job_finished': u'2020-07-31T16:05:55',
    u'job_id': u'b6191212-d347-11ea-8a47-fa163e45cbc4',
    u'job_metadata': None,
    u'job_state': u'FAILED',
    u'job_type': u'N',
    u'max_time_in_queue': None,
    u'overwrite_flag': False,
    u'priority': 3,
    u'reason': u'One or more files failed. Please have a look at the details for
    more information',
    u'retry': -1,
    u'retry_delay': 0,
    u'source_se': u'gsiftp://source',
    u'source_space_token': u'',
    u'space_token': u'',
    u'submit_host': u'',
    u'submit_time': u'2020-07-31T16:05:54',
    u'user_dn': u'/DC=org/DC=terena/DC=tcs/C=NL/O=Stichting EGI/CN=Andrea Manzi',
    u'verify_checksum': u't',
    u'vo_name': u'dteam'}