Documentation related to EGI DataHub
The DataHub allows to:
The main features offered by the DataHub are:
It is based on the OneData technology.
a virtual volume where users will organize their data. A space is supported by one or multiple Oneproviders providing actual storage resources
a central component federating providers. It takes care of Authentication and Authorization and other management tasks (like space creation). EGI DataHub is a Onezone instance.
the central Onedata Onezone instance of the EGI Federation. Single Sign On (SSO) with all the connected storage providers (Oneprovider) is guaranteed through EGI Check-in
a data management component deployed in the data centres, provisioning data and managing transfers. A Oneprovider is typically deployed at a site near the local storage resources, and can access local storage resources over multiple connectors (CEPH, POSIX,...). A default one is operated for EGI by CYFRONET.
a client application providing access to the spaces through a FUSE mount point (local POSIX access). Spaces are accessible as if they were part of the local file system. Oneclient can be used from VM, containers, desktop,...
Web interfaces and APIs are also available
Using the EGI DataHub web interface it's possible to manage the space.
Using Oneclient it's possible to mount a space locally, and access it over a POSIX interface, using files as they were stored locally. The file's blocks are downloaded on demand.
In Onedata the file distribution is done on a block basis, blocks will be replicated on the fly, and it's possible to instrument the replication.
Three different formats of metadata can be attached to files: basic (key/value), JSON and RDF. The metadata can be managed using the Web interface and the APIs. It's also possible to create indexes and query them.
It's possible to view the popularity of a file and manage smart caching.