MAN01 How to publish site information

How to publish site information

Document control

PropertyValue
TitleHow to publish Site Information
Policy GroupOperations Management Board (OMB)
Document statusApproved
Procedure StatementPublishing site information in the Information Discovery System
OwnerSDIS team

EGI profile for the use of the GLUE 2.0 Information Schema specifies how the GLUE 2.0 information schema should be used in EGI. It gives detailed guidance on what should be published, how the information should be interpreted, what kinds of uses are likely, and how the information may be validated to ensure accuracy.

Configuring a site BDII

The site BDII needs to be configured to read from every node in the site which publishes information (meaning that it runs a so-called resource BDII). In YAIM this is defined with the BDII_REGIONS variable, which contains a list of node names which in turn refer to variables called BDII_<NODE>_URL which specify the LDAP URL of each resource BDII.

Some services may have DNS aliases for multiple hosts, but the BDII_REGIONS must contain the real hostnames for each underlying node - the information in the resource BDII is different for each node, so reading it via an alias would produce inconsistent results. However, it will usually be desirable for the published endpoint URLs to contain the alias rather than the real hostname; that can often be defined with a YAIM variable for the service. For the site BDII itself this variable is SITE_BDII_HOST. (If multiple site or top BDIIs are configured identically their content will also be identical, so reading via an alias does not produce any inconsistencies.)

Most services now publish themselves, so sites should check that all relevant services are included. In particular, VOMS servers have only published themselves comparatively recently so may be missing from the configuration. If the glite-CLUSTER node type is used this must also be included. Publication has been enabled for Argus in EMI 2, so this may also need to be added. Common services which do not currently publish are APEL and Squid. See the table below for more detailed information.

It is important to realise that the site BDII itself has a resource BDII, and this must be explicitly included in the configuration, e.g. with something like

BDII_REGIONS="CE SE BDII"
(...)
BDII_BDII_URL="ldap://$SITE_BDII_HOST:2170/mds-vo-name=resource,o=grid"

In the past it was common for the site BDII to be colocated with the CE so it did not need to be listed explicitly, but if installed on a dedicated node (which is now the recommended deployment) it must be included.

To check that all expected services are published the following command can be used:

$ ldapsearch -x -h $SITE_BDII_HOST -p 2170 -b mds-vo-name=$SITE_NAME,o=grid \
  objectclass=GlueService \
  | perl -p00e 's/\r?\n //g' | grep Endpoint:

(replacing SITE_BDII_HOST; and SITE_NAME with the values for your site), which should list all the service URLs.

In addition, most services should now be published in GLUE 2 format. There is no explicit configuration needed for GLUE 2, but one thing to be aware of is that the site name (and the other parts like o=grid) in the GIIS URL field in the GOCDB must have the correct case as GLUE 2 is case-sensitive.

To verify the GLUE 2 publication use the command:

$ ldapsearch -x -h $SITE_BDII_HOST -p 2170 -b GLUE2DomainID=$SITE_NAME,o=glue \
  objectclass=GLUE2Endpoint \
  | perl -p00e 's/\r?\n //g' | grep URL:

Some services, notably storage elements, may be missing or incomplete in GLUE 2 if they are older than the EMI 2 release. The following table shows the publishing status for gLite and WLCG node types (ARC and Unicore have a different structure).

Node typeGLUE 1GLUE 2Notes
LCG-CEYesNoObsolete
CREAMYesYesFull publication only in EMI 2
CLUSTERYesYesFull publication only in EMI 2
WMSYesYes
LBYesYes
DPMYesEMI 2
dCacheYesEMI 2
StoRMYesEMI 2
LFCYesEMI 2
FTSYesEMI 2Channels not yet published in GLUE 2
HydraEMI 2EMI 2Not yet released in EMI 2
AMGAYesEMI 2
VOMSYesYes
MyProxyYesYes
ArgusNoEMI 2Internal service, publication for deployment monitoring
Site BDIIYesYes
Top BDIIYesYes
R-GMAYesNoObsolete
VOBOXYesYes
ApelNoNoInternal service, publishing not yet requested
SquidNoNoConfiguration exists but not enabled
NagiosYesYes

Federated Cloud BDII configuration

For information about configuration of a Federated Cloud BDII, please look at the EGI Information System.

GlueSite Object

These are the existing well established attributes in the GlueSite object. All of these MUST remain.

AttributeExampleSchemaNotes
GlueSiteNameRAL-LCG2Free text, no whitespaceSame as GOCDB name if in GOCDB, your choice.
GlueSiteUniqueIDRAL-LCG2Identical to your !GlueSiteNameSame as GlueSiteName
GlueSiteWebhttps://cern.ch/itFree TextValid URL about the site.
GlueSiteLatitude52.42NN.NNSite Latitute.
GlueSiteLongitude16.91NN.NNLongitude of Site.
GlueSiteDescriptionRutherford LabFree TextA long name for the site.
GlueSiteLocationDublin, IrelandTown, City, CountryAn decreasing resolution ending with Country, agree a country name within a country. i.e UK != United Kingdom. Scotland and the Balkans should write a dynamic provider.
!GlueSiteUserSupportContactmailto:helpdesk@example.comValid URLURL for getting support. A ticket
system if available.
!GlueSiteSysAdminContactxmpp://admins@jabber.orgValid URLHow to contact the admins.
!GlueSiteSecurityContactmailto:security@example.comValid URLHow to contact for security related matters.

The GlueSite object in the 1.3 Glue Schema contains an attribute GlueSiteOtherInfo. To quote.

The attribute is to be used to publish data that does not fit any other attribute of the site entity. A name=value pair or an XML structure are example[s] of usage.

All this extra configuration will be with in the static information for the glue site within the Grid Information Provider system.

Guidelines for GlueSite Object

A format for publishing useful information about sites within the !GlueSiteOtherInfo is needed, as shown in the following table.

KeyExampleTypeNotes
GRIDEGI[#validgrid List of valid grid names]Multiple ones can be defined.
WLCG_TIER1Tier level of site in WLCG context.Either 0, 1 , 2 , 3 , 4
WLCG_PARENTUK-T1-RALName of the higher (administrative) tier site in WLCGThe WLCG_NAME of the site at a higher tier with WLCG
WLCG_NAMEIT-ATLAS-federation[#lcgnames Valid WLCG Names]An official WLCG name.
WLCG_NAMEICONhttps://example.com/tier2.pngValid URLURL to WLCGNAME icon, ideally 80x80 pixels.
EGEE_ROCRussiaValid federated Operations Centre nameOnly applicable if your site is still part of a federated Operations Centre (“ROC” according to the old EGEE terminology). Name MUST match the Operations Centre name declared in GOCDB. Note. If the site is now part of a NGI, then EGI_NGI MUST be used (see below).
EGI_NGINGI_CZValid NGIMust agree with the GOC DB
EGEE_SERVICEprodprod, pps or certWhich EGEE grid your site is part of, multiple attributes is okay. Obsolete in EGI.
OLDNAMEBristoltextIf your !GlueSiteName changes at some point please record your old name here.
ICONhttps://example.com/icon.pngValid URLIcon Image for your site, ideally 80x80 pixels
BLOGhttps://scotgrid.blogspot.com/feeds/posts/defaultValid RSS or Atom FeedYour site blog if you have one
CONFIGyaimyaim, puppet, quattor, …The configuration tool(s) used at the site

Note. Keywords starting with one of the grid names are to some extent reserved for that grid.

Example

GlueSiteName: RAL-LCG2
GlueSiteOtherInfo: BLOG=https://example.com/blog/feed
GlueSiteOtherInfo: EGI_NGI=NGI_UK
GlueSiteOtherInfo: GRID=EGI
GlueSiteOtherInfo: GRID=GRIDPP
GlueSiteOtherInfo: GRID=WLCG
GlueSiteOtherInfo: ICON=https://example.com/images/tierOneSmall.png
GlueSiteOtherInfo: WLCG_PARENT=CERN-PROD
GlueSiteOtherInfo: WLCG_TIER=1

Distributed Tier1s and Tier2s

Within an WLCG context for instance there are instances of distributed Tier2s and Tier1s. If separate component sites want to exist as a single WLCG tier then they might contain common values for their WLCGNAME.

GlueSiteName: CSCS-LCG2
GlueSiteOtherInfo: CONFIG=yaim
GlueSiteOtherInfo: EGI_NGI=NGI_CH
GlueSiteOtherInfo: GRID=EGI
GlueSiteOtherInfo: GRID=WLCG
GlueSiteOtherInfo: WLCG_NAME=CH-CHIPP-CSCS
GlueSiteOtherInfo: WLCG_PARENT=FZK-LCG2
GlueSiteOtherInfo: WLCG_TIER=2

Note that WLCG_PARENT is an accounting unit defined in the MOU document, as shown in WLCG CRIC.

Established Grid Name

Short NameLong NameURL
EGIEuropean Grid Initiativehttps://www.egi.eu
EELAEurope and Latin Americahttps://www.eu-eela.eu/
WLCGWorld LHC Computing Gridhttps://cern.ch/lcg
GRIDPPUK Particle Physics Gridhttps://www.gridpp.ac.uk
UKNGSNational UK Grid Servicehttps://www.ngs.ac.uk
OSGOpen Science Grid (US)https://www.opensciencegrid.org/
NDGFNordic DataGrid Facilityhttps://www.ndgf.org/
LondonGridLondon Gridhttps://www.gridpp.ac.uk/tier2/london/
NORTHGRIDNorthern (UK) Gridhttps://www.gridpp.ac.uk/northgrid/
SCOTGRIDScottish Gridhttps://www.scotgrid.ac.uk/
SOUTHGRIDSouthern (UK) Gridhttps://www.gridpp.ac.uk/southgrid/
Academic Grid MalaysiaMalaysian Grid
UPM Campus GridUniversiti Putra Malaysiahttps://www.upm.edu.my/
AEGISAcademic and Educational Grid Initiative of Serbiahttps://www.aegis.rs/
BIGGRIDDutch e-science Gridhttps://www.biggrid.nl/
Consorzio CometaConsorzio Multi-Ente per la promozione e l’adozione di Tecnologie di calcolo Avanzato (Italy)https://www.consorzio-cometa.it/en
D-GridGerman Gridhttps://www.d-grid-gmbh.de/index.php?id=1&amp;L=1
EUMEDEU/Mediterranean Gridhttps://www.eumedgrid.eu/
GILDAGrid INFN Laboratory for Dissemination Activities (Italy)https://gilda.ct.infn.it/
GISELAGrid Initiative for e-Science virtual communities in Europe and Latin Americahttps://www.gisela-grid.eu/
GRISUGriglia del Sud (Southern Italy Grid)https://www.grisu-org.it/
NEUGRIDNeuroscience Gridhttps://neugrid4you.eu/background
RDIGRussian Data Intensive Gridhttps://grid-eng.jinr.ru/?page_id=43
SEE-GRIDSouth Eastern European GRid-enabled eInfrastructure Developmenthttps://www.see-grid.org/

Important: The EGEE Grid name was decomissioned on [[Agenda-14-02-2011|14-02-2011]]. All sites need to replace this grid name with EGI.

Being part of a grid is just a reference that your site is in some way associated with a particular Resource Infrastructure Provider either technically or as part of a collaboration. The list of Grids can be extended. Please contact operations@egi.eu to request changes.

Valid WLCG Names

The WLCG names are the site names that appear within the LCG MOU concerning commitments to LHC computing.

WLCG NameCurrent GlueSiteName
CA-TRIUMFTRIUMF-LCG2
CERNCERN-PROD
DE-KITFZK-LCG2
ES-PICpic
FR-CCIN2P3IN2P3-CC
IT-INFN-CNAFINFN-T1
NDGFNGDF-T1
NL-T1SARA-MATRIX
TW-ASGCTaiwan-LCG2
UK-T1-RALRAL-LCG2
US-FNAL-CMSUSCMS-FNAL-W1
US-T1-BNLBNL-LCG2

For the tier two names please consult WLCG CRIC. The column marked Accounting Name are the WLCG Names which in the case of Tier2s are the GOCDB names. Use your site GOCDB name as your WLCG_NAME.

Also some tier2s live under more than 1 tier1 perhaps for different for different VOs. If your tier2 has more that one WLCG_PARENT then just add two distinct records to show this. Also some tier2s do not have a WLCGNAME at all.

GlueSiteUniqueId: EENet
GlueSiteName: EENet
GlueSiteOtherInfo: GRID=WLCG
GlueSiteOtherInfo: GRID=EGI
GlueSiteOtherInfo: EGI_NGI=NGI_NL
GlueSiteOtherInfo: WLCG_TIER=2
GlueSiteOtherInfo: WLCG_PARENT=UK-T1-RAL
GlueSiteOtherInfo: WLCG_PARENT=NL-T1

Valid EGI NGI Names

The valid names are those published on GOCDB.

YAIM Instructions

YAIM will have to be updated for those sites using yaim. This will be done and submitted to sites in the normal way.

YAIM Variable and ValueResulting Glue Attribute and Value
SITE_NAME=RAL_LCG2GlueSiteName: RAL-LCG2
SITE_DESC=“Rutherford Lab”GlueSiteDescription: Rutherford Lab
SITE_EMAIL= steve@example.comGlueSiteSysAdminContact: mailto:steve@example.com
SITE_SUPPORT_EMAIL= steve@example.comGlueSiteUserSupportContact: mailto:steve@example.com
SITE_SECURITY_EMAIL= steve@example.comGlueSiteSecurityContact: mailto:steve@example.com
SITE_LOC=“Soho, London, United Kingdom”GlueSiteLocation: Soho, London, United Kingdom
SITE_LONG=52.45GlueSiteLongitude: 52.45
SITE_LAT=-12.34GlueSiteLatitude: -12.34
SITE_WEB=“https://example.com/"GlueSiteWeb: https://example.com/
SITE_OTHER_GRID=“EGI|WLCG”GlueSiteOtherInfo: GRID=EGI
GlueSiteOtherInfo: GRID=WLCG
SITE_OTHER_EGEE_ROC=“UK/I”GlueSiteOtherInfo: EGEE_ROC=UK/I
SITE_OTHER_EGI_NGI=“NGI_CZ”GlueSiteOtherInfo: EGI_NGI=NGI_CZ
SITE_OTHER_EGEE_SERVICE=“prod”GlueSiteOtherInfo: EGEE_SERVICE=prod
SITE_OTHER_WLCG_TIER=2GlueSiteOtherInfo: WLCG_TIER=2
SITE_OTHER*="|GlueSiteOtherInfo: KEY=GlueSiteOtherInfo: KEY=

If multiple values for GlueSiteOtherInfo are needed, then just delimit your values with a |. The character | must be avoided in values.

Check your own GlueSite Object

The information published can be checked through an ldap search:

$ ldapsearch -x -H ldap://$SITE_BDII_HOST:2170 \
  -b 'Mds-Vo-Name=$SITE_NAME,o=Grid' \
  '(ObjectClass=GlueSite)'

In addition, VAPOR is a tool which provides a GUI for different views of published information, including a LDAP view.

Site information in GLUE 2

The GLUE 2 equivalent of the GlueSite object is the GLUE2AdminDomain. The same information should be present although in a slightly different format, and there are separate GLUE2Contact and GLUE2Location objects.