Documentation – High-Throughput Compute

Providers: HTCondor-CE Accounting

Mon, 01 Jan 0001 00:00:00 +0000

HTCondor-CE can be configured to automatically publish accounting data to EGI Accounting service, if compute resources are provided by an HTCondor pool. Optionally, the HTCondor pool can also be configured to provide per-machine performance data for increased accounting accuracy.

Enabling accounting records on each HTCondor-CE

Each HTCondor-CE must be configured separately to create accounting data for its jobs and feed them to APEL.

See MAN09 for general configuration instructions of APEL.

Install htcondor-ce-apel, available from the EGI UMD or HTCondor RPM repositories.

Configure the APEL parser via parser.cfg to read the HTCondor-CE accounting records:

[blah]
enabled = true
dir = /var/lib/condor-ce/apel/
filename_prefix = blah

[batch]
enabled = true
dir = /var/lib/condor-ce/apel/
filename_prefix = batch
type = HTCondor

Configure the APEL client via client.cfg section [spec_updater] to know the cluster specs:
- site_name must equal your EGI Configuration Database (GOCDB) site name.
- manual_spec1 must equal the CE identifier, spec type and average spec value of the cluster. The format is:
```
manual_spec1 = <fqdn>:9619/<fqdn>-condor,<spec_type>,<spec_value>
```
  Commonly, the spec type is one of HEPscore23 or HEPSPEC. The spec value is per core.
  
  For example, if the CE has the Fully Qualified Domain Name (FQDN) my-htcondor-ce.example.com and resources average 12.5 HEPSPEC per core:
```
manual_spec1 = my-htcondor-ce.example.com:9619/my-htcondor-ce.example.com-condor,HEPSPEC,12.5
```
Start and enable the condor-ce-apel.timer unit.

Enabling per-machine performance information

By default, HTCondor-CE APEL accounting assumes that all machines in the cluster have comparable performance. If this is not the case, accounting accuracy can be improved by adding performance information per machine.

Performance information may be added to each HTCondor StartD. There are two separate ways to do so:

An absolute spec value, similar to the average spec value on to the CE. HTCondor-CE APEL accounting then weights resource usage by comparing the StartD spec value to the average spec value.

In the StartD configuration, define ApelSpecs as a new-style classad mapping of spec types and their values; multiple spec types are supported. Also add ApelSpecs to the classad attributes of the StartD:
```
# The absolute performance per core on this StartD
ApelSpecs = [HEPSPEC=14.37; HEPscore23=14.409; SI2K=2793]
STARTD_ATTRS = $(STARTD_ATTRS) ApelSpecs
```
A relative spec value, as a factor to the average spec value on to the CE. HTCondor-CE APEL accounting then weights resource usage by the relative spec factor.

In the StartD configuration, define ApelScaling as a number; values above 1 mean the performance is above average. Also add ApelScaling to the classad attributes of the StartD.
```
# The relative performance per core on this StartD
ApelScaling = 1.15
STARTD_ATTRS = $(STARTD_ATTRS) ApelScaling
```

If both ApelSpecs and ApelScaling are defined, ApelSpecs takes precedence.

Providers: Storage accounting

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Storage space usage accounting is based on the StAR (Storage Accounting Record) developed during the EMI project in conjunction with the OGF Usage Record Work Group (UR-WG). The format is documented in GFD-I.201.

EMI delivered StAR solutions for dCache and DPM in EMI-3. In both cases the storage service queries its database at a site and extracts data to populate StAR usage records. The site then uses SSM as a transport method to send the StAR records via the EGI Messaging Service to EGI Accounting repository.

Deployment instructions

Accounting script for dCache

Note

Find dCache documentation on the dCache site.

The configuration follows the same configuration system that is common to all dCache components. As with other dCache components, the default values are exposed and documented.

In general, a site should review the star.properties file (in the standard location for configuration defaults) and update their site-specific configuration, if necessary.

Information on how to set the star.properties file can be found in the dCache repository.

Map local groupid to the VO names as appropriate, for example:

star.gid-mapping = 12345=/atlas,12346=/cms,12347=/lhcb (EXAMPLE)

Running the dcache-star command generates records according to that site’s configuration.

Edit SSM’s sender.cfg to have path:/var/spool/dcache/star so that it can find and publish the records dCache generated.

Accounting script for DPM

Important

DPM/DMLite is supported until June 2023. All the RCs providing DPM have been invited to move to a different storage solution. A decommissioning and migration campaign was started for this purpose.

You need to install DPM-DMLITE 1.8.7 or higher. The Storage Accounting is implemented as a puppet module that adds cron configuration to execute the script daily.

Please be sure to have installed the star-accounting.py script v1.0.4 at least.

Accounting script for EOS

The script generating the required space accounting information is available in the eos-server package starting with release 5.0.15.

Please have a look at the EOS documentation for more information.

Install the APEL SSM software

The APEL SSM software can be installed from the UMD-4 repository.

Add the information to EGI Configuration Database

You need to add a new service endpoint for that host to EGI Configuration Database with the service type eu.egi.storage.accounting and the correct host certificate DN. The Accounting Repository takes up to an hour to update its ACL from the configuration Database and the EGI Messaging Service take up to 4 hours. If you get warnings in your SSM log about invalid username or password you can retry again after a delay. If this persists for over 4 hours, then do open a Helpdesk ticket.

Configure SSM

Set the configuration files as explained in the general documentation and in the migration instructions.

Running the Accounting Software

Create a cron job to run your accounting script followed by calling the SSM sender. We recommend that you send storage accounting data once per day. There will be a delay of up to 24 hours before you see the data you have sent reflected in the Accounting Portal.

Example:

$ cat /etc/cron.daily/dmlite-StAR-accounting
#!/bin/sh
set -e
mkdir -p /var/spool/apel/outgoing/`date +%Y%m%d`
/usr/share/dmlite/StAR-accounting/star-accounting.py --reportgroups \
 --dbhost=[hostname] \
 --dbuser=[username] \
 --dbpwd=[password] \
 --nsdbname=cns_db \
 --dpmdbname=dpm_db \
 --site=[site name] > /var/spool/apel/outgoing/`date +%Y%m%d`/`date +%Y%m%d%H%M%S`
ssmsend

You can confirm if the Accounting Repository receives your data by looking at the sites publishing storage accounting records

The page is updated on a daily basis.

Storage Accounting Data at the EGI Accounting Portal

The storage accounting view is currently available on the development instance of the Accounting Portal.

Providers: Changing the Site BDII

Mon, 01 Jan 0001 00:00:00 +0000

Moving the Site BDII to another machine

Ensure the new Site BDII is working fine and publishing all the necessary site information
- See MAN01 for general information about how to configure a Site BDII. In particular, remember that the Site BDII configuration must include the BDII node itself.
Put the old Site BDII in scheduled downtime in the Configuration Database for a couple of hours
Register the new service in the Configuration Database by adding a new Service Endpoint:
- select Site BDII in Service Type field
- fill in at least the hostname
- select Y for production and monitoring, N for beta service field
Properly modify the GIIS URL field with the new SITE-BDII ldap URL
- Note that the site name in the GIIS URL is case-sensitive
Mark the old Site BDII as not production and turn off its monitoring
Top BDIIs updates their sites list every hour so the new Site BDII should be published in less than an hour; instead Nagios updates the monitored hosts every 3 hours
When the new Site BDII is appeared on Nagios and the old one is disappeared, turn off the old Site BDII and remove it from the Configuration Database

Turning off a Site BDII co-hosted with other services

Stop and remove the service

service bdii stop
yum remove glite-BDII glite-yaim-bdii

Delete some info-providers and ldif file ( /opt/glite/etc/gip/ldif/stub-site.ldif, /opt/glite/etc/gip/site-urls.conf, /opt/glite/etc/gip/provider/glite-info-provider-site, /opt/glite/etc/gip/provider/glite-info-provider-service-bdii-site-wrapper, /opt/glite/etc/gip/ldif/glite-info-site.ldif)
Reconfigure with yaim without specifying the BDII_site profile