HTCondor-CE can be configured to automatically publish accounting data to EGI Accounting service, if compute resources are provided by an HTCondor pool. Optionally, the HTCondor pool can also be configured to provide per-machine performance data for increased accounting accuracy.
Enabling accounting records on each HTCondor-CE
Each HTCondor-CE must be configured separately to create accounting data for its jobs and feed them to APEL.
See MAN09 for general configuration instructions of APEL.
Configure the APEL parser via
parser.cfgto read the HTCondor-CE accounting records:
[blah] enabled = true dir = /var/lib/condor-ce/apel/ filename_prefix = blah [batch] enabled = true dir = /var/lib/condor-ce/apel/ filename_prefix = batch type = HTCondor
Configure the APEL client via
[spec_updater]to know the cluster specs:
site_namemust equal your EGI Configuration Database (GOCDB) site name.
manual_spec1must equal the CE identifier, spec type and average spec value of the cluster. The format is:
manual_spec1 = <fqdn>:9619/<fqdn>-condor,<spec_type>,<spec_value>
Commonly, the spec type is one of
HEPSPEC. The spec value is per core.
For example, if the CE has the Fully Qualified Domain Name (FQDN)
my-htcondor-ce.example.comand resources average 12.5 HEPSPEC per core:
manual_spec1 = my-htcondor-ce.example.com:9619/my-htcondor-ce.example.com-condor,HEPSPEC,12.5
Start and enable the
Enabling per-machine performance information
By default, HTCondor-CE APEL accounting assumes that all machines in the cluster have comparable performance. If this is not the case, accounting accuracy can be improved by adding performance information per machine.
Performance information may be added to each HTCondor StartD. There are two separate ways to do so:
An absolute spec value, similar to the average spec value on to the CE. HTCondor-CE APEL accounting then weights resource usage by comparing the
StartDspec value to the average spec value.
ApelSpecsas a new-style classad mapping of spec types and their values; multiple spec types are supported. Also add
ApelSpecsto the classad attributes of the
# The absolute performance per core on this StartD ApelSpecs = [HEPSPEC=14.37; HEPscore23=14.409; SI2K=2793] STARTD_ATTRS = $(STARTD_ATTRS) ApelSpecs
A relative spec value, as a factor to the average spec value on to the CE. HTCondor-CE APEL accounting then weights resource usage by the relative spec factor.
ApelScalingas a number; values above 1 mean the performance is above average. Also add
ApelScalingto the classad attributes of the
# The relative performance per core on this StartD ApelScaling = 1.15 STARTD_ATTRS = $(STARTD_ATTRS) ApelScaling
ApelScaling are defined,
ApelSpecs takes precedence.