The SRM protocol allows storage operations to be marshalled into a specific reservation. However, the user may choose simply not to specify a space token when uploading data or they might use version 1 of SRM which lacks support for specifying a space reservation.
Moreover, there are many, existing protocols that have no concept of space tokens. When data is transferred using these protocols no information can be included about into which space the file is to be stored.
If, for whatever reason, a file is to be uploaded into a
directory without the user explicitly specifying a space
reservation then dCache will check for the
WriteToken directory tag. If found, the
value is used as the ID of the space reservation into which the
file should be accounted.
Should a user attempt to upload a file and no space reservation can be found for it then dCache must do something. How these transfers are handled depends on whether implicit space reservation is switched on.
If implicit space reservation is switched off then dCache will attempt to match the transfer using the links not allocated to any linkgroup. A consequence of this is that some storage must be available through links not in a linkgroup. It is into this non-linkgroup storage that files transfered without a space-token will be stored.
A pool may not be shared between two different linkgroups, or between a linkgroup and a link not in that linkgroup. Because of this, having implicit space reserveration switched off requires a site admin to partition their storage space.
The alternative is to switch on implicit space reservation. With this option enabled, incoming transfers will trigger the generation of a space reservation the same size as the file. This allows all storage be placed into linkgroups.
The main problem is that there is no support for implicit
space reservation for the GSIdCap and xrootd protocols.
Work is underway in supporting implicit space reservation for
GSIdCap.
Implicit space reservation is configured by the option
srmImplicitSpaceManagerEnabled option,
within the dCacheSetup configuration file.
When space is implicitly reserved, dCache must figure out
what access-latency and retention-policy the reservation
should have. The value could come from a number of places:
the SRM v2.2 protocol allows access-latency and
retention-policy to be specified, the directory (into which a
file is to be stored) may have tags defining what
access-latency and retention-policy is appropriate.
Should all attempts at discovering a transfer-specific
access-latency or retention-policy fail, dCache will use
system-wide default values. These are defined in
dCacheSetup using the options
SpaceManagerDefaultRetentionPolicy and
SpaceManagerDefaultAccessLatency.
The default values are good for a dCache instance with a
tape system; however, for a disk-only dCache, the values are
not useful. The options should be set explicitly to
REPLICA and ONLINE.
By default, implicit space reservation only affects
SRM-initiated transfers, the ones where the end-user omitted
to specify a space token. Enabling the
SpaceManagerReserveSpaceForNonSRMTransfers
option allows non-SRM initiated transfers, such as transfers
directly to the GridFTP door.
The following fragment from dCacheSetup shows implicit
space reservations being enabled for SRM and non-SRM initiated
tranfers and sets the system-wide default values for
access-latency and retention-polcy to online and replica
respectively.
# will have no effect if srmSpaceManagerEnabled is "no" # srmImplicitSpaceManagerEnabled=yes srmImplicitSpaceManagerEnabled=yes # # ----if the transfer request come from the door, and there was not prior # space reservation made for this file, should we try to reserve # space before satisfying the request # SpaceManagerReserveSpaceForNonSRMTransfers=false SpaceManagerReserveSpaceForNonSRMTransfers=true # ----if space reservation request does not specify retention policy # we will assign this retention policy by default # SpaceManagerDefaultRetentionPolicy=CUSTODIAL SpaceManagerDefaultRetentionPolicy=REPLICA # # ----if space reservation request does not specify access latency # we will assign this access latency by default # SpaceManagerDefaultAccessLatency=NEARLINE SpaceManagerDefaultAccessLatency=ONLINE
The SRM component must be restarted before this change takes
effect.
[root@dcache-node] #/opt/d-cache/bin/dcache restart srm Using CATALINA_BASE: /opt/d-cache/libexec/apache-tomcat-5.5.20 Using CATALINA_HOME: /opt/d-cache/libexec/apache-tomcat-5.5.20 Using CATALINA_TMPDIR: /opt/d-cache/libexec/apache-tomcat-5.5.20/temp Using JRE_HOME: /usr/java/jdk1.5.0_16 Stopping srm-dcache-node(pid=29824) 0 Done Using CATALINA_BASE: /opt/d-cache/libexec/apache-tomcat-5.5.20 Using CATALINA_HOME: /opt/d-cache/libexec/apache-tomcat-5.5.20 Using CATALINA_TMPDIR: /opt/d-cache/libexec/apache-tomcat-5.5.20/temp Using JRE_HOME: /usr/java/jdk1.5.0_16 Pinging srm server to wake it up, will take few seconds ... - Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart). Attachment support is disabled. VersionInfo : v2.2 backend_type:dCache Done
It is now possible to write into dCache, specifying an access latency and retention policy but no space token, and a new space reservation is made for the file.
[user@ui-node] $srmcp -2 -protocols=gsiftp -access_latency=ONLINE -retention_policy=REPLICA file://///etc/group srm://:8443/pnfs/fzk.de/data/gks/testfile1 WARNING: SRM_PATH is defined, which might cause a wrong version of srm client to be executed WARNING: SRM_PATH=/opt/d-cache/srmdcache-node.fzk.de
It is also possible to use globus-url-copy.
This writes data without any attempt at using SRM, directly to
the GSI-FTP door. With
SpaceManagerReserveSpaceForNonSRMTransfers
enabled, this too will have a space reservation created for
the file.
[user@ui-node] $globus-url-copy file://///etc/fstab gsiftp://:22128/pnfs/fzk.de/data/gks/wurstdcache-node.fzk.de
The required access-latency and retention policy are taken either from directory tags or the dCache-wide default values. The reservation will be from a linkgroup that supports the required access-latency and retention-policy.