This section describes how to configure dCache so that the directory in which data is created determins which set of pools are available. This, along with the filesystem permissions, can be used to allow a group of people (such as a VO) access to some storage.
The dCache filesystem supports some advanced features not present with many standard filesystems, including directory tags. A directory tag, or simply tag for short, is a keyword-value pair assigned to a directory.
Subdirectories will inherit these values from their parent directory. When creating a new tag or assigning a tag a new value, this tag-value assignment will be available in all subdirectories.
You can list all the available tags by accessing the contents
of a special file: .(tags)().
Since these special filenames contains parentheses, the name must be quoted to prevent the shell from expanding them. Failure to do so will lead to potentially confusing results:
[root@dcache-node] # cat .(tags)()
-bash: syntax error near unexpected token `('We can demonstrate this by listing the tags in the VO-specific directory for VO gks
[root@dcache-node] # cat "/pnfs/fzk.de/data/gks/.(tags)()"
.(tag)(OSMTemplate)
.(tag)(sGroup)
In the above example, the directory /pnfs/fzk.de/data/gks has two
tags: OSMTemplate and
sGroup. To obtain the value of these tags,
just examine the contents of the file:
.(tag)(.
For example, the contents of file
tagname).(tag)(sGroup) contains the value of the
sGroup tag:
[root@dcache-node] # cat '.(tag)(sGroup)'
STATICThe following handy trick lists all tags and their values in the current directory.
[root@dcache-node] # grep '' `cat '.(tags)()'`
.(tag)(OSMTemplate):StoreName gks
.(tag)(sGroup):STATIC
Take care when typing it: the order is tick, tick, backtick,
tick, tick, backtick. With many shells, this may be replace
with grep "" $(cat ".(tags)()")
To assigning a value to tag
tagname.(tag)(
file. If previously the tag did not exist, it is created
automatically:
tagname)
[root@dcache-node] #echo "This is a test." > '.(tag)(foo)'[root@dcache-node] #cat '.(tags)()' .(tag)(foo) .(tag)(OSMTemplate) .(tag)(sGroup)[root@dcache-node] #cat '.(tag)(foo)' This is a test.
Certain tags have specific meaning. In particular, the two
tags OSMTemplate and
sGroup are used to determine the
store descriptor, from which the
storage class is determined. The
storage class is important as it can be used when choosing
which pools to use when a user is writing data into dCache.
The OSMTemplate tag value will typically
contains two words on a single line: the phrase
StoreName followed by some value. The
value of sGroup provides the storage group.
Together they provide the storage descriptor as
store
name:storage
group
For disk-only systems, the HSM osm is
assumed. This implies that the storage class is
store
name:storage
group@osm
For example, given the following tag values:
[root@dcache-node] # grep '' `cat '.(tags)()'`
.(tag)(OSMTemplate):StoreName gks
.(tag)(sGroup):EXAMPLE
the storage class is gks:EXAMPLE@osm.
When faced with a data operation (such as writing data into
dCache) the Pool Selection Unit (psu), part of the
PoolManager, is used to determine which pools are suitable
for that request. It is the Pool Selection Unit that must be
configured so pools are selected based on the request's
storage class.
To assign one pool exclusively to gks and another pool
exclusively to dech, the two directories /pnfs/fzk.de/data/gks and
/pnfs/fzk.de/data/dech
must be assigned different storage classes. The pool
selection unit is configured so that pools are assigned
exclusively to certain VOs.
First assign the tags so that the different parts of the namespace have different storage classes.
[root@dcache-node] #cd /pnfs/fzk.de/data Use more restrictive permissions[root@dcache-node] #chmod 775 gks dech Assign the directory gks the storage class gks:GENERATED@osm[root@dcache-node] #cd gks[root@dcache-node] #echo GENERATED > ".(tag)(sGroup)"[root@dcache-node] #echo "StoreName gks" > ".(tag)(OSMTemplate)" Assign the directory dech the storage class dech:GENERATED@osm[root@dcache-node] #cd ../dech[root@dcache-node] #echo GENERATED > ".(tag)(sGroup)"[root@dcache-node] #echo "StoreName dech" > ".(tag)(OSMTemplate)"
Next, the PoolManager must be configured so it will select
pools based on the two storage classes:
gks:GENERATED@osm and
dech:GENERATED@osm. To do this, log into
the admin interface and navigate to the PoolManager.
[user@ui-node] $ssh -c blowfish -p 22223 admin@dcache-nodeadmin@'s password: Warning: Remote host denied authentication agent forwarding. dCache Admin (VII) (user=admin) [dcache-node] (local) admin > cd PoolManagerdcache-node.fzk.de
Unit groups are sets of predicates (logic conditions) that must be satisfied before a link is acceptable for some data transfer. Unit groups contain one or more units, each of which are simple tests. At least one unit within a unitgroup must be true for the unit group to be satisfied; this is a boolean “or” operation.
The command psu ls -l ugroup
will list all currently configured unitgroups:
[dcache-node.fzk.de
Notice that the unit group gks-groups and
dech-groups have been already defined. The
unit group gks-groups is satisfied (i.e.,
is true) if the disk operation involves either the storage class
gks:STATIC@osm or the storage class
gks:GENERATED@osm. The unit group
dech-groups has similar constraints.
Links may contain one or more unit groups. When choosing a link for some data operation, all a link's unit groups must be satisfied for that link to be considered; this is a boolean “and” operation.
If a link contains unit group gks-groups
then that link will only be select if the file operation
involves a directory where the storage class is
gks:STATIC@osm or
gks:GENERATED@osm. Likewise, if a link
contains unit group dech-groups then a
similar condition holds. These unit groups are sufficient for
our needs.
Next, look at what links are currently defined.
[] (PoolManager) admin > psu ls -l linkdcache-node.fzk.degks-linkreadPref : 20 cachePref : 20 writePref : 20 p2pPref : -1 section : None linkGroup : None UGroups : gks-groups (links=1;units=2) world-net (links=4;units=1) default-link readPref : 10 cachePref : 10 writePref : 10 p2pPref : -1 section : None linkGroup : None UGroups : any-protocol (links=1;units=1) any-store (links=1;units=7) world-net (links=4;units=1)dech-linkreadPref : 20 cachePref : 20 writePref : 20 p2pPref : -1 section : None linkGroup : None UGroups : dech-groups (links=1;units=2) world-net (links=4;units=1) dteam-link readPref : 20 cachePref : 20 writePref : 20 p2pPref : -1 section : None linkGroup : None UGroups : dteam-groups (links=1;units=2) world-net (links=4;units=1)
Note how the link gks-link contains the unit group
gks-groups. Recall that this ensures that
this link is only ever selected if the file operation involves
the storage class gks:STATIC@osm or
gks:GENERATED@osm. This means that, for
gks-link to be available, the file operation must be within
the directory that has tags defining one of the two acceptable
storage classes. For our configuration, storage class
gks:GENERATED@osm is only available within
the directory /pnfs/fzk.de/data/gks.
The link dech-link contains the unit group
dech-group, so has a similar condition: it
is only selectable for file operations within the directory
/pnfs/fzk.de/data/dech
These two links are ideal for assigning pools such that they have exclusive use for their respective VOs. Next, we see how the pools are assigned to different links by listing the poolgroups.
[] (PoolManager) admin > psu ls -l pgroup Note that these three poolgroups are currently empty ResilientPools linkList : poolList : gks linkList :dcache-node.fzk.degks-link(pref=20/20/-1/20;;ugroups=2;pools=1) poolList : dech linkList :dech-link(pref=20/20/-1/20;;ugroups=2;pools=1) poolList : All pools have been assigned to poolgroup default. default linkList : default-link (pref=10/10/-1/10;;ugroups=3;pools=1) poolList :(enabled=true;active=10;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled)_4dcache-node(enabled=true;active=9;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled)_3dcache-node(enabled=true;active=8;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled)_2dcache-node(enabled=true;active=11;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled) dteam linkList : dteam-link (pref=20/20/-1/20;;ugroups=2;pools=1) poolList :_1dcache-node
Notice that the current configuration has all pools assigned to
the default poolgroup. This poolgroup is
assigned to the link default-link, so is
available for all file operations.
There are poolgroups assigned to the two VO-exclusive links
(gks-link and dech-link): the poolgroups gks and
dech respectively. However, these poolgroups are
currently empty, not having any pools assigned to them.
To reserve a pool for the gks VO it must be removed from the
default poolgroup and assigned to the
gks poolgroup. Doing this makes the pool available
for the gks-link link, which will be selected preferentially
for operations involving the gks VO storage classes
(dech:STATIC@osm and
dech:GENERATED@osm).
The following will assign pool dcache-nodedcache-nodedcache-nodedcache-nodedefault poolgroup. This
allows end-users to continue operations whilst site-admins can
redeploy storage as necessary.
Assignexclusively to VO gks [_1dcache-node] (PoolManager) admin > psu removefrom pgroup defaultdcache-node.fzk.de[_1dcache-node] (PoolManager) admin > psu addto pgroup gksdcache-node.fzk.deDo the same for VO dech and pool_1dcache-node. [_2dcache-node] (PoolManager) admin > psu removefrom pgroup defaultdcache-node.fzk.de[_2dcache-node] (PoolManager) admin > psu addto pgroup dechdcache-node.fzk.deBe sure to save the new configuration. [_2dcache-node] (PoolManager) admin > savedcache-node.fzk.de
To test that this works, try copying a file into dCache as gks and dech. The following test gives an indication that dCache is configured correctly.
Copy a file three times as a member of VO gks[user@ui-node] $voms-proxy-init -voms gks Enter GRID pass phrase: Your identity:Your DN is hereCreating temporary proxy ................................................. Done Contacting dgrid-voms.fzk.de:15025 [/O=GermanGrid/OU=FZK/CN=host/dgrid-voms.fzk.de] "gks" Done Creating proxy .................................... Done Your proxy is valid untilTue Aug 26 04:08:53 2008[user@ui-node] $dccp /bin/bash gsidcap://:22128/pnfs/fzk.de/data/gks/test-psu-1dcache-node.fzk.de[user@ui-node] $dccp /bin/bash gsidcap://:22128/pnfs/fzk.de/data/gks/test-psu-2dcache-node.fzk.de[user@ui-node] $dccp /bin/bash gsidcap://:22128/pnfs/fzk.de/data/gks/test-psu-3 Then copy a file three times as a member of VO dechdcache-node.fzk.de[user@ui-node] $voms-proxy-init -voms dech Enter GRID pass phrase: Your identity:Your DN is hereCreating temporary proxy ................................................. Done Contacting glite-io.scai.fraunhofer.de:15025 [/O=GermanGrid/OU=Fraunhofer SCAI/CN=host/glite-io.scai.fraunhofer.de] "dech" Done Creating proxy .................................... Done Your proxy is valid untilTue Aug 26 04:08:53 2008[user@ui-node] $dccp /bin/bash gsidcap://:22128/pnfs/fzk.de/data/dech/test-psu-1dcache-node.fzk.de[user@ui-node] $dccp /bin/bash gsidcap://:22128/pnfs/fzk.de/data/dech/test-psu-2dcache-node.fzk.de[user@ui-node] $dccp /bin/bash gsidcap://:22128/pnfs/fzk.de/data/dech/test-psu-3dcache-node.fzk.de
Now log back into the admin interface and ask the
PnfsManager in which pool these files were stored.
First check the gks files: [] (PnfsManager) admin > pnfsidof /pnfs/fzk.de/data/gks/test-psu-1 0000DF487493131E45259EFDC83FA0B0A945 [dcache-node.fzk.de] (PnfsManager) admin > cacheinfoof 0000DF487493131E45259EFDC83FA0B0A945dcache-node.fzk.de[_1dcache-node] (PnfsManager) admin > pnfsidof /pnfs/fzk.de/data/gks/test-psu-2 000053E0C67AA35840278B2215981055A522 [dcache-node.fzk.de] (PnfsManager) admin > cacheinfoof 000053E0C67AA35840278B2215981055A522dcache-node.fzk.de[_1dcache-node] (PnfsManager) admin > pnfsidof /pnfs/fzk.de/data/gks/test-psu-3 00003CB28EDC51AC457B8DD32F5E8B9F8AD9 [dcache-node.fzk.de] (PnfsManager) admin > cacheinfoof 00003CB28EDC51AC457B8DD32F5E8B9F8AD9dcache-node.fzk.de... and check the dech files: [_1dcache-node] (PnfsManager) admin > pnfsidof /pnfs/fzk.de/data/dech/test-psu-1 0000FC5F80F56A79498082246E61562BAF86 [dcache-node.fzk.de] (PnfsManager) admin > cacheinfoof 0000FC5F80F56A79498082246E61562BAF86dcache-node.fzk.de[_2dcache-node] (PnfsManager) admin > pnfsidof /pnfs/fzk.de/data/dech/test-psu-2 0000C429D1C185A34FCB9E17FDB1B4C8E40C [dcache-node.fzk.de] (PnfsManager) admin > cacheinfoof 0000C429D1C185A34FCB9E17FDB1B4C8E40Cdcache-node.fzk.de[_2dcache-node] (PnfsManager) admin > pnfsidof /pnfs/fzk.de/data/dech/test-psu-3 0000DDF1AD8142024A86B1BDFA2E0D3B19EF [dcache-node.fzk.de] (PnfsManager) admin > cacheinfoof 0000DDF1AD8142024A86B1BDFA2E0D3B19EFdcache-node.fzk.de_2dcache-node