emacsxterm at Startupgrepfindcmake Makes make DoxargsThe mail program Alpine uses your physical e-mail address (like jdoe@mail.desy.de) in the “From:” field of your outgoing mails by default. However, the usage of this physical e-mail address is strongly discouraged by the DESY IT group – see the web page on Mail Accounts at DESY. To make Alpine use your unique e-mail address (like john.doe@desy.de) instead, you can do the following:
In the main menu of Alpine, select “Setup” and then “Config”.
Look for a field named “customized-hdrs” and enter something like in the example below. Make sure you don’t forget the “From:” part.
Find the “alt-addresses” field (far down) and enter your unique e-mail address again. This will let Alpine know that this address belongs to yourself – see the help for more information on the slightly different treatment of such mails. (Type Control-G while the item is highlighted.)
Exit the setup and confirm your changes. You may have to quit and restart Alpine for the new settings to take effect.
customized-hdrs = From: John Doe <john.doe@desy.de>
···
alt-addresses = john.doe@desy.deNew e-mails that you compose after this should use the customised “From:” field in their header. You can check this by activating the display of “Rich Hdrs” in the message composer. (Type Control-R while the cursor is in the header section.)
Many commands in the Geant4 user interface have parameters with default values – either fixed or taken from the current value. You can display these default values by preceding the command name with a question mark on the command prompt:
/gun/position 1 2 3 m
?/gun/position
Current value(s) of the parameter(s) : 100 200 300 cmPlease note that this will only work if the method “G4UImessenger::GetCurrentValue” is implemented properly.
You can use the default value by replacing a parameter with an exclamation mark. This can be useful if you only want to change one of the rear parameters – maybe even without knowing the default values of those at the front:
/vis/sceneHandler/create OGLIX
/vis/viewer/create ! ! 200Note that this will only work if the parameter is marked as “omittable”. Maybe you remember that programs like PAW used the same convention.
All commands in the user interface and their parameters will be explained when you put “help” in front. A single “help” will show you the whole command tree with the possibility to navigate around.
help /event/stack/clearBe aware that the actual content of useful information in this built-in help may vary. However, the tree view will at least show you which commands (with which parameters) exist in the first place.
You can Connect to Your Windows Network Drives directly via Samba:
Open the dialog “Connect to Server…”, select “Windows share”. The server is either “netapp91.desy.de” or “netapp92.desy.de” (try both).
Your Windows home directory (“My Documents”) has a share named “username$” or “groupnameusername$”, where “username” is your AFS username and “groupname” is the name of your working group. Don’t forget the trailing dollar sign. The directory of your group has a share named “groupname$” on one of the servers.
The domain name is “WIN” (case-insensitive). Enter your AFS username and password to log in. Your Windows network drive will appear on the desktop like any other removable volume.
Note that this will only work from within the DESY network. If you want to access your Windows home directory from the outside world, you’ll have to use a VPN connection – see the DESY IT pages on Virtual Private Network at DESY.
OpenOffice has the interesting feature that its documents are simply zip-compressed archives. In order to extract, say, images from an OpenOffice presentation, you can unzip the odp or sxi file and look out for a directory named “Pictures”. Note that you should use the “-d” option of unzip or put the OpenOffice document into its own directory first – otherwise, the files extracted from the archive will pollute your current working directory.
The DESY SMTP server (smtp.desy.de) will not let you send mails to non-DESY addresses when your mail client is running on a host that is not inside the DESY network. If you have written a mail on a non-DESY machine and want to send it to a recipient with a non-DESY address, you can use authenticated SMTP instead:
In your mail client, set the outgoing mail host to smtp-auth.desy.de, enable a secure connection over SSL, make sure you connect to port 587 (instead of the usual port 25), enable authentication by username and password, and finally enter your DESY username and your common DESY password. Note that you need a Windows account to use authenticated SMTP – but you should have one by default, even if you have never used it. For further information, see the DESY IT pages on authenticated SMTP and on the Configuration of Mail Programs in general.
Another possibility is to use a VPN connection to get an IP address within the DESY network. To do this, you need a VPN account and the proper client software on your local machine – see the DESY IT pages on Virtual Private Network at DESY.
If you are logged in on a DL5 or SLD3 machine and you want to connect to another of those computers via SSH, you will usually have to enter your AFS password for each remote login. However, you can instead use the command kinit to obtain a Kerberos ticket-granting ticket – after that, you can login to other DL5 and SLD3 machines without retyping your password each time. Use klist and kdestroy to display and delete the tickets you own, respectively.
You should use the command “type kinit” to check that you are using the program version located at /opt/products/bin/kinit, because there are other versions available which may not work correctly.
Note that this behaviour is actually caused by a bug in the graphical login system of the DESY environment: If you log in on a tty, you will be given a ticket-granting ticket automatically and you won’t have to invoke kinit by hand. The same is true for SLD4.
The X Window System has the possibility to define a “compose character” key, which can be used to enter characters that are not directly available on the keyboard. Under DESY Linux, this “compose character” key is predefined to be Shift + Control. Since this key combination is a bit unhandy to use, you may want to choose another key for the same purpose: Good candiates are the “Windows” and “Menu” keys, which are present on most modern keyboards, but which are usually useless under non-Windows systems.
You can use the following command to define the right “Windows” key as a “compose character” key:
xmodmap -e "keycode 116 = Multi_key"You will typically add this command to your login script. If you prefer the right “Menu” key, enter the keycode 117 instead. You can also use the program xev to find out about the keycodes of other keys on your keyboard. Note that you can have more than one key working as a “compose character”.
Composing characters is straightforward: Hit the “compose character” key (you don’t need to keep it pressed down), then type two characters that make up the composed character. The useful key combinations are often quite intuitive: Just think of the character glyph you want and imagine two simple glyphs of which it could (roughly) be composed – this will usually work, provided that the desired character exists in the current character set.
Typical combinations are all accented characters (' + a → á, ` + E → È, ~ + n → ñ, " + O → Ö, ^ + u → û, and so on), but there are also more exotic combinations like C + , → Ç, a + e → æ, s + s → ß, 3 + 4 → ¾, d + - → đ, ^ + 2 → ², = + L → ₤, > + > → », ( + r → ®, _ + a → ª, x + o → ¤, and others. There is often more than one way to compose a certain character and sometimes the order of the two keystrokes doesn’t even matter. If you enter an invalid combination, nothing will happen at all (except maybe a warning beep). You can find a list of all possible key combinations in the file /usr/X11R6/lib/X11/locale/your_character_set/Compose, where your_character_set is typically something like iso8859-1 or iso8859-15 or maybe utf-8.
Depending on your system software, there may also be other services for entering special characters. For example, if SCIM is running, try hitting Control + Shift + U and enter the hexadecimal codepoint of a Unicode character, finished by Return or Space. The cursor should first change to underlined mode, and then the code should be replaced by the according Unicode character.
When you start Alpine, you will usually be asked for your username and your password. You can do the following in order to avoid confirming your username each time:
In the main menu of Alpine, select “Setup” and then “Config”.
Look for a field named “inbox-path”. It will usually contain <No Value Set: using "{imap.desy.de/ssl}inbox">, which is the default setting provided by the DESY computing environment.
Hit the Return key (or C) to change the value of the entry. When asked for the name of the inbox server, append the string “/user=” and your username to the previous setting. Keep “inbox” as the folder to be used as your inbox. After that, the setup screen should contain a line like in the example below.
Exit the Alpine setup and confirm your changes.
inbox-path = {imap.desy.de/ssl/user=username}inboxWhen you quit and restart Alpine now, you will immediately be asked for your password.
Be aware, however, that this procedure may potentially pose a security risk: If you are used to entering your password immediately after startup, but you run Alpine on another system that behaves differently, you might enter the password when you are in fact asked for the username. In that case, the password would – at least for a moment – be visible on the screen.
Did you ever want to log in to your computer in the morning (or whenever), noticing that the screen resolution had mysteriously changed over night? If you just log in as usual, the font sizes in your graphical user interface may become messed up (illegibly small or needlessly large). Therefore, instead of logging in, restart the X Server by hitting Control + Alt + Backspace. Chances are that things will show up again in their usual size afterwards.
If you’d like to use a digital certificate to sign and encrypt e-mails with Mail, you first have to convert the certificate and your private key to a format that can be imported into your keychain. Open “Terminal”, make sure you have your certificate and private key files at hand, and use the openssl tool to create a PKCS#12 (or p12) file:
openssl pkcs12 -export -in usercert.pem -inkey userkey.pem -out usercert.p12 -name your_nameReplace your_name with your full name (enclosed in quotes), if you like. You will first be asked for your PEM pass phrase, then you’ll have to enter an arbitrary password (twice) by which the contents of the PKCS#12 file should be protected. See the manpage of pkcs12(1) for details.
Import the PKCS#12 file into your keychain. You can either select “Import…” from the “File” menu in the “Keychain Access” utility or simply open the file from the Finder – “Keychain Access” should be launched automatically. You’ll have to enter the password of the PKCS#12 file that you created just a moment ago. When asked for the keychain to which the certificate should be added, select your personal one (or a dedicated keychain that you only use for your personal certificates, if you wish). After that, your certificate and the corresponding private key should appear in your keychain.
Your certificate must have been signed by a CA, but some of these authorities may be unknown and untrusted by default. This is not a major problem when you work with your certificate in a web browser, but the full chain of trust (including all CAs down to the Root CA) must be established in order to use the certificate in a mail client. Therefore, go to the web site of the CA that signed your certificate (e. g. the GridKa CA) and get the corresponding CA root certificate, if needed. Note that you should choose PEM format if possible.
Import the certificate of your Root CA with “Keychain Access”. When asked where the certificate should be added, you should first verify the SHA1 and/or MD5 fingerprint of the certificate you are about to import (click on “View Certificates” to browse the contents of the file). Then select “X509Anchors” as the destination.
Now you should be ready to sign and encrypt your mail using your digital certificate. Open “Mail” and make sure that the sender address you are using is the same as in your certificate. On the right-hand side of the toolbar in the message composing window, you should now see a small star-shaped icon, which you can select to digitally sign your message. If you have a certificate of at least one of your mail recipients in your keychain, another padlock-shaped icon will appear with which you can also encrypt the message.
Your personal keychain is directly accessible with “Keychain Access”, but if you’d like to have a closer look at the “X509Anchors” file, you can inspect it with “Add Keychain…” from the “File…” menu. Open /System/Library/Keychains/X509Anchors to browse the contents of this file with “Keychain Access”. Once you’re done, you can remove it again with “File → Delete Keychain “X509Anchors””. Make sure you select “Delete References” in the dialogue that will pop up in order not to delete the file itself.
If you observe that the memory consumption of a program continuously increases with run time, chances are that you have a memory leak – memory is allocated dynamically by some piece of the code (using the new operator or a function like malloc), but it is never freed afterwards. Depending on how often this happens and how large the allocated blocks are, your program runs out of memory more or less quickly.
Blocks that are allocated only once and not released properly in the end are practically harmless, but if a memory leak occurs in a loop or in a function that is called again and again, you’ll have to find and fix it. If you have already checked your program for typical mistakes (calling new without a subsequent delete) and couldn’t find anything suspicious, you may want to use some dedicated tool to track down memory leaks.
One such tool is Valgrind, which comes – among other things – with a module to check for memory errors (like leaks). Valgrind is preinstalled in the DESY environment under /opt/products/valgrind/3.2.0, other versions are available in the FLC group directory under /afs/desy.de/group/flc/opt/valgrind.
To use Valgrind, type the command valgrind and append the name of your program plus any command line parameters that it may take. Valgrind will then watch over the execution of your program and monitor the memory consumption. After your program has terminated, you’ll get a report on the standard error stream. To track down memory leaks, you should use the option --leak-check=full and possibly --leak-resolution=high (or try med and see what you get).
valgrind --leak-check=full --leak-resolution=high ./bin/Linux-g++/Mokka ./mokka-steerThe report produced by Valgrind will tell you which function calls (in which stack context) caused the memory leaks. Look out for those block counts that increase with run time! For best results, you should recompile your program with debugging enabled – in that case Valgrind will be able to tell you the source file and the line number that is responsible for the problem.
For further information, consult the manpage of valgrind(1) and the Valgrind Documentation. If you have difficulties in understanding the output report, go ahead and ask your friendly computer administrator or somebody else who has a bit of experience in debugging.
emacsDon’t.
To copy a whole directory recursively from one host to another, you would usually use the scp command with the “-r” option (and possibly “-p” as well). However, there is also another possibility that uses the tar utility and sends the data over an SSH tunnel:
ssh hostname tar -c -f - -C parent_directory_path directory_name | tar -xYou may also use the “-z” option (on both sides) to get gzip compression during the transfer. The effect of this pipe construct is basically equivalent to the usual command
scp -rp hostname:parent_directory_path/directory_name .but it has two major advantages: First, using ssh plus tar can be significantly faster than using scp (especially when the directory contains a large number of small files), and second, scp has the known problem that it will always dereference symbolic links – this will not happen with tar unless you use the “-h” option.
Depending on your particular needs, you may also want to try out the rsync tool – see the manpage of rsync(1) and the rsync web pages for details.
xterm at StartupThe HEPiX shell scripts control the startup of your X session. By default, an xterm will be launched right after login (even before your window manager is started). To get rid of this window, create a file ~/.hepix/xclients. If this file is empty, no further applications will be started after logging in.
When you print mails with Alpine, the output will get some rudimentary formatting (header with username, hostname, timestamp, and page number) from the printer spooler, but you can get much nicer results (header with subject, sender, and username; footer with timestamp and page numbering; plus SMTP header highlighting and possible n-up printing) using the a2ps tool. To make Alpine use this tool to process your printing output, do the following:
In the main menu of Alpine, select “Setup” and then “Printer”.
Look for a field named “Printer List” in the section “Personally selected print command” and hit the “A” key to add a printer. Enter an arbitrary name to identify this setting and hit the Return key to accept it. As the command for the printer, enter “a2ps --pretty-print=mail”. a2ps is rather smart in detecting the “language” of its input data, but with the additional option you’ll be on the safe side. a2ps will produce 2-up output by default, but you can also explicitly use one of the options “-1”, “-2”, “-3”, … to set the number of logical pages you’d like to get on one physical page of paper.
You may add further entries to the printer list, for example with different options of a2ps.
Make sure to “Select” your favourite setting (or hit the Return key). The top section of the setup screen should then state this setting as the default printer. Exit the setup and confirm your changes.
Whenever you print a message now, you’ll be asked whether you’d like to use your custom printer setting. If you have multiple settings defined, you may cycle through them with the “Prev Printer” and “Next Printer” commands. You also select “CustomPrint” to choose yet another output filter on the fly. The standard output of the printing command – be it a2ps or anything else – will be displayed on the screen afterwards.
See the manpage of a2ps(1) for more information, and keep in mind that Alpine provides detailed online help in every situation.
It goes without saying that we all hate it, but sometimes there may be no way to avoid using Microsoft Windows. For these rare cases, the DESY IT group provides the Windows Terminal Server, which gives you the possibility to log in to a remote Windows system with the rdesktop command. You’ll need certain resource access permissions in the DESY Registry to use the Terminal Server – in case of doubt, contact the User Consulting Office or your local administrator.
You should log in to adterm.desy.de, which also provides an easy interface to your local disks and to the AFS (your home directory and the DESY cell should already appear as network disks).
rdesktop -r disk:cdrom=/media/cdrom,scratch=/scratch adterm.desy.deSee the manpage of rdesktop(1) for further information about the various options of this command, for example “-f” for full-screen display or “-a” to adjust the colour depth. The key combination Control + Alt + Return can still be used to toggle between full-screen mode and windowed mode. Most of the other keystrokes will be caught and if your keyboard has them, even the Windows-specific keys between Control and Alt should work properly. Control + Alt + Delete will act as expected, but Control + Alt + Backspace will kill your local X Server – avoid it.
Once logged in to the Terminal Server, you may read the online documentation available in the Start Menu. When you’re finished with your session, use the Start Menu to log off properly – don’t just close the rdesktop window.
You can let cron execute scheduled commands for you. These commands will run as “you” (i. e. with your Unix user and group ID) and they will have the same access permissions on files (set with chmod) as if you had executed them directly. However, cronjobs will not automatically get a valid Kerberos ticket, and therefore they can access the AFS only with the permissions of “system:anyuser”, which are usually very restricted.
If you need higher privileges (e. g. in order to write to the AFS or to read from a private area), you can use the K5cron service. It will execute your commands with a valid Kerberos ticket, thus granting you fully authorised access not only to the AFS, but also to other kerberised applications.
To use K5cron, simply install your cronjobs with k5crontab instead of the usual crontab. Since the K5cron service is provided centrally, you’ll have to specify the host on which the command should be run – the hostname has to be given between the time specification and the command itself. The first time you edit your k5crontab, you will see some helpful explanations. For details, refer to the manpages of k5crontab(1) and crontab(5). Note that you will need a valid Kerberos ticket to use k5crontab – use the kinit command to obtain such a ticket if necessary.
You can get rid of your cronjobs by deleting them from your k5crontab. (Note that the k5crontab command lacks the “-r” option of the usual crontab.) When no jobs are left, you will get an error message telling you that “you have no valid entries in your crontab”. You can safely ignore this message – your cronjobs will not be executed anymore.
PS: CERN offers a similar service named acron, which is also revisited in the CNL October 2009.
Section 7.5.1 of the gLite 3 User Guide tells you that you should use the commands lcg-gt and lcg-sd to obtain and release a TURL for a given SURL, respectively.
The first command is called in the form “lcg-gt surl protocol”. Depending on the type of storage, it will return three lines of output: a TURL for the given SURL, the request ID, and the file ID – see the manpage of lcg-gt(1) for details.
lcg-gt srm://srm-dcache.desy.de/pnfs/desy.de/ilc/generated/2007-04-01/file19bcd3a1-49d3-40df-8f9f-7d9b61a352bd gsiftp
gsiftp://dcache20.desy.de:2811//pnfs/desy.de/ilc/generated/2007-04-01/file19bcd3a1-49d3-40df-8f9f-7d9b61a352bd
-2147247363
-2147247362However, with the second command there are two problems: Contrary to the example in the user guide, it has to be called in the form “lcg-sd surl reqid fileid oflag”, with a SURL instead of a TURL and an additional argument at the end (“0” if you obtained the TURL via lcg-gt) – again, see the manpage of lcg-sd(1) for details.
Furthermore, lcg-sd will not work with negative values for the request ID or the file ID since it misinterprets them as (unknown) command line options. Quoting or escaping the arguments won’t help you in this case because this is not a shell-related problem. Instead you can put one additional argument consisting only of two minus signs in front of the numbers. This will indicate that everything which follows should really be treated as an argument and not as an option. (The same works for many other commands – see the manpage of getopt(3) for details.)
lcg-sd srm://srm-dcache.desy.de/pnfs/desy.de/ilc/generated/2007-04-01/file19bcd3a1-49d3-40df-8f9f-7d9b61a352bd -2147247363 -2147247362 0
usage: lcg-sd [-t timeout] surl reqid|token fileid oflag
lcg-sd srm://srm-dcache.desy.de/pnfs/desy.de/ilc/generated/2007-04-01/file19bcd3a1-49d3-40df-8f9f-7d9b61a352bd -- -2147247363 -2147247362 0In the first case, all you get is a generic error message. In the second case, the command succeeds without any further output.
EPS files that have been created with professional graphics programs (e. g. Illustrator) sometimes contain large amounts of unneeded data, such as predefined sets of functions that are actually never invoked. They may also come with a data format that some applications (e. g. gv or xfig) cannot understand correctly. You can use the tool eps2eps to reprocess such files – you will usually get files that are smaller, can be interpreted faster, and comply with the standard. If you wish, you can also switch to another PostScript Language Level. See the manpage of ps2ps(1) for more information.
An impressive example is the ILC Logo, which uses up the ridiculous amount of approximately 600 kB of disk space and which is not even standard EPS. If you pass the file through eps2eps, the remnant will have a size of no more than 5 kB and will contain perfectly valid EPS code – very nice.
Maybe you have noticed that the find command doesn’t work reliably in the AFS. Use the option “-noleaf” to fix this – find will run slower, but it should now work correctly. Note that “-follow” already implies “-noleaf”, and see the manpage of find(1) for further details.
Did you ever create (possibly by accident) a symbolic link in the AFS that pointed to a target starting with a percent sign (%) or a number sign (#)? Such directory entries show rather weird effects: Their properties (access permissions, ownership, modification time, etc.) seem completely damaged, and all you’ll get from them is an error “No such device” – you cannot even delete them anymore! The reason is that such symbolic links are used as an internal representation for mountpoints in the AFS, so you’ll need the command fs rmmount to dispose of them.
ln -s %foo bar
rm bar
rm: cannot remove `bar': No such device
fs rmmount barNote that this is an idiosyncrasy only of the AFS – see the OpenAFS Administration Guide for details.
DESY has a central LDAP server running on the host ldap.desy.de, port 389 (without SSL encryption). The only commonly useful search base (which is also accessible to everybody within the DESY network) is “ou=People,o=DESY,c=DE”. As an example, you can use these values in your favourite mail client to look up mail addresses (for Alpine the corresponding settings are even provided by default).
Apart from that, you can also use the ldapsearch tool to talk to the LDAP server directly. Make sure you select the correct host (option “-h”), simple authentication (option “-x”), and a suitable search base (option “-b”). Following these arguments, you can specify any valid LDAP search filter and the attributes that you’d like to query:
ldapsearch -h ldap.desy.de -x -b ou=People,o=DESY,c=DE uid=vogel
ldapsearch -h ldap.desy.de -x -b ou=People,o=DESY,c=DE objectClass=person mailYou should enclose more complicated filters in quotes because they will often contain characters that are treated specially by the shell. See RFC 4515 for details about the string representation of LDAP search filters.
Note that you can make use of SSH port forwarding if your computer is not within the DESY network: Forward an arbitrary local port to port 389 of ldap.desy.de when connecting e. g. to flcl01.desy.de:
ssh -L 38389:ldap.desy.de:389 flcl01.desy.deYou can then query that port on localhost from another (local) shell:
ldapsearch -h localhost -p 38389 -x -b ou=People,o=DESY,c=DE uid=vogelHowever, this doesn’t work with bastion.desy.de as the destination host.
In case you have an LCD screen, the graphics driver may enable subpixel rendering as soon as you connect your screen to the digital output of the graphics card. This means that the red, green, and blue components of each pixel are controlled separately in order to produce an even more precise rendering, thereby accepting slightly coloured fringes for the benefit of a higher pseudo-resolution. You may consider this a good thing, but you may also find it rather ugly.
In order to avoid subpixel rendering, you can either fall back to the analog output of your graphics card or adjust the settings of your graphics driver: Find the entry “Driver” in the section “Device” of the file /etc/X11/xorg.conf or /etc/X11/XF86Config and consult the corresponding manpage in section 4. See if you can switch subpixel rendering off. As an example, the “radeon” driver would accept the option
Option "SubPixelOrder" "NONE"This line has to go right after the “Driver” entry of your X configuration, but note that superuser privileges are needed to edit this file.
Many e-mail clients are able to handle encrypted incoming mails and decrypt them automatically, provided they have access to your certificate and private key. Newer versions of Alpine also support mail encryption, but this is not enabled in the DESY environment – therefore, if you receive an encrypted mail, you’ll have to decrypt it by hand.
First, save the contents of the mail (typically with MIME type “Application/PKCS7-MIME” and a name like “smime.p7m”) to a file. Then use openssl to decrypt that file:
openssl smime -decrypt -in smime.p7m -inform der -recip .globus/usercert.pem -inkey .globus/userkey.pem -out smime.outThe option “-in” takes the name of the encrypted input file, with the option “-out” you can specify the name of the decrypted output file. Since most mail clients encode outgoing encrypted mail according to the DER format, you’ll often have to use the option “-inform der”. The options “-recip” and “-inkey” specify the location of your certificate and your private key, respectively. You’ll then be prompted for the pass phrase of your private key.
After decrypting the file, you may end up with a mail body containing all kinds of additional MIME information that you’ll have to handle by hand. If there are base64-encoded attachments, openssl can decode them – put the base64-encoded data block into a file and run:
openssl base64 -d -in infile -out outfileIf you are unsure about the contents of the resulting decrypted and decoded file, the Unix file utility may be able to give you a clue.
grepSimilar to ls, the grep utility supports the option “--color” to make its output more legible. The default behaviour is to colour the matching parts of the output in bold (or bright) red. However, you can set the environment variable “GREP_COLOR” to modify the style of highighting – the value may be any number (or pair of numbers) that would usually go between “\e[” and “m” of a terminal escape sequence. For example, if you’d like to have the matches underlined, set GREP_COLOR=4. To always get this output, you can use the environment variable GREP_OPTIONS:
export GREP_COLOR=4
export GREP_OPTIONS=--color=autoUsing “--color=auto” will only insert the additional escape sequences if the output is going directly to a terminal, but not if it is written to a file or passed to a pipe. Note that newer versions of grep offer much more fine-grained control of the output colouring through the variable GREP_COLORS.
See the manpage of grep(1) and a list of Terminal Control Escape Sequences for further information.
You can make use of Kerberos authentication to log in to DESY hosts – if you own a valid Kerberos ticket of the DESY realm, you will be granted access without being asked for your password. Mac OS X already comes with preinstalled Kerberos software, but you need to adjust a few settings in order to use it:
Copy the file /etc/krb5.conf from a DESY host to /Library/Preferences/edu.mit.Kerberos on your local machine.
You may wish to edit the file with a plain-text editor and remove the [appdefaults] section and all entries dealing with WIN.DESY.DE and IFH.DE.
A graphical user interface for Kerberos can be found under /System/Library/CoreServices/Kerberos. If you like, you can create an alias in /Applications/Utilities to access this application more conveniently. If you prefer to work in the Terminal, you can as well use the command-line tools kinit, klist, and kdestroy.
You should now be ready to obtain a Kerberos ticket for the realm DESY.DE and then log in to a DESY host with ssh (or copy files with scp) without being asked for your password again. Keep in mind that only certain hosts are reachable through the DESY firewall from outside the DESY network – you should use bastion.desy.de for remote access.
In order to forward your local ticket to the remote host, enable the option GSSAPIDelegateCredentials in your SSH configuration file (typically ~/.ssh/config). For security reasons, you should forward tickets only to hosts that are trusted by you.
Host bastion bastion.desy.de
HostName bastion.desy.de
User username
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yesFor further information about Kerberos for Macintosh and its configuration, consult the Frequently Asked Questions and the Preferences Documentation of the developers at MIT. For details about the Kerberos configuration file, see the manpage of krb5.conf(5).
When a new shell is started, it reads and executes commands from certain system-wide and user-specific initialisation scripts (typically /etc/profile, ~/.profile, ~/.bashrc, or similar files). You will usually know what your personal scripts are supposed to do, but it may be hard to follow what’s actually happening in the global scripts. It can help to start a shell in verbose mode – then you’ll see all commands echoed to stderr as they are read and executed by the shell:
bash --login --verbose -o xtrace
zsh --login --verbose --xtraceYou can also omit the “--login” option to start a simple non-login shell, which will typically read fewer (or maybe even other) files than a full-blown login shell.
To find out which files (existing as well as non-existing) are accessed by the shell, you can start the process with strace and watch out for file-related system calls:
strace -f -e trace=file bash --login 2> bash.strace <<< exitMore information can be found in the manpage of your favourite shell and in a DESY-specific manpage about the login environment, hepenv(1).
You can define default options for MySQL-related programs in a user-specific configuration file ~/.my.cnf in your home directory. For example, if you find yourself starting the MySQL monitor mysql with the same command-line options again and again, you could define:
[mysql]
host=polui01.in2p3.fr
user=consult
password=consultNote that you can disable all predefined settings (global and personal) with the --no-defaults option. Keep in mind that you can always display the current settings for any MySQL-related command by appending --help as the last option on the command line – the order of the options does matter!
Be aware that you should not store secret passwords on disk. See the section on Using Option Files in the MySQL Reference Manual and the respective manpages for details.
The shell initialisation scripts in the DESY computing environment try to ensure a proper functioning of the “Backspace” (backward delete) and “Delete” (forward delete) keys. Nevertheless, it can still happen that Backspace does not work anymore and produces only “^?”. Instead, Control-H will typically act as the backward delete key then. In order to restore the original setting, you can try to re-set the corresponding terminal property:
stty erase 127Also watch out for possible readline key bindings that can be defined with the bind command of your shell. Results may vary!
If your terminal goes off to some weird graphics display mode (e. g. by accidentally dumping arbitrary binary data to standard output), try to type:
resetwhich might – in your situation – rather look similar to something like:
_␊┌┌> ␊_␊├Chances are that your terminal will be all right afterwards. If reset is not avaliable on your system, you can try “tput reset” or “echo -ne "\ec"” instead.
You can influence the window title of your terminal emulator (xterm and most of its friends) with the escape sequence “\e]0;title\a”, where “\e” represents the Escape character (ASCII 0x1B), “\a” represents the Bell character (ASCII 0x07), and title can be a text of your choice (the closing bracket, the digit zero, and the semicolon have to be entered literally).
In KDE’s terminal emulator Konsole, you can also set the title of your current session tab with “\e]30;tab_title\a”, but note that this is unlikely to work with any other terminal emulator – the sequence will either be ignored, or it may have unwanted side effects. In Gnome’s Terminal, you can set the window title and the title of the corresponding panel button (in the Window List) independently with “\e]2;window_title\a” and “\e]1;panel_title\a” – even though you’d probably find that rather confusing.
Consult the documentation of your shell to learn about its prompting capabilities. If you include terminal control sequences in your prompt string (typically stored in the variable “PS1”), you can – for example – change the window title according to your current working directory. Using bash, you could extend (not replace!) PS1 with the string “\[\e]0;[\h] \w\a\]” in order to have the hostname (enclosed by brackets) and the current working directory displayed in the window title. Note that bash will treat the character sequences “\e”, “\h”, “\w”, and “\a” specially and replace them with the Escape character, the hostname, the current working directory, and the Bell character respectively. The enclosing markers “\[” and “\]” are needed to indicate a character sequence that will not appear on the terminal itself – if you omit them, line breaking will behave oddly. Other shells should have similar (but probably slightly different) ways to accomplish the same.
Have a look at the manpage of console_codes(4) to learn about even more ways to mess up your terminal.
Depending on your language settings, ls will mix filenames starting with lowercase letters, uppercase letters, underscores, and dots. This can be influenced by the LC_COLLATE environment variable: If you set this variable to “C”, filenames will be sorted in strict ASCII ordering (i. e. dot < digits < uppercase < underscore < lowercase). You can define a shell alias if you only want to modify the behaviour of ls:
alias ls='LC_COLLATE=C ls' # … plus possibly other things such as “--color=auto”Export the variable to your environment if you want to change the sorting in other programs, too. (Such as sort – believe it or not!) See the manpages of locale(7) and strcoll(3) for some further information.
findYou can find dangling (a. k. a. orphaned or broken) symbolic links with the “-xtype” test of the find utility. Unlike any other file (including symlinks that point to another valid symlink), dangling symlinks will match “-xtype l”.
find /usr/lib -xtype lMake sure you don’t use the “-follow” or “-L” options, or the behaviour of find will change. Consult the manpage of find(1) for details.
If you can access some resource via HTTP with your web browser, but you cannot retrieve it with a command-line tool like curl or wget, you may need a session cookie that is issued to you by the HTTP server. You can query the server once just to get the cookie (curl option “-c”), and then query it again (presenting the cookie that you just received, curl option “-b”) to get the actual data. If you don’t want to store the cookie in a dedicated “cookie jar” file, you can even pass it through a pipe, using “-” as a symbol for stdout and stdin respectively:
curl -c - url | curl -b - urlwget has similar options – don’t forget to “--keep-session-cookies” for this purpose! See the manpages of curl(1) or wget(1) for details.
Is your computer part of a batch cluster, and are you annoyed by batch jobs that run in the background while you’re trying to work? Batch systems normally run their jobs with a high “nice” value, but the jobs can still have unpleasant side effects if they use slow input/output operations (e. g. over the network or through massive disk access). If this interferes with your interactive work too strongly, you can take a simple countermeasure:
Run a background process that produces a high CPU load, but make sure that it has a high “nice” value and that it doesn’t perform any operations that could block your system. This way, the process will behave really nicely (i. e. it will give way as soon as any other process is supposed to run), but it will make your machine extremely unattractive to the batch system because your system load is always close to (or even slightly above) the maximum. If you have more than one CPU, you may need to run more than one such process. Good candidates are dd (copying data from /dev/zero to /dev/null), yes (writing to /dev/null), shell loops (doing nothing), or whatever you like.
nice -n 19 dd if=/dev/zero of=/dev/null &Keep in mind that this method will not necessarily scare away batch jobs that are already running. Also be aware that it may not be environmentally friendly, because a CPU running under full load will typically use more power than an idle one.
There is no immediate way to tell which directory entries are hard links to the same file – you only know how many hard links for each file exist in total. But you can use find to list all files that have more than one hard link, display their inode numbers with the “-ls” action, and then run the output through sort to see which filenames belong to the same inode (first column of numbers). Since hard links cannot span across filesystems (and inode numbers are unique only per filesystem), you should use the “-xdev” option of find.
find / -xdev -type f -links +1 -ls | sort -nChances are that you will not encounter many of such multiple hard links – only few tools and packages (e. g. ext2/ext3 tools, gzip/bzip2 tools, zoneinfo data) seem to favour hard links over symbolic links these days.
cmake Makes make DoYou certainly know and love cmake, but you probably have wondered what those auto-generated makefiles really do – particularly if it’s not what they’re supposed to. To see every command echoed to the terminal before it gets executed, run make with an additional variable VERBOSE=1:
cmake ..
make VERBOSE=1The same works if VERBOSE=1 is defined as an environment variable. There is also a CMake variable named CMAKE_VERBOSE_MAKEFILE, which will cause makefiles always to be verbose.
Documentation is sparse on this issue, but it works.
In order to distinguish 32-bit and 64-bit versions of executables and shared objects, you can use the ELF header of such files. The easiest way to access this information is probably the “file” utility:
file /lib*/libm-*.soIf your version of file is not smart enough, you can also try “objdump -f” or “readelf -h” to inspect the ELF header. In case you prefer to work on a lower level, check the fifth byte (i. e. offset 4) of the object file: a value of 1 means 32-bit, a value of 2 means 64-bit.
See the manpage of elf(5) for details (particularly “EI_CLASS”), and have a look at the “magic” file /usr/share/file/magic to see how file tries to recognise the different formats.
Where will gcc search for headers and libraries if you don’t specify any “-I” and “-L” options? The preprocessor cpp and the linker ld (which are used internally by gcc) can tell you if they are run in verbose mode:
cpp -v /dev/null
ld --verboseIn the case of the preprocessor, look for “#include "..." search starts here:”, “#include <...> search starts here:”, and “End of search list.”. In the case of the linker, look for “SEARCH_DIR("...")” statements in the configuration script that gets printed out.
The default search paths for the dynamic linker ld.so (which finds shared libraries at runtime) are given in the top-level file /etc/ld.so.conf and typically in further files inside the directory /etc/ld.so.conf.d. See the manpage of ld.so(8) for more information on the loading of shared libraries.
SSH with public-key authentication allows you to log in to a remote host via ssh (or to transfer files with scp or sftp) without entering your password. To use public-key authentication, first create a public/private key pair with “ssh-keygen -t rsa” on your local host, then use ssh-copy-id to transfer the public key (usually named “~/.ssh/id_rsa.pub”) to the file “~/.ssh/authorized_keys” on the remote host.
Take care of the access permissions for your private key – it must not be readable by anyone but you. ssh-keygen will automatically set the Unix access permissions (“mode bits”) to “600”, but keep in mind that this won’t help you if the file is stored in the AFS. Make sure that the ACL of the “~/.ssh” directory does not grant read access to anybody else, or move the private key to another, protected directory and create a symbolic link that points to it.
When the private key is generated, you will be asked for a passphrase to protect it. On the one hand, an unprotected private key with an empty passphrase is dangerous, except you happen to trust all system administrators and everybody who might have access to backup data. On the other hand, a private key with a passphrase doesn’t seem to make much sense because you’ll need to enter its passphrase instead of the password for the remote system. The solution is the SSH agent, which can store passphrase(s) for your private key(s) in a secure way. Chances are that an ssh-agent is already running in local session after login. Just execute ssh-add once, type the passphrase of your private key, and you’re ready to go. Whenever you use ssh, scp, or sftp now, the SSH agent will provide the private key automatically. See the manpages of ssh-add(1) and ssh-agent(1) for further information.
If you already have an existing public/private key pair, you can still add a passphrase to the private key by running “ssh-keygen -p”. Note that there is no need to make any changes to the “authorized_keys” file on the remote host(s) afterwards – this file contains only the public key, which never needs to be protected. The passphrase-protected private key will still match the initial public key.
Keep in mind that public-key authentication is often not a good idea when AFS is involved, because you will not get an AFS token on the remote machine. Instead, try to obtain a Kerberos ticket for the remote host on your local host with kinit, and then log in via SSH with GSSAPI authentication instead of public-key authentication. Have a look at the manpages of ssh_config(5) and ssh(1) for details on ssh, and kinit(1) and krb5.conf(5) for further information on Kerberos.
If you are sure that you want to access an AFS host via public-key authentication, keep in mind that “~/.ssh/authorized_keys” has to be readable for system:anyuser, because you will not have a token at the time you try to log in. Because “~/.ssh” should not be world-readable, you should only grant the “lookup” permission to system:anyuser, put “authorized_keys” into another, public directory, and create a symbolic link that points to it.
Do you need a custom Kerberos 5 configuration file, e. g. to access a realm that is not included in the system-wide /etc/krb5.conf? Just edit your personal copy of krb5.conf and let the environment variable KRB5_CONFIG point to it.
Interestingly, none of the Kerberos 5 manpages mentions the existence of this variable – only sudo(8) knows that it is potentially dangerous for sudo purposes.
If you happen to have some spare time, you may want to have a look at:
boot(7) – general description of boot sequencecrontab(5) – tables for driving cronenviron(5) – user environmentglob(7) – globbing pathnameshier(7) – description of the file system hierarchyld.so(8) – dynamic linker/loaderoperator(7) – C operator precedence and order of evaluationpath_resolution(7) – find the file referred to by a filenameproc(5) – process information pseudo-filesystemreadline(3) – get a line from a user with editingregex(7) – POSIX.2 regular expressionssignal(7) – list of available signalsstdin(3) – standard I/O streamssuffixes(7) – list of file suffixesunicode(7) – the Universal Character Setunits(7) – decimal and binary prefixesutf-8(7) – an ASCII-compatible multi-byte Unicode encodingThose pages contain interesting pieces of information, but maybe you haven’t stumbled across them before – most of them live in the quiet sections 5 and 7 where not many users come by.
xargsA common mistake is to underestimate the power of xargs. It can not only compose argument lists as if you had typed them on the command line, but it can also run processes in parallel. Use the -P option to specify how many invocations of your command should be run in parallel, and use the -n option to limit the number of arguments per invocation – otherwise you’ll likely end up with one process, nevertheless. This can be helpful if you have multiple CPUs or if the execution speed is limited by external factors.
lfc-ls $LFC_HOME | xargs -P 10 -n 1 -i lcg-cp lfn:$LFC_HOME/{} file:$PWD/{}See the manpage of xargs(1) for detailed information.
Inside the AFS, you can easily identify volume mountpoints by an arcane property: stat(2) says that they are directories, but they nevertheless have even inode numbers (unlike regular directories in the AFS). The reason is that mountpoints are internally represented by symlinks, not directories. You can’t use find alone to do the job, but an additional grep for even numbers will help:
find ~ -noleaf -fstype afs -type d -ls | grep '^[0-9]*[02468]\>'
# or with a bit cleaner output:
find ~ -noleaf -fstype afs -type d -printf '%i\t%p\n' | grep '^[0-9]*[02468]\>' | cut -f2Always remember to use the “-noleaf” option when traversing the AFS with find. If you’re sure that you’re searching inside the AFS you can omit the additional test “-fstype afs”.
Alpine supports digital signatures and encryption through S/MIME, and Grid certificates (e. g. from GridKa) can be used for S/MIME, but putting both together will not work in all cases. The reason is that Alpine tries to match the mail address of the sender (or receiver) with a mail address from its certificate store, but it searches in the wrong place: Alpine expects the address as a part of the “Subject” DN, but certificates from GridKa contain the subject’s mail address in an additional “X509v3 Subject Alternative Name” field. As a consequence, you will get the error message
[Couldn't find the certificate needed to sign.]when trying to sign an outgoing mail, or
[Couldn't verify S/MIME signature: certificate verify error]when reading an incoming signed mail.
This can be solved by using a patched version of the S/MIME code for Alpine. Instead of manually searching the mail address in the “Subject” field, the modified version uses the predefined X.509v3 function “X509_get1_email()”, which is able to find the address in both fields. I have submitted that patch a while ago, but up to now it’s only available in the SVN snapshots.
The Geant4 build system can not yet deal with multi-core machines properly (as of version 9.3.p01) – there is no straightforward way to specify that multiple make jobs should be run, and if so, not all target libraries will be made by the Configure script. A possible workaround for this is:
Set the “-j” option in the MAKEFLAGS environment variable and run Configure for the first time. Compilation should work properly, but you will probably see a few error messages concerning files named “obj.last”.
Run Configure a second time (without arguments) to generate the “env.sh” setup script. Specify the Geant4 build directory as your G4WORKDIR and source the setup script.
Invoke make for the “source” subdirectory, making sure that MAKEFLAGS is not set this time. You should see that there is “nothing to be done” in most cases, but several libraries will still be built. The libmap tool will run automatically in the end, and you’re done.
MAKEFLAGS=-j2 ./Configure -build
./Configure
export G4WORKDIR=$PWD
source env.sh
make -C sourceThe second iteration should be rather quick, so you’ll almost save half (three quarters, seven eighths, …) of the total time needed for building Geant4.
When you’re working in a subshell, you may want to go back to the parent for a moment without exiting and losing your current environment. The shell is immune to Control + Z, but you can send it a suspension signal (SIGSTOP) via kill, or you can simply use the built-in command suspend. You can return to the suspended shell with the fg command. An interactive shell cannot run in the background, so you shouldn’t use the bg command or send a continuation signal (SIGCONT). Don’t suspend a shell that has no interactive parent. See the manpages of bash(1) (sections “Job Control” and “Special Parameters”) and signal(7) for further details.
If you find yourself doing this often, it’s probably better and easier to use a tool like screen for managing several shell sessions in parallel.
There may be occasions when you want to exchange stdout and stderr, e. g. to pipe stderr through a filter without throwing away stdout completely. You can do this with the help of a third, auxiliary file descriptor:
cmd 3>&1 1>&2 2>&3- | filterNow the filter will process the original contents of stderr, while the original contents of stdout still are still printed to the terminal. Note that the synchronisation of the two streams will probably be lost due to the different buffering behaviour of stdout and stderr.
Keep in mind that redirections typically make most sense if you read them right-to-left. Look at the chapter on “Redirection” in the manpage of your favourite shell and consult the manpage of stdout(3) for details.
If you want to convert HTML markup from uppercase to lowercase (possibly for a transition to XHTML), the following regular expressions for vim may be helpful. The first converts all element names (in start and end tags), the second converts attribute names (but only if they are not minimised), the third terminates empty elements by appending a slash to their tags (do this only if you want XHTML).
%s/<\/\?\w\+[ >]/\L&/g
%s/\(<[^>]\+\)\@<=\<[[:alnum:]_-]\+="/\L&/g
%s/<\(\(area\|base\|basefont\|br\|col\|frame\|hr\|img\|input\|isindex\|link\|meta\|param\)\>\([^>]*[^/]\)\?\)>/<\1\/>/gMore information about the differences between HTML and XHTML can be found in the XHTML specification.
If you want to pick one of several similar items (e. g. directories with different versions of the same software) it can be an easy solution to have a symlink that points to the item you want. However, you need to pay attention when you want to bend the link to another target, especially when the targets are directories. The following will not work as expected:
ln -s foo-v3 foo # creates foo -> foo-v3
ln -s foo-v4 foo # creates foo-v3/foo -> foo-v4 (dangling)Make sure you use the “-n” option of ln to prevent the old symlink from being treated as a directory. Also use the “-f” option to force the old symlink to be overwritten.
ln -s foo-v3 foo # creates foo -> foo-v3
ln -sfn foo-v4 foo # creates foo -> foo-v4Beware: “ln -sf” can also overwrite regular files, not only symlinks. Also keep in mind that “ln -sf” is not strictly atomic: internally, it removes the old symlink first and then creates a new one, so there is a short period of time where no symlink exists at all. If this is a problem, consider using “mv -T” instead. If you have “mv” aliased to “mv -i”, you’ll also need the “-f” option.
ln -s foo-v3 foo # creates foo -> foo-v3
ln -s foo-v4 bar # creates bar -> foo-v4
mv -T bar foo # yields foo -> foo-v4 (atomic)Instead of the simple “bar”, you’ll probably want to create a unique temporary name that is unlikely to clash with other, existing directory entries.
Suppose you have a list of unordered items (one per line), some of which occur multiple times. You want to shorten the list such that each item is unique, and the items should be ordered by their first occurrence in the original list. An example of this problem may be to find the correct order of “\bibitem{}” entries in a simple LaTeX bibliography environment.
The uniq command will not help you here because it only acts on adjacent lines – that’s why it is usually preceded by sort. sort, on the other hand, destroys the original order. One solution is to use awk:
cmd | awk '++seen[$0] == 1'
cmd | awk '!seen[$0]++' # even shorter, more obscure, less flexibleWhy does it work?
The special variable $0 contains the whole input record (i. e. the entire line).
The variable seen is an associative array that counts the number of occurrences of each line. Its name is arbitrary, it does not have to be “seen”.
Like all other variables, array elements can be created on the fly and will be initialised as empty. awk will only complain about this when running in --lint mode.
Patterns (i. e. “selectors”) can not only be regular expressions, but also anything that evaluates to zero (false) or a non-zero value (true).
If the action for a given pattern is omitted, the default is equivalent to { print $0 }.
Remember that it’s always a good idea to enclose awk scripts on the command line in single quotes.
When invoking the spell-checker from within Alpine, it is currently not possible to select a language on the fly. One solution is to invoke a wrapper script instead, let the user select a language, and then invoke Aspell with the selected language. Here’s an example:
#!/bin/bash
PS3="Please select a language for spell-checking: "
select SPELL_LANG in de_DE en_GB en_US ; do
test -n "${SPELL_LANG}" && exec aspell --dont-backup --mode=email --lang="${SPELL_LANG}" check "$@"
doneA list of available dictionaries may also be generated with “aspell dump dicts”, but note that this list may turn out rather long and will probably contain many near-duplicates.
This solution approach was originally posted on the Alpine-info mailing list.
This page is valid XHTML — Last change: 2013-08-05 by Adrian Vogel, to be contacted via Michael Steder <michael.steder@desy.de>