CrystFEL

Information

Documentation

CrystFEL processing best practice

This page contains our most recent recommendations to processing SFX data using CrystFEL, as well as some hints and tips.

Please see the update log at the end of this page to check when these guidelines have been changed.

General

Read all the messages written to the terminal by the various CrystFEL programs. They contain important information and useful advice.

Peak search

Good peak detection is essential for indexing, so it's worth spending a long time optimising the peak search. Experiment with and use whichever combination you like from the peak detection options, which are: --peaks=zaef, --peaks=hdf5, --threshold, --gradient, --min-snr, --median-filter, --filter-noise, --no-use-saturated , --no-revalidate and --check-hdf5-snr. None of these options can do anything other than increase or decrease the indexing success rate.
Note that --use-saturated has been the default since CrystFEL 0.5.1. This option states that saturated peaks are to be given to the indexing procedure. It does not mean that their intensities will be integrated and included in the final results.
--check-hdf5-snr was the default behaviour prior to CrystFEL 0.5.3. Since then, the default behaviour has been not to check the SNR of peaks if you use --peaks=hdf5. If your indexing rate dropped after upgrading to CrystFEL 0.5.3, add this option to restore the old behaviour.

Detector geometry

Geometry refinement can make a big difference! It is worth spending some time on it.

Indexing

When initially determining the unit cell parameters, try indexing with mosflm first. Mosflm is the only indexing method which can guess the lattice type.
Once you've determined the cell parameters, use as many indexing methods as you can, provided that they all use -cell, -axes or comb. The more, the merrier! You can sometimes increase the indexing rate by combining multiple invocations of the same indexing program, for example: --indexing=mosflm-latt,mosflm-nolatt. The easiest way to get a good list of indexing methods is to simply omit the --indexing argument. Indexamajig will then automatically determine the available methods.
Set the unit cell tolerances (--tolerance) wide enough to capture the entire distributions of all the parameters, unless you have some reason to restrict it. The widths of the distributions vary between samples, so you have to measure and then configure this yourself.
Don't use indexamajig's --tolerance parameter to separate subtly different batches of crystals. There are several "gotchas" here, such as that the tolerances are applied to the reciprocal cell parameters, if applicable after converting a centered cell to a primitive one. The --tolerance parameter is only intended as a coarse tolerance to exclude outliers. If you need to do finer filtering, use either stream_grep or a custom script on the stream afterwards.
If you are using a very old version of CrystFEL, avoid using -bad anywhere in your list of indexing methods. Add this option only if it's really necessary.
Do not use -comb if once of the cell parameters is a simple multiple of the others, e.g. if a is approximately equal to 2b. This happens with tetragonal lysozyme, so don't assume it won't happen to you. -comb is the default for many indexing methods, so specify -axes explicity.
If you are using a very old version of CrystFEL, avoid the options --integrate-found and --min-integration-snr. Newer versions don't have either of these options.
If you are using CrystFEL 0.5.3 or later, use the Unit Cell Explorer instead of the old cell-please script. The Unit Cell Explorer does a much better job of quantifying the cell parameters and deals with centered lattices properly.

Integration

Use --integration=rings-nocen-nosat-norescut. This is the default. --integration=rings implies all the other options nocen-nosat-norescut.
If you are using CrystFEL 0.5.3 or later, set max_adu to a high value in the geometry file (or omit it altogether), and instead use process_hkl --max-adu during the merging stage to set the saturation cutoff. See the merging section below for more advice on saturation levels for CSPAD detectors.
If you are using CrystFEL 0.5.3 or later, cautiously try adding -cen to the integration option, i.e. --integration=rings-cen. This may improve the low resolution data, but could result in a subtle bias of the intensities of high resolution or systematically absent reflections towards positive values.
If you are using CrystFEL 0.5.2 or earlier, the use of -cen is not recommended. The criterion for when the integration region would be centred was made tighter between 0.5.2 and 0.5.3. In CrystFEL 0.5.1, there was a severe bug in the centering procedure.
Test the application of a per-crystal resolution cutoff by adding -rescut to the integration options. This is not dangerous, but might reduce the data quality. It might also increase it. Plase consider sending us feedback on whether this improves or compromises the data quality.
If you are using CrystFEL version 0.6.0 or later, the parameters for spot prediction (divergence, bandwidth and reciprocal space profile size) will be determined automatically. If there seem to be too many or too few predicted spots, try setting these parameters manually using indexamajig --fix-profile-radius, --fix-bandwidth and --fix-divergence. Please consider sending us feedback on this feature.
If you see a lot of warnings about n implausibly negative reflections, use --int-diag=implausible to inspect the peaks directly and see what's going on. The most common cause is one or two bad pixels with very low values in a region of high background.

Merging

Merge using partialator -i my.stream -o my.hkl -y sym --model=unity --iterations=1.
Merge Friedel pairs to get higher data quality. To do this, give a centrosymmetric point group for sym. The symmetry chart lists the centrosymmetric version (the Laue Class) of each point group. Refer to the front page of the manual if you get confused about how to specify your point group on the command line.
Specify the unique axis if your lattice is monoclinic, because there are at least two common conventions. CrystFEL assumes unique axis c, and other software probably assumes unique axis b. To merge according to point group 2/m (i.e. point group 2 with merging of Friedel pairs), use -y 2/m_uab.
Check the saturation level using scripts/peakogram-stream and set --max-adu appropriately. See the tutorial for an example. If you're processing CSPAD (LCLS) data, the saturation level will be close to 14000, but it varies between datasets.
Don't use --sum, --max-only, --no-polarisation, --min-snr or --reference. These options are for diagnostic testing only.
For monochromatic synchrotron data, try post-refinement in partialator using --model=offset. After merging, run the scripts plot-pr and plot-pr-contourmap to evaluate where the refinement has found the global minimum for most crystals, and if the plot of observed vs calculated partiality is close to a straight line through the origin.

Figures of merit

Calculate at least R_split and CC^* in resolution shells.
Do not use the option --sigma-cutoff for check_hkl or compare_hkl without a good reason.

Update log

18 September 2019: Added advice about --model=offset for monochromatic synchrotron data.
13 March 2017: Added advice about not using --tolerance as an isomorphism filter.
16 November 2016: Changed the recommended merging program from process_hkl to partialator, and updated advice about saturation levels.
8 June 2015: Advice regarding partialator updated.
20 February 2015: Advice added about automatic prediction parameter determination.
9 January 2015: Advice about saturation values for CSPAD altered: 14000 is now the recommended value, changed from 3500.