Information


Documentation



partialator: scaling and post-refinement of partial reflections

Synopsis

partialator -i input.stream -o output.hkl -g detector.geom -b beamline.beam -y pointgroup [options] ...
partialator --help

Description

partialator merges reflections by scaling and post refinement, accounting for the partialities of the reflections. That means that it models the geometry of diffraction for each pattern (crystal orientation, unit cell parameters, X-ray bandwidth and so on) and attempts to optimise the geometrical parameters to make the fully integrated intensities calculated using the model agree as closely as possible between the many patterns.

See the usage cases below for examples of commands to merge reflections in different ways, for example with and without partiality or scaling.

In addition to the output reflection list, partialator will write a file called partialator.params which contains the scaling factors determined for each crystal. After each iteration, a file will be written called pgraph-itern.dat which contains the observed and calculated partialities for a randomly chosen set of "free" reflections which were not included in the refinement.

partialator will also write datasets merged using two halves of the total data using the same filename as you give for -o or --output, with 1 and 2 appended. For example, partialator.hkl1 and partialator.hkl2. Compare these two files with each other to calculate figures of merit such as Rsplit and CC1/2. See the section on Custom Dataset Splitting for more discussion on this topic.

Options

-i filename
--input=filename
Give the name of the input stream.
-o filename
--output=filename
Give the name of the output file. The default is --output=partialator.hkl.
-y pointgroup
--symmetry=pointgroup
Merge according to symmetry pointgroup.
-n n
--iterations=n
Run n cycles of scaling and post refinement.
--no-scale
Disable the scaling part of the refinement calculation.
--no-pr
Disable the orientation/physics model part of the refinement calculation.
-m model
--model=model
Specify the partiality model. See the list below for possible choices.
-j n
Run n analyses in parallel.
--no-polarisation
Disable the polarisation correction.
--max-adu=n
Include reflections only if their peak values were less than n. That means, n is the saturation value of the detector. The default is infinity, i.e. no cutoff.
--min-measurements=n
Include a reflection in the output only if it appears at least least n times. The default is --min-measurements=2.
--push=res=n
merge reflections which are up to n nm-1 higher than the apparent resolution limit of each individual crystal. n can be negative to merge lower than the apparent resolution limit. The default is --push-res=inf, which means no resolution cutoff at all.
--start-after=n
Ignore the first n crystals in the input. The default is --start-after=0, i.e. start at the beginning.
--stop-after=n
Stop processing after n crystals have been successfully read. The default is --stop-after=0, which means to process all the patterns from the start point to the end of the input (see --start-after).
--no-free
Disable cross-validation (for testing only).
--custom-split=filename
Read a list of filenames, event IDs and dataset IDs from filename. See the section on Custom Dataset Splitting below.
--max-rel-B=n
Reject crystals if the absolute values of their relative Debye-Waller ("B") factors are more than n square Angstroms. The default is --max-rel-B=100.
--output-every-cycle
Write out the per-crystal parameters and reflection lists after every cycle of refinement, instead of only at the end. The intermediate reflection lists and parameter filenames will be prefixed with iterN_ (note the trailing underscore), where N is the iteration number. If you use --custom-split, intermediate results will also be output for each custom datset.
--no-logs
Do not write the extensive log files needed for plotting contour maps and spectrum graphs. This makes the process a lot faster, but you probably do want these logs to check that post-refinement is working reasonably.
-w pg

Set the apparent symmetry of the crystals. The ambiguity operator will be determined by comparing this to the actual symmetry.

If you prefer (or the scenario demands it), you can specify the ambiguity operator directly using --operator.

--operator=op

Specify the indexing ambiguity operator. Example: --operator=k,h,-l.

If you prefer, you can specify the ambiguity operator by specifying the apparent symmetry using -w.

--force-bandwidth=bw
--force-radius=R
Set the X-ray bandwidth or initial profile radius for all crystals before proceeding, overriding the values from the stream. Bandwidth is given as a fraction, i.e. --force-bandwidth=0.0013 means 0.13 percent (approximate FWHM). Radius is given in nm^-1.

Partiality models

The available partiality models are:

xsphere

The volume of intersection between a sphere centered on each reciprocal lattice point and the part of reciprocal space excited by the Ewald sphere taking into account the finite bandwidth and convergence angle. A "source coverage factor" is included to take into account the spectral brightness of the effective source for the reflection.

This model is the same as that described in Acta Cryst. D71 (2015) p1400.

unity
Fix all partialities at 1.

Usage cases

Merging without scaling, partialities or post-refinement:
partialator -i my.stream -o my.hkl -y mypointgroup --model=unity --iterations=0
Merging without partialities or post-refinement, but with scaling
partialator -i my.stream -o my.hkl -y mypointgroup --model=unity --iterations=1
(Use a higher number of iterations to increase the accuracy of scaling, but at a cost of more CPU time and possibly more rejected crystals)
Merging with partialities, but without post-refinement and without scaling:
partialator -i my.stream -o my.hkl -y mypointgroup --model=xsphere --iterations=0
Mering with partialities, with scaling but without post-refinement:
partialator -i my.stream -o my.hkl -y mypointgroup --model=xsphere --iterations=1 --no-pr
Merging with partialities, post-refinement and scaling:
partialator -i my.stream -o my.hkl -y mypointgroup --model=xsphere --iterations=1
(Use a higher number of iterations to increase the accuracy of scaling and post-refinement, but at a cost of more CPU time and possibly more rejected crystals)
Merging with partialities and post-refinement, but without scaling:
This would be a strange thing to want to do, however:
partialator -i my.stream -o my.hkl -y mypointgroup --model=xsphere --iterations=1 --no-scale
(Use a higher number of iterations to increase the accuracy of post-refinement, but at a cost of more CPU time and possibly more rejected crystals)

Custom dataset splitting

When performing a time-resolved experiment (for example), it is preferable to ensure that the data for all time points has been processed identically. Rather than processing each time point independently with separate runs of partialator, it is better to process them all together and do the splitting into time points just before the final output. Consider, for example, the case of simple scaling (without a B factor): when merging independently, the resulting datasets would probably end up with different overall scaling factors. When comparing the results, you would need to take this difference into account. In practice, most programs can do that job easily, but what about if a B factor is included? And what if partialities are included - how unique is the solution?

With partialator --custom-split, you can provide a separate text file containing a list of filenames, event numbers and dataset names, one event (detector frame) per line, with the fields separated by any number of spaces, commas or tabs. For each unique dataset name, a separate reflection list will be output. All crystals will be refined together, but they will be merged according to the dataset names you give. The parameters (scaling factors, partialities etc) determined during the joint refinement will be applied. For each dataset, a separate pair of split half-datasets will also be written, allowing you to calculate figures of merit such as Rsplit and CC1/2 for each one.

If the overall output filename (given with -o or --output) were merged.hkl, then a dataset named dataset would be written to merged-dataset.hkl. The corresponding half-datasets would be written to merged-dataset.hkl1 and merged-dataset.hkl2.

Note that the filenames and event names must match exactly what is written into the stream as the Image filename and Event, taking into account options such as indexamajig --prefix and --basename. You should therefore check that the numbers of crystals in each dataset, which will be written on the terminal by partialator, match your expectations and that no patterns have been "lost". There is no requirement for every event in the list to appear in the stream, nor for every event in the stream to belong to one of the datasets. If an event is listed for more than one dataset, the results are "undefined".

If you do not have event IDs for your data, i.e. if you have one detector frame per input file, simply leave out the event IDs from the custom split file.

Finally, note that the main and all custom split datasets, and also all the half-datasets, are subject to --min-measurements.

Author

This page was written by Thomas White.

Reporting bugs

Report bugs to taw@physics.org

Copyright and Disclaimer

Copyright © 2012-2015 Deutsches Elektronen-Synchrotron DESY, a research centre of the Helmholtz Association.

Please read the AUTHORS file in the CrystFEL source code distribution for a full list of contributions and contributors.

CrystFEL is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CrystFEL. If not, see http://www.gnu.org/licenses/.

See also

crystfel indexamajig process_hkl partial_sim

If CrystFEL is installed on your computer, you can read this manual page offline using the command man partialator.