Xdscc12

XDSCC12 is a program for generating delta-CC_1/2 and delta-CC_1/2-ano values for XDS_ASCII.HKL (written by XDS), or for XSCALE.HKL containing several files of type XDS_ASCII.HKL after scaling in XSCALE (with MERGE=FALSE).

It implements the method described in Assmann, Brehm and Diederichs (2016) Identification of rogue datasets in serial crystallography. J. Appl. Cryst. 49, 1021 [1], and it does this not only for the individual datasets in XSCALE.HKL, but also for individual frames, or groups of frames, of a single dataset collected with the rotation method and processed by XDS.

The program can be downloaded for Linux or Mac.

Usage (this text can be obtained with xdscc12 -h):

xdscc12 KD 2019-04-30. Academic use only; no redistribution. -h option shows options.
Please cite Assmann, G., Brehm, W., Diederichs, K. (2016) J.Appl.Cryst. 49, 1021-1028
running 'xdscc12 -h' on 20190502 at 16:11:46 +0200
usage: xdscc12 [-dmin <lowres>] [-dmax <highres>] [-nbin <nbin>] [-mode <1 or 2>] [-<abcdefstwz>] [-r <ref>] FILE_NAME
dmin (default 999A), dmax (default 1A) and nbin (default 10) have the usual meanings.
mode can be 1 (equal volumes of resolution shells) or 2 (increasing volumes; default).
  -r: next parameter: ASCII reference file with lines: h,k,l,Fcalc or h,k,l,Fcalc+,Fcalc-
      this allows calculation of CC of isomorphous signal with reference
  -s: read two columns from reference: Fcalc(+), Fcalc(-). 
      this allows calculation of CC of anomalous signal with that of reference
  -t: total oscillation (degree) to batch fine-sliced frames into
 FILE_NAME can be XDS or XSCALE reflection file
 other options can be combined (e.g. -def), and switch the following off:
  -a: individual isomorphous summary values
  -b: individual (Fisher-transformed) delta-CC1/2 values
  -c: individual delta-CC1/2 reflection numbers
  -d: individual anomalous summary values
  -e: individual (Fisher-transformed) delta-CC1/2ano values
  -f: individual delta-CC1/2ano reflection numbers
  -w: weighting of intensities with their sigmas
  -z: no Fisher transformation of delta-CC1/2 values

The program output in the terminal window is terse but supposed to be self-explanatory; it can (and most often should) be saved or re-directed to a file.

xdscc12 ... > xdscc12.log  #  or xdscc12 ... | tee xdscc12.log

If xdscc12 is used with XDS_ASCII.HKL, the isomorphous delta-CC_1/2 of a batch of frames (width chosen with the -t option) relative to all data is most easily visualized via XDSGUI (Statistics tab); the delta-CC_1/2-ano may be plotted with e.g. gnuplot after grepping the relevant lines from the output.

If xdscc12 is used with a XSCALE.HKL generated from multiple datasets, the output lines show the contribution of each dataset toward the total CC_1/2. Negative numbers indicate a worsening of the overall signal. In this case, the program writes a file called XSCALE.INP.rename_me which shows statistics of delta-CC_1/2 and delta-CC_1/2-ano values, and has a sorted list on INPUT_FILEs - the first one provides the best data set, and the last one is the worst one. This XSCALE.INP.rename_me can then be edited (i.e. deleting a few data sets with very negative delta-CC_1/2), and renamed to XSCALE.INP.

Statistics are given (in resolution shells) for the isomorphous and the anomalous signal.

To find outlier data sets in XSCALEd data, you may try the -w option. This can be helpful since the a and b parameters of the error model are adjusted by XSCALE such that the sigmas are very high, which may reduce the delta-CC_1/2 signal.

A complete description of how to process serial crystallography data with XDS/XSCALE is given in SSX. A program that implements the method of Brehm and Diederichs (2014) and theory of Diederichs (2017) is xscale_isocluster.

To remove bad frames from a XDS_ASCII.HKL file, you can use the program exclude_data_range_from_XDS_ASCII.HKL, or re-INTEGRATE with the keyword EXCLUDE_DATA_RANGE in XDS.INP.

Correlation against a reference data set (-r <reference> option)

The correlation of the experimental data set against the user-supplied reference data is shown in the lines starting with r. To prepare a reference data set if the refinement was done with phenix.refine, one could use e.g.

mtz2various hklin 2bn3_refine_001.mtz hklout temp.hkl <<eof
OUTPUT USER *
LABIN FC=F-model PHIC=PHIF-model
END
eof

- the column corresponding to PHIC will not be used by xdscc12. Alternatively,

sftools
read mymodel_001.mtz
write temp.hkl format(3i5,f10.3) col F-model
y
quit

Reference data with anomalous signal (additional -s option)

The correlation of the anomalous difference of the experimental data set against the anomalous signal of the user-supplied reference data is shown in the lines starting with s. A simple way to obtain Fcalc(+) and Fcalc(-) is to run phenix.refine with options (in case of S as anomalous scatterer)

refinement.input.xray_data.labels="F(+),SIGF(+),F(-),SIGF(-),merged"  refinement.refine.anomalous_scatterers.group.selection="element S" strategy=individual_sites+individual_adp+group_anomalous+occupancies

and then

sftools <<eof
read mymodel_001.mtz
write anom-reference.hkl format(3i5,2f10.3) col "F-model(+)" "F-model(-)"
y
quit
eof

in which case sftools outputs only the acentric reflections - only those have anomalous differences. XDSCC12 then has to be run with the -s -r anom-reference.hkl option.