Xdscc12

XDSCC12 is a program for generating delta-CC_1/2 and delta-CC_1/2-ano values for XDS_ASCII.HKL (written by XDS), or for XSCALE.HKL containing several files of type XDS_ASCII.HKL after scaling in XSCALE (with MERGE=FALSE).

It implements the method described in Assmann, Brehm and Diederichs (2016) Identification of rogue datasets in serial crystallography. J. Appl. Cryst. 49, 1021 [1], and it does this not only for the individual datasets in XSCALE.HKL, but also for individual frames, or groups of frames, of a single dataset collected with the rotation method and processed by XDS.

The program can be downloaded for Linux or Mac.

Usage (this text can be obtained with xdscc12 -h):

xdscc12 KD 2019-04-30. Academic use only; no redistribution. -h option shows options.
Please cite Assmann, G., Brehm, W., Diederichs, K. (2016) J.Appl.Cryst. 49, 1021-1028
running 'xdscc12 -h' on 20190502 at 16:11:46 +0200
usage: xdscc12 [-dmin <lowres>] [-dmax <highres>] [-nbin <nbin>] [-mode <1 or 2>] [-<abcdefstwz>] [-r <ref>] FILE_NAME
dmin (default 999A), dmax (default 1A) and nbin (default 10) have the usual meanings.
mode can be 1 (equal volumes of resolution shells) or 2 (increasing volumes; default).
  -r: next parameter: ASCII reference file with lines: h,k,l,Fcalc or h,k,l,Fcalc+,Fcalc-
      this allows calculation of CC of isomorphous signal with reference
  -s: read two columns from reference: Fcalc(+), Fcalc(-). 
      this allows calculation of CC of anomalous signal with that of reference
  -t: total oscillation (degree) to batch fine-sliced frames into
 FILE_NAME can be XDS or XSCALE reflection file
 other options can be combined (e.g. -def), and switch the following OFF:
  -a: individual isomorphous summary values
  -b: individual (Fisher-transformed) delta-CC1/2 values
  -c: individual delta-CC1/2 reflection numbers
  -d: individual anomalous summary values
  -e: individual (Fisher-transformed) delta-CC1/2ano values
  -f: individual delta-CC1/2ano reflection numbers
  -w: weighting of intensities with their sigmas
  -z: Fisher transformation of delta-CC1/2 values

The program output in the terminal window is terse but supposed to be self-explanatory; it can (and most often should) be saved or re-directed to a file.

xdscc12 ... > xdscc12.log  #  or xdscc12 ... | tee xdscc12.log

All statistics (tables) produced by xdscc12 may be visualized with e.g. gnuplot, after grepping the relevant lines from the output. If xdscc12 is used with a XDS_ASCII.HKL reflection file (from XDS), the isomorphous delta-CC_1/2 of a batch of frames (width chosen with the -t option) relative to all data is most easily visualized via XDSGUI (Statistics tab). Negative numbers indicate a worsening of the overall signal.

If xdscc12 is used with a XSCALE.HKL generated from multiple datasets, the output lines show the contribution of each dataset toward the total CC_1/2. In this case, the program writes a file called XSCALE.INP.rename_me which shows statistics of delta-CC_1/2 and delta-CC_1/2-ano values, and has a sorted enumeration of the INPUT_FILEs - the first of these provides the best data set, and the last one is the worst one. This XSCALE.INP.rename_me can then be edited (i.e. for deleting a few data sets with very negative delta-CC_1/2), and renamed to XSCALE.INP.

Statistics are given (in resolution shells) for the isomorphous and the anomalous signal. In case of SSX data (which have few reflections per data set, compared to complete data sets), we typically use nbin of 1 in xdscc12.

To find out about the influence of the a and b parameters of the XDS/XSCALE-adjusted error model, you may try the -w option; this assigns the same sigma to all reflections. Likewise, the Fisher transformation, which serves to make changes in CC_1/2 comparable across resolution ranges, may be switched off for testing purposes, with the -z option.

Correlation against a reference data set (-r <reference> option)

The correlation of the experimental data set against the user-supplied reference data is shown in the lines starting with r. To prepare a reference data set if the refinement was done with phenix.refine, one could use e.g.

mtz2various hklin 2bn3_refine_001.mtz hklout temp.hkl <<eof
OUTPUT USER *
LABIN FC=F-model PHIC=PHIF-model
END
eof

- the column corresponding to PHIC will not be used by xdscc12. Alternatively,

sftools
read mymodel_001.mtz
write temp.hkl format(3i5,f10.3) col F-model
y
quit

Reference data with anomalous signal (additional -s option)

The correlation of the anomalous difference of the experimental data set against the anomalous signal of the user-supplied reference data is shown in the lines starting with s. A simple way to obtain Fcalc(+) and Fcalc(-) is to run phenix.refine with options (in case of S as anomalous scatterer)

refinement.input.xray_data.labels="F(+),SIGF(+),F(-),SIGF(-),merged"  refinement.refine.anomalous_scatterers.group.selection="element S" strategy=individual_sites+individual_adp+group_anomalous+occupancies

and then

sftools <<eof
read mymodel_001.mtz
write anom-reference.hkl format(3i5,2f10.3) col "F-model(+)" "F-model(-)"
y
quit
eof

in which case sftools outputs only the acentric reflections - only those have anomalous differences. XDSCC12 then has to be run with the -s -r anom-reference.hkl option.

See also

A complete description of how to process serial crystallography data with XDS/XSCALE is given in SSX.

A program that implements the method of Brehm and Diederichs (2014) and theory of Diederichs (2017) is xscale_isocluster.

To remove bad frames from a XDS_ASCII.HKL file, you can use the program exclude_data_range_from_XDS_ASCII.HKL, or re-INTEGRATE with the keyword EXCLUDE_DATA_RANGE in XDS.INP.