|
|
| Line 491: |
Line 491: |
| </pre> | | </pre> |
| to copy the script from this website into an executable file generate_XDS.INP in your current directory. | | to copy the script from this website into an executable file generate_XDS.INP in your current directory. |
|
| |
| ==Variant script (xds_generate_all)==
| |
| This script includes some minor modifications of the Generate_XDS.INP script - for instance, it tells the user the energy, as well as the wavelength of the X-rays, and gives the user the beam center in mm as well as pixels - values that may be useful when switching between HKL2000, MOSFLM and XDS.
| |
|
| |
| It also generates two executable shell scripts in the same directory as XDS.INP.
| |
|
| |
| The first of these scripts, xds_graph.sh, extracts various statistics from INTEGRATE.LP and CORRECT.LP, pipes them out to a logfile, INTEGRATE_STATS.LP, and plots them using loggraph. This script does not have some of the more useful features of XDSSTAT, such as calculation of per frame Rmeas values, but it does give a quick overview of various data quality parameters by resolution and image number. This script will only work with recent versions of XDS (after calculation of CC(1/2) - not sure exactly which version).
| |
|
| |
| The second of these scripts, xds_to_ccp4.sh, will take XDS_ASCII.HKL and generate a CCP4-format MTZ file, containing anomalous data (as F(+)/F(-) and DANO/SigDANO) and a test set of 5% of reflections for calculation of the free R-factor. It will also generate an unmerged SHELX format HKL file, for input into SHELXC/D/E (e.g. via the HKL2MAP GUI).
| |
|
| |
| Both these shell scripts can by installed system wide by transferring them to a directory in your path, for example the directory containing your XDS binaries.
| |
|
| |
| Here is the script:
| |
| <pre>
| |
| #!/bin/bash
| |
| # purpose: xds_generate_all
| |
| #
| |
| # tested with some datasets from ALS, SSRL, SLS, ESRF and BESSY; only MARCCD, ADSC/SMV, PILATUS detectors;
| |
| # for other detectors, values marked with XXX must be manually filled in.
| |
| #
| |
| # revision 0.03 . Kay Diederichs 2/2010
| |
| # revision 0.04 . Kay Diederichs 4/2010 - include alternative ORGX, ORGY calculations for ADSC
| |
| # revision 0.05 . Kay Diederichs 5/2010 - grep for "Corrected" in addition to "marccd"; needed for BESSY
| |
| # revision 0.06 . KD 6/2010 - add UNTRUSTED_RECTANGLE and UNTRUSTED_ELLIPSE; use `whereis catmar` and so on
| |
| # revision 0.07 . KD 6/2010 - decide about ORGX/Y info in MAR header being pixels or mm; other fixes
| |
| # revision 0.08 . KD 6/2010 - fixes for Pilatus 6M
| |
| # revision 0.09 . KD 6/2010 - get rid of requirement for mccd_xdsparams.pl and/or catmar; rather use "od"
| |
| # revision 0.10 . Tim Gruene 7/2010 - set link 'images' to image directory if path exceeds 72 characters
| |
| # revision 0.11 . KD 7/2010 - for MarCCD: look for distance info at different byte position
| |
| # revision 0.12 . KD 7/2010 - fix for negative PHISTART
| |
| # revision 0.13 . KD 8/2010 - store correct NX NY QX QY in XDS.INP
| |
| # revision 0.14 . KD 1/2011 - SENSOR_THICKNESS for Pilatus; MINIMUM_NUMBER_OF_PIXELS_IN_A_SPOT=3
| |
| # revision 0.15 . KD 2/2011 - add comment for -ive sign of APS 19-ID and Australian Synchrotron rotation axis
| |
| # revision 0.16 . KD 3/2011 - SENSOR_THICKNESS=0.01 for ADSC and MarCCD. Add comment about SILICON=
| |
| # revision 0.17 . KD 3/2011 - make it work for .bz2 frames; improve screen output
| |
| # revision 0.18 . KD 4/2011 - faster by doing "strings" only once; revert "images/${1##/*/}" "correction"
| |
| # revision 0.19 . KD 6/2011 - bugfix for 0.18
| |
| # revision 0.20 . KD 7/2011 - redirect stderr of /bin/ls to /dev/null
| |
| # revision 0.21 . KD 11/2011 - SEPMIN, CLUSTER_RADIUS hints; read NX NY from header (for Pilatus 2M)
| |
| # revision 0.22 . KD 12/2011 - Pilatus 2M UNTRUSTED_RECTANGLE lines, SENSOR_THICKNESS from header
| |
| # revision 0.23 . KD 1/2012 - add UNTRUSTED_QUADRILATERAL, remove MINIMUM_ZETA (0.05 is default now)
| |
| # revision 0.24 . KD 3/2012 - remove revision 0.10 since XDS now takes much longer paths
| |
| # revision 0.25 . KD 3/2012 - remove revision 0.22 for PSI Pilatus 2M; see http://www.globalphasing.com/autoproc/wiki/index.cgi?TroubleShootingKnownIssues
| |
| # revision 0.26 Oliver Clarke 10/12 - Added generation of shell scripts for conversion to MTZ and visualisation of statistics. Other minor alterations to XDS.INP.
| |
| # revision 0.27 Oliver Clarke 10/12 - xds_graph.sh will now run xdsstat, if available, and plot Rmeas and Rd per image.
| |
| # revision 0.28 Oliver Clarke 10/12 - xds_graph was not plotting h00, 0k0, 00l reflections when a reference dataset was present. Fixed.
| |
| # revision 0.29 KD 11/2012 - EXCLUDE_RESOLUTION_RANGE lines and generic Pilatus Flat_field test
| |
| # revision 0.30 Keitaro 11/2012 - for MarCCD: read oscillation range from the position 1024+736 (fix for omega rotation) XXX for STARTING_ANGLE=, needs more work to support both phi and omega rotation
| |
| REVISION="0.30 (28-Nov-2012)"
| |
| #
| |
| # usage: e.g. xds_generate_all "frms/mydata_1_???.img"
| |
| # make sure to have the two quotation marks !
| |
| # the ? are wildcards for the frame numbers.
| |
| #
| |
| # limitations:
| |
| # - frame numbers are assumed to start with 1 and run consecutively
| |
| #
| |
| # known problems:
| |
| # - for ADSC detectors, there are at least three ways to obtain ORGX and ORGY values from the header (see below);
| |
| # - the same might be a problem for MAR headers, too (not sure about this)
| |
| #
| |
| # notes for debugging of the script:
| |
| # - add the -v option to the first line, to see where an error occurs
| |
| # - comment out the removal of tmp1 and tmp2 in the last line
| |
| #
| |
| # ====== Start of script ======
| |
| echo generate_XDS.INP version $REVISION . Obtain the latest version from
| |
| echo http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/generate_XDS.INP
| |
| if [ "$1" == "help" ] || [ "$1" == "-help" ] || [ "$1" == "-h" ]; then
| |
| echo usage: xds_generate_all \"frms/mydata_1_???.img\" \(_with_ the quotation marks!\)
| |
| echo if the frames are compressed with bzip2, leave out the .bz2 extension!
| |
| exit
| |
| fi
| |
| #
| |
| # defaults:
| |
| #
| |
| DETECTOR="XXX MINIMUM_VALID_PIXEL_VALUE=XXX OVERLOAD=XXX"
| |
| ORGX=XXX
| |
| ORGY=XXX
| |
| ORGXMM=XXX
| |
| ORGYMM=XXX
| |
| DETECTOR_DISTANCE=XXX
| |
| OSCILLATION_RANGE=XXX
| |
| X_RAY_WAVELENGTH=XXX
| |
| X_RAY_ENERGY=XXX
| |
| STARTING_ANGLE=XXX
| |
| QX=XXX
| |
| QY=XXX
| |
| NX=XXX
| |
| NY=XXX
| |
| NXMM=XXX
| |
| NYMM=XXX
| |
| SENSOR_THICKNESS=0
| |
| # see how we are called:
| |
| NAME_TEMPLATE_OF_DATA_FRAMES="$1"
| |
| # list frames matching the wildcards in NAME_TEMPLATE_OF_DATA_FRAMES
| |
| # don't accept the "direct beam" shot at SLS/Pilatus PX-I and PX-II
| |
| /bin/ls -C1 $1 $1.bz2 2>/dev/null | egrep -v "_00000.cbf|_000.img" > tmp1 || exit 1
| |
|
| |
| # we can continue - the frames are found
| |
|
| |
| # set upper limit of DATA_RANGE to number of frames (see "limitations" above)
| |
| DATA_RANGE=`wc -l tmp1 | awk '{print $1}'`
| |
|
| |
| # set upper limit of SPOT_RANGE to half of DATA_RANGE, but not less than 1
| |
| SPOT_RANGE=`echo "scale=0; $DATA_RANGE/2" | bc -l`
| |
| SPOT_RANGE=`echo "if ($SPOT_RANGE<1) 1;if ($SPOT_RANGE>1) $SPOT_RANGE" | bc -l`
| |
|
| |
| echo DATA_RANGE=1 $DATA_RANGE
| |
|
| |
| # find out detector type
| |
| DET=XXX
| |
| FIRSTFRAME=`head -1 tmp1`
| |
| echo $FIRSTFRAME | grep -q bz2 && bzcat $FIRSTFRAME > tmp1 && FIRSTFRAME=tmp1
| |
| strings $FIRSTFRAME > tmp2
| |
| egrep -q 'marccd|Corrected' tmp2 && DET=mccd
| |
| grep -q PILATUS tmp2 && DET=pilatus
| |
| grep -q BEAM_CENTER_X tmp2 && DET=adsc
| |
| # identify other detector types in the same way (MAR IP would be straightforward)
| |
|
| |
| # parse ASCII header of first frame
| |
|
| |
| if [ "$DET" == "XXX" ]; then
| |
| echo "this is not a MAR, ADSC/SMV or PILATUS detector - fill in XXX values manually!"
| |
| DETECTOR="XXX MINIMUM_VALID_PIXEL_VALUE=XXX OVERLOAD=XXX"
| |
|
| |
| # find parameters of first frame
| |
| elif [ "$DET" == "mccd" ]; then
| |
| echo Data from a MarCCD detector
| |
|
| |
| DETECTOR="CCDCHESS MINIMUM_VALID_PIXEL_VALUE= 1 OVERLOAD= 65500"
| |
| SENSOR_THICKNESS=0.01
| |
| # use first frame of dataset to obtain parameters
| |
|
| |
| # offsets are documented; values can be find in mccd_xdsparams.pl script
| |
| let SKIP=1024+80
| |
| NX=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| let SKIP=$SKIP+4
| |
| NY=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
|
| |
| let SKIP=1720
| |
| DETECTOR_DISTANCE=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| DETECTOR_DISTANCE=`echo "scale=3; $DETECTOR_DISTANCE/1000" | bc -l`
| |
|
| |
| let SKIP=1024+256+128+256+4
| |
| ORGX=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| ORGX=`echo "scale=2; $ORGX/1000" | bc -l `
| |
| let SKIP=$SKIP+4
| |
| ORGY=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| ORGY=`echo "scale=2; $ORGY/1000" | bc -l `
| |
|
| |
| let SKIP=1024+256+128+256+44
| |
| PHISTART=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| STARTING_ANGLE=`echo "scale=3; ($PHISTART/1000)" | bc -l`
| |
| let SKIP=1024+736
| |
| OSCILLATION_RANGE=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| OSCILLATION_RANGE=`echo "scale=3; $OSCILLATION_RANGE/1000" | bc -l`
| |
|
| |
| let SKIP=1024+256+128+256+128+4
| |
| QX=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| QX=`echo "scale=10; $QX/1000000" |bc -l `
| |
| let SKIP=$SKIP+4
| |
| QY=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| QY=`echo "scale=10; $QY/1000000" |bc -l `
| |
|
| |
| let SKIP=1024+256+128+256+128+128+12
| |
| X_RAY_WAVELENGTH=$(od -t dI -j $SKIP -N 4 $FIRSTFRAME | head -1 | awk '{print $2}')
| |
| X_RAY_WAVELENGTH=`echo "scale=5; $X_RAY_WAVELENGTH/100000" | bc -l`
| |
| X_RAY_ENERGY=`echo "12398.5/$X_RAY_WAVELENGTH" | bc -l | awk '{printf "%5.1f", $1}'`
| |
| # at most BLs, ORGX and ORGY are in pixels, but sometimes in mm ... guess:
| |
| NXBYFOUR=`echo "scale=0; $NX/4" | bc -l `
| |
| ORGXINT=`echo "scale=0; $ORGX/1" | bc -l `
| |
| if [ $ORGXINT -lt $NXBYFOUR ]; then
| |
| ORGXMM=$ORGX
| |
| ORGYMM=$ORGY
| |
| NXMM=`echo "$NX*$QX" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| NYMM=`echo "$NY*$QY" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| ORGX=`echo "scale=1; $ORGX/$QX" | bc -l`
| |
| ORGY=`echo "scale=1; $ORGY/$QY" | bc -l`
| |
| echo MARCCD detector: header ORGX, ORGY seem to be in mm ... converting to pixels
| |
| else
| |
| ORGXMM=`echo "$ORGX*$QX" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| ORGYMM=`echo "$ORGY*$QY" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| NXMM=`echo "$NX*$QX" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| NYMM=`echo "$NY*$QY" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| echo MARCCD detector: header ORGX, ORGY seem to be in pixel units
| |
| fi
| |
|
| |
| elif [ "$DET" == "adsc" ]; then
| |
|
| |
| DETECTOR="ADSC MINIMUM_VALID_PIXEL_VALUE= 1 OVERLOAD= 65000"
| |
| echo Data from ADSC detector. Obtaining ORGX, ORGY depends on beamline setup:
| |
| SENSOR_THICKNESS=0.01
| |
| sed s/\;// tmp2 > tmp1
| |
| mv tmp1 tmp2
| |
|
| |
| # find X_RAY_WAVELENGTH:
| |
| X_RAY_WAVELENGTH=`grep WAVELENGTH tmp2 | head -1 | sed s/WAVELENGTH=//`
| |
|
| |
| # find NX, QX, ORGX and ORGY:
| |
| X_RAY_ENERGY=`echo "12398.5/$X_RAY_WAVELENGTH" | bc -l | awk '{printf "%5.1f", $1}'`
| |
| STARTING_ANGLE=`grep PHI tmp2 | sed s/PHI=//`
| |
| NX=`grep SIZE1 tmp2 | tail -1 | sed s/SIZE1=//`
| |
| QX=`grep PIXEL_SIZE tmp2 | sed s/PIXEL_SIZE=//`
| |
| # FIXME - next 2 lines should be done properly, from header
| |
| NY=$NX
| |
| QY=$QX
| |
| NXMM=`echo "$NX*$QX" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| NYMM=`echo "$NY*$QY" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| BEAM_CENTER_X=`grep BEAM_CENTER_X tmp2 | sed s/BEAM_CENTER_X=//`
| |
| BEAM_CENTER_Y=`grep BEAM_CENTER_Y tmp2 | sed s/BEAM_CENTER_Y=//`
| |
| # fix 2010-04-26 - tell user about possible ORGX, ORGY alternatives -
| |
| # at ESRF and ... (pls fill in!) the following should be used:
| |
| ORGX=`echo "scale=1; $BEAM_CENTER_Y/$QX" | bc -l`
| |
| ORGY=`echo "scale=1; $BEAM_CENTER_X/$QX" | bc -l`
| |
| echo - at ESRF BLs use: ORGX=$ORGX ORGY=$ORGY
| |
| # this 2nd alternative convention should be used at the following beamlines (pls complete the list): ALS 5.0.3, ...
| |
| ORGX=`echo "scale=1; $NX-$BEAM_CENTER_X/$QX" | bc -l `
| |
| ORGY=`echo "scale=1; $BEAM_CENTER_Y/$QX" | bc -l `
| |
| echo - at e.g. ALS 5.0.3 use: ORGX=$ORGX ORGY=$ORGY
| |
| # this 3rd alternative convention should be used at the following beamlines (pls complete the list): ALS 8.2.2, ...
| |
| ORGX=`echo "scale=1; $BEAM_CENTER_X/$QX" | bc -l `
| |
| ORGY=`echo "scale=1; $NX-$BEAM_CENTER_Y/$QX" | bc -l `
| |
| ORGXMM=`echo "$ORGX*$QX" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| ORGYMM=`echo "$ORGY*$QY" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| echo - at e.g. ALS 8.2.2 use: ORGX=$ORGX ORGY=$ORGY - this is written to XDS.INP
| |
| # the latter alternative is written into the generated XDS.INP ! You have to correct this manually in XDS.INP, or adjust this script.
| |
| # find DETECTOR_DISTANCE and OSCILLATION_RANGE:
| |
| DETECTOR_DISTANCE=`grep DISTANCE tmp2 | sed s/DISTANCE=//`
| |
| OSCILLATION_RANGE=`grep OSC_RANGE tmp2 | sed s/OSC_RANGE=//`
| |
|
| |
| elif [ "$DET" == "pilatus" ]; then
| |
| DETECTOR="PILATUS MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD= 1048576 !PILATUS"
| |
| QX=0.172 QY=0.172
| |
| echo Data from a Pilatus detector
| |
| sed s/#// tmp2 > tmp1
| |
| mv tmp1 tmp2
| |
|
| |
| # find SENSOR_THICKNESS:
| |
| SENSOR_THICKNESS=`grep thickness tmp2 | sed -e s/'Silicon sensor, thickness'// | awk '{print $1*1000}'`
| |
| # find X_RAY_WAVELENGTH:
| |
| X_RAY_WAVELENGTH=`grep Wavelength tmp2 | sed -e s/Wavelength// -e s/A// | awk '{print $1}'`
| |
|
| |
| X_RAY_ENERGY=`echo "12398.5/$X_RAY_WAVELENGTH" | bc -l | awk '{printf "%5.1f", $1}'`
| |
| # find NX and NY; 2463/2527 is 6M, 1475/1679 is 2M
| |
| NX=`grep X-Binary-Size-Fastest-Dimension tmp2 | awk '{print $2}'`
| |
| NY=`grep X-Binary-Size-Second-Dimension tmp2 | awk '{print $2}'`
| |
| NXMM=`echo "$NX*$QX" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| NYMM=`echo "$NY*$QY" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| # find ORGX and ORGY:
| |
| ORGX=`grep Beam_xy tmp2 | sed -e s/\(// -e s/\)// -e s/\,// | awk '{print $2}'`
| |
| ORGY=`grep Beam_xy tmp2 | sed -e s/\(// -e s/\)// -e s/\,// | awk '{print $3}'`
| |
| ORGXMM=`echo "$ORGX*$QX" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| ORGYMM=`echo "$ORGY*$QY" | bc -l | awk '{printf "%3.2f", $1}'`
| |
| # find DETECTOR_DISTANCE and OSCILLATION_RANGE:
| |
| DETECTOR_DISTANCE=`awk '/distance/{print $2}' tmp2`
| |
| DETECTOR_DISTANCE=`echo "$DETECTOR_DISTANCE*1000" | bc -l`
| |
|
| |
| OSCILLATION_RANGE=`awk '/Angle/{print $2}' tmp2`
| |
|
| |
| else
| |
| echo should never come here
| |
| exit 1
| |
| fi
| |
|
| |
| echo ORGX= $ORGX ORGY= $ORGY - check these values with adxv !
| |
| echo DETECTOR_DISTANCE= $DETECTOR_DISTANCE
| |
| echo OSCILLATION_RANGE= $OSCILLATION_RANGE
| |
| echo X-RAY_WAVELENGTH= $X_RAY_WAVELENGTH
| |
| # now we know everything that is required to generate XDS.INP
| |
|
| |
| cat > XDS.INP << eof
| |
| !****************************************************************************************
| |
| ! Written by generate_XDS.INP version $REVISION
| |
| !
| |
| ! A few notes on usage and modification of this input file:
| |
| !
| |
| ! If XDS fails after IDXREF with an error indicating that
| |
| ! an insufficient percentage of reflections were indexed,
| |
| ! but you are confident that the indexing solution is
| |
| ! correct, rerun XDS with the JOB keyword changed to
| |
| ! "DEFPIX INTEGRATE CORRECT".
| |
| !
| |
| ! After the first round of integration (in a triclinic cell)
| |
| ! it is often helpful to reintegrate using the globally
| |
| ! refined cell parameters and the correct space group.
| |
| ! You can do this by renaming GXPARM.XDS to XPARM.XDS
| |
| ! - e.g. by the command "cp GXPARM.XDS XPARM.XDS" -
| |
| ! and rerunning DEFPIX, INTEGRATE and CORRECT.
| |
| !
| |
| ! If you are trying to determine the optimal starting point for
| |
| ! data collection starting from a single image, first complete
| |
| ! a run of XDS using default parameters. Then, rerun XDS using
| |
| ! the correct cell and spacegroup as described above
| |
| ! ("cp GXPARM.XDS XPARM.XDS"), but on the second run of XDS, give
| |
| ! the JOB keyword as "JOB=DEFPIX XPLAN".
| |
| !
| |
| ! Be warned that the starting spindle angles given by XPLAN are
| |
| ! relative to the first image given - XDS assumes that the first image
| |
| ! given has a phi of 0 unless explicitly told otherwise. If this is not
| |
| ! the case (e.g. you are indexing from a snapshot taken at phi=90,
| |
| ! you will need to add the actual phi (as reported in the image header)
| |
| ! to the values given by XPLAN in order for these values to make sense.
| |
| !****************************************************************************************
| |
| !
| |
| !****************************************************************************************
| |
| !General parameters:
| |
| !****************************************************************************************
| |
|
| |
| JOB= XYCORR INIT COLSPOT IDXREF DEFPIX INTEGRATE CORRECT
| |
|
| |
| ORGX= $ORGX ORGY= $ORGY ! check these values with adxv!
| |
| !In mm, ORGX=$ORGXMM and ORGY=$ORGYMM
| |
|
| |
| DETECTOR_DISTANCE= $DETECTOR_DISTANCE
| |
| OSCILLATION_RANGE= $OSCILLATION_RANGE
| |
|
| |
| X-RAY_WAVELENGTH= $X_RAY_WAVELENGTH
| |
| !X-ray energy is $X_RAY_ENERGY eV (calculated from wavelength)
| |
|
| |
| NAME_TEMPLATE_OF_DATA_FRAMES=$NAME_TEMPLATE_OF_DATA_FRAMES
| |
|
| |
| !REFERENCE_DATA_SET=xxx/XDS_ASCII.HKL ! e.g. to ensure consistent indexing.
| |
| !XDS will *only* use the reference dataset if the space group and cell
| |
| !parameters are specified below.
| |
|
| |
| DATA_RANGE=1 $DATA_RANGE
| |
| SPOT_RANGE=1 $SPOT_RANGE
| |
|
| |
| !BACKGROUND_RANGE=1 10 !Default is first five degrees.
| |
|
| |
| SPACE_GROUP_NUMBER=0 ! 0 if unknown
| |
| UNIT_CELL_CONSTANTS= 70 80 90 90 90 90 ! put correct values if known
| |
|
| |
| INCLUDE_RESOLUTION_RANGE=50 0 ! After CORRECT, insert high resol limit; re-run CORRECT
| |
| ! Ice rings can be excluded from indexing and scaling with the keyword
| |
| ! EXCLUDE_RESOLUTION_RANGE, e.g. EXCLUDE_RESOLUTION_RANGE=2.28 2.22
| |
|
| |
| !SECONDS=60 !Uncomment this line to tell XDS to wait 1min for the next image before aborting.
| |
|
| |
| !****************************************************************************************
| |
| !Parameters important for processing anomalous data:
| |
| !****************************************************************************************
| |
|
| |
| FRIEDEL'S_LAW=FALSE ! This acts only on the CORRECT step. Even if the anomalous signal
| |
| ! in your dataset is very small, it still may be useful for
| |
| ! calculating an anomalous difference map.
| |
|
| |
| STRICT_ABSORPTION_CORRECTION=FALSE !Change this to TRUE if the anomalous signal
| |
| ! is strong: in that case, in CORRECT.LP the three
| |
| ! "CHI^2-VALUE OF FIT OF CORRECTION FACTORS" values
| |
| ! will be significantly> 1, e.g. 1.5.
| |
|
| |
| WFAC1=1.0 !This parameter controls rejection of misfits during scaling. Sometimes,
| |
| ! strong anomalous pairs may be rejected as misfits, in which case increasing
| |
| ! WFAC1 (e.g. to 1.5) may improve anomalous signal.
| |
|
| |
| !****************************************************************************************
| |
| !Regions of the detector to be excluded during indexing and integration:
| |
| !****************************************************************************************
| |
| ! exclude (mask) untrusted areas of detector, e.g. beamstop shadow :
| |
| !UNTRUSTED_RECTANGLE= !1800 1950 2100 2150 ! x-min x-max y-min y-max ! repeat
| |
| !UNTRUSTED_ELLIPSE= !2034 2070 1850 2240 ! x-min x-max y-min y-max ! if needed
| |
| !UNTRUSTED_QUADRILATERAL= !x1 y1 x2 y2 x3 y3 x4 y4 ! see documentation
| |
|
| |
| TRUSTED_REGION=0.00 1.2 ! Partially use corners of detectors; 1.41421=full use, 1.0=edge
| |
|
| |
| !****************************************************************************************
| |
| !Other parameters:
| |
| !****************************************************************************************
| |
| VALUE_RANGE_FOR_TRUSTED_DETECTOR_PIXELS=6000. 30000. ! Often 7000 or 8000 is ok.
| |
| STRONG_PIXEL=4 ! COLSPOT: only use strong reflections (default is 3; 6 may be better for strong data)
| |
| MINIMUM_NUMBER_OF_PIXELS_IN_A_SPOT=3 ! default of 6 is sometimes too high
| |
| MAXIMUM_ERROR_OF_SPOT_POSITION=3 ! Increasing may help for poor quality data
| |
|
| |
| MINPK=75 !Increase (e.g. MINPK=98) to improve data accuracy at the expense of completeness.
| |
|
| |
| DELPHI=5 !Size of integration wedge. Increasing to 10 or 20 may be helpful in some cases.
| |
|
| |
| NUMBER_OF_PROFILE_GRID_POINTS_ALONG_ALPHA/BETA=13 ! Default is 9 - Increasing may improve data
| |
| NUMBER_OF_PROFILE_GRID_POINTS_ALONG_GAMMA=13 ! accuracy, particularly if finely-sliced on phi,
| |
| ! and does not seem to have any downsides.
| |
|
| |
| MINIMUM_ZETA=0.05 ! Controls how close to the blind region (about phi) reflections should be
| |
| ! integrated. 0.05 is the default; increasing MINIMUM_ZETA to 0.15 *may*
| |
| ! improve data quality by removing unreliable reflections near the phi
| |
| ! axis, but will reduce completeness in low symmetry space groups.
| |
|
| |
| SEPMIN=6 ! Reduce if spots are close together (due to a long axis). Consider
| |
| CLUSTER_RADIUS=3 ! increasing if crystal is split and unit cell dimensions are relatively short.
| |
|
| |
| ! For bad or low resolution data remove the "!" in the following line (default is ALL):
| |
| ! REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS ! DISTANCE
| |
| REFINE(INTEGRATE)=CELL BEAM ORIENTATION ! AXIS DISTANCE (If integration is unstable, comment out this line.
| |
| ! REFINE(CORRECT)=CELL BEAM ORIENTATION AXIS DISTANCE ! Default is: refine everything
| |
|
| |
| ROTATION_AXIS=1 0 0 !At e.g. Australian Synchrotron, SERCAT ID-22 (?), APS 19-ID (?), BM30A this needs to be -1 0 0
| |
|
| |
| !****************************************************************************************
| |
| ! Parameters specifically for this detector and beamline (shouldn't need changing):
| |
| !****************************************************************************************
| |
| DETECTOR= $DETECTOR
| |
| SENSOR_THICKNESS= $SENSOR_THICKNESS
| |
| ! attention CCD detectors: for very high resolution (better than 1A) make sure to specify SILICON
| |
| ! as about 32* what CORRECT.LP suggests (absorption of phosphor is much higher than that of silicon)
| |
|
| |
| NX= $NX NY= $NY QX= $QX QY= $QY ! to make CORRECT happy if frames are unavailable
| |
| !In mm, NX=$NXMM and NY=$NYMM
| |
|
| |
| DIRECTION_OF_DETECTOR_X-AXIS=1 0 0
| |
| DIRECTION_OF_DETECTOR_Y-AXIS=0 1 0
| |
| INCIDENT_BEAM_DIRECTION=0 0 1
| |
| FRACTION_OF_POLARIZATION=0.98 ! better value is provided by beamline staff!
| |
| POLARIZATION_PLANE_NORMAL=0 1 0
| |
| !used by DEFPIX and CORRECT to exclude ice-reflections / ice rings - uncomment if necessary
| |
| !EXCLUDE_RESOLUTION_RANGE= 3.93 3.87 !ice-ring at 3.897 Angstrom
| |
| !EXCLUDE_RESOLUTION_RANGE= 3.70 3.64 !ice-ring at 3.669 Angstrom
| |
| !EXCLUDE_RESOLUTION_RANGE= 3.47 3.41 !ice-ring at 3.441 Angstrom
| |
| !EXCLUDE_RESOLUTION_RANGE= 2.70 2.64 !ice-ring at 2.671 Angstrom
| |
| !EXCLUDE_RESOLUTION_RANGE= 2.28 2.22 !ice-ring at 2.249 Angstrom
| |
| !EXCLUDE_RESOLUTION_RANGE= 2.102 2.042 !ice-ring at 2.072 Angstrom - strong
| |
| !EXCLUDE_RESOLUTION_RANGE= 1.978 1.918 !ice-ring at 1.948 Angstrom - weak
| |
| !EXCLUDE_RESOLUTION_RANGE= 1.948 1.888 !ice-ring at 1.918 Angstrom - strong
| |
| !EXCLUDE_RESOLUTION_RANGE= 1.913 1.853 !ice-ring at 1.883 Angstrom - weak
| |
| !EXCLUDE_RESOLUTION_RANGE= 1.751 1.691 !ice-ring at 1.721 Angstrom - weak
| |
| eof
| |
| if [ "$DET" == "pilatus" ]; then
| |
| if [ $NX == "1475" ]; then
| |
| if ! grep -q FF_p2m0109_E12398_T6199_vrf_m0p20.tif tmp2 ; then
| |
| cat >> XDS.INP << eof
| |
| !EXCLUSION OF VERTICAL DEAD AREAS OF THE PILATUS 2M DETECTOR
| |
| UNTRUSTED_RECTANGLE= 486 496 0 1680
| |
| UNTRUSTED_RECTANGLE= 980 990 0 1680
| |
| !EXCLUSION OF HORIZONTAL DEAD AREAS OF THE PILATUS 2M DETECTOR
| |
| UNTRUSTED_RECTANGLE= 0 1476 194 214
| |
| UNTRUSTED_RECTANGLE= 0 1476 406 426
| |
| UNTRUSTED_RECTANGLE= 0 1476 618 638
| |
| UNTRUSTED_RECTANGLE= 0 1476 830 850
| |
| UNTRUSTED_RECTANGLE= 0 1476 1042 1062
| |
| UNTRUSTED_RECTANGLE= 0 1476 1254 1274
| |
| UNTRUSTED_RECTANGLE= 0 1476 1466 1486
| |
| eof
| |
| fi
| |
| # elif [ $NX == "2463" ]; then
| |
| # cat >> XDS.INP << eof
| |
| #eof
| |
| fi
| |
| fi
| |
| touch xds_to_ccp4.sh
| |
| echo "#!/bin/bash
| |
| #Removes old log file if present
| |
| rm xds_to_ccp4.log
| |
| #Makes new log file
| |
| touch xds_to_ccp4.log
| |
| #Pipe stdout and stderr to both logfile and tty
| |
| tail -f -n1 xds_to_ccp4.log &
| |
| exec > xds_to_ccp4.log 2>&1
| |
| echo \"
| |
|
| |
|
| |
| If this message is all you see,
| |
| something has probably gone wrong.
| |
| Are you sure XDS_ASCII.HKL is present
| |
| in the current directory?
| |
|
| |
|
| |
| \"
| |
| #Make first MTZ file (with DANO/SIGDANO)
| |
| echo \"INPUT_FILE=XDS_ASCII.HKL
| |
| OUTPUT_FILE=temp.hkl CCP4
| |
| MERGE=TRUE
| |
| GENERATE_FRACTION_OF_TEST_REFLECTIONS=0.05
| |
| FRIEDEL'S_LAW=FALSE\" > XDSCONV.INP
| |
| xdsconv
| |
| f2mtz HKLOUT temp.mtz<F2MTZ.INP
| |
|
| |
| #Make second MTZ file (with F(+)/F(-))
| |
| echo \"INPUT_FILE=XDS_ASCII.HKL
| |
| OUTPUT_FILE=temp2.hkl CCP4_F
| |
| MERGE=TRUE
| |
| GENERATE_FRACTION_OF_TEST_REFLECTIONS=0.05
| |
| FRIEDEL'S_LAW=FALSE\" > XDSCONV.INP
| |
| xdsconv
| |
| f2mtz HKLOUT temp2.mtz<F2MTZ.INP
| |
|
| |
| #Make SHELX format unmerged hkl file
| |
| echo \"INPUT_FILE=XDS_ASCII.HKL
| |
| OUTPUT_FILE=xds_shelx.hkl SHELX
| |
| MERGE=FALSE
| |
| FRIEDEL'S_LAW=FALSE\" > XDSCONV.INP
| |
| xdsconv
| |
|
| |
| #cad the two previously generated MTZ files together
| |
| cad HKLIN1 temp.mtz HKLIN2 temp2.mtz HKLOUT xds_ccp4_merged.mtz<<EOF
| |
| LABIN FILE 1 E1=FP E2=SIGFP E3=DANO E4=SIGDANO E5=FreeRflag
| |
| LABIN FILE 2 E1=F(+) E2=SIGF(+) E3=F(-) E4=SIGF(-)
| |
| END
| |
| EOF
| |
|
| |
| #Remove temporary files
| |
| rm temp.mtz
| |
| rm temp2.mtz
| |
| rm temp.hkl
| |
| rm temp2.hkl
| |
| rm F2MTZ.INP
| |
|
| |
| echo \"
| |
|
| |
| ********************************XDS_TO_CCP4**********************************
| |
|
| |
| This script takes XDS_ASCII.HKL and uses XDSCONV and F2MTZ to
| |
| generate a merged CCP4-format MTZ file named
| |
| \\\"xds_ccp4_merged.mtz\\\" with a test (free) set
| |
| labeled \\\"FreeRflag\\\" constituting 5% of reflections.
| |
|
| |
| The space group of the output MTZ file will be the same as
| |
| that present in XDS_ASCII.HKL. It is probably worth checking
| |
| that this is the correct space group - either manually inspect
| |
| the systematic absences in CORRECT.LP, or run XDS_ASCII.HKL
| |
| through the CCP4 program POINTLESS (\\\"pointless xdsin XDS_ASCII.HKL\\\")
| |
|
| |
| Anomalous data is kept, if present, and output as both
| |
| DANO/SIGDANO and F(+)/F(-).
| |
|
| |
| This script also generates an unmerged SHELX hkl file,
| |
| suitable for input into SHELXC/D/E, named \\\"xds_shelx.hkl\\\".
| |
|
| |
| A log of all processes is present in xds_to_ccp4.log.
| |
|
| |
| Parts of this script have been shamelessly copied from
| |
| the XDS Wiki, at http://www.strucbio.biologie.uni-konstanz.de/xdswiki/
| |
|
| |
| ******************************************************************************
| |
| \"" > xds_to_ccp4.sh
| |
| touch xds_graph.sh
| |
| echo "#!/bin/bash
| |
| #Makes new log file
| |
| touch INTEGRATE_STATS.LP
| |
| #Pipe stdout and stderr to both logfile and tty
| |
| tail -f -n1 INTEGRATE_STATS.LP &
| |
| exec > INTEGRATE_STATS.LP 2>&1
| |
| if command -v xdsstat >/dev/null; then
| |
| echo \"\" | xdsstat > XDSSTAT.LP
| |
| echo \"\" | xdsstat 100 6 > XDSSTAT.LP_low
| |
| echo \"
| |
| \\\$TABLE:Rmeas vs image (to 6A):
| |
| \\\$SCATTER:Rmeas vs image for low resolution reflections:A:1,2: \\$\\$
| |
| IMAGE RMEAS
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| grep ' L\$' XDSSTAT.LP_low | awk '{print \$1,\$9}'
| |
| echo \" \\$\\$ \"
| |
| echo \"
| |
| \\\$TABLE:Rmeas vs image (all):
| |
| \\\$SCATTER:Rmeas vs image (all reflections):A:1,2: \\$\\$
| |
| IMAGE RMEAS
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| grep ' L$' XDSSTAT.LP | awk '{print \$1,\$9}'
| |
| echo \" \\$\\$ \"
| |
| echo \"
| |
| \\\$TABLE:Rd vs image (to 6A):
| |
| \\\$SCATTER:Rd (from XDSSTAT) for low resolution reflections:A:1,2: \\$\\$
| |
| IMAGE RD
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| grep 'DIFFERENCE' XDSSTAT.LP_low | awk '{print \$1,\$7}'
| |
| echo \" \\$\\$ \"
| |
| echo \"
| |
| \\\$TABLE:Rd vs image (all):
| |
| \\\$SCATTER:Rd (from XDSSTAT) for all reflections:A:1,2: \\$\\$
| |
| IMAGE RD
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| grep 'DIFFERENCE' XDSSTAT.LP | awk '{print \$1,\$7}'
| |
| echo \" \\$\\$ \"
| |
| fi
| |
|
| |
| echo \"
| |
| \\\$TABLE: XDS Integration Statistics :
| |
| \\\$SCATTER:Scale factors by image:A:1,3: :Number of overloads by image:A:1,5: :Number of strong reflections by image:A:1,7: :Number of rejects by image:A:1,8: :Mosaicity by image:A:1,10: \\$\\$
| |
| IMAGE IER SCALE NBKG NOVL NEWALD NSTRONG NREJ SIGMAB SIGMAR
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| egrep \"[0-9]\.[0-9][0-9][0-9][0-9] [0-9]\.[0-9][0-9][0-9][0-9]\" INTEGRATE.LP
| |
| echo \" \\$\\$ \"
| |
| echo \"
| |
| \\\$TABLE: XDS Scaling Statistics by Resolution (X-axis is resolution in Angstroems) :
| |
| \\\$GRAPHS:Chi-Squared by resolution:A:2,4: :Observed and expected R-factors by resolution (as %):A:2,5,6: :Rejects by resolution:A:2,9: :Rejects by resolution as a percentage of total reflections:A:2,10: \\$\\$
| |
| LOW_RES Resolution(Angstroems) I/Sigma Chi^2 R-FACTOR(OBSERVED) R-FACTOR(EXPECTED) NUMBER_OF_REFLECTIONS ACCEPTED REJECTS REJECT_PERCENT
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| awk '/RESOLUTION RANGE I\/Sigma Chi\^2 R\-FACTOR R\-FACTOR NUMBER ACCEPTED REJECTED/, /\-\-\-\-\-\-\-\-\-/' CORRECT.LP | egrep -v \"[a-z,A-Z,] | \-\-|\-99\.9\" | egrep \"[0-9]\" | awk '{print \$1,\$2,\$3,\$4,\$5,\$6,\$7,\$8,\$9,(\$9/(\$8+\$9))*100}'
| |
| echo \" \\$\\$\"
| |
| echo \"
| |
| \\\$TABLE: Overall data quality by Resolution (X-axis is resolution in Angstroems) :
| |
| \\\$GRAPHS:Completeness vs resolution :A:1,6: :Redundancy vs resolution :A:1,4: :I/Sigma by resolution:A:1,10: :Rmeas by resolution:A:1,11: :Anomalous correlation by resolution:A:1,13: :Significance of the anomalous signal by resolution:A:1,14: \\$\\$
| |
| Resolution(Angstroems) OBSERVED_REFLECTIONS UNIQUE_REFLECTIONS REDUNDANCY POSSIBLE_REFLECTIONS COMPLETENESS R-FACTOR(OBSERVED) R-FACTOR(EXPECTED) COMPARED I/SIGMA R-meas CC(1/2) Anomalous_Correlation SigAno Nano
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| egrep -B25 \" WILSON STATISTICS \" CORRECT.LP | egrep -v \"[a-z,A-Z]|\*\*\*\" | egrep \" \" | awk '{ gsub(/[%*]/,\" \"); print }' | awk '{print \$1,\$2,\$3,\$2/\$3,\$4,\$5,\$6,\$7,\$8,\$9,\$10,\$11,\$12,\$13,\$14}'
| |
| echo \" \\$\\$ \"
| |
|
| |
| echo \"
| |
| \\\$TABLE: I/Sigma of h00 reflections (for identification of systematic absences):
| |
| \\\$GRAPHS:I/Sigma vs h :A:1,5: \\$\\$
| |
| h k l RES I/SIGMA NUMBER
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| awk '/REFLECTIONS OF TYPE H/,/COMP/' CORRECT.LP | egrep \"[1-9]\" | awk '{print \$1,\$2,\$3,\$4,\$7,\$8}' | awk '(\$1 > 0)'
| |
| echo \"\\$\\$\"
| |
|
| |
| echo \"
| |
| \\\$TABLE: I/Sigma of 0k0 reflections (for identification of systematic absences):
| |
| \\\$GRAPHS:I/Sigma vs k :A:2,5: \\$\\$
| |
| h k l RES I/SIGMA NUMBER
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| awk '/REFLECTIONS OF TYPE H/,/COMP/' CORRECT.LP | egrep \"[1-9]\" | awk '{print \$1,\$2,\$3,\$4,\$7,\$8}' | awk '(\$2 > 0)'
| |
| echo \"\\$\\$\"
| |
|
| |
| echo \"
| |
| \\\$TABLE: I/Sigma of 00l reflections (for identification of systematic absences):
| |
| \\\$GRAPHS:I/Sigma vs l :A:3,5: \\$\\$
| |
| h k l RES I/SIGMA NUMBER
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| awk '/REFLECTIONS OF TYPE H/,/COMP/' CORRECT.LP | egrep \"[1-9]\" | awk '{print \$1,\$2,\$3,\$4,\$7,\$8}' | awk '(\$3 > 0)'
| |
| echo \"\\$\\$\"
| |
|
| |
| echo \"
| |
| \\\$TABLE:Beam center (ORGX/ORGY) plotted vs wedge (DELPHI):
| |
| \\\$GRAPHS:ORGX and ORGY vs batch (DELPHI) :A:1,2,3: \\$\\$
| |
| N ORGX ORGY
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| egrep \"DETECTOR ORIGIN\" INTEGRATE.LP | grep -n \"\" | awk '{ gsub(/[a-z,A-Z,),(,:]/,\"\"); print }'
| |
| echo \"\\$\\$\"
| |
|
| |
| echo \"
| |
| \\\$TABLE:Unit cell parameters plotted vs wedge:
| |
| \\\$GRAPHS:Unit cell lengths a, b and c plotted vs wedge:A:1,2,3,4: :Unit cell angles alpha, beta and gamma plotted vs wedge:A:1,5,6,7: \\$\\$
| |
| N a b c ALPHA BETA GAMMA
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| egrep \"UNIT CELL PARAMETERS\" INTEGRATE.LP | grep -n \"\" | awk '{ gsub(/[a-z,A-Z,),(,:]/,\"\"); print }'
| |
| echo \"\\$\\$\"
| |
|
| |
| echo \"
| |
| \\\$TABLE:Standard deviation of spot position (pixels) plotted vs wedge:
| |
| \\\$GRAPHS:Standard deviation of spot position plotted vs wedge:A:1,2: \\$\\$
| |
| N SD
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| egrep \"SPOT POSITION\" INTEGRATE.LP | grep -n \"\" | awk '{ gsub(/[a-z,A-Z,),(,:]/,\"\"); print }'
| |
| echo \"\\$\\$\"
| |
|
| |
| echo \"
| |
| \\\$TABLE:Detector distance (mm) plotted vs wedge (DELPHI):
| |
| \\\$GRAPHS:Detector distance plotted vs batch:A:1,2: \\$\\$
| |
| N DIST
| |
| \\$\\$ \\$\\$
| |
| \"
| |
| egrep \"DETECTOR DISTANCE \\(mm\\)\" INTEGRATE.LP | grep -n \"\" | awk '{ gsub(/[a-z,A-Z,),(,:]/,\"\"); print }'
| |
|
| |
| echo \"\\$\\$\"
| |
|
| |
| loggraph INTEGRATE_STATS.LP">xds_graph.sh
| |
| chmod +x xds_to_ccp4.sh
| |
| chmod +x xds_graph.sh
| |
| echo "XDS.INP is ready for use. The file has only the most important keywords.
| |
| Full documentation, including complete detector templates, is at
| |
| http://www.mpimf-heidelberg.mpg.de/~kabsch/xds . More documentation in XDSwiki
| |
| After running xds, inspect, using XDS-Viewer, at least the beamstop mask in
| |
| BKGPIX.cbf, and the agreement of predicted and observed spots in FRAME.cbf!
| |
|
| |
| Two shell scripts have been generated that may be of use.
| |
|
| |
| The first, xds_graph.sh, will plot various statistics after an XDS run.
| |
| After both INTEGRATE and CORRECT have finished, run it by typing \"./xds_graph.sh\"
| |
| without the quotation marks. This script uses loggraph to plot data, so you will
| |
| need CCP4 to see the graphical output. All the raw data is piped out to a log file,
| |
| INTEGRATE_STATS.LP.
| |
|
| |
| If XDSSTAT is present in your path, xds_graph will also plot some of the statistics it
| |
| calculates - specifically the Rmeas and Rd per image, for all reflections and only
| |
| low resolution reflections.
| |
|
| |
| The second script, xds_to_ccp4.sh, will take XDS_ASCII.HKL and generate both a CCP4
| |
| format MTZ file with anomalous data retained (as F(+)/F(-) and DANO/SigDANO)
| |
| and an unmerged SHELX format hkl file for input to SHELXC/D/E. Run it after CORRECT
| |
| by typing \"./xds_to_ccp4.sh\" at the prompt."
| |
|
| |
| rm -f tmp1 tmp2
| |
| </pre>
| |
|
| |
| Use this script in the same manner as Generate_XDS.INP. The two shell scripts created should be executable; If they are not, then make them so (chmod +x ./xxx.sh).
| |