2,684
edits
(→The script: KD 27.4.22 always print out detector serial number) |
|||
(38 intermediate revisions by the same user not shown) | |||
Line 10: | Line 10: | ||
== The script == | == The script == | ||
<pre> | <pre> | ||
Line 114: | Line 115: | ||
# revision 0.96 . KD 03/10/2020 make script echo detector serial number if inverted ROTATION_AXIS is detected. | # revision 0.96 . KD 03/10/2020 make script echo detector serial number if inverted ROTATION_AXIS is detected. | ||
# revision 0.97 . KD 19/10/2020 add UNTRUSTED_RECTANGLEs for Eiger2 (which has a few pixels less than Eiger (thanks to Andreas Förster) | # revision 0.97 . KD 19/10/2020 add UNTRUSTED_RECTANGLEs for Eiger2 (which has a few pixels less than Eiger (thanks to Andreas Förster) | ||
REVISION=" | # revision 0.98 . Thomas Hauß (HZB) 06/11/2020 switch Python print syntax to Python3 | ||
# revision 0.99 . Aaron Finke (CHESS) 07/11/2020 add ROTATION_AXIS=-1 0 0 for EIGER2 16M detector at MX beamline ID7B2 (CHESS) | |||
# revision 1.00 . Gleb Bourenkov 10/11/2020 add ROTATION_AXIS=0 -1 0 for beamline P14 (EMBL Hamburg) detectors Eiger 16M S/N E-32-0107; previously Eiger 4M S/N E-08-0107 and PILATUS 6M-F, S/N 60-0115-F | |||
# revision 1.01 . KD 07/12/2020 ROTATION_AXIS=0 -1 0 for Pilatus3 2M, S/N 24-0118 at ID23-2 (http://www.globalphasing.com/autoproc/wiki/index.cgi?BeamlineSettings) | |||
# revision 1.02 . KD 11/01/2021 recognize mar555 detector as mar345 (thanks to Thomas Hauß, HZB) | |||
# revision 1.03 . Feng Yu 18/7/2021 fix/expand information about Shanghai Synchrotron Radiation Facility (SSRF) | |||
# revision 1.04 . Zhipu Luo 13/08/2021 extract ORGX,ORGY for electron diffraction from SMV header if the wavelength value starts with 0.0 | |||
# revision 1.05 . KD specialcase extraction of number of images for BNL detectors E-32-0101 and E-18-0104 | |||
# revision 1.06 . KD fix URLs in output. Availability of Apple M1 processor dectris-neggia-Apple-arm64.so . LC_ALL=C . | |||
# revision 1.07 . KD, Thomas Hauß, Gleb Bourenkov 25/10/2021. Detector moved from Petra P14 to P13 | |||
# revision 1.08 . KD, Thomas Hauß, Feng Yu 7/11/2021 fix TZ in timestamps for SSRF detectors | |||
# revision 1.09 . KD, Feng Yu 7/11/2021 fix detector number BNL E-18-0121 to be E-18-0104 | |||
# revision 1.10 . KD NeXus header for Eiger | |||
# revision 1.11 . KD / Helena Taberman ROTATION_AXIS=-1 0 0 for Eiger .cbf data from Petra P14 | |||
# revision 1.12 . KD / Helena Taberman correct rev 1.11 to be 0 -1 0 | |||
REVISION="1.12 (25-MAR-2022)" | |||
# | # | ||
Line 122: | Line 138: | ||
# | # | ||
# known problems: | # known problems: | ||
# revision 1.10 . KD implement NeXus for Eiger | |||
# - for ADSC detectors, there are at least three ways to obtain ORGX and ORGY values from the header (see below); | # - for ADSC detectors, there are at least three ways to obtain ORGX and ORGY values from the header (see below); | ||
# - the same might be a problem for MAR headers, too (not sure about this) | # - the same might be a problem for MAR headers, too (not sure about this) | ||
Line 127: | Line 144: | ||
# | # | ||
# notes for debugging of the script: | # notes for debugging of the script: | ||
# - add the -x option to | # - add the -x option to #!/bin/bash in first line, to see where an error occurs | ||
# - comment out the removal of tmp1 and tmp2 in the last line | # - comment out the removal of tmp1 and tmp2 in the last line | ||
# | # | ||
Line 138: | Line 155: | ||
exit | exit | ||
fi | fi | ||
# make sure the locale does not interfere with e.g. awk calculations: | |||
LC_ALL="C";export LC_ALL | |||
# | # | ||
# defaults: | # defaults: | ||
Line 161: | Line 180: | ||
# default DIRECTION_OF_DETECTOR_X-AXIS | # default DIRECTION_OF_DETECTOR_X-AXIS | ||
DIRECTION_OF_DETECTOR_X_AXIS="1 0 0" | DIRECTION_OF_DETECTOR_X_AXIS="1 0 0" | ||
# default DIRECTION_OF_DETECTOR_Y-AXIS | |||
DIRECTION_OF_DETECTOR_Y_AXIS="0 1 0" | |||
# default FRACTION_OF_POLARIZATION | # default FRACTION_OF_POLARIZATION | ||
pol_frac=0.98 | pol_frac=0.98 | ||
Line 167: | Line 188: | ||
SEPMIN=7.0 | SEPMIN=7.0 | ||
CLUSTER_RADIUS=3.5 | CLUSTER_RADIUS=3.5 | ||
REFINE_CORRECT="CELL BEAM ORIENTATION AXIS POSITION ! Default is: refine everything" | |||
dname=`echo "$1" | xargs dirname` | dname=`echo "$1" | xargs dirname` | ||
Line 196: | Line 218: | ||
data_last=`cut -b $pos1-$pos2 tmp1 | tail -n1 | bc` | data_last=`cut -b $pos1-$pos2 tmp1 | tail -n1 | bc` | ||
DATA_RANGE="$data_first $data_last" | DATA_RANGE="$data_first $data_last" | ||
echo DATA_RANGE=$DATA_RANGE | |||
# set SPOT_RANGE to first half of DATA_RANGE | # set SPOT_RANGE to first half of DATA_RANGE | ||
Line 207: | Line 230: | ||
FIRSTFRAME=`head -1 tmp1` | FIRSTFRAME=`head -1 tmp1` | ||
echo $FIRSTFRAME | grep "\.h5$" && is_h5=1 || is_h5=0 | echo $FIRSTFRAME | grep "\.h5$" && is_h5=1 || is_h5=0 | ||
# find out detector type | # find out detector type | ||
Line 238: | Line 252: | ||
grep -q "^SOURCE_WAVELENGTH= *1" tmp2 && DET=dtrek | grep -q "^SOURCE_WAVELENGTH= *1" tmp2 && DET=dtrek | ||
grep -q BEAM_CENTER_X tmp2 && DET=adsc | grep -q BEAM_CENTER_X tmp2 && DET=adsc | ||
grep -q mar345 tmp2 && DET=MAR345 | grep -q mar345 tmp2 && DET=MAR345 | ||
# rev. 1.02: in one case, mar555 data could be processed pretending it is mar345, so: | |||
grep -q mar555 tmp2 && DET=MAR345 | |||
grep -q BRUKER tmp2 && grep -q CBF tmp2 && DET=Bruker-cbf | grep -q BRUKER tmp2 && grep -q CBF tmp2 && DET=Bruker-cbf | ||
grep -q CMOS1 tmp2 && DET=adsc-CMOS1 | grep -q CMOS1 tmp2 && DET=adsc-CMOS1 | ||
grep -q MARCONTROL tmp2 && DET=OLDMAR | grep -q MARCONTROL tmp2 && DET=OLDMAR | ||
grep -q "WAVELENGTH=0.0" tmp2 && DET=experimental-ED | |||
else | else | ||
h5dump -d "/entry/instrument/detector/description" $FIRSTFRAME | grep -i Eiger > /dev/null && DET=eiger | h5dump -d "/entry/instrument/detector/description" $FIRSTFRAME | grep -i Eiger > /dev/null && DET=eiger | ||
Line 461: | Line 478: | ||
DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=/-/` | DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=/-/` | ||
OSCILLATION_RANGE=`grep OMEGA_DELTA tmp2 | sed s/OMEGA_DELTA=//` | OSCILLATION_RANGE=`grep OMEGA_DELTA tmp2 | sed s/OMEGA_DELTA=//` | ||
DETECTOR_X_AXIS="-1 0 0" | |||
elif [ "$DET" == "experimental-ED" ]; then | |||
DETECTOR="ADSC MINIMUM_VALID_PIXEL_VALUE= 1 OVERLOAD= 65000 ! OVERLOAD is really unknown " | |||
echo --- Electron diffraction data in SMV format! | |||
SENSOR_THICKNESS=0.01 | |||
sed s/\;// tmp2 > tmp1 | |||
mv tmp1 tmp2 | |||
X_RAY_WAVELENGTH=`grep WAVELENGTH tmp2 | head -1 | sed s/WAVELENGTH=//` | |||
NX=`grep SIZE1 tmp2 | tail -1 | sed s/SIZE1=//` | |||
QX=`grep PIXEL_SIZE tmp2 | sed s/PIXEL_SIZE=//` | |||
NY=`grep SIZE2 tmp2 | tail -1 | sed s/SIZE2=//` | |||
QY=$QX | |||
BEAM_CENTER_X=`grep BEAM_CENTER_X tmp2 | sed s/BEAM_CENTER_X=//` | |||
BEAM_CENTER_Y=`grep BEAM_CENTER_Y tmp2 | sed s/BEAM_CENTER_Y=//` | |||
QXY=`echo "scale=1; $QX*($BEAM_CENTER_X+$BEAM_CENTER_Y)" | bc -l` | |||
if [ "$QXY" "<" "$BEAM_CENTER_X" ] ; then | |||
ORGX=`echo "scale=2; $BEAM_CENTER_Y/$QX" | bc -l` | |||
ORGY=`echo "scale=2; $BEAM_CENTER_X/$QX" | bc -l` | |||
else | |||
ORGX=`echo "scale=2; $BEAM_CENTER_X/1" | bc -l` | |||
ORGY=`echo "scale=2; $BEAM_CENTER_Y/1" | bc -l` | |||
fi | |||
DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=//` | |||
OSCILLATION_RANGE=`grep OSC_RANGE tmp2 | sed s/OSC_RANGE=//` | |||
STARTING_ANGLE=`grep OSC_START tmp2 | sed s/OSC_START=//` | |||
echo --- For TIMEPIX detector, please fix ROTATION_AXIS yourself! | |||
DIRECTION_OF_DETECTOR_X_AXIS="1 0 0" | |||
REFINE_CORRECT="ORIENTATION CELL AXIS BEAM ! for ED, no POSITION when CELL is refined" | |||
elif [ "$DET" == "pilatus" ]; then | elif [ "$DET" == "pilatus" ]; then | ||
Line 478: | Line 523: | ||
# find SENSOR_THICKNESS: | # find SENSOR_THICKNESS: | ||
SENSOR_THICKNESS=`grep thickness tmp2 | sed -e s/' | SENSOR_THICKNESS=`grep thickness tmp2 | sed -e s/'Si.* sensor, thickness'// | awk '{print $1*1000}'` | ||
# find X_RAY_WAVELENGTH: | # find X_RAY_WAVELENGTH: | ||
X_RAY_WAVELENGTH=`grep Wavelength tmp2 | sed -e s/Wavelength// -e s/A// | awk '{print $1}'` | X_RAY_WAVELENGTH=`grep Wavelength tmp2 | sed -e s/Wavelength// -e s/A// | awk '{print $1}'` | ||
Line 529: | Line 574: | ||
fi | fi | ||
fi | fi | ||
# | # PETRA P14: raw data from Eiger are stored as CBF files so this is treated as Pilatus | ||
if [ "$DET_SN" == "Dectris Eiger 4M, E-08-0107" -o "$DET_SN" == "PILATUS 6M-F, S/N 60-0115-F" \ | |||
-o "$DET_SN" == "Dectris EIGER2 CdTe 16M, E-32-0129" ] ; then | |||
rotation_axis="0 -1 0" | |||
echo ROTATION_AXIS="0 -1 0" at PETRA P14 | |||
fi | |||
# similarly, Eiger 16M at PETRA P14 until May 22, 2021 | |||
if [ "$DET_SN" == "Dectris Eiger 16M, E-32-0107" ] ; then | |||
egrep -q '^# 2020-|^# 2021-0[1-5]|^# 2021-06-[01]|^# 2021-06-2[01]' tmp2 || isatP13=1 | |||
if [ "$isatP13" == 1 ] ; then | |||
echo using the default ROTATION_AXIS=1 0 0 at PETRA P13 | |||
else | |||
rotation_axis="0 -1 0" | |||
echo ROTATION_AXIS="0 -1 0" at PETRA P14 | |||
fi | |||
fi | |||
# ESRF ID23-2: | |||
if [ "$DET_SN" == "PILATUS3 2M, S/N 24-0118, ESRF ID23" ] ; then | |||
rotation_axis="0 -1 0" | |||
echo ROTATION_AXIS="0 -1 0" at ESRF ID23-2 | |||
fi | |||
elif [ "$DET" == "eiger" ]; then | elif [ "$DET" == "eiger" ]; then | ||
nframes=`h5dump -d "/entry/instrument/detector/detectorSpecific/nimages" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'` | |||
DATA_RANGE="1 $nframes" | |||
SPOT_RANGE="1 `echo "scale=0; if (${nframes}<2) 1; if (${nframes}>1) ${nframes}/2"|bc -l`" | |||
SN=`h5dump -d "/entry/instrument/detector/detector_number" $FIRSTFRAME | awk '/\(0\): /{print $2}' | sed s/\"//g` | |||
echo detector serial number is $SN | |||
# find out if HDF5 from Diamond (DLS=1) or Dectris (DLS=0) | # find out if HDF5 from Diamond (DLS=1) or Dectris (DLS=0) | ||
DLS=0 | DLS=0 | ||
Line 537: | Line 608: | ||
if [ "$DLS" == 1 ]; then | if [ "$DLS" == 1 ]; then | ||
echo Eiger HDF5 from Diamond | echo Eiger HDF5 from Diamond | ||
# unfortunately h5dump 1.10 is required to get this right for the DLS .h5 files. This version is available at DLS but | # unfortunately h5dump 1.10 is required to get this right for the DLS .h5 files. This version is available at DLS but maybe not elsewhere | ||
OVERLOAD=`h5dump -d "/entry/instrument/detector/saturation_value" $FIRSTFRAME | awk '/\(0\):/{print $2}'` | OVERLOAD=`h5dump -d "/entry/instrument/detector/saturation_value" $FIRSTFRAME | awk '/\(0\):/{print $2}'` | ||
# v0.95: fix the next 2 lines by taking care of negative values with the \- , and stop after first "(0)" | # v0.95: fix the next 2 lines by taking care of negative values with the \- , and stop after first "(0)" | ||
Line 545: | Line 616: | ||
# rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"` | # rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"` | ||
# the above gives -1 0 0 for DLS data instead of the correct 1 0 0, so commented out for now | # the above gives -1 0 0 for DLS data instead of the correct 1 0 0, so commented out for now | ||
# comment while implementing rev 1.10: this is because the DLS headers are NeXus, so DIRECTION_OF_DETECTOR_X/Y_AXIS must be adjusted. | |||
else | else | ||
echo Eiger HDF5 from Dectris | echo Eiger HDF5 from Dectris | ||
Line 551: | Line 623: | ||
# STARTING_ANGLE: the \- was introduced in version 0.91 to allow negative values : | # STARTING_ANGLE: the \- was introduced in version 0.91 to allow negative values : | ||
STARTING_ANGLE=`h5dump -d "/entry/sample/goniometer/omega_start" $FIRSTFRAME | awk '/\(0\): [\-0-9]/{print $2}'` | STARTING_ANGLE=`h5dump -d "/entry/sample/goniometer/omega_start" $FIRSTFRAME | awk '/\(0\): [\-0-9]/{print $2}'` | ||
# If rotation | # /entry/sample/goniometer/omega_start is missing in some eiger2 detectors (e.g. Eiger2 9M with fw version release-2020.2.1 and SIMPLON API 1.8) (Feng YU 2021-07-18) | ||
if [ "$STARTING_ANGLE" == "" ]; then | |||
echo "/entry/sample/goniometer/omega_start not found, try /entry/sample/goniometer/omega" | |||
STARTING_ANGLE=`h5dump -d /entry/sample/goniometer/omega $FIRSTFRAME | grep "(0):" | head -n 1 | awk '{print $2}' | sed -e "s/,//g"` | |||
fi | |||
echo "STARTING_ANGLE=$STARTING_ANGLE" | |||
# If rotation axis set (NeXus) | |||
rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"` | rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"` | ||
# | # EIGER2 16M CHESS ID7B2 has S/N E-32-0123 (A. Finke 2020-11-07) v0.99 | ||
SN= | if [ "$SN" == "E-32-0123" ]; then | ||
rotation_axis="-1 0 0" | |||
echo CHESS ID7B2 with inverted rotation axis | |||
fi | |||
# SSRF BL17U1 and SSRF BL10U2 (Feng YU 2021-07-18) | |||
# Eiger X 16M (S/N E-32-0111) was installed at SSRF BL17U1 from Oct 2017 to Feb 2021. After Feb 2021, it was moved back to SSRF BL10U2. | |||
# 2021-03-01 00:00:00 time stamp is 1614528000 | |||
if [ "$SN" == "E-32-0111" ]; then | if [ "$SN" == "E-32-0111" ]; then | ||
rotation_axis="-1 0 0" | collection_time=`h5dump -d "/entry/instrument/detector/detectorSpecific/data_collection_date" $FIRSTFRAME | grep "(0):" | awk '{print $2}' | sed -e "s/\.\(.*\)//g; s/\"//g"` | ||
echo SSRF BL17U1 with inverted rotation axis | if [ `uname -s` == "Darwin" ]; then | ||
fi | collection_timestamp=`date -j -f "%Y-%m-%dT%H:%M:%S" $collection_time +%s` | ||
elif [ `uname -s` == "Linux" ]; then | |||
collection_timestamp=`date -d $collection_time +%s` | |||
else | |||
collection_timestamp=0 | |||
fi | |||
if [ $collection_timestamp -eq 0 ]; then | |||
is_rotation_axis_set=1 | |||
rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0` | |||
if [ "$is_rotation_axis_set" == "1" ]; then | |||
rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"` | |||
echo "The rotation axis of SSRF BL10U2 (Eiger X 16M) is $rotation_axis" | |||
else | |||
rotation_axis="-1 0 0 ! Cannot determine rotation axis. SSRF BL17U1: -1 0 0; SSRF BL02U1: 0 -1 0" | |||
fi | |||
elif [ $collection_timestamp -ge 0 ] && [ $collection_timestamp -le 1614528000 ]; then | |||
# SSRF BL17U1 | |||
rotation_axis="-1 0 0" | |||
echo "SSRF BL17U1 (Eiger X 16M) with inverted rotation axis" | |||
else | |||
# SSRF BL10U2 | |||
is_rotation_axis_set=1 | |||
rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0` | |||
if [ "$is_rotation_axis_set" == "1" ]; then | |||
rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"` | |||
echo "The rotation axis of SSRF BL10U2 (Eiger X 16M) is $rotation_axis" | |||
else | |||
rotation_axis="0 -1 0" | |||
echo "SSRF BL10U2 (Eiger X 16M) with vertical rotation axis" | |||
fi | |||
fi | |||
fi | |||
# SSRF BL02U1 (Feng YU 2021-07-18) | |||
# EIGER2 S 9M SSRF BL02U1 has S/N E-18-0121 | |||
if [ "$SN" == "E-18-0121" ]; then | |||
is_rotation_axis_set=1 | |||
rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0` | |||
if [ "$is_rotation_axis_set" == "1" ]; then | |||
rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"` | |||
echo "The rotation axis of SSRF BL02U1 (Eiger2 S 9M) is $rotation_axis" | |||
else | |||
rotation_axis="1 0 0" | |||
echo "SSRF BL02U1 (Eiger2 S 9M) with horizontal rotation axis" | |||
fi | |||
fi | |||
# revision 1.05 specialcase nframes for Eiger detectors at BNL | |||
if [ "$SN" == "E-18-0104" -o "$SN" == "E-32-0101" ]; then | |||
echo specialcase nframes for Eiger detectors at BNL: | |||
nframes=`h5dump -A -g "/entry/data" $FIRSTFRAME | grep "DATASPACE SIMPLE" | sed -e "s/,.*//" | awk '{a+=$5}END{print a}'` | |||
DATA_RANGE="1 $nframes" | |||
SPOT_RANGE="1 `echo "scale=0; if (${nframes}<2) 1; if (${nframes}>1) ${nframes}/2"|bc -l`" | |||
fi | |||
fi | |||
# rev 1.10: check for NeXus header. If found, its geometry will overwrite any rotation_axis set until here. | |||
NeXus=0 | |||
h5dump -d "/entry/definition" $FIRSTFRAME | grep -q NXmx && NeXus=1 | |||
if [ "$NeXus" == "1" ]; then | |||
echo NeXus header found. This defines DIRECTION_OF_DETECTOR_X/Y-AXIS and ROTATION_AXIS. | |||
DIRECTION_OF_DETECTOR_X_AXIS=$(h5dump -d "/entry/instrument/detector/module/fast_pixel_direction" $FIRSTFRAME 2>/dev/null | grep "(0):" | tail -1 | sed -e "s/^.*://; s/,//g") | |||
DIRECTION_OF_DETECTOR_Y_AXIS=$(h5dump -d "/entry/instrument/detector/module/slow_pixel_direction" $FIRSTFRAME 2>/dev/null | grep "(0):" | tail -1 | sed -e "s/^.*://; s/,//g") | |||
rotation_axis=$(h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g") | |||
else | |||
echo no NeXus header found. | |||
fi | fi | ||
echo DATA_RANGE=$DATA_RANGE | |||
DETECTOR="EIGER MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD= $OVERLOAD" | DETECTOR="EIGER MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD= $OVERLOAD" | ||
QX=`h5dump -d "/entry/instrument/detector/x_pixel_size" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'` | QX=`h5dump -d "/entry/instrument/detector/x_pixel_size" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'` | ||
Line 587: | Line 734: | ||
#let SKIP=768 | #let SKIP=768 | ||
#NX=$(od -t x -j $SKIP -N 4 $FIRSTFRAME |awk 'NR==1{print toupper($2)}'|perl -nle '@array= $_ =~/.{2}/g; print "ibase=16;obase=A;".join("",reverse @array)'|bc) | #NX=$(od -t x -j $SKIP -N 4 $FIRSTFRAME |awk 'NR==1{print toupper($2)}'|perl -nle '@array= $_ =~/.{2}/g; print "ibase=16;obase=A;".join("",reverse @array)'|bc) | ||
NX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(768);print "%.4d"%struct.unpack(">i",f.read(4))') | NX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(768);print("%.4d"%struct.unpack(">i",f.read(4)))') | ||
NY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(772);print "%.4d"%struct.unpack(">i",f.read(4))') | NY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(772);print("%.4d"%struct.unpack(">i",f.read(4)))') | ||
DETECTOR_DISTANCE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(344);print("-%.4f"%struct.unpack(">f",f.read(4)))') | |||
DETECTOR_DISTANCE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(344);print "-%.4f"%struct.unpack(">f",f.read(4))') | ORGX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(540);print("%.4f"%struct.unpack(">f",f.read(4)))') | ||
ORGY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(544);print("%.4f"%struct.unpack(">f",f.read(4)))') | |||
ORGX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(540);print "%.4f"%struct.unpack(">f",f.read(4))') | OSCILLATION_RANGE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(524);phis,phie=struct.unpack(">ff",f.read(8));print("%.4f"%(phie-phis))') | ||
ORGY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(544);print "%.4f"%struct.unpack(">f",f.read(4))') | QX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(776);print("%.6f"%struct.unpack(">f",f.read(4)))') | ||
QY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(780);print("%.6f"%struct.unpack(">f",f.read(4)))') | |||
OSCILLATION_RANGE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(524);phis,phie=struct.unpack(">ff",f.read(8));print "%.4f"%(phie-phis)') | X_RAY_WAVELENGTH=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(292);print("%.6f"%struct.unpack(">f",f.read(4)))') | ||
QX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(776);print "%.6f"%struct.unpack(">f",f.read(4))') | |||
QY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(780);print "%.6f"%struct.unpack(">f",f.read(4))') | |||
X_RAY_WAVELENGTH=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(292);print "%.6f"%struct.unpack(">f",f.read(4))') | |||
elif [ "$DET" == "dtrek" ]; then | elif [ "$DET" == "dtrek" ]; then | ||
Line 695: | Line 837: | ||
DETECTOR="MAR345 MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD=130000" | DETECTOR="MAR345 MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD=130000" | ||
NX=`awk '/FORMAT/{print $2}' tmp2` | NX=`awk '/FORMAT/{print $2}' tmp2` | ||
NY=$NX | # next line is rev 1.02 (previously NY= $NX): | ||
NY=`awk -v NX=$NX '/FORMAT/{print $4/NX}' tmp2` | |||
QX=`awk '/PIXEL/{print $3/1000.}' tmp2` | QX=`awk '/PIXEL/{print $3/1000.}' tmp2` | ||
QY=`awk '/PIXEL/{print $5/1000.}' tmp2` | QY=`awk '/PIXEL/{print $5/1000.}' tmp2` | ||
Line 848: | Line 991: | ||
! however, if the spots from COLSPOT extend to 2A then POSITION could, and if 1.5A POSITION should be refined | ! however, if the spots from COLSPOT extend to 2A then POSITION could, and if 1.5A POSITION should be refined | ||
REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS ! add POSITION if high resolution, or DETECTOR_DISTANCE inaccurate | REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS ! add POSITION if high resolution, or DETECTOR_DISTANCE inaccurate | ||
REFINE(INTEGRATE)= POSITION BEAM ORIENTATION ! AXIS CELL . If 1.5A or higher it is ok to refine CELL | REFINE(INTEGRATE)= POSITION BEAM ORIENTATION ! AXIS CELL . If 1.5A or higher it is ok to refine CELL (unless electron diffraction) | ||
REFINE(CORRECT)= $REFINE_CORRECT | |||
! parameters specifically for this detector and beamline: | ! parameters specifically for this detector and beamline: | ||
DETECTOR= $DETECTOR | DETECTOR= $DETECTOR | ||
Line 865: | Line 1,007: | ||
if [ -e /usr/local/lib64/dectris-neggia.so ]; then | if [ -e /usr/local/lib64/dectris-neggia.so ]; then | ||
echo LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP | echo LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP | ||
echo LIB= line was written to XDS.INP | echo LIB= line was written to XDS.INP . For Apple ARM64 processors, you must modify the name. | ||
else | else | ||
echo !LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP | echo !LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP | ||
Line 906: | Line 1,048: | ||
cat >> XDS.INP << eof | cat >> XDS.INP << eof | ||
DIRECTION_OF_DETECTOR_X-AXIS=$DIRECTION_OF_DETECTOR_X_AXIS | DIRECTION_OF_DETECTOR_X-AXIS=$DIRECTION_OF_DETECTOR_X_AXIS | ||
DIRECTION_OF_DETECTOR_Y-AXIS= | DIRECTION_OF_DETECTOR_Y-AXIS=$DIRECTION_OF_DETECTOR_Y_AXIS | ||
eof | eof | ||
fi | fi | ||
Line 942: | Line 1,084: | ||
! and does not seem to have any downsides. | ! and does not seem to have any downsides. | ||
eof | eof | ||
if [ $NX == "1028" -a $NY == "1062" ]; then | if [ "$NX" == "1028" -a "$NY" == "1062" ]; then | ||
# Eiger2 1M ; v0.97 numbers from Andreas Förster | # Eiger2 1M ; v0.97 numbers from Andreas Förster | ||
cat >> XDS.INP << eof | cat >> XDS.INP << eof | ||
UNTRUSTED_RECTANGLE= 0 1029 512 551 | UNTRUSTED_RECTANGLE= 0 1029 512 551 | ||
eof | eof | ||
elif [ $NX == "1475" ]; then | elif [ "$NX" == "1475" ]; then | ||
if ! grep -q Flat_field tmp2 ; then | if ! grep -q Flat_field tmp2 ; then | ||
cat >> XDS.INP << eof | cat >> XDS.INP << eof | ||
Line 980: | Line 1,122: | ||
eof | eof | ||
fi | fi | ||
elif [ $NX == "2068" -a $NY == "2162" ]; then | elif [ "$NX" == "2068" -a "$NY" == "2162" ]; then | ||
# Eiger2 4M ; v0.97 numbers from Andreas Förster | # Eiger2 4M ; v0.97 numbers from Andreas Förster | ||
cat >> XDS.INP << eof | cat >> XDS.INP << eof | ||
Line 990: | Line 1,132: | ||
UNTRUSTED_RECTANGLE= 0 2069 1612 1651 | UNTRUSTED_RECTANGLE= 0 2069 1612 1651 | ||
eof | eof | ||
elif [ $NX == "2463" ]; then | elif [ "$NX" == "2463" ]; then | ||
# Pilatus 6M | # Pilatus 6M | ||
# FIXME: here we could test if a Flat_field correction was applied like we do for 2M | # FIXME: here we could test if a Flat_field correction was applied like we do for 2M | ||
Line 1,010: | Line 1,152: | ||
UNTRUSTED_RECTANGLE= 0 2464 2315 2333 | UNTRUSTED_RECTANGLE= 0 2464 2315 2333 | ||
eof | eof | ||
elif [ $NX == "3110" -a $NY == "3269" ]; then | elif [ "$NX" == "3110" -a "$NY" == "3269" ]; then | ||
# Eiger 9M | # Eiger 9M | ||
cat >> XDS.INP << eof | cat >> XDS.INP << eof | ||
Line 1,023: | Line 1,165: | ||
UNTRUSTED_RECTANGLE= 0 3110 2717 2757 | UNTRUSTED_RECTANGLE= 0 3110 2717 2757 | ||
eof | eof | ||
elif [ $NX == "3108" -a $NY == "3262" ]; then | elif [ "$NX" == "3108" -a "$NY" == "3262" ]; then | ||
# Eiger2 9M ; v0.97 numbers from Andreas Förster | # Eiger2 9M ; v0.97 numbers from Andreas Förster | ||
cat >> XDS.INP << eof | cat >> XDS.INP << eof | ||
Line 1,036: | Line 1,178: | ||
UNTRUSTED_RECTANGLE= 0 3109 2712 2751 | UNTRUSTED_RECTANGLE= 0 3109 2712 2751 | ||
eof | eof | ||
elif [ $NX == "4150" -a $NY == "4371" ]; then | elif [ "$NX" == "4150" -a "$NY" == "4371" ]; then | ||
# Eiger 16M | # Eiger 16M | ||
cat >> XDS.INP << eof | cat >> XDS.INP << eof | ||
Line 1,052: | Line 1,194: | ||
UNTRUSTED_RECTANGLE= 3109 3122 0 4371 | UNTRUSTED_RECTANGLE= 3109 3122 0 4371 | ||
eof | eof | ||
elif [ $NX == "4148" -a $NY == "4362" ]; then | elif [ "$NX" == "4148" -a "$NY" == "4362" ]; then | ||
# Eiger2 16M ; v0.97 numbers from Andreas Förster | # Eiger2 16M ; v0.97 numbers from Andreas Förster | ||
cat >> XDS.INP << eof | cat >> XDS.INP << eof | ||
Line 1,071: | Line 1,213: | ||
fi | fi | ||
echo XDS.INP is ready for use. The file has only the most important keywords. | echo XDS.INP is ready for use. The file has only the most important keywords. | ||
echo Full documentation, including complete detector templates, | echo Full documentation, including complete detector templates, at xds.mr.mpg.de . | ||
echo | echo More documentation in strucbio.biologie.uni-konstanz.de/xdswiki/index.php . | ||
echo After running xds, inspect | echo After running xds, inspect at least the agreement of predicted and observed | ||
echo spots in FRAME.cbf! | |||
rm -f tmp1 tmp2 | rm -f tmp1 tmp2 | ||
</pre> | </pre> | ||
Line 1,089: | Line 1,231: | ||
See also [[Generate_XDS.INP#Dependencies]] below, and the [[Installation]] article. | See also [[Generate_XDS.INP#Dependencies]] below, and the [[Installation]] article. | ||
== Copying generate_XDS.INP from XDSwiki webserver == | |||
On Linux: | |||
wget https://strucbio.biologie.uni-konstanz.de/pub/linux_bin/generate_XDS.INP | |||
chmod a+x generate_XDS.INP | |||
On Mac: | |||
curl -o generate_XDS.INP https://strucbio.biologie.uni-konstanz.de/pub/linux_bin/generate_XDS.INP | |||
chmod a+x generate_XDS.INP | |||
See also [[Installation]]. | |||
== Obtaining generate_XDS.INP from this webpage == | == Obtaining generate_XDS.INP from this webpage == | ||
Line 1,147: | Line 1,298: | ||
* The script tries to interpret the header of the frames, so is currently limited to Dectris (Pilatus, Eiger), ADSC (Quantum), Rigaku (several types), MAR (CCD and image plate) detectors, and one Bruker detector. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like ROTATION_AXIS, OSCILLATION_RANGE, X-RAY_WAVELENGTH, DETECTOR_DISTANCE, and XORG, YORG. For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. | * The script tries to interpret the header of the frames, so is currently limited to Dectris (Pilatus, Eiger), ADSC (Quantum), Rigaku (several types), MAR (CCD and image plate) detectors, and one Bruker detector. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like ROTATION_AXIS, OSCILLATION_RANGE, X-RAY_WAVELENGTH, DETECTOR_DISTANCE, and XORG, YORG. For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. | ||
* The authors have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has an anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing. | * The authors have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has an anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing. | ||
* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. | * At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. The frame headers do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction. Pls also see the article [[Beamline notes]]. | ||
* Sometimes, the x- and y- values of the primary beam position recorded in the header should be used for ORGY and ORGX (i.e reversed) instead of as ORGX and ORGY. For ADSC, this has been implemented in the script for a number of beamlines. | * Sometimes, the x- and y- values of the primary beam position recorded in the header should be used for ORGY and ORGX (i.e reversed) instead of as ORGX and ORGY. For ADSC, this has been implemented in the script for a number of beamlines. | ||
* there are apparently several flavours of HDF5 files produced at Diamond Light Source. They differ e.g. in the naming of the header items. This means that items like NX, NY, DETECTOR_DISTANCE and number of images cannot always be determined by the [[generate_XDS.INP]] script. Example: The data at /dls/i04-1/data/2021/mx28114-9/processing/Lenye_Diamini/ThiL/ThiL found during the CCP4 2021 online Cape Town workshop. A workaround is to use e.g. xia2 pipeline=3dii to process these files, and - if needed - extract those items from its output files, e.g. from DEFAULT/NATIVE/SWEEP1/index/XDS.INP . | |||
== See also == | == See also == |