Generate XDS.INP: Difference between revisions

Jump to navigation Jump to search

Generate XDS.INP (view source)

Revision as of 17:38, 27 April 2022

7,696 bytes added ,  27 April 2022
→‎The script: KD 27.4.22 always print out detector serial number
(→‎The script: rev 1.01)
(→‎The script: KD 27.4.22 always print out detector serial number)
(31 intermediate revisions by the same user not shown)
Line 10: Line 10:


== The script ==
== The script ==
<pre>
<pre>


Line 117: Line 118:
# revision 0.99 . Aaron Finke (CHESS) 07/11/2020 add ROTATION_AXIS=-1 0 0 for EIGER2 16M detector at MX beamline ID7B2 (CHESS)
# revision 0.99 . Aaron Finke (CHESS) 07/11/2020 add ROTATION_AXIS=-1 0 0 for EIGER2 16M detector at MX beamline ID7B2 (CHESS)
# revision 1.00 . Gleb Bourenkov 10/11/2020 add ROTATION_AXIS=0 -1 0 for beamline P14 (EMBL Hamburg) detectors Eiger 16M S/N E-32-0107; previously Eiger 4M S/N E-08-0107 and PILATUS 6M-F, S/N 60-0115-F
# revision 1.00 . Gleb Bourenkov 10/11/2020 add ROTATION_AXIS=0 -1 0 for beamline P14 (EMBL Hamburg) detectors Eiger 16M S/N E-32-0107; previously Eiger 4M S/N E-08-0107 and PILATUS 6M-F, S/N 60-0115-F
# revision 1.01 . KD 07/12/2020 ROTATION_AXIS=0 -1 0 for Pilatus3 X 2M, S/N 24-0118 at ID23-2 (http://www.globalphasing.com/autoproc/wiki/index.cgi?BeamlineSettings)
# revision 1.01 . KD 07/12/2020 ROTATION_AXIS=0 -1 0 for Pilatus3 2M, S/N 24-0118 at ID23-2 (http://www.globalphasing.com/autoproc/wiki/index.cgi?BeamlineSettings)
REVISION="1.01 (07-Dec-2020)"
# revision 1.02 . KD 11/01/2021 recognize mar555 detector as mar345 (thanks to Thomas Hauß, HZB)
# revision 1.03 . Feng Yu 18/7/2021 fix/expand information about Shanghai Synchrotron Radiation Facility (SSRF)
# revision 1.04 . Zhipu Luo 13/08/2021 extract ORGX,ORGY for electron diffraction from SMV header if the wavelength value starts with 0.0
# revision 1.05 . KD specialcase extraction of number of images for BNL detectors E-32-0101 and E-18-0104
# revision 1.06 . KD fix URLs in output. Availability of Apple M1 processor dectris-neggia-Apple-arm64.so . LC_ALL=C .
# revision 1.07 . KD, Thomas Hauß, Gleb Bourenkov 25/10/2021. Detector moved from Petra P14 to P13
# revision 1.08 . KD, Thomas Hauß, Feng Yu 7/11/2021 fix TZ in timestamps for SSRF detectors
# revision 1.09 . KD, Feng Yu 7/11/2021 fix detector number BNL E-18-0121 to be E-18-0104
# revision 1.10 . KD NeXus header for Eiger
# revision 1.11 . KD / Helena Taberman ROTATION_AXIS=-1 0 0 for Eiger .cbf data from Petra P14
# revision 1.12 . KD / Helena Taberman correct rev 1.11 to be 0 -1 0
REVISION="1.12 (25-MAR-2022)"


#                                                                                                             
#                                                                                                             
Line 126: Line 138:
#                                                                                                             
#                                                                                                             
# known problems:                                                                                             
# known problems:                                                                                             
# revision 1.10 . KD implement NeXus for Eiger
# - for ADSC detectors, there are at least three ways to obtain ORGX and ORGY values from the header (see below);
# - for ADSC detectors, there are at least three ways to obtain ORGX and ORGY values from the header (see below);
# - the same might be a problem for MAR headers, too (not sure about this)  
# - the same might be a problem for MAR headers, too (not sure about this)  
Line 131: Line 144:
#                                                                                                                 
#                                                                                                                 
# notes for debugging of the script:                                                                             
# notes for debugging of the script:                                                                             
# - add the -x option to the first line, to see where an error occurs                                             
# - add the -x option to #!/bin/bash in first line, to see where an error occurs                                             
# - comment out the removal of tmp1 and tmp2 in the last line                                                     
# - comment out the removal of tmp1 and tmp2 in the last line                                                     
#                                                                                                                 
#                                                                                                                 
Line 142: Line 155:
   exit                                                                                                           
   exit                                                                                                           
fi                                                                                                               
fi                                                                                                               
# make sure the locale does not interfere with e.g. awk calculations:
LC_ALL="C";export LC_ALL
#                                                                                                                 
#                                                                                                                 
# defaults:                                                                                                       
# defaults:                                                                                                       
Line 165: Line 180:
# default DIRECTION_OF_DETECTOR_X-AXIS
# default DIRECTION_OF_DETECTOR_X-AXIS
DIRECTION_OF_DETECTOR_X_AXIS="1 0 0"
DIRECTION_OF_DETECTOR_X_AXIS="1 0 0"
# default DIRECTION_OF_DETECTOR_Y-AXIS
DIRECTION_OF_DETECTOR_Y_AXIS="0 1 0"
# default FRACTION_OF_POLARIZATION
# default FRACTION_OF_POLARIZATION
pol_frac=0.98
pol_frac=0.98
Line 171: Line 188:
SEPMIN=7.0
SEPMIN=7.0
CLUSTER_RADIUS=3.5
CLUSTER_RADIUS=3.5
REFINE_CORRECT="CELL BEAM ORIENTATION AXIS POSITION ! Default is: refine everything"


dname=`echo "$1" | xargs dirname`
dname=`echo "$1" | xargs dirname`
Line 200: Line 218:
  data_last=`cut -b $pos1-$pos2 tmp1 | tail -n1 | bc`
  data_last=`cut -b $pos1-$pos2 tmp1 | tail -n1 | bc`
  DATA_RANGE="$data_first $data_last"
  DATA_RANGE="$data_first $data_last"
echo DATA_RANGE=$DATA_RANGE
   
   
  # set SPOT_RANGE to first half of DATA_RANGE
  # set SPOT_RANGE to first half of DATA_RANGE
Line 211: Line 230:
FIRSTFRAME=`head -1 tmp1`
FIRSTFRAME=`head -1 tmp1`
echo $FIRSTFRAME | grep "\.h5$" && is_h5=1 || is_h5=0
echo $FIRSTFRAME | grep "\.h5$" && is_h5=1 || is_h5=0
if [ "$is_h5" == 1 ]; then
# nframes=`h5dump -A -g "/entry/data" $FIRSTFRAME | grep "DATASPACE  SIMPLE" | sed -e "s/,.*//" | awk '{a+=$5}END{print a}'`
nframes=`h5dump -d "/entry/instrument/detector/detectorSpecific/nimages" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
DATA_RANGE="1 $nframes"
SPOT_RANGE="1 `echo "scale=0; if (${nframes}<2) 1; if (${nframes}>1) ${nframes}/2"|bc -l`"
fi
echo DATA_RANGE=$DATA_RANGE


# find out detector type
# find out detector type
Line 242: Line 252:
  grep -q "^SOURCE_WAVELENGTH= *1" tmp2 && DET=dtrek
  grep -q "^SOURCE_WAVELENGTH= *1" tmp2 && DET=dtrek
  grep -q BEAM_CENTER_X tmp2      && DET=adsc                                 
  grep -q BEAM_CENTER_X tmp2      && DET=adsc                                 
  grep -q mar345 tmp2 && DET=MAR345    
  grep -q mar345 tmp2 && DET=MAR345
# rev. 1.02: in one case, mar555 data could be processed pretending it is mar345, so:
grep -q mar555 tmp2 && DET=MAR345
  grep -q BRUKER tmp2 && grep -q CBF tmp2  && DET=Bruker-cbf   
  grep -q BRUKER tmp2 && grep -q CBF tmp2  && DET=Bruker-cbf   
  grep -q CMOS1 tmp2 && DET=adsc-CMOS1   
  grep -q CMOS1 tmp2 && DET=adsc-CMOS1   
  grep -q MARCONTROL tmp2 && DET=OLDMAR                  
  grep -q MARCONTROL tmp2 && DET=OLDMAR
grep -q "WAVELENGTH=0.0" tmp2 && DET=experimental-ED                                   
else
else
  h5dump -d "/entry/instrument/detector/description" $FIRSTFRAME | grep -i Eiger > /dev/null && DET=eiger
  h5dump -d "/entry/instrument/detector/description" $FIRSTFRAME | grep -i Eiger > /dev/null && DET=eiger
Line 465: Line 478:
   DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=/-/`                                                                     
   DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=/-/`                                                                     
   OSCILLATION_RANGE=`grep OMEGA_DELTA tmp2 | sed s/OMEGA_DELTA=//`   
   OSCILLATION_RANGE=`grep OMEGA_DELTA tmp2 | sed s/OMEGA_DELTA=//`   
   DIRECTION_OF_DETECTOR_X_AXIS="-1 0 0"                                                                 
   DETECTOR_X_AXIS="-1 0 0"  
                                                                  
elif [ "$DET" == "experimental-ED" ]; then
  DETECTOR="ADSC MINIMUM_VALID_PIXEL_VALUE= 1 OVERLOAD= 65000  ! OVERLOAD is really unknown "
  echo --- Electron diffraction data in SMV format!
  SENSOR_THICKNESS=0.01                                                       
  sed s/\;// tmp2 > tmp1                                                     
  mv tmp1 tmp2                                                               
  X_RAY_WAVELENGTH=`grep WAVELENGTH tmp2 | head -1 | sed s/WAVELENGTH=//`
  NX=`grep SIZE1 tmp2 | tail -1 | sed s/SIZE1=//`
  QX=`grep PIXEL_SIZE tmp2 | sed s/PIXEL_SIZE=//`
  NY=`grep SIZE2 tmp2 | tail -1 | sed s/SIZE2=//`                                           
  QY=$QX 
  BEAM_CENTER_X=`grep BEAM_CENTER_X tmp2 | sed s/BEAM_CENTER_X=//`
  BEAM_CENTER_Y=`grep BEAM_CENTER_Y tmp2 | sed s/BEAM_CENTER_Y=//`
  QXY=`echo "scale=1; $QX*($BEAM_CENTER_X+$BEAM_CENTER_Y)" | bc -l`
      if [ "$QXY" "<" "$BEAM_CENTER_X" ] ; then
      ORGX=`echo "scale=2; $BEAM_CENTER_Y/$QX" | bc -l`
      ORGY=`echo "scale=2; $BEAM_CENTER_X/$QX" | bc -l`
      else
      ORGX=`echo "scale=2; $BEAM_CENTER_X/1" | bc -l`
      ORGY=`echo "scale=2; $BEAM_CENTER_Y/1" | bc -l`
      fi
  DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=//`                                                                   
  OSCILLATION_RANGE=`grep OSC_RANGE tmp2 | sed s/OSC_RANGE=//`
  STARTING_ANGLE=`grep OSC_START tmp2 | sed s/OSC_START=//`
  echo --- For TIMEPIX detector, please fix ROTATION_AXIS yourself!
  DIRECTION_OF_DETECTOR_X_AXIS="1 0 0"
  REFINE_CORRECT="ORIENTATION CELL AXIS BEAM ! for ED, no POSITION when CELL is refined"


elif [ "$DET" == "pilatus" ]; then
elif [ "$DET" == "pilatus" ]; then
Line 482: Line 523:


       # find SENSOR_THICKNESS:
       # find SENSOR_THICKNESS:
       SENSOR_THICKNESS=`grep thickness tmp2 | sed -e s/'Silicon sensor, thickness'// | awk '{print $1*1000}'`
       SENSOR_THICKNESS=`grep thickness tmp2 | sed -e s/'Si.* sensor, thickness'// | awk '{print $1*1000}'`
       # find X_RAY_WAVELENGTH:                                                                               
       # find X_RAY_WAVELENGTH:                                                                               
       X_RAY_WAVELENGTH=`grep Wavelength tmp2 | sed -e s/Wavelength// -e s/A// | awk '{print $1}'`             
       X_RAY_WAVELENGTH=`grep Wavelength tmp2 | sed -e s/Wavelength// -e s/A// | awk '{print $1}'`             
Line 534: Line 575:
       fi
       fi
# PETRA P14: raw data from Eiger are stored as CBF files so this is treated as Pilatus
# PETRA P14: raw data from Eiger are stored as CBF files so this is treated as Pilatus
       if [ "$DET_SN" == "Dectris Eiger 16M, E-32-0107" -o "$DET_SN" == "Dectris Eiger 4M, E-08-0107" -o "$DET_SN" == "PILATUS 6M-F, S/N 60-0115-F" ] ; then
       if [ "$DET_SN" == "Dectris Eiger 4M, E-08-0107" -o "$DET_SN" == "PILATUS 6M-F, S/N 60-0115-F" \
                                        -o "$DET_SN" == "Dectris EIGER2 CdTe 16M, E-32-0129" ] ; then
         rotation_axis="0 -1 0"
         rotation_axis="0 -1 0"
         echo ROTATION_AXIS="0 -1 0" at PETRA P14
         echo ROTATION_AXIS="0 -1 0" at PETRA P14
      fi
# similarly, Eiger 16M at PETRA P14 until May 22, 2021
      if [ "$DET_SN" == "Dectris Eiger 16M, E-32-0107" ] ; then
        egrep -q '^# 2020-|^# 2021-0[1-5]|^# 2021-06-[01]|^# 2021-06-2[01]' tmp2 || isatP13=1
        if [ "$isatP13" == 1 ] ; then
          echo using the default ROTATION_AXIS=1 0 0 at PETRA P13
        else
          rotation_axis="0 -1 0"
          echo ROTATION_AXIS="0 -1 0" at PETRA P14
        fi
       fi
       fi
# ESRF ID23-2:
# ESRF ID23-2:
       if [ "$DET_SN" == "Pilatus3 X 2M, S/N 24-0118" ] ; then
       if [ "$DET_SN" == "PILATUS3 2M, S/N 24-0118, ESRF ID23" ] ; then
         rotation_axis="0 -1 0"
         rotation_axis="0 -1 0"
         echo ROTATION_AXIS="0 -1 0" at ESRF ID23-2
         echo ROTATION_AXIS="0 -1 0" at ESRF ID23-2
Line 545: Line 597:


elif [ "$DET" == "eiger" ]; then
elif [ "$DET" == "eiger" ]; then
nframes=`h5dump -d "/entry/instrument/detector/detectorSpecific/nimages" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
DATA_RANGE="1 $nframes"
SPOT_RANGE="1 `echo "scale=0; if (${nframes}<2) 1; if (${nframes}>1) ${nframes}/2"|bc -l`"
SN=`h5dump -d "/entry/instrument/detector/detector_number" $FIRSTFRAME | awk '/\(0\): /{print $2}' | sed s/\"//g`
echo detector serial number is $SN
# find out if HDF5 from Diamond (DLS=1) or Dectris (DLS=0)
# find out if HDF5 from Diamond (DLS=1) or Dectris (DLS=0)
   DLS=0
   DLS=0
Line 558: Line 616:
   # rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"`
   # rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"`
   # the above gives -1 0 0 for DLS data instead of the correct 1 0 0, so commented out for now
   # the above gives -1 0 0 for DLS data instead of the correct 1 0 0, so commented out for now
  # comment while implementing rev 1.10: this is because the DLS headers are NeXus, so DIRECTION_OF_DETECTOR_X/Y_AXIS must be adjusted.
   else
   else
     echo Eiger HDF5 from Dectris
     echo Eiger HDF5 from Dectris
Line 564: Line 623:
   # STARTING_ANGLE:  the \- was introduced in version 0.91 to allow negative values :
   # STARTING_ANGLE:  the \- was introduced in version 0.91 to allow negative values :
     STARTING_ANGLE=`h5dump -d "/entry/sample/goniometer/omega_start" $FIRSTFRAME | awk '/\(0\): [\-0-9]/{print $2}'`
     STARTING_ANGLE=`h5dump -d "/entry/sample/goniometer/omega_start" $FIRSTFRAME | awk '/\(0\): [\-0-9]/{print $2}'`
   # If rotation vector set (NeXus)
  # /entry/sample/goniometer/omega_start is missing in some eiger2 detectors (e.g. Eiger2 9M with fw version release-2020.2.1 and SIMPLON API 1.8) (Feng YU 2021-07-18)
    if [ "$STARTING_ANGLE" == "" ]; then
  echo "/entry/sample/goniometer/omega_start not found, try /entry/sample/goniometer/omega"
  STARTING_ANGLE=`h5dump -d /entry/sample/goniometer/omega $FIRSTFRAME | grep "(0):" | head -n 1 | awk '{print $2}' | sed -e "s/,//g"`
    fi
    echo "STARTING_ANGLE=$STARTING_ANGLE"
   # If rotation axis set (NeXus)
     rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"`
     rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"`
   # Eiger 16M SSRF BL17U1 has S/N E-32-0111;  EIGER2 16M CHESS ID7B2 has S/N E-32-0123 (A. Finke 2020-11-07) v0.99
   # EIGER2 16M CHESS ID7B2 has S/N E-32-0123 (A. Finke 2020-11-07) v0.99
    SN=`h5dump -d "/entry/instrument/detector/detector_number" $FIRSTFRAME | awk '/\(0\): /{print $2}' | sed s/\"//g`
     if [ "$SN" == "E-32-0123" ]; then
     if [ "$SN" == "E-32-0111" ] || [ "$SN" == "E-32-0123" ]; then
       rotation_axis="-1 0 0"
       rotation_axis="-1 0 0"
       echo SSRF BL17U1 or CHESS ID7B2 with inverted rotation axis
       echo CHESS ID7B2 with inverted rotation axis
     fi  
    fi
  # SSRF BL17U1 and SSRF BL10U2 (Feng YU 2021-07-18)
  # Eiger X 16M (S/N E-32-0111) was installed at SSRF BL17U1 from Oct 2017 to Feb 2021. After Feb 2021, it was moved back to SSRF BL10U2.
# 2021-03-01 00:00:00 time stamp is 1614528000
    if [ "$SN" == "E-32-0111" ]; then
      collection_time=`h5dump -d "/entry/instrument/detector/detectorSpecific/data_collection_date" $FIRSTFRAME | grep "(0):" | awk '{print $2}' | sed -e "s/\.\(.*\)//g; s/\"//g"`
      if [ `uname -s` == "Darwin" ]; then
        collection_timestamp=`date -j -f "%Y-%m-%dT%H:%M:%S" $collection_time +%s`
      elif [ `uname -s` == "Linux" ]; then
        collection_timestamp=`date -d $collection_time +%s`
      else
        collection_timestamp=0
      fi
      if [ $collection_timestamp -eq 0 ]; then
        is_rotation_axis_set=1
        rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0`
        if [ "$is_rotation_axis_set" == "1" ]; then
          rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"`
          echo "The rotation axis of SSRF BL10U2 (Eiger X 16M) is $rotation_axis"
        else
          rotation_axis="-1 0 0    ! Cannot determine rotation axis. SSRF BL17U1: -1 0 0; SSRF BL02U1: 0 -1 0"
        fi
      elif [ $collection_timestamp -ge 0 ] && [ $collection_timestamp -le 1614528000 ]; then
        # SSRF BL17U1
        rotation_axis="-1 0 0"
        echo "SSRF BL17U1 (Eiger X 16M) with inverted rotation axis"
      else
        # SSRF BL10U2
        is_rotation_axis_set=1
        rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0`
        if [ "$is_rotation_axis_set" == "1" ]; then
          rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"`
          echo "The rotation axis of SSRF BL10U2 (Eiger X 16M) is $rotation_axis"
        else
          rotation_axis="0 -1 0"
          echo "SSRF BL10U2 (Eiger X 16M) with vertical rotation axis"
        fi
      fi
    fi
  # SSRF BL02U1 (Feng YU 2021-07-18)
  # EIGER2 S 9M SSRF BL02U1 has S/N E-18-0121
    if [ "$SN" == "E-18-0121" ]; then
      is_rotation_axis_set=1
      rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0`
      if [ "$is_rotation_axis_set" == "1" ]; then
        rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"`
        echo "The rotation axis of SSRF BL02U1 (Eiger2 S 9M) is $rotation_axis"
      else
        rotation_axis="1 0 0"
        echo "SSRF BL02U1 (Eiger2 S 9M) with horizontal rotation axis"
      fi
    fi
# revision 1.05 specialcase nframes for Eiger detectors at BNL
    if [ "$SN" == "E-18-0104" -o "$SN" == "E-32-0101" ]; then
      echo specialcase nframes for Eiger detectors at BNL:
      nframes=`h5dump -A -g "/entry/data" $FIRSTFRAME | grep "DATASPACE  SIMPLE" | sed -e "s/,.*//" | awk '{a+=$5}END{print a}'`
      DATA_RANGE="1 $nframes"
      SPOT_RANGE="1 `echo "scale=0; if (${nframes}<2) 1; if (${nframes}>1) ${nframes}/2"|bc -l`"
     fi
  fi
# rev 1.10: check for NeXus header. If found, its geometry will overwrite any rotation_axis set until here.
  NeXus=0
  h5dump -d "/entry/definition" $FIRSTFRAME | grep -q NXmx && NeXus=1
  if [ "$NeXus" == "1" ]; then
    echo NeXus header found. This defines DIRECTION_OF_DETECTOR_X/Y-AXIS and ROTATION_AXIS.
    DIRECTION_OF_DETECTOR_X_AXIS=$(h5dump -d "/entry/instrument/detector/module/fast_pixel_direction" $FIRSTFRAME 2>/dev/null | grep "(0):" | tail -1 | sed -e "s/^.*://; s/,//g")
    DIRECTION_OF_DETECTOR_Y_AXIS=$(h5dump -d "/entry/instrument/detector/module/slow_pixel_direction" $FIRSTFRAME 2>/dev/null | grep "(0):" | tail -1 | sed -e "s/^.*://; s/,//g")
    rotation_axis=$(h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g")
  else
    echo no NeXus header found.
   fi
   fi
  echo DATA_RANGE=$DATA_RANGE
   DETECTOR="EIGER MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD= $OVERLOAD"
   DETECTOR="EIGER MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD= $OVERLOAD"
   QX=`h5dump -d "/entry/instrument/detector/x_pixel_size" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
   QX=`h5dump -d "/entry/instrument/detector/x_pixel_size" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
Line 600: Line 734:
   #let SKIP=768
   #let SKIP=768
   #NX=$(od -t x -j $SKIP -N 4 $FIRSTFRAME |awk 'NR==1{print toupper($2)}'|perl -nle '@array= $_ =~/.{2}/g; print "ibase=16;obase=A;".join("",reverse @array)'|bc)
   #NX=$(od -t x -j $SKIP -N 4 $FIRSTFRAME |awk 'NR==1{print toupper($2)}'|perl -nle '@array= $_ =~/.{2}/g; print "ibase=16;obase=A;".join("",reverse @array)'|bc)
!  NX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(768);print "%.4d"%struct.unpack(">i",f.read(4))')
!  NY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(772);print "%.4d"%struct.unpack(">i",f.read(4))')
   NX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(768);print("%.4d"%struct.unpack(">i",f.read(4)))')
   NX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(768);print("%.4d"%struct.unpack(">i",f.read(4)))')
   NY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(772);print("%.4d"%struct.unpack(">i",f.read(4)))')
   NY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(772);print("%.4d"%struct.unpack(">i",f.read(4)))')
!  DETECTOR_DISTANCE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(344);print "-%.4f"%struct.unpack(">f",f.read(4))')
   DETECTOR_DISTANCE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(344);print("-%.4f"%struct.unpack(">f",f.read(4)))')
   DETECTOR_DISTANCE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(344);print("-%.4f"%struct.unpack(">f",f.read(4)))')
!  ORGX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(540);print "%.4f"%struct.unpack(">f",f.read(4))')
!  ORGY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(544);print "%.4f"%struct.unpack(">f",f.read(4))')
   ORGX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(540);print("%.4f"%struct.unpack(">f",f.read(4)))')
   ORGX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(540);print("%.4f"%struct.unpack(">f",f.read(4)))')
   ORGY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(544);print("%.4f"%struct.unpack(">f",f.read(4)))')
   ORGY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(544);print("%.4f"%struct.unpack(">f",f.read(4)))')
!  OSCILLATION_RANGE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(524);phis,phie=struct.unpack(">ff",f.read(8));print "%.4f"%(phie-phis)')
   OSCILLATION_RANGE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(524);phis,phie=struct.unpack(">ff",f.read(8));print("%.4f"%(phie-phis))')
   OSCILLATION_RANGE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(524);phis,phie=struct.unpack(">ff",f.read(8));print("%.4f"%(phie-phis))')
!  QX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(776);print "%.6f"%struct.unpack(">f",f.read(4))')
!  QY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(780);print "%.6f"%struct.unpack(">f",f.read(4))')
   QX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(776);print("%.6f"%struct.unpack(">f",f.read(4)))')
   QX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(776);print("%.6f"%struct.unpack(">f",f.read(4)))')
   QY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(780);print("%.6f"%struct.unpack(">f",f.read(4)))')
   QY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(780);print("%.6f"%struct.unpack(">f",f.read(4)))')
!  X_RAY_WAVELENGTH=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(292);print "%.6f"%struct.unpack(">f",f.read(4))')
   X_RAY_WAVELENGTH=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(292);print("%.6f"%struct.unpack(">f",f.read(4)))')
   X_RAY_WAVELENGTH=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(292);print("%.6f"%struct.unpack(">f",f.read(4)))')


Line 717: Line 837:
  DETECTOR="MAR345  MINIMUM_VALID_PIXEL_VALUE=0  OVERLOAD=130000"
  DETECTOR="MAR345  MINIMUM_VALID_PIXEL_VALUE=0  OVERLOAD=130000"
  NX=`awk '/FORMAT/{print $2}' tmp2`
  NX=`awk '/FORMAT/{print $2}' tmp2`
  NY=$NX
# next line is rev 1.02 (previously NY= $NX):
  NY=`awk -v NX=$NX '/FORMAT/{print $4/NX}' tmp2`
  QX=`awk '/PIXEL/{print $3/1000.}' tmp2`
  QX=`awk '/PIXEL/{print $3/1000.}' tmp2`
  QY=`awk '/PIXEL/{print $5/1000.}' tmp2`
  QY=`awk '/PIXEL/{print $5/1000.}' tmp2`
Line 870: Line 991:
! however, if the spots from COLSPOT extend to 2A then POSITION could, and if 1.5A POSITION should be refined
! however, if the spots from COLSPOT extend to 2A then POSITION could, and if 1.5A POSITION should be refined
REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS  ! add POSITION if high resolution, or DETECTOR_DISTANCE inaccurate
REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS  ! add POSITION if high resolution, or DETECTOR_DISTANCE inaccurate
REFINE(INTEGRATE)= POSITION BEAM ORIENTATION ! AXIS CELL . If 1.5A or higher it is ok to refine CELL
REFINE(INTEGRATE)= POSITION BEAM ORIENTATION ! AXIS CELL . If 1.5A or higher it is ok to refine CELL (unless electron diffraction)
! REFINE(CORRECT)=CELL BEAM ORIENTATION AXIS POSITION ! Default is: refine everything
REFINE(CORRECT)= $REFINE_CORRECT
 
! parameters specifically for this detector and beamline:
! parameters specifically for this detector and beamline:
DETECTOR= $DETECTOR
DETECTOR= $DETECTOR
Line 887: Line 1,007:
     if [ -e /usr/local/lib64/dectris-neggia.so ]; then
     if [ -e /usr/local/lib64/dectris-neggia.so ]; then
       echo LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP
       echo LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP
       echo LIB= line was written to XDS.INP
       echo LIB= line was written to XDS.INP . For Apple ARM64 processors, you must modify the name.
     else
     else
       echo !LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP
       echo !LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP
Line 928: Line 1,048:
  cat >> XDS.INP << eof
  cat >> XDS.INP << eof
DIRECTION_OF_DETECTOR_X-AXIS=$DIRECTION_OF_DETECTOR_X_AXIS
DIRECTION_OF_DETECTOR_X-AXIS=$DIRECTION_OF_DETECTOR_X_AXIS
DIRECTION_OF_DETECTOR_Y-AXIS=0 1 0
DIRECTION_OF_DETECTOR_Y-AXIS=$DIRECTION_OF_DETECTOR_Y_AXIS
eof
eof
  fi
  fi
Line 964: Line 1,084:
!                                                  and does not seem to have any downsides.  
!                                                  and does not seem to have any downsides.  
eof
eof
   if [ $NX == "1028"  -a $NY == "1062" ]; then
   if [ "$NX" == "1028"  -a "$NY" == "1062" ]; then
# Eiger2 1M ; v0.97 numbers from Andreas Förster
# Eiger2 1M ; v0.97 numbers from Andreas Förster
     cat >> XDS.INP << eof
     cat >> XDS.INP << eof
UNTRUSTED_RECTANGLE=    0 1029    512  551
UNTRUSTED_RECTANGLE=    0 1029    512  551
eof
eof
   elif [ $NX == "1475" ]; then
   elif [ "$NX" == "1475" ]; then
     if ! grep -q Flat_field tmp2 ; then
     if ! grep -q Flat_field tmp2 ; then
     cat >> XDS.INP << eof
     cat >> XDS.INP << eof
Line 1,002: Line 1,122:
eof
eof
     fi
     fi
   elif [ $NX == "2068"  -a $NY == "2162" ]; then
   elif [ "$NX" == "2068"  -a "$NY" == "2162" ]; then
# Eiger2 4M ; v0.97 numbers from Andreas Förster
# Eiger2 4M ; v0.97 numbers from Andreas Förster
     cat >> XDS.INP << eof
     cat >> XDS.INP << eof
Line 1,012: Line 1,132:
  UNTRUSTED_RECTANGLE=    0 2069  1612 1651
  UNTRUSTED_RECTANGLE=    0 2069  1612 1651
eof
eof
   elif [ $NX == "2463" ]; then
   elif [ "$NX" == "2463" ]; then
# Pilatus 6M
# Pilatus 6M
# FIXME: here we could test if a Flat_field correction was applied like we do for 2M
# FIXME: here we could test if a Flat_field correction was applied like we do for 2M
Line 1,032: Line 1,152:
UNTRUSTED_RECTANGLE=  0 2464  2315 2333
UNTRUSTED_RECTANGLE=  0 2464  2315 2333
eof
eof
   elif [ $NX == "3110"  -a $NY == "3269" ]; then
   elif [ "$NX" == "3110"  -a "$NY" == "3269" ]; then
# Eiger 9M
# Eiger 9M
     cat >> XDS.INP << eof
     cat >> XDS.INP << eof
Line 1,045: Line 1,165:
UNTRUSTED_RECTANGLE= 0 3110 2717 2757
UNTRUSTED_RECTANGLE= 0 3110 2717 2757
eof
eof
   elif [ $NX == "3108"  -a $NY == "3262" ]; then
   elif [ "$NX" == "3108"  -a "$NY" == "3262" ]; then
# Eiger2 9M ; v0.97 numbers from Andreas Förster
# Eiger2 9M ; v0.97 numbers from Andreas Förster
     cat >> XDS.INP << eof
     cat >> XDS.INP << eof
Line 1,058: Line 1,178:
UNTRUSTED_RECTANGLE=    0 3109  2712 2751
UNTRUSTED_RECTANGLE=    0 3109  2712 2751
eof
eof
   elif [ $NX == "4150" -a $NY == "4371" ]; then
   elif [ "$NX" == "4150" -a "$NY" == "4371" ]; then
# Eiger 16M
# Eiger 16M
     cat >> XDS.INP << eof
     cat >> XDS.INP << eof
Line 1,074: Line 1,194:
  UNTRUSTED_RECTANGLE= 3109 3122      0 4371
  UNTRUSTED_RECTANGLE= 3109 3122      0 4371
eof
eof
   elif [ $NX == "4148" -a $NY == "4362" ]; then
   elif [ "$NX" == "4148" -a "$NY" == "4362" ]; then
# Eiger2 16M ; v0.97 numbers from Andreas Förster
# Eiger2 16M ; v0.97 numbers from Andreas Förster
     cat >> XDS.INP << eof
     cat >> XDS.INP << eof
Line 1,093: Line 1,213:
fi
fi
echo XDS.INP is ready for use. The file has only the most important keywords.
echo XDS.INP is ready for use. The file has only the most important keywords.
echo Full documentation, including complete detector templates, is at
echo Full documentation, including complete detector templates, at xds.mr.mpg.de .
echo http://www.mpimf-heidelberg.mpg.de/~kabsch/xds . More documentation in XDSwiki
echo More documentation in strucbio.biologie.uni-konstanz.de/xdswiki/index.php .
echo After running xds, inspect, using XDS-Viewer, at least the beamstop mask in
echo After running xds, inspect at least the agreement of predicted and observed  
echo BKGPIX.cbf, and the agreement of predicted and observed spots in FRAME.cbf!
echo spots in FRAME.cbf!
rm -f tmp1 tmp2
rm -f tmp1 tmp2
</pre>
</pre>
Line 1,178: Line 1,298:
* The script tries to interpret the header of the frames, so is currently limited to Dectris (Pilatus, Eiger), ADSC (Quantum), Rigaku (several types), MAR (CCD and image plate) detectors, and one Bruker detector. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like ROTATION_AXIS, OSCILLATION_RANGE, X-RAY_WAVELENGTH, DETECTOR_DISTANCE, and XORG, YORG. For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates].
* The script tries to interpret the header of the frames, so is currently limited to Dectris (Pilatus, Eiger), ADSC (Quantum), Rigaku (several types), MAR (CCD and image plate) detectors, and one Bruker detector. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like ROTATION_AXIS, OSCILLATION_RANGE, X-RAY_WAVELENGTH, DETECTOR_DISTANCE, and XORG, YORG. For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates].
* The authors have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has an anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing.
* The authors have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has an anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing.
* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. We have only just started the article [[Beamline notes]], and the frame headers do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction.
* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. The frame headers do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction. Pls also see the article [[Beamline notes]].
* Sometimes, the x- and y- values of the primary beam position recorded in the header should be used for ORGY and ORGX (i.e reversed) instead of as ORGX and ORGY. For ADSC, this has been implemented in the script for a number of beamlines.
* Sometimes, the x- and y- values of the primary beam position recorded in the header should be used for ORGY and ORGX (i.e reversed) instead of as ORGX and ORGY. For ADSC, this has been implemented in the script for a number of beamlines.
* there are apparently several flavours of HDF5 files produced at Diamond Light Source. They differ e.g. in the naming of the header items. This means that items like NX, NY, DETECTOR_DISTANCE and number of images cannot always be determined by the [[generate_XDS.INP]] script. Example: The data at /dls/i04-1/data/2021/mx28114-9/processing/Lenye_Diamini/ThiL/ThiL found during the CCP4 2021 online Cape Town workshop. A workaround is to use e.g. xia2 pipeline=3dii to process these files, and - if needed - extract those items from its output files, e.g. from DEFAULT/NATIVE/SWEEP1/index/XDS.INP .


== See also ==
== See also ==
Kay
Bureaucrats
2,652

edits

Retrieved from "https://wiki.uni-konstanz.de/xds/index.php/Special:MobileDiff/4227...4401"
Cookies help us deliver our services. By using our services, you agree to our use of cookies.
More information

Navigation menu