2VB1: Difference between revisions

Jump to navigation Jump to search

2VB1 (view source)

Revision as of 15:13, 24 March 2020

1,087 bytes added ,  24 March 2020
m
no edit summary
No edit summary
mNo edit summary
 
(10 intermediate revisions by 2 users not shown)
Line 6: Line 6:
# edit [[XDS.INP]] and change/add the following:
# edit [[XDS.INP]] and change/add the following:
  ORGX=3130 ORGY=3040  ! for ADSC, header values are subject to interpretation; these values from visual inspection
  ORGX=3130 ORGY=3040  ! for ADSC, header values are subject to interpretation; these values from visual inspection
  UNTRUSTED_RECTANGLE=1 3160 3000 3070  ! <xmin xmax ymin ymax> to mask shadow of beamstop; XDS-viewer to find out
! the following is for masking the beamstop shadow in sweeps c-d
  UNTRUSTED_RECTANGLE=0 3189 2960 3087 ! use XDS-viewer of ADXV to find the values
! the following is for sweeps e-h
UNTRUSTED_RECTANGLE=1 3160 3000 3070
  TRUSTED_REGION=0 1.5 ! we want the whole detector area
  TRUSTED_REGION=0 1.5 ! we want the whole detector area
  ROTATION_AXIS=-1 0 0 ! at this beamline the spindle goes backwards!
  ROTATION_AXIS=-1 0 0 ! at this beamline the spindle goes backwards!
Line 12: Line 15:
  ! significant for hi-res data; 34.812736=32*(value for silicon as printed to CORRECT.LP if SILICON= not given)
  ! significant for hi-res data; 34.812736=32*(value for silicon as printed to CORRECT.LP if SILICON= not given)
  MAXIMUM_NUMBER_OF_PROCESSORS=4 ! for fast processing on a machine with many cores (e.g. for 16 cores)
  MAXIMUM_NUMBER_OF_PROCESSORS=4 ! for fast processing on a machine with many cores (e.g. for 16 cores)
  MAXIMUM_NUMBER_OF_JOBS=6 ! "overcommit" the available cores but on the whole this produces results faster (see below)
  MAXIMUM_NUMBER_OF_JOBS=6 ! "overcommit" the available cores but on the whole this produces results faster
  SPACE_GROUP_NUMBER=1                  ! this is known
  SPACE_GROUP_NUMBER=1                  ! this is known
  UNIT_CELL_CONSTANTS=  27.07 31.25 33.76 87.98 108.00 112.11  ! from 2vb1
  UNIT_CELL_CONSTANTS=  27.07 31.25 33.76 87.98 108.00 112.11  ! from 2vb1
Line 245: Line 248:
  INCLUDE_RESOLUTION_RANGE=30 0.65
  INCLUDE_RESOLUTION_RANGE=30 0.65


=== XSCALE.LP main table ===
=== XSCALE.LP tables ===


The error model is adjusted by XSCALE:
    a        b          ISa    ISa0  INPUT DATA SET
7.094E+00  1.294E-04  33.00  38.03 ../a/XDS_ASCII.HKL                               
7.476E+00  1.170E-04  33.81  38.95 ../b/XDS_ASCII.HKL                               
7.453E+00  1.598E-04  28.98  38.00 ../c/XDS_ASCII.HKL                               
6.539E+00  1.640E-04  30.54  39.08 ../d/XDS_ASCII.HKL                               
7.304E+00  1.342E-04  31.94  37.69 ../e/XDS_ASCII.HKL                               
8.201E+00  1.574E-04  27.83  35.58 ../f/XDS_ASCII.HKL                               
8.182E+00  1.759E-04  26.36  27.60 ../g/XDS_ASCII.HKL                               
7.717E+00  3.694E-04  18.73  21.93 ../h/XDS_ASCII.HKL                               
and there are about 1500 rejected reflections. It is reassuring to note that the error model works well; the ISa goes down toward sweep h probably because the crystal degrades. But see also the "a posterior remarks" below - sweep h is the one that is most affected by a shadow on the detector.


  SUBSET OF INTENSITY DATA WITH SIGNAL/NOISE >= -3.0 AS FUNCTION OF RESOLUTION
  SUBSET OF INTENSITY DATA WITH SIGNAL/NOISE >= -3.0 AS FUNCTION OF RESOLUTION
Line 252: Line 266:
   LIMIT    OBSERVED  UNIQUE  POSSIBLE    OF DATA  observed  expected                                      Corr
   LIMIT    OBSERVED  UNIQUE  POSSIBLE    OF DATA  observed  expected                                      Corr
   
   
     2.91      15799   2114     2147      98.5%      2.3%      2.5%    15787   73.42     2.6%    1.1%  -15%  0.705   1969
     2.91      16170   2112     2147      98.4%      2.2%      2.4%    16157   78.96     2.5%    1.1%  -12%  0.741   2023
     2.06      39607   3830     3856      99.3%      2.5%      2.8%    39602   81.49     2.6%    0.9%   -11%  0.750   3794
     2.06      40349   3831     3856      99.4%      2.4%      2.7%    40345   84.89     2.6%    0.9%   -9%  0.764   3803
     1.68      64423   5068      5087      99.6%      3.1%      3.3%    64415   82.27     3.3%    1.0%   -3%  0.843   5018
     1.68      65329   5068      5087      99.6%      3.1%      3.2%    65321   83.77     3.3%    1.0%     0%  0.847   5020
     1.45      72869   6147      6163      99.7%      3.2%      3.5%    72867   77.43     3.4%    1.0%    0%  0.833   6055
     1.45      73373   6147      6163      99.7%      3.2%      3.5%    73371   78.02     3.4%    1.0%    2%  0.842   6053
     1.30      71079   6652     6657      99.9%      3.3%      3.5%    71079   70.69     3.4%    1.1%    8%  0.865   6506
     1.30      71196   6651     6657      99.9%      3.2%      3.5%    71196   71.07     3.4%    1.1%    4%  0.857   6503
     1.19      74584   7287      7298      99.8%      3.2%      3.4%    74575   66.78     3.4%    1.2%    5%  0.870   7060
     1.19      74542   7287      7298      99.8%      3.2%      3.4%    74534   67.06     3.3%    1.2%    5%  0.854   7060
     1.10      84893   8268     8278      99.9%      3.5%      3.7%    84865   62.98     3.6%    1.3%    5%  0.858   7983
     1.10      84918   8269     8278      99.9%      3.4%      3.7%    84891   63.24     3.6%    1.3%    7%  0.853   7988
     1.03      87893   8585     8603      99.8%      4.2%      4.4%    87859   56.04     4.4%    1.5%    4%  0.828   8238
     1.03      87890   8584     8603      99.8%      4.1%      4.4%    87855   56.26     4.4%    1.5%    5%  0.818   8231
     0.97      92833   9457     9465      99.9%      5.2%      5.6%    92810   48.70     5.5%    1.7%    6%  0.802   9010
     0.97      92917   9460     9465      99.9%      5.2%      5.6%    92894   48.90     5.5%    1.7%    4%  0.795   9010
     0.92      83981   9911      9927      99.8%      5.7%      6.3%    83954   41.48     6.0%    2.1%    5%  0.785   9362
     0.92      83994   9911      9927      99.8%      5.7%      6.3%    83969   41.67     6.0%    2.0%    6%  0.787   9358
     0.88      74101   9620      9621      100.0%      6.3%      7.2%    74083   35.53     6.7%    2.6%    5%  0.785   9041
     0.88      74100   9620      9621      100.0%      6.3%      7.1%    74082   35.74     6.7%    2.5%    4%  0.772   9040
     0.84      81383   11511    11518      99.9%      6.8%      7.7%    81361   30.26     7.3%    3.3%    1%  0.760  10616
     0.84      81322   11511    11518      99.9%      6.9%      7.7%    81300   30.43     7.3%    3.3%    1%  0.760  10609
     0.81      67616   10240     10247      99.9%      7.1%      7.8%    67596   25.84     7.7%    4.2%    1%  0.782   9368
     0.81      67539   10239     10247      99.9%      7.1%      7.7%    67518   25.96     7.7%    4.2%    2%  0.779   9364
     0.78      74077   11807    11817      99.9%      7.2%      7.3%    74049   22.26     7.8%    5.2%    1%  0.797   10697
     0.78      73980   11807    11817      99.9%      7.1%      7.3%    73951   22.34     7.7%    5.3%    2%  0.799   10699
     0.75      86236   13831    13839      99.9%      8.5%      8.7%    86206   18.77    9.3%    6.7%    2%  0.809  12497
     0.75      86111   13831    13839      99.9%      8.4%      8.6%    86076   18.77    9.2%    6.8%    2%  0.809  12496
     0.73      64601   10481    10488      99.9%      10.4%    10.5%    64573   15.77   11.3%    8.2%    2%  0.810   9375
     0.73      64554   10481    10488      99.9%      10.3%    10.4%    64525   15.73   11.3%    8.2%    3%  0.815   9384
     0.71      71886   11727    11741      99.9%      12.8%    13.0%    71835   13.05   14.0%    10.6%    2%  0.800   10420
     0.71      71891   11727    11741      99.9%      12.8%    13.0%    71844   12.95   14.0%    10.6%    3%  0.810   10436
     0.69      80233   13156     13163       99.9%      16.5%    16.9%    80130   10.32   18.1%    13.7%    1%  0.796   11661
     0.69      80168   13157     13163     100.0%      16.6%    16.9%    80065   10.16   18.2%    14.1%    2%  0.799   11662
     0.67      84259   14746     14766      99.9%      22.0%    22.5%    84056   7.61   24.1%    19.6%    3%  0.789   12468
     0.67      84431   14747     14766      99.9%      22.2%    22.7%    84231   7.44   24.4%    19.7%    3%  0.798   12520
     0.65      60775   15579     16551      94.1%      27.5%    30.3%    59893   4.49   31.7%    32.3%    1%  0.723    8936
     0.65      61031   15592     16551      94.2%      27.6%    30.6%    60165   4.36   31.8%    33.1%    1%  0.723    9005
     total    1433128 190017   191232      99.4%      3.3%      3.5%  1431595   33.18     3.5%    3.5%    2%  0.801  170074
     total    1435805 190032   191232      99.4%      3.1%      3.3%  1434290   33.42     3.3%    3.1%    3%  0.801  170264


If two more resolution shells are added, they look like -
    0.64      23276    7411      9155      81.0%      35.0%    40.6%    22324    2.90    41.7%    47.9%    3%  0.683    3204
    0.63      18044    6488      9647      67.3%      42.2%    49.7%    16630    2.22    50.7%    60.9%    -5%  0.643    2437
So there is still useful signal beyond 0.65 A.


Remark: The first frames of sweeps g and h show a shadow in one corner of the detector. Nothing was done by me to exclude this shadow from processing (but one should do so if the resolution should be expanded beyond 0.65 A which the XSCALE statistics suggest to be possible). There is however no facility in XDS to exclude bad areas of specific frames in a dataset; one would need to chop the dataset into two parts.
== Some ''a posteriori'' remarks ==
 
* For sweeps e-h one should use TRUSTED_REGION= 0 1.2 since that already gives 0.626 A in the corners.
 
* The first and last frames of sweeps g and h show a shadow in one corner of the detector. Nothing was done by me to exclude this shadow from processing (but one should do so at least if the resolution should be expanded beyond 0.65 A which the XSCALE statistics suggest to be possible). <br> One could experiment with MINIMUM_VALID_PIXEL_VALUE= 40 (or so) instead of 1 - I'd probably try that, but of course one does not want to exclude valid pixels so the result has to be carefully checked. <br> Anyway, there is no general facility in XDS to exclude bad areas of ''specific'' frames in a dataset; one needs to chop the dataset into parts and deal with each shadow separately.


== Comparison of data processing: published (2006) ''vs'' XDS results ==
== Comparison of data processing: published (2006) ''vs'' XDS results ==
Line 293: Line 315:


<tr><b>
<tr><b>
<td> published(2006) </td>
<td> published (2006) </td>
<td> 30-0.65Å (0.67-0.65Å) </td>
<td> 30-0.65Å (0.67-0.65Å) </td>
<td> 1331953 (12764) </td>
<td> 1331953 (12764) </td>
Line 304: Line 326:


<tr><b>
<tr><b>
<td> XDS </td>
<td> XDS Version Dec 06, 2010 </td>
<td> 30-0.65Å (0.67-0.65Å) </td>
<td> 30-0.65Å (0.67-0.65Å) </td>
<td> 1433128 (60775) </td>
<td> 1435805 (61031) </td>
<td> 190017 (15579) </td>
<td> 190032 (15592) </td>
<td> 7.5 (3.9) </td>
<td> 7.5 (3.9) </td>
<td> 99.4 (94.1) </td>
<td> 99.4 (94.2) </td>
<td> 3.3 (27.5) </td>
<td> 3.1 (27.6) </td>
<td> 33.2 (4.5) </td>
<td> 33.4 (4.4) </td>
</b></tr>
</b></tr>


</table>
</table>


== Availability of data from XDS processing ==
I changed XSCALE.INP to have
!FRIEDEL'S_LAW=TRUE  ! by commenting it out XSCALE will use FRIEDEL'S_LAW=FALSE
!                      since this is how the data were processed
RESOLUTION_SHELLS=2.91 2.06 1.68 1.45 1.30 1.19 1.10 1.03 0.97 0.92 0.88 0.84 0.80 0.76 0.73 0.70 0.67 0.65 0.64 0.63


== timings for processing sweep "e" as a function of MAXIMUM_NUMBER_OF_PROCESSORS and MAXIMUM_NUMBER_OF_JOBS ==
and ran XSCALE again, to get a file with reflections to 0.63 A.  
 
The following is going to be rather technical! If you are only interested in crystallography, skip this.
 
Using
MAXIMUM_NUMBER_OF_PROCESSORS=2
MAXIMUM_NUMBER_OF_JOBS=8
we observe for the INTEGRATE step:
total cpu time used              2063.6 sec
total elapsed wall-clock time      296.1 sec
 
Using
MAXIMUM_NUMBER_OF_PROCESSORS=1
MAXIMUM_NUMBER_OF_JOBS=16
the times are
total cpu time used              2077.1 sec
total elapsed wall-clock time      408.2 sec
 
Using
MAXIMUM_NUMBER_OF_PROCESSORS=4
MAXIMUM_NUMBER_OF_JOBS=4
the times are
total cpu time used              2102.8 sec
total elapsed wall-clock time      315.6 sec


Using
Conversion to other program systems is performed with XDSCONV. XDSCONV.INP for producing a MTZ file with intensities and anomalous signal is:
MAXIMUM_NUMBER_OF_PROCESSORS=16 ! the default for xds_par on a 16-core machine
  INPUT_FILE= lys-xds.ahkl
  MAXIMUM_NUMBER_OF_JOBS=1 ! the default
  OUTPUT_FILE=temp.hkl CCP4_I
the times are
total cpu time used              2833.4 sec
  total elapsed wall-clock time      566.5 sec
but please note that this actually only uses 10 processors, since the default DELPHI=5
and the OSCILLATION_RANGE is 0.5°.


Using
After running xdsconv, I cut-and-paste the screen output:
  MAXIMUM_NUMBER_OF_PROCESSORS=4
  f2mtz HKLOUT temp.mtz<F2MTZ.INP
  MAXIMUM_NUMBER_OF_JOBS=8
  cad HKLIN1 temp.mtz HKLOUT output_file_name.mtz<<EOF
(thus overcommitting the available cores by a factor of 2) the times are
LABIN FILE 1 ALL
  total cpu time used              2263.5 sec
  END
  total elapsed wall-clock time      320.8 sec
  EOF


Using
and obtain output_file_name.mtz which I mv to [https://{{SERVERNAME}}/pub/xds-datared/2vb1/xds-hewl-I.mtz xds-hewl-I.mtz]. SFCHECK statistics for this file are [https://{{SERVERNAME}}/pub/xds-datared/2vb1/sfcheck_XXXX.pdf here].
MAXIMUM_NUMBER_OF_PROCESSORS=4
MAXIMUM_NUMBER_OF_JOBS=6
(thus overcommitting the available cores, but less severely) the times are
total cpu time used              2367.6 sec
total elapsed wall-clock time      267.2 sec


Thus,  
Similarly, using OUTPUT_FILE=temp.hkl CCP4 I obtained a file with amplitudes, [https://{{SERVERNAME}}/pub/xds-datared/2vb1/xds-hewl-F.mtz xds-hewl-F.mtz]
MAXIMUM_NUMBER_OF_PROCESSORS=4
MAXIMUM_NUMBER_OF_JOBS=6
performs best for a 2-Xeon X5570 (HT enabled, thus 16 cores) machine with 24GB of memory and a RAID1 consisting of 2 1TB SATA disks. It should be noted that the dataset has 27GB, and in 296 seconds this means 92 MB/s continuous reading. The processing time is thus limited by the disk access, not by the CPU. And no, the data are not simply read from RAM (tested by "echo 3 > /proc/sys/vm/drop_caches" before the XDS run).
Retrieved from "https://wiki.uni-konstanz.de/xds/index.php/Special:MobileDiff/2345...4081"
Cookies help us deliver our services. By using our services, you agree to our use of cookies.
More information

Navigation menu