CORRECT.LP
Statistics of reflections
Near the top of CORRECT.LP we find:
531781 REFLECTIONS ON FILE "INTEGRATE.HKL"
0 CORRUPTED REFLECTION RECORDS (IGNORED)
0 REFLECTIONS INCOMPLETE OR OUTSIDE IMAGE RANGE 1 ... 1799
0 OVERLOADED REFLECTIONS (IGNORED)
81 REFLECTIONS OUTSIDE ACCEPTED RESOLUTION RANGES
OR TOO CLOSE TO ROTATION AXIS (IGNORED)
531700 REFLECTIONS ACCEPTED
Statistics of observations
XDS, like SCALA and d*TREK, gives statistics about unaveraged and averaged quantities, but in different tables. The unaveraged values are in a table that is fine-grained in terms of resolution, at the beginning of CORRECT.LP. The Sigma values in that table are corrected to match the RMS scatter.
The table that has information about the averaged data (suitably weighted) is repeated several times. It is less fine-grained in resolution (9 shells, and overall). [if a user wants this table in fine-grained form, s/he can use XSCALE].
The way the tables are printed is the same for both types of tables: at first the definitions of the quantities in the table are given, and then the table itself is printed.
Specifically, the heading of the table which talks about the unaveraged data looks like this:
I/Sigma = mean intensity/Sigma of a reflection in shell
Chi2 = goodness of fit between sample variances of
symmetry-related intensities and their errors
(Chi2 = 1 for perfect agreement)
R-FACTOR
observed = (SUM(ABS(I(h,i)-I(h))))/(SUM(I(h,i)))
expected = expected R-FACTOR derived from Sigma(I)
NUMBER = number of reflections in resolution shell
used for calculation of R-FACTOR
ACCEPTED = number of accepted reflections
REJECTED = number of rejected reflections (MISFITS),
recognized by comparison with symmetry-related
reflections.
and then the table itself is:
RESOLUTION RANGE I/Sigma Chi2 R-FACTOR R-FACTOR NUMBER ACCEPTED REJECTED
observed expected
48.268 17.853 9.63 0.97 5.06 6.10 865 868 44
17.853 13.079 10.02 0.97 5.22 6.14 1301 1305 81
13.079 10.812 9.83 1.10 5.56 5.94 1374 1388 99
10.812 9.423 9.88 1.09 5.32 6.03 1820 1825 108
9.423 8.460 9.56 1.07 6.03 6.21 2087 2101 167
.... (many resolution shells deleted for brevity)
Statistics of unique reflections
Later tables talk about the averaged intensities:
R-FACTOR
observed = (SUM(ABS(I(h,i)-I(h))))/(SUM(I(h,i)))
expected = expected R-FACTOR derived from Sigma(I)
COMPARED = number of reflections used for calculating R-FACTOR
I/SIGMA = mean of intensity/Sigma(I) of unique reflections
(after merging symmetry-related observations)
Sigma(I) = standard deviation of reflection intensity I
estimated from sample statistics
R-meas = redundancy independent R-factor (intensities)
Diederichs & Karplus (1997), Nature Struct. Biol. 4, 269-275.
CC(1/2) = percentage of correlation between intensities from
random half-datasets. Correlation significant at
the 0.1% level is marked by an asterisk.
Karplus & Diederichs (2012), Science 336, 1030-33
Anomal = percentage of correlation between random half-sets
Corr of anomalous intensity differences. Correlation
significant at the 0.1% level is marked.
SigAno = mean anomalous difference in units of its estimated
standard deviation (|F(+)-F(-)|/Sigma). F(+), F(-)
are structure factor estimates obtained from the
merged intensity observations in each parity class.
Nano = Number of unique reflections used to calculate
Anomal_Corr & SigAno. At least two observations
for each (+ and -) parity are required.
and the table itself is
NOTE: Friedel pairs are treated as different reflections.
SUBSET OF INTENSITY DATA WITH SIGNAL/NOISE >= -3.0 AS FUNCTION OF RESOLUTION
RESOLUTION NUMBER OF REFLECTIONS COMPLETENESS R-FACTOR R-FACTOR COMPARED I/SIGMA R-meas CC(1/2) Anomal SigAno Nano
LIMIT OBSERVED UNIQUE POSSIBLE OF DATA observed expected Corr
5.72 23750 7284 7488 97.3% 6.6% 6.6% 23666 14.59 7.9% 99.3* 33* 1.043 3033
4.06 41574 12997 13384 97.1% 10.0% 8.3% 41476 11.40 12.1% 98.3* 45* 1.341 5775
3.32 56679 16961 17336 97.8% 16.8% 15.4% 56494 6.49 20.1% 97.9* 31* 1.079 7697
2.88 67173 20272 20497 98.9% 38.4% 39.0% 66875 2.91 45.9% 93.1* 19* 0.840 9333
2.57 79365 23100 23197 99.6% 77.6% 85.3% 79063 1.46 92.1% 75.3* 5 0.701 10761
2.35 86431 25554 25631 99.7% 128.9% 146.7% 86014 0.86 153.2% 54.7* 3 0.633 11894
2.18 83863 27529 27946 98.5% 197.0% 230.0% 81669 0.49 237.7% 31.6* -1 0.575 11422
2.04 51338 23815 29966 79.5% 286.2% 343.0% 43478 0.26 361.1% 15.1* 0 0.526 5523
1.92 25803 15877 31898 49.8% 483.3% 577.5% 17026 0.12 635.3% 3.8 2 0.519 1856
total 515976 173389 197343 87.9% 27.8% 29.3% 495761 2.89 33.5% 98.2* 19* 0.781 67294
NUMBER OF REFLECTIONS IN SELECTED SUBSET OF IMAGES 531700
NUMBER OF REJECTED MISFITS 15698
NUMBER OF SYSTEMATIC ABSENT REFLECTIONS 0
NUMBER OF ACCEPTED OBSERVATIONS 516002
NUMBER OF UNIQUE ACCEPTED REFLECTIONS 173398
Why is there a discrepancy between "total 515976 173389" versus "NUMBER OF ACCEPTED OBSERVATIONS 516002", and "NUMBER OF UNIQUE ACCEPTED REFLECTIONS 173398" ?? The reason is that the higher numbers include even those reflections with I<3*sigma(I), whereas the numbers in the table refer only to those reflections which should be used downstream (for phasing and refinement). Indeed, XDSCONV filters out observations with I<-3*sigma(I).
At the bottom of CORRECT.LP we find:
NUMBER OF UNIQUE ALIEN REFLECTIONS WITH A Z-SCORE ABOVE LIMIT 162 (ALIENS ABOVE LIMIT (REJECT_ALIEN= 20.0) ARE MARKED INVALID) NUMBER OF REFLECTION RECORDS ON OUTPUT FILE "XDS_ASCII.HKL" 531700 NUMBER OF ACCEPTED OBSERVATIONS (INCLUDING SYSTEMATIC ABSENCES) 515712 NUMBER OF REJECTED MISFITS & ALIENS (marked by -1*SIGMA(IOBS)) 15988
The file XDS_ASCII.HKL actually has all 531700 reflections.