ISa: Difference between revisions

From XDSwiki
Jump to navigation Jump to search
(Undo revision 2176 by Kay (Talk))
No edit summary
Line 2: Line 2:


==An estimate for the overall quality of an experimental setup==
==An estimate for the overall quality of an experimental setup==
A useful Unix command that prints a single number that depends on the overall quality of an experimental setup (beam, crystal, spindle, detector, cryo, software, ...) is
awk '/a          b/{getline;print 1/($1*$2)^0.5}' CORRECT.LP
This will give you the upper limit of I/sigma(I) for any reflection in your dataset - even if your crystal is great, all reflections are bound to be worse than that.


Why does the command give you such a useful value? It just finds the line " a          b " in [[CORRECT.LP]], grabs the values of "a" and "b" from the next line, and prints out 1/sqrt(a*b). The values a and b appear in the formula v(I)=a*(v0(I)+b*I^2) which is used by CORRECT to adjust the variances of the intensities, to match their experimental spread. For strong and well-measured reflections, the variance is dominated by the systematic error that is introduced by any beam /spindle / detector /cryo or other instability or malfunction. For weak reflections, v0(I), the variance from counting statistics, dominates. The value for v(I) that the formula gives, will be higher than v'(I)=a*b*I^2 by an amount a*v0(I). Therefore, I/sigma(I) = I/sqrt(v(I)) will be lower than I/sqrt(v'(I)) = 1/sqrt(a*b) which is what the Unix command prints out.
A single number that depends on the overall quality of an experimental setup (beam, crystal, spindle, detector, cryo, software, ...) is the upper limit of I/sigma(I) for any reflection in your dataset - even if your crystal is great, all reflections are bound to be worse than that.  


What might go wrong with this simple measure? Sometimes, e.g. if too few strong reflections exist in the dataset, b might come out negative. In that case the Unix command prints out "nan" which means "not a number" and indicates that it could not calculate the square root of a negative number.  
This number is called <math>{I/Sigma(I)}^{asymptotic}</math> ([Diederichs, Acta Cryst. (2010). D66, 733-740 http://dx.doi.org/10.1107/S0907444910014836])
 
What is that number? Scaling procedures (like the ones used in XDS, SCALA, SCALEPACK and dscaleaverage The variance of a reflection comes from random errors, and from systematic influences The values a and b appear in the formula v(I)=a*(v0(I)+b*I^2) which is used by CORRECT to adjust the variances of the intensities, to match their experimental spread. For strong and well-measured reflections, the variance is dominated by the systematic error that is introduced by any beam /spindle / detector /cryo or other instability or malfunction. For weak reflections, v0(I), the variance from counting statistics, dominates. The value for v(I) that the formula gives, will be higher than v'(I)=a*b*I^2 by an amount a*v0(I). Therefore, I/sigma(I) = I/sqrt(v(I)) will be lower than I/sqrt(v'(I)) = 1/sqrt(a*b) which is what the Unix command prints out.
 
=== Versions of XDS before May 10, 2010 ===
A Unix command to obtain it from CORRECT.LP is
awk '/a *b *I/{getline;print ($1*($2+4e-4))^-0.5}' CORRECT.LP
 
It just finds the line " a          b            INPUT DATA SET" in [[CORRECT.LP]], grabs the values of "a" and "b" from the next line, and prints out 1/sqrt(a*(b+0.0004)).
 
The 0.0004 stems from the fact that the INTEGRATE step does its own adjustment of the variances, and the two adjustments (in INTEGRATE and CORRECT) have to be combined.
 
=== Versions of XDS since May 10, 2010 ===
 
Newer versions print out 1/sqrt(a*(b+0.0004)) as "ISa".  


As you can see from the formula, low values of a and b are ''good'' in the sense that a high upper limit of I/sigma(I) results. If e.g. the crystal is badly split or broken, or reflections are too close on the detector, or the data reduction is not good (wrong parameters), then the values of a and b are elevated.
As you can see from the formula, low values of a and b are ''good'' in the sense that a high upper limit of I/sigma(I) results. If e.g. the crystal is badly split or broken, or reflections are too close on the detector, or the data reduction is not good (wrong parameters), then the values of a and b are elevated.


If your crystal is good (and no matter ''how'' good your crystal is!), then a and b will reflect the quality of the other components of the experimental setup (e.g. beamline stability). I have seen values around 20 for good crystals that still allowed my to solve a MAD structure, but that required high multiplicity of observations. Values around 30 allowed me to solve a sulfur-SAD structure at medium resolution (diffraction to 2.3 A, anomalous signal to 3 A). On the bright side, I have also seen a value of 87.6 for Z. Dauter's 0.98A Proteinase K (2ID8) sulfur-SAD data from J. Holton's APS/22-ID beamline.
If your crystal is good (and no matter ''how'' good your crystal is!), then a and b will reflect the quality of the other components of the experimental setup (e.g. beamline stability).  
 
== Practical considerations ==
In practice, both crystal quality and beamline quality limit the value of the asymptotic I/SigmaI . A good crystal (even with elevated mosaicity and medium resolution) should give a high value of the asymptotic I/SigmaI on a good beamline.
I have seen values around 20 for good crystals that still allowed my to solve a MAD structure, but that required high multiplicity of observations. Values around 30 allowed me to solve a sulfur-SAD structure at medium resolution (diffraction to 2.3 A, anomalous signal to 3 A).  
I have also seen a value around 40 for Z. Dauter's 0.98A Proteinase K (2ID8) sulfur-SAD data from J. Holton's APS/22-ID beamline, and recently even higher values were obtained at the SLS, beamline X06SA, with a Pilatus detector.
On the other hand, I have sometimes obtained values less than 10 with good test crystals. It is always good to discuss this with the people who are responsible for the beamline. They might know what is broken, or might be able to find out what went wrong.

Revision as of 16:50, 29 May 2010

CORRECT is the scaling step of XDS.

An estimate for the overall quality of an experimental setup

A single number that depends on the overall quality of an experimental setup (beam, crystal, spindle, detector, cryo, software, ...) is the upper limit of I/sigma(I) for any reflection in your dataset - even if your crystal is great, all reflections are bound to be worse than that.

This number is called [math]\displaystyle{ {I/Sigma(I)}^{asymptotic} }[/math] ([Diederichs, Acta Cryst. (2010). D66, 733-740 http://dx.doi.org/10.1107/S0907444910014836])

What is that number? Scaling procedures (like the ones used in XDS, SCALA, SCALEPACK and dscaleaverage The variance of a reflection comes from random errors, and from systematic influences The values a and b appear in the formula v(I)=a*(v0(I)+b*I^2) which is used by CORRECT to adjust the variances of the intensities, to match their experimental spread. For strong and well-measured reflections, the variance is dominated by the systematic error that is introduced by any beam /spindle / detector /cryo or other instability or malfunction. For weak reflections, v0(I), the variance from counting statistics, dominates. The value for v(I) that the formula gives, will be higher than v'(I)=a*b*I^2 by an amount a*v0(I). Therefore, I/sigma(I) = I/sqrt(v(I)) will be lower than I/sqrt(v'(I)) = 1/sqrt(a*b) which is what the Unix command prints out.

Versions of XDS before May 10, 2010

A Unix command to obtain it from CORRECT.LP is

awk '/a *b *I/{getline;print ($1*($2+4e-4))^-0.5}' CORRECT.LP

It just finds the line " a b INPUT DATA SET" in CORRECT.LP, grabs the values of "a" and "b" from the next line, and prints out 1/sqrt(a*(b+0.0004)).

The 0.0004 stems from the fact that the INTEGRATE step does its own adjustment of the variances, and the two adjustments (in INTEGRATE and CORRECT) have to be combined.

Versions of XDS since May 10, 2010

Newer versions print out 1/sqrt(a*(b+0.0004)) as "ISa".

As you can see from the formula, low values of a and b are good in the sense that a high upper limit of I/sigma(I) results. If e.g. the crystal is badly split or broken, or reflections are too close on the detector, or the data reduction is not good (wrong parameters), then the values of a and b are elevated.

If your crystal is good (and no matter how good your crystal is!), then a and b will reflect the quality of the other components of the experimental setup (e.g. beamline stability).

Practical considerations

In practice, both crystal quality and beamline quality limit the value of the asymptotic I/SigmaI . A good crystal (even with elevated mosaicity and medium resolution) should give a high value of the asymptotic I/SigmaI on a good beamline. I have seen values around 20 for good crystals that still allowed my to solve a MAD structure, but that required high multiplicity of observations. Values around 30 allowed me to solve a sulfur-SAD structure at medium resolution (diffraction to 2.3 A, anomalous signal to 3 A). I have also seen a value around 40 for Z. Dauter's 0.98A Proteinase K (2ID8) sulfur-SAD data from J. Holton's APS/22-ID beamline, and recently even higher values were obtained at the SLS, beamline X06SA, with a Pilatus detector. On the other hand, I have sometimes obtained values less than 10 with good test crystals. It is always good to discuss this with the people who are responsible for the beamline. They might know what is broken, or might be able to find out what went wrong.