Data quality

What is the resolution of my dataset?

First of all, it is limited by completeness. In practical terms this means that the highest resolution you can get is the resolution at the edge of the detector. If you collected enough frames, you may be able to squeeze out 0.1A if you process data all the way to the corner. Usually the detector is positioned close enough to the crystal so that you don't have any diffraction at the edge and then resolution limits should be chosen based on strength of the diffraction.

This limit is commonly based on average [math]\displaystyle{ I/\sigma }[/math]. Examples of such choices are:

• [math]\displaystyle{ I/\sigma=1 }[/math] in the highest resolution shell
• [math]\displaystyle{ I/\sigma=2 }[/math] in the highest resolution shell
• at least 50% of reflections in the highest resolution shell have [math]\displaystyle{ I/\sigma }[/math] > 2
• ...

Some of these choices are more liberal than others (and so will result in higher resolution values). It is probably not worthwhile to argue which choice is the best, since it is indeed a matter of personal preference.

There is probably not much reason to limit resolution by R_merge. When the resolution limit is selected based on R_merge being less than a certain cutoff, the argument is that in higher resolution shells the variation among independent measurements of the intensity of the same reflection is too high. But such variation is indeed bound to be high for weak reflections. R_merge may and should be used as the measure of the overall data consistency (e.g. of two independent datasets the one that has higher R_merge probably is noisier).

Of course you can achieve lower R-factors in refinement by setting the resolution limit based on some cutoff value of R_merge. It is perfectly OK to aspire low R-factors, but to achieve this by throwing away good data isn't. The better strategy probably is to choose a generous high resolution limit early during structure solution, and to decide near the end of the refinement, by inspecting maps and comparing model R-factors at different resolutions, at which resolution the useful signal vanishes in the noise.

Improved indicators for data quality

R_merge is the wrong quantity to look at altogether, because

• it depends on the multiplicity (unfortunately often called redundancy): the higher the multiplicity, the higher R_merge becomes
• it assesses data consistency, not the quality of the reduced data

This has been discussed by Diederichs and Karplus, who suggest a multiplicity-independant version called R_meas, which unfortunately is not used by everyone because the formula gives higher values than R_merge. R-factors for data quality assessment were also suggested by Diederichs and Karplus, and Weiss and Hilgenfeld . Weiss showed that these R-factors are indeed strongly correlated with the quality of the data.