Difficult datasets

The following suggestions apply not only to small-molecule datasets, but also to very weak or low-resolution, or otherwise difficult macromolecular datasets.

These datasets have few (or few strong, or few well-defined) reflections per frame. Therefore, the multitude of parameters describing the diffraction experiment needs to be reduced (in refinement one would say: to avoid overfitting). This means that some parameters may need adjustment.

optimizing the parameters for INTEGRATE

DELPHI=45 ! (or 60, or even 90; the default is 5) to base reflection profiles and refinements on more reflections - try this first if you get error messages about incomplete profiles in the INTEGRATE step, or if the geometric parameters during INTEGRATE refinements "run away"
REFINE(INTEGRATE)= ! do not refine anything in INTEGRATE; be sure to recycle GXPARM.XDS to XPARM.XDS. Also try REFINE(INTEGRATE)=ORIENTATION CELL ! maybe add BEAM, but probably AXIS or DISTANCE should not be refined.

Furthermore, you may grab the lines e.g.

BEAM_DIVERGENCE=   2.067  BEAM_DIVERGENCE_E.S.D.=   0.207
REFLECTING_RANGE=  2.303  REFLECTING_RANGE_E.S.D.=  0.329

from INTEGRATE.LP, and insert them into XDS.INP . The latter is accomplished easily with

grep _E INTEGRATE.LP | tail -2 > x
grep -v _E.S.D XDS.INP >> x
mv x XDS.INP

or similar. These values then serve as a Bayesian prior for the values that will actually be used for integrating.

If you used larger OSCILLATION_RANGE than 1°, you should increase MAXIMUM_ERROR_OF_SPINDLE_POSITION= (default is 2°) because you expect larger deviations of experimentally determined and calculated phi positions of reflections. Same might apply to MAXIMUM_ERROR_OF_SPOT_POSITION= ; the default is 3 pixels which may be too low to account for broad reflections. If you don't do this, many reflections that violate these limits will not be used for geometry refinement, which may turn out to be unstable.

optimizing the parameters for CORRECT

NBATCH= (total rotation range)/45 ! to reduce the number of scale factors; the value of NBATCH is shown as NXBIN in CORRECT.LP for the ABSORB and DECAY correction; default is (total rotation range)/5. Other values, like (total rotation range)/60 or (total rotation range)/90 might also be tried. Try to match NBATCH with DELPHI (see above).
CORRECTIONS= ABSORB ! don't try to correct for MODULATION and DECAY in scaling
FRIEDEL'S_LAW=TRUE unless the multiplicity is high. A clear sign that FRIEDEL'S_LAW=TRUE is required is that the CHI^2-VALUE OF FIT OF CORRECTION FACTORS is significantly below 1 (e.g. 0.5)

optimizing the parameters for XSCALE

If you use XSCALE to merge several XDS datasets, you should make sure that you specify in XSCALE.INP the same values for NBATCH and CORRECTIONS as you do, in XDS.INP, for the CORRECT step.