Low dose data - Revision history

Kay: fix link

2019-10-22T10:28:12Z

fix link

← Older revision		Revision as of 12:28, 22 October 2019
Line 3:		Line 3:
	In XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.		In XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.

	This seems to work well down to an average of 0.05 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://~~homes~~.mpimf-heidelberg.mpg.de~~/~kabsch/xds~~/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.		This seems to work well down to an average of 0.05 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.

	Even if this feature is used, DEFPIX may stop with the (misleading) error message <pre>!!! ERROR !!! WRONG DETECTOR PARAMETERS</pre> If this happens, omit DEFPIX from the JOB= list of parameters, and copy BKGINIT.cbf to BKGPIX.cbf (DEFPIX would have to write BKGPIX.cbf, but this has to be done manually).

	See also: [[Difficult datasets]]		See also: [[Difficult datasets]]

Kay: add DEFPIX "wrong detector parameters" workaround (thanks to Tim Grüne!)

2019-06-04T19:57:43Z

add DEFPIX "wrong detector parameters" workaround (thanks to Tim Grüne!)

← Older revision		Revision as of 21:57, 4 June 2019
Line 5:		Line 5:
	This seems to work well down to an average of 0.05 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.		This seems to work well down to an average of 0.05 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.

	~~The following problem was fixed in the March 1, 2015 version of XDS: "Below 0.5 counts/pixel~~, ~~the I/sigma values become overly optimistic. Data frames which have been collected too finely/weakly~~ may ~~be added together using~~ the ~~[http:~~/~~/homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/merge2cbf_program.html merge2cbf] program of the XDS package. This is currently only relevant for~~ the ~~Pilatus and Eiger detectors; a typical low background on other types~~ of ~~detectors is more on the order of 10 counts/pixel (Holton~~ and ~~Fraenkel (2010) Acta Cryst~~. (~~2010)~~. ~~D66~~, ~~393–408; [http://dx.doi.org/10.1107/S0907444910007262]~~)."		Even if this feature is used, DEFPIX may stop with the (misleading) error message <pre>!!! ERROR !!! WRONG DETECTOR PARAMETERS</pre> If this happens, omit DEFPIX from the JOB= list of parameters, and copy BKGINIT.cbf to BKGPIX.cbf (DEFPIX would have to write BKGPIX.cbf, but this has to be done manually).

	See also: [[Difficult datasets]]		See also: [[Difficult datasets]]

Kay at 10:46, 9 March 2017

2017-03-09T10:46:41Z

← Older revision		Revision as of 12:46, 9 March 2017
Line 3:		Line 3:
	In XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.		In XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.

	This seems to work ~~very~~ well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.		This seems to work well down to an average of 0.05 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.

	The following problem was fixed in the March 1, 2015 version of XDS: "Below 0.5 counts/pixel, the I/sigma values become overly optimistic. Data frames which have been collected too finely/weakly may be added together using the [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/merge2cbf_program.html merge2cbf] program of the XDS package. This is currently only relevant for the Pilatus ~~detector~~; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262])."		The following problem was fixed in the March 1, 2015 version of XDS: "Below 0.5 counts/pixel, the I/sigma values become overly optimistic. Data frames which have been collected too finely/weakly may be added together using the [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/merge2cbf_program.html merge2cbf] program of the XDS package. This is currently only relevant for the Pilatus and Eiger detectors; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262])."

	See also: [[Difficult datasets]]		See also: [[Difficult datasets]]

Kay at 16:41, 2 March 2015

2015-03-02T16:41:14Z

← Older revision		Revision as of 18:41, 2 March 2015
Line 5:		Line 5:
	This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.		This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.

	~~Another~~ problem ~~is that below~~ 0.5 counts/pixel, the I/sigma values become overly optimistic. Whether this is due to the intensities being too high or the sigmas too low has not yet been investigated; likely it's the latter. This effect is ''not'' mitigated by use of [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR= DATA_RANGE_FIXED_SCALE_FACTOR] .		The following problem was fixed in the March 1, 2015 version of XDS: "Below 0.5 counts/pixel, the I/sigma values become overly optimistic. Data frames which have been collected too finely/weakly may be added together using the [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/merge2cbf_program.html merge2cbf] program of the XDS package. This is currently only relevant for the Pilatus detector; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262])."

	~~CC1/2 (printed out in CORRECT.LP since version of March 15, 2012), on the other hand, appears to be more realistic.~~

	~~It is therefore recommended that~~
	* exposure time and flux is high enough to ensure at least 0.5 counts per pixel, on average.
	* fine-slicing is performed according to the findings in the open-access paper [http://journals.iucr.org/d/issues/2012/01/00/wd5161/index.html] which means that the OSCILLATION_RANGE should be no less than half the REFLECTING_RANGE_E.S.D. ; choosing OSCILLATION_RANGE about the same as REFLECTING_RANGE_E.S.D. should result in data that are close to optimal.

	Data frames which have been collected too finely/weakly may be added together using the [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/merge2cbf_program.html merge2cbf] program of the XDS package.

	~~All of this~~ is currently only relevant for the Pilatus detector; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262]).

	See also: [[Difficult datasets]]		See also: [[Difficult datasets]]

Kay at 16:04, 9 January 2013

2013-01-09T16:04:47Z

← Older revision		Revision as of 18:04, 9 January 2013
Line 1:		Line 1:
	The background counts per pixel are printed in INIT.LP, or can be estimated by visualizing the frames with [[XDS-Viewer]] or [[adxv]].		The average of the background counts per pixel are printed in INIT.LP, or can be estimated by visualizing the frames with [[XDS-Viewer]] or [[adxv]].

	~~With the current version of~~ XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.		In XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.

	This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.		This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.

Kay: change text to use DATA_RANGE_FIXED_SCALE_FACTOR instead of FIXED_SCALE_FACTOR

2012-08-15T11:27:52Z

change text to use DATA_RANGE_FIXED_SCALE_FACTOR instead of FIXED_SCALE_FACTOR

← Older revision		Revision as of 13:27, 15 August 2012
Line 3:		Line 3:
	With the current version of XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.		With the current version of XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.

	This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#~~FIXED_SCALE_FACTOR~~= ~~FIXED_SCALE_FACTOR~~] has been introduced to handle this situation.		This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR== DATA_RANGE_FIXED_SCALE_FACTOR] has been introduced to handle this situation.

	Another problem is that below 0.5 counts/pixel, the I/sigma values become overly optimistic. Whether this is due to the intensities being too high or the sigmas too low has not yet been investigated; likely it's the latter. This effect is ''not'' mitigated by use of [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#~~FIXED_SCALE_FACTOR~~= ~~FIXED_SCALE_FACTOR~~] .		Another problem is that below 0.5 counts/pixel, the I/sigma values become overly optimistic. Whether this is due to the intensities being too high or the sigmas too low has not yet been investigated; likely it's the latter. This effect is ''not'' mitigated by use of [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#DATA_RANGE_FIXED_SCALE_FACTOR= DATA_RANGE_FIXED_SCALE_FACTOR] .

	CC1/2 (printed out in CORRECT.LP since version of March 15, 2012), on the other hand, appears to be more realistic.		CC1/2 (printed out in CORRECT.LP since version of March 15, 2012), on the other hand, appears to be more realistic.

Kay at 16:44, 30 April 2012

2012-04-30T16:44:00Z

← Older revision		Revision as of 18:44, 30 April 2012
Line 5:		Line 5:
	This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#FIXED_SCALE_FACTOR= FIXED_SCALE_FACTOR] has been introduced to handle this situation.		This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#FIXED_SCALE_FACTOR= FIXED_SCALE_FACTOR] has been introduced to handle this situation.

	Another problem ~~in this regime is that the background of a reflection~~ is ~~calculated from pixels around those~~ that ~~contribute to the reflection~~. ~~If many of the background pixels are zero~~, ~~the background becomes (relatively) less smooth, and empirically we find that~~ the I/sigma values ~~that XDS calculates may~~ become overly optimistic. Whether use of [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#FIXED_SCALE_FACTOR= FIXED_SCALE_FACTOR] ~~mitigates this problem is not known yet~~.		Another problem is that below 0.5 counts/pixel, the I/sigma values become overly optimistic. Whether this is due to the intensities being too high or the sigmas too low has not yet been investigated; likely it's the latter. This effect is ''not'' mitigated by use of [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#FIXED_SCALE_FACTOR= FIXED_SCALE_FACTOR] .

	~~It is therefore recommended that fine-slicing is performed according to the findings~~ in ~~the open-access paper [http://journals~~.~~iucr.org/d/issues/~~2012~~/01/00/wd5161/index.html] which means that~~ the ~~OSCILLATION_RANGE should~~ be ~~no less than half the REFLECTING_RANGE_E.S.D. ; choosing OSCILLATION_RANGE about the same as REFLECTING_RANGE_E.S.D. should result in data that are close to optimal~~.		CC1/2 (printed out in CORRECT.LP since version of March 15, 2012), on the other hand, appears to be more realistic.

	Data frames which have been collected "too finely" may be added together using the [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/merge2cbf_program.html merge2cbf] program of the XDS package.		It is therefore recommended that
			* exposure time and flux is high enough to ensure at least 0.5 counts per pixel, on average.
			* fine-slicing is performed according to the findings in the open-access paper [http://journals.iucr.org/d/issues/2012/01/00/wd5161/index.html] which means that the OSCILLATION_RANGE should be no less than half the REFLECTING_RANGE_E.S.D. ; choosing OSCILLATION_RANGE about the same as REFLECTING_RANGE_E.S.D. should result in data that are close to optimal.

			Data frames which have been collected too finely/weakly may be added together using the [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/merge2cbf_program.html merge2cbf] program of the XDS package.

	All of this is currently only relevant for the Pilatus detector; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262]).		All of this is currently only relevant for the Pilatus detector; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262]).

	See also: [[Difficult datasets]]		See also: [[Difficult datasets]]

Kay at 09:27, 27 April 2012

2012-04-27T09:27:24Z

← Older revision		Revision as of 11:27, 27 April 2012
Line 12:		Line 12:

	All of this is currently only relevant for the Pilatus detector; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262]).		All of this is currently only relevant for the Pilatus detector; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262]).

			See also: [[Difficult datasets]]

Kay at 09:22, 27 April 2012

2012-04-27T09:22:01Z

← Older revision		Revision as of 11:22, 27 April 2012
Line 1:		Line 1:
			The background counts per pixel are printed in INIT.LP, or can be estimated by visualizing the frames with [[XDS-Viewer]] or [[adxv]].

	With the current version of XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.		With the current version of XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.

Kay: Created page with "With the current version of XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step. This seems to work very well do..."

2011-12-28T18:05:31Z

Created page with "With the current version of XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step. This seems to work very well do..."

New page

With the current version of XDS, the background of a data frame is used for scaling adjacent frames relative to each other in the INTEGRATE step.

This seems to work very well down to an average of 0.5 counts/pixel. If, however, the crystal is exposed so shortly/weakly that the average drops significantly below this value, scaling becomes impossible - the ratio of averages which are around zero is obviously undefined. The keyword [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#FIXED_SCALE_FACTOR= FIXED_SCALE_FACTOR] has been introduced to handle this situation.

Another problem in this regime is that the background of a reflection is calculated from pixels around those that contribute to the reflection. If many of the background pixels are zero, the background becomes (relatively) less smooth, and empirically we find that the I/sigma values that XDS calculates may become overly optimistic. Whether use of [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#FIXED_SCALE_FACTOR= FIXED_SCALE_FACTOR] mitigates this problem is not known yet.

It is therefore recommended that fine-slicing is performed according to the findings in the open-access paper [http://journals.iucr.org/d/issues/2012/01/00/wd5161/index.html] which means that the OSCILLATION_RANGE should be no less than half the REFLECTING_RANGE_E.S.D. ; choosing OSCILLATION_RANGE about the same as REFLECTING_RANGE_E.S.D. should result in data that are close to optimal.

Data frames which have been collected "too finely" may be added together using the [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/merge2cbf_program.html merge2cbf] program of the XDS package.

All of this is currently only relevant for the Pilatus detector; a typical low background on other types of detectors is more on the order of 10 counts/pixel (Holton and Fraenkel (2010) Acta Cryst. (2010). D66, 393–408; [http://dx.doi.org/10.1107/S0907444910007262]).