Xscale - Revision history

Kay: /* keywords unique to XSCALE */ REIDX_ISET

2024-03-20T10:11:30Z

keywords unique to XSCALE: REIDX_ISET

← Older revision		Revision as of 12:11, 20 March 2024
Line 51:		Line 51:

	=== keywords unique to XSCALE ===		=== keywords unique to XSCALE ===
			* REIDX_ISET= ! re-index data from the most recent INPUT_FILE
	* MERGE= ! average intensities from all input files, applies to output file		* MERGE= ! average intensities from all input files, applies to output file
	* WEIGHT= ! applies to input file		* WEIGHT= ! applies to input file

Kay: /* Scaling many datasets */

2022-02-06T08:09:47Z

Scaling many datasets

← Older revision		Revision as of 10:09, 6 February 2022
Line 112:		Line 112:
	and check the column "ACCEPTED REFLECTIONS". Then remove the dataset(s) with fewest accepted reflections, and re-run the program. Repeat if necessary.		and check the column "ACCEPTED REFLECTIONS". Then remove the dataset(s) with fewest accepted reflections, and re-run the program. Repeat if necessary.

	~~The latest~~ XSCALE ~~(March 1, 2015)~~ makes it explicit which dataset(s) it cannot scale; it prints out e.g. "no common reflections with data set 197"~~. If you get this message for many datasets, I suggest to have a line~~		XSCALE makes it explicit which dataset(s) it cannot scale; it prints out e.g. "no common reflections with data set 197".
	~~MINIMUM_I/SIGMA=2 ! reduce to 1, or 0.5, or 0.25, or 0.125, or ... to lower the cutoff~~
	~~after each INPUT_FILE= line, to increase the number of reflections available for scaling. However, MINIMUM_I/SIGMA= should not be decreased needlessly below its default of 3~~.

	~~Old versions of~~ XSCALE may also finish with the error message !!! ERROR !!! INACCURATE SCALING FACTORS. This usually indicates that one or more datasets are linearly depending on others (this happens if the ''same'' data are included more than once as INPUT_FILE), or are pure noise~~. The latest version of XSCALE (March 1, 2015) copes much better with this situation; I have not seen this error message any more~~.		XSCALE may also finish with the error message !!! ERROR !!! INACCURATE SCALING FACTORS. This usually indicates that one or more datasets are linearly depending on others (this happens if the ''same'' data are included more than once as INPUT_FILE), or are pure noise.

Kay: fix links

2019-10-22T13:27:33Z

fix links

← Older revision		Revision as of 15:27, 22 October 2019
Line 5:		Line 5:
	One advantage of using XSCALE for a single dataset is that the user can specify the number and limits of the resolution shells. Another is that zero-dose extrapolation can be done.		One advantage of using XSCALE for a single dataset is that the user can specify the number and limits of the resolution shells. Another is that zero-dose extrapolation can be done.

	At the XDS website, there is a short and a long commented example of [http://~~www~~.mpimf-heidelberg.mpg.de~~/~kabsch/xds~~/html_doc/INPUT_templates/XSCALE.INP XSCALE.INP]		At the XDS website, there is a short and a long commented example of [http://xds.mpimf-heidelberg.mpg.de/html_doc/INPUT_templates/XSCALE.INP XSCALE.INP]

	----		----

Kay: fix link

2019-05-23T12:28:07Z

fix link

← Older revision		Revision as of 14:28, 23 May 2019
Line 1:		Line 1:
	== Simple and advanced usage ==		== Simple and advanced usage ==

	[http://~~www~~.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xscale_parameters.html XSCALE] is the stand-alone scaling program of the XDS suite. It scales reflection files (typically called XDS_ASCII.HKL) produced by XDS. Since the CORRECT step of XDS ''already scales'' an individual dataset, XSCALE is only ''needed'' if several datasets should be scaled relative to another. However, it does not deterioriate (over-fit) a dataset if it is "scaled again" in XSCALE, since the supporting points of the scalefactors are at the same positions in detector and batch space.		[http://xds.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xscale_parameters.html XSCALE] is the stand-alone scaling program of the XDS suite. It scales reflection files (typically called XDS_ASCII.HKL) produced by XDS. Since the CORRECT step of XDS ''already scales'' an individual dataset, XSCALE is only ''needed'' if several datasets should be scaled relative to another. However, it does not deterioriate (over-fit) a dataset if it is "scaled again" in XSCALE, since the supporting points of the scalefactors are at the same positions in detector and batch space.

	One advantage of using XSCALE for a single dataset is that the user can specify the number and limits of the resolution shells. Another is that zero-dose extrapolation can be done.		One advantage of using XSCALE for a single dataset is that the user can specify the number and limits of the resolution shells. Another is that zero-dose extrapolation can be done.

Kay: /* Simple and advanced usage */

2016-07-02T07:28:44Z

Simple and advanced usage

← Older revision		Revision as of 09:28, 2 July 2016
Line 20:		Line 20:
	FRIEDEL'S_LAW=FALSE		FRIEDEL'S_LAW=FALSE
	STRICT_ABSORPTION_CORRECTION=TRUE ! see XDSwiki:Tips_and_Tricks		STRICT_ABSORPTION_CORRECTION=TRUE ! see XDSwiki:Tips_and_Tricks
	INPUT_FILE= ../fae-rh/xds_2/XDS_ASCII.HKL		! the star in front of the file name indicates that it is the reference wrt falloff
			INPUT_FILE= *../fae-rh/xds_2/XDS_ASCII.HKL
	FRIEDEL'S_LAW=FALSE		FRIEDEL'S_LAW=FALSE
	STRICT_ABSORPTION_CORRECTION=TRUE		STRICT_ABSORPTION_CORRECTION=TRUE

Kay: /* keywords unique to XSCALE */

2015-09-07T13:35:43Z

keywords unique to XSCALE

← Older revision		Revision as of 15:35, 7 September 2015
Line 56:		Line 56:
	* DOSE_RATE= ! (optional for radiation damage correction f.i.r.)		* DOSE_RATE= ! (optional for radiation damage correction f.i.r.)
	* 0-DOSE_SIGNIFICANCE_LEVEL= ! (optional for radiation damage correction f.i.r.)		* 0-DOSE_SIGNIFICANCE_LEVEL= ! (optional for radiation damage correction f.i.r.)
			* SAVE_CORRECTION_IMAGES= ! Default is TRUE. If FALSE, don't write DECAY.cbf MODPIX.cbf ABSORP*.cbf

	== Radiation damage correction ==		== Radiation damage correction ==

Kay: /* Possible problems in scaling many datasets */

2015-03-04T11:31:45Z

Possible problems in scaling many datasets

← Older revision		Revision as of 13:31, 4 March 2015
Line 102:		Line 102:


	== ~~Possible problems in scaling~~ many datasets ==		== Scaling many datasets ==
	XSCALE may finish with an error message !!! ERROR !!! INSUFFICIENT NUMBER OF COMMON STRONG REFLECTIONS . This usually indicates that one or more datasets have too few reflections. Please inspect the table		When scaling e.g. hundreds of partial datasets, XSCALE may finish with an error message !!! ERROR !!! INSUFFICIENT NUMBER OF COMMON STRONG REFLECTIONS . This usually indicates that one or more datasets have too few reflections. Please inspect the table
	<nowiki>		<nowiki>
	DATA MEAN REFLECTIONS INPUT FILE NAME		DATA MEAN REFLECTIONS INPUT FILE NAME

Kay: /* Possible problems in scaling many datasets */

2015-03-04T11:30:31Z

Possible problems in scaling many datasets

← Older revision		Revision as of 13:30, 4 March 2015
Line 102:		Line 102:


	=== Possible problems in scaling many datasets ===		== Possible problems in scaling many datasets ==
	XSCALE may finish with an error message !!! ERROR !!! INSUFFICIENT NUMBER OF COMMON STRONG REFLECTIONS . This usually indicates that one or more datasets have too few reflections. Please inspect the table		XSCALE may finish with an error message !!! ERROR !!! INSUFFICIENT NUMBER OF COMMON STRONG REFLECTIONS . This usually indicates that one or more datasets have too few reflections. Please inspect the table
	<nowiki>		<nowiki>

Kay: /* Scaling many datasets */

2015-03-04T11:29:35Z

Scaling many datasets

← Older revision		Revision as of 13:29, 4 March 2015
Line 100:		Line 100:
	Another example: by defining STARTING_DOSE=-78.* one would tell the program to calculate, by interpolation, those intensity values that correspond to those that would be obtained near frame 78.		Another example: by defining STARTING_DOSE=-78.* one would tell the program to calculate, by interpolation, those intensity values that correspond to those that would be obtained near frame 78.

	~~== Scaling many datasets ==~~

	The program has no internal limit for the number of datasets. However, many items are calculated for ''each pair of datasets''. This results in some component of the CPU time being quadratic in the number of datasets. Nevertheless, it is perfectly possible to scale >1000 datasets.

	~~If this is too slow, one could use an ''incremental'' way:~~

	~~# Scale the first (say) 100 datasets together, including any low-resolution datasets~~
	~~# For the next step, use the OUTPUT_FILE from the previous step as the first INPUT_FILE (with WFAC1=2), and the next (say) 99 datasets as further INPUT_FILEs.~~
	~~# Go to step 2. as long as you have more datasets~~

	This should reduce wallclock requirements, and simply keeps adding datasets to an ever-growing merged dataset. The nice thing is that you can monitor how the completeness keeps growing with each step, and once good completeness is obtained, CC1/2 should start growing.

	~~Example showing the principle:~~
	~~<nowiki>~~
	~~# first step~~
	~~cat <<EOF>XSCALE.INP~~
	~~OUTPUT_FILE=1to100.ahkl~~
	~~INPUT_FILE=../1/XDS_ASCII.HKL~~
	~~INPUT_FILE=../2/XDS_ASCII.HKL~~
	~~! insert lines for INPUT_FILEs 3..100~~
	~~EOF~~
	~~xscale_par~~
	~~mv XSCALE.LP XSCALE_1to100.LP~~

	~~# second step~~
	~~cat <<EOF>XSCALE.INP~~
	~~OUTPUT_FILE=1to200.ahkl~~
	~~INPUT_FILE=1to100.ahkl~~
	~~WFAC1=2 ! avoid rejecting more outliers~~
	~~INPUT_FILE=../101/XDS_ASCII.HKL~~
	~~INPUT_FILE=../102/XDS_ASCII.HKL~~
	~~! insert lines for INPUT_FILEs 103..200~~
	~~EOF~~
	~~xscale_par~~
	~~mv XSCALE.LP XSCALE_1to200.LP~~

	~~# third step~~
	~~cat <<EOF>XSCALE.INP~~
	~~OUTPUT_FILE=1to300.ahkl~~
	~~INPUT_FILE=1to200.ahkl~~
	~~WFAC1=2 ! avoid rejecting more outliers~~
	~~INPUT_FILE=../201/XDS_ASCII.HKL~~
	~~INPUT_FILE=../202/XDS_ASCII.HKL~~
	~~! insert lines for INPUT_FILEs 203..300~~
	~~EOF~~
	~~xscale_par~~
	~~mv XSCALE.LP XSCALE_1to300.LP~~
	~~...~~
	~~</nowiki>~~

	=== Possible problems in scaling many datasets ===		=== Possible problems in scaling many datasets ===

Kay: /* Possible problems in scaling many datasets */

2015-03-03T09:35:40Z

Possible problems in scaling many datasets

← Older revision		Revision as of 11:35, 3 March 2015
Line 158:		Line 158:
	and check the column "ACCEPTED REFLECTIONS". Then remove the dataset(s) with fewest accepted reflections, and re-run the program. Repeat if necessary.		and check the column "ACCEPTED REFLECTIONS". Then remove the dataset(s) with fewest accepted reflections, and re-run the program. Repeat if necessary.

	XSCALE may also finish with the error message !!! ERROR !!! INACCURATE SCALING FACTORS. This usually indicates that one or more datasets are linearly depending on others (this happens if the ''same'' data are included more than once as INPUT_FILE), or are pure noise. ~~I have an experimental~~ version of XSCALE ~~that prints out the numbers of these bad datasets~~.		The latest XSCALE (March 1, 2015) makes it explicit which dataset(s) it cannot scale; it prints out e.g. "no common reflections with data set 197". If you get this message for many datasets, I suggest to have a line
			MINIMUM_I/SIGMA=2 ! reduce to 1, or 0.5, or 0.25, or 0.125, or ... to lower the cutoff
			after each INPUT_FILE= line, to increase the number of reflections available for scaling. However, MINIMUM_I/SIGMA= should not be decreased needlessly below its default of 3.

			Old versions of XSCALE may also finish with the error message !!! ERROR !!! INACCURATE SCALING FACTORS. This usually indicates that one or more datasets are linearly depending on others (this happens if the ''same'' data are included more than once as INPUT_FILE), or are pure noise. The latest version of XSCALE (March 1, 2015) copes much better with this situation; I have not seen this error message any more.