Wishlist: Difference between revisions

From XDSwiki
Jump to navigation Jump to search
mNo edit summary
 
(57 intermediate revisions by 5 users not shown)
Line 1: Line 1:
This is a collection of things users feel [[XDS]] should handle differently from how the program currently does.  
This is a collection of things users feel [[XDS]] should handle differently from how the program currently does.  


On this page (only), it would seem useful if a user who edits this puts his "signature with timestamp" (second button from the right in the menu bar above) after the sentence s/he inserts.
On this page (only), it would seem useful if a user who edits this puts his "signature with timestamp" (second button from the right in the edit menu bar) after the sentence s/he inserts.




== Bugs ==


== Important ==
* FRAME.cbf does not have distance and wavelength information, as well as direct beam position - would be good to have those for adxv usage.
 
* When scaling multiple datasets with multiple OUTPUT entries within the same XSCALE.INP file, the reported statistics within XSCALE.LP does respect the corresponding INCLUDE_RESOLUTION_RANGE= keyword entries. --[[User:Mnfodje|Mnfodje]] June 24, 2011 <br> ''Comment: XSCALE finds the highest high-resolution cutoff of all output datasets, and prints out statistics for all datasets up to this resolution. Depending on the user-input values of INCLUDE_RESOLUTION_RANGE= , this may lead to tables that e.g. have 0 reflections in the high-resolution bin(s). This does not affect the data that are output; the only number (I think) that is wrong in those tables is the value for the total completeness. The workaround to get that value obviously is running XSCALE separately for each resolution-truncated dataset. --[[User:Kay|Kay]] 18:21, 25 June 2011 (CEST)''
* none so far
* In previous versions of XDS, you could integrate a partial dataset which contains only frames 1-2,46-47,90-91 by specifying a DATA_RANGE=1 91. Although this complained about missing images, it still worked and we used this for screening by collecting only those frames and naming the images accordingly.  Starting with the Dec 2011 version, this no longer works and the integration step stops with the error message ''!!!ERROR!!! CANNOT OPEN OR READ FILE bin2_XX.tmp''. Checking the directory, I can see that the file bin2_XX.tmp exists. After some testing, I can confirm that this happens if all the images within any single integration batch are missing.  If the number of images missing is equal to or greater than DELPHI/OSCILLATION_RANGE, the error appears.  The number of the bin file seems to correspond to the batch number  and it does not matter where in the dataset the images are missing from. As a result it is no longer possible to process partial datasets for screening, or to exclude a section of bad frames  in the dataset which is larger than DELPHI/OSCILLATION_RANGE. --[[User:Mnfodje|Mnfodje]] 17:09, 2 February 2012 (CET) ''Comment: Thanks for reporting the bug. Versions of XDS downloaded after Feb 3, 2012 have a fix for this, but they still identify themselves as 'VERSION January 19, 2012' --[[User:Kay|Kay]] 10:18, 8 February 2012 (CET)''
 
 


== Would be nice to have ==
== Would be nice to have ==
* Add the REIDX keyword to XDSCONV. This would enable scaling in a centered spacegroup, and using P1 for the final dataset.
* Add a SKIP_RANGE keyword similar to SPOT_RANGE which enables skipping multiple ranges of frames from the specified DATA_RANGE. XDS should just ignore those frames as if they were supposed to exist but do not. --[[User:Mnfodje|Mnfodje]] 15:07, 1 November 2010 (UTC). ''Comment: there is EXCLUDE_DATA_RANGE for this purpose. This parameter is used by INIT, COLSPOT, INTEGRATE, but not by CORRECT which means you must re-INTEGRATE. If you want to exclude a data range from an existing XDS_ASCII.HKL, run [[exclude_data_range_from_XDS_ASCII.HKL]].''


* A breakdown of R<sub>meas</sub> by frame number, as currently only available from [[XDSSTAT]]. --[[User:Kay|Kay]] 14:00, 12 November 2007 (CET)
* if the first frame of DATA_RANGE cannot be found in CORRECT (because the frames have already been deleted or so), xds should pick up QX, QY, NX, NY from XDS.INP. ''Fixed in the version of Dec 06, 2010''


* Templates of XDS.INP currently have
* A breakdown of R<sub>meas</sub> by frame number, as currently only available from [[XDSSTAT]].  
!EXCLUDE_RESOLUTION_RANGE= 3.93 3.87 !ice-ring at 3.897 Angstrom
!EXCLUDE_RESOLUTION_RANGE= 3.70 3.64 !ice-ring at 3.669 Angstrom
!EXCLUDE_RESOLUTION_RANGE= 3.47 3.41 !ice-ring at 3.441 Angstrom
!EXCLUDE_RESOLUTION_RANGE= 2.70 2.64 !ice-ring at 2.671 Angstrom
!EXCLUDE_RESOLUTION_RANGE= 2.28 2.22 !ice-ring at 2.249 Angstrom


This list may/could be completed by
* INCLUDE_RESOLUTION_RANGE= should be respected by IDXREF because this could be used to prevent ice rings from disturbing the indexing. Drawback: one would have to change INCLUDE_RESOLUTION_RANGE= after running IDXREF.<br />Probably better: EXCLUDE_RESOLUTION_RANGE= should be respected by IDXREF. Drawback: geometrical parameters (in particular the direct beam position) are unrefined when resolution is calculated in IDXREF. ''Solution: Both INCLUDE_RESOLUTION_RANGE and EXCLUDE_RESOLUTION_RANGE are evaluated by IDXREF since version March-30, 2013.''
 
* a keyword REJECT_ALIEN= with a default of 20. This would be the cutoff for automatic rejection of "aliens". See [[Optimization#Wilson%20outliers%20(aliens)]]--[[User:Kay|Kay]] 10:29, 27 November 2007 (CET). '' This was implemented in January 2010.''
!EXCLUDE_RESOLUTION_RANGE= 2.102 2.042 !ice-ring at 2.072 Angstrom - should be strong
* Automated Direct Beam Position Recognition from Direct Beam Shots: The [[Features that are not officially documented|undocumented]] DIRB option should be announced and extended. I would suggest the following: a new input keyword "NAME_OF_DIRB_FRAME=" instead of interactive input is used to specifiy the direct beam shot frame. As a default "NAME_OF_DIRB_FRAME=" should use frame 0 of the current Name_Template_of_data_frames selection. Avoiding interactive input would facilitate automated script based processing and avoid the use of erroneous direct beam position from headers or manual input.--[[User:Tmaier|Tmaier]] 13:24, 13 December 2007 (CET)
!EXCLUDE_RESOLUTION_RANGE= 1.978 1.918 !ice-ring at 1.948 Angstrom - may be weak
* Option to write out FRAME.PCK for visual control of specific images: this is available from the "tools" tab of XDSGUI.
!EXCLUDE_RESOLUTION_RANGE= 1.948 1.888 !ice-ring at 1.918 Angstrom - should be strong
* Option to produce harvest files from CORRECT and XSCALE for deposition of structures.--[[User:Graeme|Graeme]] 18 February 2008. Clarifying this in response to question: Mosflm and Scala both produce files which contain information which can be included at deposition - this is produced as separate Harvest files, based on the statistics already generated by the programs. These harvest files are in mmCIF format, and I will upload a couple if I can figure out how. This does require a certain amount of administrative information (a "name" for the project and dataset) but could be easily rectified by adding a keyword: HARVEST_PROJECT= HARVEST_DATASET= which could then produce PROJECT_DATASET.xds or .xscale depending on the program. [[User:Graeme|Graeme]] 30 May 2008.
!EXCLUDE_RESOLUTION_RANGE= 1.913 1.853 !ice-ring at 1.883 Angstrom - may be weak
* In CORRECT.LP, there is a section "SUMMARY OF DATA SET STATISTICS FOR VARIOUS SUBSETS OF INCLUDED DATA IMAGES", which outputs data set statistics for a subset of the image with an image increment. They are very useful to determine where I should stop for processing.  For example, if I found that the Rmeas and I/sigma of image 1-180 are worse than those of image 1-160.  I then will only process image 1-160, because the last 20 images doesn't add more signal to my data. However, the image increment is automatically determined by CORRECT, which is a rather large number. It would be nice to have an option to set the image increment by users to help determine when the crystal dies. --[[User:Jzhu|Jzhu]] 04:53, 27 June 2011 (CEST) ''Comment: CORRECT prints statistics for 9 pieces of the DATA_RANGE. In the absence of an option to modify that number, the workaround is probably to run XDSSTAT to get Rmeas and I/sigma of the individual frames.'' --[[User:Kay|Kay]] 11:08, 2 April 2012 (CEST)
!EXCLUDE_RESOLUTION_RANGE= 1.751 1.691 !ice-ring at 1.721 Angstrom - may be weak
 
Please note that I did not optimize the widths (in particular the last one should probably be something like 1.741 1.701 or 1.731 1.711), and that the ranges of the 2nd, 3rd and 4th ring overlap (and thus could be combined and narrowed to something like 1.958 1.873).
--[[User:Kay|Kay]] 14:30, 12 November 2007 (CET)
 
* IDXREF should respect INCLUDE_RESOLUTION_RANGE= because this could be used to prevent ice rings from disturbing the indexing. An ugly workaround currently is to set TRUSTED_REGION=0 <some small value> for INIT where <some small value> is calculated such that ice rings are excluded - but that means that after successful indexing, INIT needs to be re-run.<br /> One might also suggest that EXCLUDE_RESOLUTION_RANGE= should be respected but this appears less useful, because geometrical parameters are not refined, and they need to be accurate because the ice rings are narrow. Found by Clemens Vonrhein --[[User:Kay|Kay]] 12:16, 16 November 2007 (CET)


== Cosmetical ==
== Cosmetical ==


* CORRECT should print out MAXIMUM_NUMBER_OF_PROCESSORS= , and the actual number of processors used. --[[User:Kay|Kay]] 13:55, 12 November 2007 (CET)
* CORRECT should print out MAXIMUM_NUMBER_OF_PROCESSORS= , and the actual number of processors used. --[[User:Kay|Kay]] 13:55, 12 November 2007 (CET)
* CORRECT should print out NBATCH= ; one currently has to "grep NXBIN CORRECT.LP|head -1" to find out.
* INTEGRATE should print out NUMBER_OF_PROFILE_GRID_POINTS_ALONG_ALPHA/BETA= and NUMBER_OF_PROFILE_GRID_POINTS_ALONG_GAMMA=, as well as  BEAM_DIVERGENCE= BEAM_DIVERGENCE_E.S.D.=    REFLECTING_RANGE= REFLECTING_RANGE_E.S.D.= if these are given in XDS.INP. The latter enable to reproduce a given run just from the INTEGRATE.LP file; the former is less important since one can simply count the number of points along alpha/beta and gamma, respectively, in the printed profiles.


* CORRECT should print out MINIMUM_I/SIGMA= . --[[User:Kay|Kay]] 14:24, 13 November 2007 (CET)
== Notes ==
 
See [[old wishes]] to find out which things were implemented.
* BACKGROUND_RANGE= should have a better default (e.g., the first five frames of the DATA_RANGE). The default currently appears to be 0 0. --[[User:Kay|Kay]] 14:45, 12 November 2007 (CET)

Latest revision as of 14:52, 3 July 2019

This is a collection of things users feel XDS should handle differently from how the program currently does.

On this page (only), it would seem useful if a user who edits this puts his "signature with timestamp" (second button from the right in the edit menu bar) after the sentence s/he inserts.


Bugs

  • FRAME.cbf does not have distance and wavelength information, as well as direct beam position - would be good to have those for adxv usage.
  • When scaling multiple datasets with multiple OUTPUT entries within the same XSCALE.INP file, the reported statistics within XSCALE.LP does respect the corresponding INCLUDE_RESOLUTION_RANGE= keyword entries. --Mnfodje June 24, 2011
    Comment: XSCALE finds the highest high-resolution cutoff of all output datasets, and prints out statistics for all datasets up to this resolution. Depending on the user-input values of INCLUDE_RESOLUTION_RANGE= , this may lead to tables that e.g. have 0 reflections in the high-resolution bin(s). This does not affect the data that are output; the only number (I think) that is wrong in those tables is the value for the total completeness. The workaround to get that value obviously is running XSCALE separately for each resolution-truncated dataset. --Kay 18:21, 25 June 2011 (CEST)
  • In previous versions of XDS, you could integrate a partial dataset which contains only frames 1-2,46-47,90-91 by specifying a DATA_RANGE=1 91. Although this complained about missing images, it still worked and we used this for screening by collecting only those frames and naming the images accordingly. Starting with the Dec 2011 version, this no longer works and the integration step stops with the error message !!!ERROR!!! CANNOT OPEN OR READ FILE bin2_XX.tmp. Checking the directory, I can see that the file bin2_XX.tmp exists. After some testing, I can confirm that this happens if all the images within any single integration batch are missing. If the number of images missing is equal to or greater than DELPHI/OSCILLATION_RANGE, the error appears. The number of the bin file seems to correspond to the batch number and it does not matter where in the dataset the images are missing from. As a result it is no longer possible to process partial datasets for screening, or to exclude a section of bad frames in the dataset which is larger than DELPHI/OSCILLATION_RANGE. --Mnfodje 17:09, 2 February 2012 (CET) Comment: Thanks for reporting the bug. Versions of XDS downloaded after Feb 3, 2012 have a fix for this, but they still identify themselves as 'VERSION January 19, 2012' --Kay 10:18, 8 February 2012 (CET)

Would be nice to have

  • Add the REIDX keyword to XDSCONV. This would enable scaling in a centered spacegroup, and using P1 for the final dataset.
  • Add a SKIP_RANGE keyword similar to SPOT_RANGE which enables skipping multiple ranges of frames from the specified DATA_RANGE. XDS should just ignore those frames as if they were supposed to exist but do not. --Mnfodje 15:07, 1 November 2010 (UTC). Comment: there is EXCLUDE_DATA_RANGE for this purpose. This parameter is used by INIT, COLSPOT, INTEGRATE, but not by CORRECT which means you must re-INTEGRATE. If you want to exclude a data range from an existing XDS_ASCII.HKL, run exclude_data_range_from_XDS_ASCII.HKL.
  • if the first frame of DATA_RANGE cannot be found in CORRECT (because the frames have already been deleted or so), xds should pick up QX, QY, NX, NY from XDS.INP. Fixed in the version of Dec 06, 2010
  • A breakdown of Rmeas by frame number, as currently only available from XDSSTAT.
  • INCLUDE_RESOLUTION_RANGE= should be respected by IDXREF because this could be used to prevent ice rings from disturbing the indexing. Drawback: one would have to change INCLUDE_RESOLUTION_RANGE= after running IDXREF.
    Probably better: EXCLUDE_RESOLUTION_RANGE= should be respected by IDXREF. Drawback: geometrical parameters (in particular the direct beam position) are unrefined when resolution is calculated in IDXREF. Solution: Both INCLUDE_RESOLUTION_RANGE and EXCLUDE_RESOLUTION_RANGE are evaluated by IDXREF since version March-30, 2013.
  • a keyword REJECT_ALIEN= with a default of 20. This would be the cutoff for automatic rejection of "aliens". See Optimization#Wilson outliers (aliens)--Kay 10:29, 27 November 2007 (CET). This was implemented in January 2010.
  • Automated Direct Beam Position Recognition from Direct Beam Shots: The undocumented DIRB option should be announced and extended. I would suggest the following: a new input keyword "NAME_OF_DIRB_FRAME=" instead of interactive input is used to specifiy the direct beam shot frame. As a default "NAME_OF_DIRB_FRAME=" should use frame 0 of the current Name_Template_of_data_frames selection. Avoiding interactive input would facilitate automated script based processing and avoid the use of erroneous direct beam position from headers or manual input.--Tmaier 13:24, 13 December 2007 (CET)
  • Option to write out FRAME.PCK for visual control of specific images: this is available from the "tools" tab of XDSGUI.
  • Option to produce harvest files from CORRECT and XSCALE for deposition of structures.--Graeme 18 February 2008. Clarifying this in response to question: Mosflm and Scala both produce files which contain information which can be included at deposition - this is produced as separate Harvest files, based on the statistics already generated by the programs. These harvest files are in mmCIF format, and I will upload a couple if I can figure out how. This does require a certain amount of administrative information (a "name" for the project and dataset) but could be easily rectified by adding a keyword: HARVEST_PROJECT= HARVEST_DATASET= which could then produce PROJECT_DATASET.xds or .xscale depending on the program. Graeme 30 May 2008.
  • In CORRECT.LP, there is a section "SUMMARY OF DATA SET STATISTICS FOR VARIOUS SUBSETS OF INCLUDED DATA IMAGES", which outputs data set statistics for a subset of the image with an image increment. They are very useful to determine where I should stop for processing. For example, if I found that the Rmeas and I/sigma of image 1-180 are worse than those of image 1-160. I then will only process image 1-160, because the last 20 images doesn't add more signal to my data. However, the image increment is automatically determined by CORRECT, which is a rather large number. It would be nice to have an option to set the image increment by users to help determine when the crystal dies. --Jzhu 04:53, 27 June 2011 (CEST) Comment: CORRECT prints statistics for 9 pieces of the DATA_RANGE. In the absence of an option to modify that number, the workaround is probably to run XDSSTAT to get Rmeas and I/sigma of the individual frames. --Kay 11:08, 2 April 2012 (CEST)

Cosmetical

  • CORRECT should print out MAXIMUM_NUMBER_OF_PROCESSORS= , and the actual number of processors used. --Kay 13:55, 12 November 2007 (CET)
  • CORRECT should print out NBATCH= ; one currently has to "grep NXBIN CORRECT.LP|head -1" to find out.
  • INTEGRATE should print out NUMBER_OF_PROFILE_GRID_POINTS_ALONG_ALPHA/BETA= and NUMBER_OF_PROFILE_GRID_POINTS_ALONG_GAMMA=, as well as BEAM_DIVERGENCE= BEAM_DIVERGENCE_E.S.D.= REFLECTING_RANGE= REFLECTING_RANGE_E.S.D.= if these are given in XDS.INP. The latter enable to reproduce a given run just from the INTEGRATE.LP file; the former is less important since one can simply count the number of points along alpha/beta and gamma, respectively, in the printed profiles.

Notes

See old wishes to find out which things were implemented.