2,684
edits
XDSGUI: Difference between revisions
→Known bugs, problems and workarounds
| (39 intermediate revisions by the same user not shown) | |||
| Line 36: | Line 36: | ||
Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are shown in the XDS.INP tab. Step-by-step: | Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are shown in the XDS.INP tab. Step-by-step: | ||
* "Load" a raw frame or a CBF file (e.g. FRAME.cbf) to have it displayed. The pane can be "dragged" with the left mouse button; the mouse wheel zooms. The parameters in the XDS.INP tab are taken for resolution calculations (i.e. the frame header is not being interpreted in any way). | * "Load" a raw frame or a CBF file (e.g. FRAME.cbf) to have it displayed. The pane can be "dragged" with the left mouse button; the mouse wheel zooms. The parameters in the XDS.INP tab are taken for resolution calculations (i.e. the frame header is not being interpreted in any way). | ||
* if XDS.INP does not yet exist, click "generate XDS.INP" (this ''will'' read the header). Check the XDS.INP tab afterwards but then go back to the Frame tab. Note that the current generate_XDS.INP works well for Pilatus, ADSC, Mar and some Rigaku detectors; for other kinds of detectors the values marked XXX in XDS.INP have to be filled in manually. | * if XDS.INP does not yet exist, click "generate XDS.INP" (this ''will'' read the header). Check the XDS.INP tab afterwards but then go back to the Frame tab. Note that the current generate_XDS.INP works well for Pilatus, ADSC, Mar and some Rigaku detectors; for other kinds of detectors the values marked XXX in XDS.INP have to be filled in manually. The [[Eiger]] detector is not yet implemented in the script, but this will eventually come. | ||
* left-click on "Untrusted areas" -> a pulldown menu appears | * left-click on "Untrusted areas" -> a pulldown menu appears | ||
* left-click on (say) "Untrusted Rectangle (2 clicks)" | * left-click on (say) "Untrusted Rectangle (2 clicks)" | ||
| Line 52: | Line 52: | ||
=== XYCORR === | === XYCORR === | ||
this just shows the output of the XYCORR step - normally not important to look at. | |||
=== INIT === | === INIT === | ||
this just shows the output of the INIT step - you can find out e.g. what the average background counts are. Not crucial to look at, though. | |||
=== COLSPOT === | === COLSPOT === | ||
=== IDXREF === | this just shows the output of the COLSPOT step - normally not crucial to look at. But you could find out about the number of strong reflections on each frame of the SPOT_RANGE. This may be interesting to see radiation damage, or change of crystal volume in the beam. | ||
=== [[IDXREF]] === | |||
detailed explanation of output: see [[IDXREF.LP]]. | |||
The right side of the tab shows the indexed (black) and not-indexed (red) reflections of [[SPOT.XDS]]. | |||
=== DEFPIX === | === DEFPIX === | ||
this just shows the output of the DEFPIX step - normally not important to look at. | |||
=== INTEGRATE === | === INTEGRATE === | ||
| Line 70: | Line 78: | ||
The border between the left and right side of the window may be adjusted with the mouse.<br> | The border between the left and right side of the window may be adjusted with the mouse.<br> | ||
=== | === tools === | ||
offers possibilities for running short scripts. | offers possibilities for running short scripts. | ||
| Line 76: | Line 84: | ||
Several such scripts are pre-defined; the user may create her own scripts. If scripts are modified, they are saved to ~/.xds-gui . The names of the buttons (e.g. "User defined command 1") can be changed by editing ~/.xds-gui . | Several such scripts are pre-defined; the user may create her own scripts. If scripts are modified, they are saved to ~/.xds-gui . The names of the buttons (e.g. "User defined command 1") can be changed by editing ~/.xds-gui . | ||
* The first item of the left panel ("Show frame with predicted spots") generates the predicted pattern of reflections for a user-specified frame, overlaid on the frame, for display with [[XDS-viewer]]. The file FRAME.cbf (produced by INTEGRATE) is renamed to FRAME_$X.cbf (where X is the user-specified frame number) and remains in the temp subdirectory. It may of course be opened in the FRAME tab, but starting XDS-viewer automatically has the advantage that several frames with predictions may be inspected on the screen, at the same time. Please note: if the XDS directory resides in a FAT32 filesystem (which is often the case on a USB stick or disk), then "ln -s" should be replaced by "cp -p" since FAT32 does not support symlinks. Also note: for the script to work correctly, NAME_TEMPLATE_OF_DATA_FRAMES in XDS.INP has to specify an absolute, not a relative path. | * The first item of the left panel ("Show frame with predicted spots") generates the predicted pattern of reflections for a user-specified frame, overlaid on the frame, for display with [[XDS-viewer]]. The file FRAME.cbf (produced by INTEGRATE) is renamed to FRAME_$X.cbf (where X is the user-specified frame number) and remains in the temp subdirectory. It may of course be opened in the FRAME tab, but starting XDS-viewer automatically has the advantage that several frames with predictions may be inspected on the screen, at the same time. Please note: if the XDS directory resides in a FAT32 filesystem (which is often the case on a USB stick or disk), then "ln -s" (of the script line) should be replaced by "cp -p" since FAT32 does not support symlinks. Also note: for the script to work correctly, NAME_TEMPLATE_OF_DATA_FRAMES in XDS.INP has to specify an absolute, not a relative path. | ||
* The second item ("Optimizing data quality") offers commands that manipulate [[XDS.INP]] in several ways. Please note: the popup "XDS.INP has been changed externally" is emitted by the Qt system and cannot be switched off. It appears if one of the scripts changes XDS.INP while it is opened by XDSGUI (which is always the case) . Thus, one should simply press the "Reload" button. The user-definable | * The second item ("Optimizing data quality") offers commands that manipulate [[XDS.INP]] in several ways. Please note: the popup "XDS.INP has been changed externally" is emitted by the Qt system and cannot be switched off. It appears if one of the scripts changes XDS.INP while it is opened by XDSGUI (which is always the case) . Thus, one should simply press the "Reload" button. The user-definable command may be used e.g. for | ||
awk '/BEAM_DIVERGENCE=/{print "",$1,2*$2,$3,$4}' INTEGRATE.LP>x; grep -v BEAM_DIVERGENCE XDS.INP>>x; mv x XDS.INP | |||
This multiplies the estimate of BEAM_DIVERGENCE in INTEGRATE.LP by 2 and thus provides more pixels to the background estimation (for another INTEGRATE). This is useful for weakly exposed data sets which have mostly pixels with zero counts. Of course, any other factor (other than 2) may be tried. | |||
* The third item ("Saving and comparing good results") offers commands to save/restore the current data processing files to/from a "save" directory. Make sure to replace "xdiff" with "xxdiff" or "tkdiff", if one of the latter is available. | * The third item ("Saving and comparing good results") offers commands to save/restore the current data processing files to/from a "save" directory. Make sure to replace "xdiff" with "xxdiff" or "tkdiff", if one of the latter is available. | ||
| Line 86: | Line 95: | ||
phenix.xtriage XDS_ASCII.HKL | phenix.xtriage XDS_ASCII.HKL | ||
=== | === statistics === | ||
This tab allows to run [[XDSSTAT]] with a mouseclick. As soon as XDSSTAT.LP exists, the main plots derived from it are displayed. | This tab allows to run [[XDSSTAT]] and [[XDSCC12]] with a mouseclick. | ||
As soon as XDSSTAT.LP exists, the main plots derived from it are displayed (second and further panes). The "view" button allows to run [[XDS-viewer]] to visualize the control images that XDSSTAT generates, namely anom.pck, scales.pck, rf.pck, misfits.pck, corr.pck, rlps.pck, peaks.pck and nobs.pck . | |||
Likewise, as soon as the output file XDSCC12.LP exists, a plot derived from those lines starting with "a:" and "b:" will be generated. In black, this shows the overall contribution of a frame, or of several frames pooled, to the CC<sub>1/2</sub> value of those reflections that this frame (or these frames) contributes to. The rainbow-coloured curves divide the total resolution range into a user-selectable number (chosen with the -nbin option of XDSCC12) of resolution ranges; blue is low resolution and red is high resolution. | |||
Fine-sliced frames (0.1°) can be pooled into 1°-pseudoframes by using the -t option, to increase the number of reflections contributing to the plot and to make it less noisy. | |||
=== XDSCONV === | === XDSCONV === | ||
| Line 100: | Line 114: | ||
== Availability == | == Availability == | ||
The program is under development and probably has bugs. If it crashes, it should simply be restarted. A crash of the program ''does not interfere'' with the operation of [[XDS]]; likewise, closing the program window does not influence any XDS run started from XDSGUI. | The [http://sourceforge.net/projects/xdsgui/ program's source code] is released under the terms of the [https://gnu.org/licenses/old-licenses/gpl-2.0.txt GPLv2]; it uses the [http://www.qt.io/ Qt library] which was released under the terms of the [https://gnu.org/licenses/lgpl.txt LGPL]. The program is under development and probably has bugs. If it crashes, it should simply be restarted. A crash of the program ''does not interfere'' with the operation of [[XDS]]; likewise, closing the program window does not influence any XDS run started from XDSGUI. | ||
=== Dependencies === | === Dependencies === | ||
XDSGUI depends on [[generate_XDS.INP]], [[XDS-viewer]], and of course [[XDS]] (both the xds and xds_par binaries!), and [[ | XDSGUI depends on [[generate_XDS.INP]], [[XDS-viewer]], and of course [[XDS]] (both the xds and xds_par binaries!), [[XDSCONV]], and [[2cbf]]. The ''statistics'' tab requires [[XDSSTAT]] and [[XDSCC12]]. [[Eiger]] data processing requires <code>h5dump</code> and <code>H5ToXds</code> (or equivalent). One of the items in the <code>tools</code> tab calls a graphical file comparison program; if the default (xdiff) is not available, [http://furius.ca/xxdiff xxdiff] or [http://sourceforge.net/projects/tkdiff tkdiff] or [http://kdiff3.sourceforge.net/ kdiff3] or [http://meldmerge.org/ meld] may be used - please adjust the commandline below the button accordingly. See also [[Installation]]. | ||
Technically, the word "dependency" means that these scripts/programs should be in your $PATH - an 'alias' is often not good enough because the TOOLS commands use bash as a shell, and that shell does not inherit the aliases from e.g. (t)csh. | Technically, the word "dependency" means that these scripts/programs should be in your $PATH - an 'alias' is often not good enough because the ''TOOLS'' commands use <code>bash</code> as a shell, and that shell does not inherit the aliases from e.g. (t)csh. | ||
=== Installation === | === Installation === | ||
The current version of the program can be downloaded, by academic users, for Linux [ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.rhel6.64 64bit] (compiled on RHEL6), [ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.rhel6.32 32bit] (compiled on RHEL6 | The current version of the program can be downloaded, by academic users, for Linux [ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.rhel6.64 64bit] (compiled on RHEL6), [ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.rhel6.32 32bit] (compiled on RHEL6), or [ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.dmg Mac] (works on OSX 10.6 and up). | ||
Industrial users of XDS: pls contact me directly. | Industrial users of XDS: pls contact me directly. | ||
| Line 119: | Line 133: | ||
'''and to start the GUI from a terminal window, by typing xdsgui'''. In a similar way, XDS-viewer should be put into the $PATH under the name xds-viewer, e.g. using | '''and to start the GUI from a terminal window, by typing xdsgui'''. In a similar way, XDS-viewer should be put into the $PATH under the name xds-viewer, e.g. using | ||
sudo ln -s /Applications/XDS-Viewer.app/Contents/MacOS/xds-viewer-bin /usr/local/bin/xds-viewer | sudo ln -s /Applications/XDS-Viewer.app/Contents/MacOS/xds-viewer-bin /usr/local/bin/xds-viewer | ||
==== Libraries that the program depends on ==== | |||
In case of Fedora or Redhat Enterprise Linux distributions, you may have to | |||
yum install qt qt-x11 | |||
if you get the error message | |||
xdsgui: error while loading shared libraries: libQtGui.so.4: cannot open shared object file: No such file or directory | |||
or similar for libQtCore.so.4. | |||
'''Ubuntu 14.04''' has libmng2 instead of libmng. Therefore, on 32bit systems you should try | '''Ubuntu 14.04''' has libmng2 instead of libmng. Therefore, on 32bit systems you should try | ||
| Line 126: | Line 147: | ||
if you get the error message | if you get the error message | ||
xdsgui: error while loading shared libraries: libmng.so.1: cannot open shared object file: No such file or directory | xdsgui: error while loading shared libraries: libmng.so.1: cannot open shared object file: No such file or directory | ||
On the '''Mac''' (e.g. Yosemite) one needs [https://itunes.apple.com/us/app/xcode/id497799835?ls=1&mt=12 Xcode] installed, seemingly before XDSGUI is installed. Otherwise one may obtain a popup error “You can’t open the application because it is not supported on this type of Mac.” | |||
== Known bugs, problems and workarounds == | == Known bugs, problems and workarounds == | ||
# ''Question concerning TOOLS: "Show frame..." on Mac: it creates everything shown in the commandline but it seem to us that everything runs in the background. Therefore xds-viewer does not open anywhere as it is not seen. Also, it would be great if this image together with the predictions would be re-directed to the FRAME tab.'' <br> Answer: On Linux, the xds-viewer window is brought to the foreground automatically, whereas on the Mac this does not seem to happen. However, an icon for xds-viewer appears in the dock (on the right) and I can double-click it to see the xds-viewer window. I have no idea how to bring xds-viewer to the foreground automatically, and I need input from people who know Macs and tell me how to do it. The reason why we do not open automatically in the FRAME tab is that one can have several xds-viewer windows open at the same time, and compare the patterns. As a '''workaround''', you can manually open the resulting temp/FRAME_$X.cbf in the FRAME tab. | # ''Question concerning TOOLS: "Show frame..." on Mac: it creates everything shown in the commandline but it seem to us that everything runs in the background. Therefore xds-viewer does not open anywhere as it is not seen. Also, it would be great if this image together with the predictions would be re-directed to the FRAME tab.'' <br> Answer: On Linux, the xds-viewer window is brought to the foreground automatically, whereas on the Mac this does not seem to happen. However, an icon for xds-viewer appears in the dock (on the right) and I can double-click it to see the xds-viewer window. I have no idea how to bring xds-viewer to the foreground automatically, and I need input from people who know Macs and tell me how to do it. The reason why we do not open automatically in the FRAME tab is that one can have several xds-viewer windows open at the same time, and compare the patterns. As a '''workaround''', you can manually open the resulting temp/FRAME_$X.cbf in the FRAME tab. | ||
# ''Another question concerning TOOLS: "Further analyses" on Mac: again pointless runs nicely but there is no output for the user.'' <br> Answer: Could it be that you started xdsgui by double-clicking its icon in the Finder? In that case, there is no output visible, because there is no console. Unfortunately, for all the Tools, the output is in the console window where xdsgui was started | # ''Another question concerning TOOLS: "Further analyses" on Mac: again pointless runs nicely but there is no output for the user.'' <br> Answer: Could it be that you started xdsgui by double-clicking its icon in the Finder? In that case, there is no output visible, because there is no console. Unfortunately, for all the Tools, the output is in the console window where xdsgui was started. Thus, you should really start xdsgui in a console window. | ||
# both generate_XDS.INP and [[XDS]] do not seem to like spaces ("blanks") in file/directory names. While frame names rarely have a space in them, directories might, e.g. if an external disk has the label "John's USB disk" then this would be mounted on Linux as /media/John's USB disk/ and that would result in a error message from generate_XDS.INP and/or XDS. The '''workaround''' is to use a symlink on a different hard disk | # both generate_XDS.INP and [[XDS]] do not seem to like spaces ("blanks") in file/directory names. While frame names rarely have a space in them, directories might, e.g. if an external disk has the label "John's USB disk" then this would be mounted on Linux as /media/John's USB disk/ and that would result in a error message from generate_XDS.INP and/or XDS. The '''workaround''' is to use a symlink on a different hard disk. | ||
# ''Problem description: On the Mac, after loading frames, by clicking “generate XDS.INP”, the program gives some strange symbol “Ô in XDS.INP. And the more you click “save” button, the more “Ô appeared. This looks like <br>SPACE_GROUP_NUMBER=0 à ! 0 if unknown <br>UNIT_CELL_CONSTANTS= 70 80 90 90 90 90 à .''<br> The problem is due to the “Rich text” format in TextEdit when saving "generate_XDS.INP". It is solved by re-downloading the script, and changing format to Plain - everything is working now. | # ''Problem description: On the Mac, after loading frames, by clicking “generate XDS.INP”, the program gives some strange symbol “Ô in XDS.INP. And the more you click “save” button, the more “Ô appeared. This looks like <br>SPACE_GROUP_NUMBER=0 à ! 0 if unknown <br>UNIT_CELL_CONSTANTS= 70 80 90 90 90 90 à .''<br> The problem is due to the “Rich text” format in TextEdit when saving "generate_XDS.INP". It is solved by re-downloading the script, and changing format to Plain - everything is working now. | ||
# on a Mac, you may find a stream of the following error messages in the console window: <code>Jan 31 14:35:17 xdsgui[20939] <Error>: Error: this application, or a library it uses, has passed an invalid numeric value (NaN, or not-a-number) to CoreGraphics API and this value is being ignored. Please fix this problem. Jan 31 14:35:17 xdsgui[20939] <Error>: If you want to see the backtrace, please set CG_NUMERICS_SHOW_BACKTRACE environmental variable.</code> This sounds terrible, but is actually harmless. Since it is a message from Qt (which XDSGUI uses for graphics), ''it does not compromise the XDS data processing in any way''. Just suppress the printout of these messages, by <code>xdsgui 2>&1 | egrep -v 'CoreGraphics|CG_NUMERICS'</code> | |||
If you find a bug, please send email to Kay dot Diederichs at uni-konstanz dot de , | If you find a bug, please send email to Kay dot Diederichs at uni-konstanz dot de , with enough information/data to reproduce the bug. | ||
== Limitations == | == Limitations == | ||
* The program depends on [[generate_XDS.INP]] to interpret the header of the frames, so is currently limited to Dectris (Pilatus), ADSC (Quantum), Rigaku (several types) | * The program depends on [[generate_XDS.INP]] to interpret the header of the frames, so is currently limited to Dectris (Pilatus), ADSC (Quantum), Rigaku (several types), MAR (CCD and image plate) detectors, and the Bruker PHOTON II. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like oscillation range, wavelength, distance, and direct beam position (or rather: point of detector that is closest to the crystal). For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. | ||
* The authors of [[generate_XDS.INP]] have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has a anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing. | * The authors of [[generate_XDS.INP]] have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has a significant anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing. | ||
* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. We have no exhaustive list of these beamlines, and the frame headers do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction. | * At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs, or even 0 -1 0 (vertical and backwards) like for some measurements at Diamond's ID24. We have no exhaustive list of these beamlines, and the frame headers ususally do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction. | ||
* The display of large frames in the FRAME tab may be slow over the network. | * The display of large frames in the FRAME tab may be slow over the network. | ||
* Eiger data may be processed like this: a) run <code>generate_XDS.INP /path/to/XXX_master.h5</code> in an empty directory, to produce XDS.INP. b) start XDSGUI in the same directory (or navigate there, from its ''Projects'' tab) c) you will not be able to use the ''Frame'' tab for looking at the raw data, but you could use the tab to look at FRAME.cbf or other XDS-produced images. d) you should use/check/modify XDS.INP as usual; all the other tabs work normally. | |||
* The green INCLUDE_RESOLUTION and red semi-transparent EXCLUDE_RESOLUTION_RANGE circles in the Frame tab as well as all resolution values in the Frame and IDXREF tabs are calculated from values in XDS.INP (rather than from the refined values in XPARM.XDS, which XDS uses). For the (curved) Pilatus 12M, the calculations in XDSGUI do not give the correct results. | |||
== News == | == News == | ||
| Line 159: | Line 185: | ||
update June 2, 2014: fix bugs | update June 2, 2014: fix bugs | ||
update December 11, 2014: fix bugs; SETTING SYMMETRY-BASED for [[pointless]] (in the TOOLS tab); radiation damage (R_d) plot with recommended limit shown in the XDSSTAT tab | |||
update Sep 17, 2015: belatedly updated xdsgui.rhel6.* - this version correctly visualizes the tables of INTEGRATE.LP which have a different format since XDS version June-2015 | |||
update May 20, 2016: many bug fixes, rename XDSSTAT tab to <code>statistics</code> tab, include delta-CC<sub>1/2</sub> plot. | |||
update Jan 31, 2017: many bug fixes. Now works with Eiger HDF5 - select the *master.h5 file in the FRAME tab. | |||
== See also == | == See also == | ||
[[Installation]] | [[Installation]] | ||