27
edits
XDSGUI: Difference between revisions
→XDS.INP
| (34 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
XDSGUI is a GUI (graphical user interface) for XDS that is supposed to help both novice and experienced users. | XDSGUI is a GUI (graphical user interface) for XDS that is supposed to help both novice and experienced users. | ||
It graphically displays the ASCII and [http://www.bernstein-plus-sons.com/software/CBF cbf] files that XDS writes, and can run useful shell commands with a simple mouse click. The design goal of the program is to enable XDS data processing without the commandline, and to supply additional graphical information, in a simple, user-modifiable and user-extensible way. | It graphically displays the ASCII and [http://www.bernstein-plus-sons.com/software/CBF cbf] files that XDS writes, and can run useful shell commands with a simple mouse click. The design goal of the program is to enable XDS data processing without the commandline, and to supply additional graphical information, in a simple, user-modifiable and user-extensible way. | ||
XDSGUI gives access to George Sheldrick's SHELXC, SHELXD and SHELXE programs, via the [[XDSGUI#SHELX|SHELX]] tab, and to Isabel Uson's ARCIMBOLDO programs, via the [[XDSGUI#ARCIMBOLDO|ARCIMBOLDO]] tab. | |||
== How to use XDSGUI == | == How to use XDSGUI == | ||
The program may be used to analyze graphically any existing directory with files earlier written by XDS, or may be used to create these files. It should be started in a terminal window, which will later show the output of the programs that are run (XDS, pointless, XDSCONV, ...). | The program may be used to analyze graphically any existing directory with files earlier written by XDS, or may be used to create these files. '''It should be started in a terminal window''', which will later show the output of the programs that are run (XDS, pointless, XDSCONV, ...). | ||
The user is supposed to use the tabs on the top, from left to right. | The user is supposed to use the tabs on the top, from left to right. | ||
| Line 20: | Line 21: | ||
=== Technical aspects of XDSGUI usage === | === Technical aspects of XDSGUI usage === | ||
* the simple editor operating in the | * the simple editor operating in the tabs understands ctrl-X (cut), ctrl-c (copy), ctrl-v (paste), ctrl-z (undo) and ctrl-Z (redo). | ||
* ctrl-+ (control-plus) and ctrl-- (control-minus) enlarge or decrease the font size of characters. | * ctrl-+ (control-plus) and ctrl-- (control-minus) enlarge or decrease the font size of characters. | ||
* flexibility of operation: all commands in the | * flexibility of operation: all commands in the [[XDSGUI#TOOLS|TOOLS]] tab may be modified by the user, simply by modifying the (bash) code in the text field below the button. The user-changed command is automagically and permanently stored in ~/.xds-gui . The user may also change what is written on a button, by editing ~/.xds-gui. It should be obvious what to change. | ||
* Menu / Settings is used for | * Menu / Settings is used for specifying: In "Paths" the "generic library" (e.g. /usr/local/lib64/dectris-neggia.so) for reading HDF5 files and for binaries (xds_par, xscale_par, shelxc/d/e, ...), in "Appearance" (font size and default plot height), and in "User Buttons" the names for user defined tool buttons. The Settings menu can be opened on the Mac by clicking upper-left on "xdsgui", then "Preferences ...", or pressing the "Command" (⌘) key and the "," key. Changed settings are automatically stored in ~/.xdsgui . | ||
== Tabs == | == Tabs == | ||
| Line 35: | Line 36: | ||
=== Frame === | === Frame === | ||
to display existing 2-dimensional data frames, or to start XDS data processing by displaying a raw data frame. In the latter case, the "generate XDS.INP" button runs the [[generate_XDS.INP]] script to create a first XDS.INP from the header of the raw data frame that is being displayed. (Note: this button needs to be used only once per dataset; all later changes to XDS.INP are done by different means - manually or scripted) The blue circle(s) delineate(s) the area(s) of the detector within TRUSTED_REGION; the green circles correspond to INCLUDE_RESOLUTION_RANGE, and the red hatched regions correspond to EXCLUDE_RESOLUTION_RANGEs. The locations of the circles are not as accurate as those that XDS uses internally, because | to display existing 2-dimensional data frames, or to start XDS data processing by displaying a raw data frame. In the latter case, the "generate XDS.INP" button runs the [[generate_XDS.INP]] script to create a first XDS.INP from the header of the raw data frame that is being displayed. (Note: this button needs to be used only once per dataset; all later changes to XDS.INP are done by different means - manually or scripted) The blue circle(s) delineate(s) the area(s) of the detector within TRUSTED_REGION; the green circles correspond to INCLUDE_RESOLUTION_RANGE, and the red [[XDSGUI#Technical aspects of XDSGUI usage|hatched]] regions correspond to EXCLUDE_RESOLUTION_RANGEs. The locations of the circles are not as accurate as those that XDS uses internally, because only values from XDS.INP, not the refined ones from XPARM.XDS) are used to calculate the resolution, and do not currently account for segmented or curved detectors. After XDS has indexed spots, they can be visualized with the *predictions* button. | ||
Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four ( | Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED_QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are placed in the XDS.INP tab. Step-by-step: | ||
* "Load" a | * "Load" a data file (e.g. FRAME.cbf) or, in case of Eiger HDF5 data, a frame based on a *master.h5 file to have it displayed. The pane can be "dragged" with the left mouse button; the mouse wheel zooms. The parameters in the XDS.INP tab are taken for resolution calculations (i.e. the frame header is not being interpreted in any way). For HDF5 data, the *master.h5 file must be loaded; this will automatically show its first frame (note that for this to work, the path to the correct [[XDSGUI#Technical aspects of XDSGUI usage|"generic library"]] must be specified in Menu/Settings). The frame number can be chosen with the selection box on the right, or the up-down arrows. | ||
* if XDS.INP does not yet exist, click "generate XDS.INP" (this ''will'' read the header). Check the XDS.INP tab afterwards but then go back to the Frame tab. Note that the current [[generate_XDS.INP]] works well for Pilatus, ADSC, Mar, [[Eiger]] and some Rigaku detectors; for other kinds of detectors the values marked XXX in XDS.INP have to be filled in manually. | * if XDS.INP does not yet exist, click "generate XDS.INP" (this ''will'' read the header). Check the XDS.INP tab afterwards but then go back to the Frame tab. Note that the current [[generate_XDS.INP]] works well for Pilatus, ADSC, Mar, [[Eiger]] and some Rigaku detectors; for other kinds of detectors the values marked XXX in XDS.INP have to be filled in manually. | ||
* left-click on "Untrusted areas" -> a pulldown menu appears | * left-click on "Untrusted areas" -> a pulldown menu appears | ||
* left-click on (say) "Untrusted Rectangle (2 clicks)" | * left-click on (say) "Untrusted Rectangle (2 clicks)" | ||
| Line 46: | Line 47: | ||
* with two more right-clicks you get another rectangle, and so on | * with two more right-clicks you get another rectangle, and so on | ||
* you can choose "Untrusted Ellipse (2 clicks)" or "Untrusted Quadrilateral (4 clicks)" and these work in exactly the same way | * you can choose "Untrusted Ellipse (2 clicks)" or "Untrusted Quadrilateral (4 clicks)" and these work in exactly the same way | ||
* if you remove | * if you remove an UNTRUSTED_* line from XDS.INP then its shape will be removed from the Frame tab. | ||
* You can remove the recently added UNTRUSTED_** line by pressing Ctrl-Z (Command-Z on Mac), useful for re-drawing if you made an error. | |||
For best performance, it is recommended to run the program locally rather than through a ssh network connection. | |||
=== XDS.INP === | === XDS.INP === | ||
this shows the current [[XDS.INP]] file. Coloured items correspond to the circles of the FRAME tab. Changes can be made with a simple editor, and the new version can be saved. A run of xds_par can be started (and killed). <br> | this shows the current [[XDS.INP]] file. Coloured items correspond to the circles of the FRAME tab. Changes can be made with a simple editor, and the new version can be saved. A run of xds_par can be started (and killed). <br> | ||
The XDS.INP tab is directly connected with the FRAME tab; changing a value in the XDS.INP tab results in immediate change in the FRAME tab. This means that e.g. an UNTRUSTED area that was positioned wrongly can be edited or simply removed in the XDS.INP tab. <br> | The XDS.INP tab is directly connected with the FRAME tab; changing a value in the XDS.INP tab results in immediate change in the FRAME tab. This means that e.g. an UNTRUSTED area that was positioned wrongly can be edited or simply removed in the XDS.INP tab. <br> | ||
The origin of XDS.INP does not matter - it may e.g. be derived from a detector template, or written by the beamline software, or obtained by clicking the "generate_XDS.INP" button in the FRAME tab. | The origin of XDS.INP does not matter - it may e.g. be derived from a detector template, or written by the beamline software, or obtained by clicking the "[[generate_XDS.INP]]" button in the FRAME tab. | ||
=== XYCORR === | === XYCORR === | ||
| Line 87: | Line 89: | ||
Several such scripts are pre-defined; the user may create her own scripts. If scripts are modified, they are saved to ~/.xds-gui . The names of the buttons (e.g. "User defined command 1") can be changed by editing ~/.xds-gui . | Several such scripts are pre-defined; the user may create her own scripts. If scripts are modified, they are saved to ~/.xds-gui . The names of the buttons (e.g. "User defined command 1") can be changed by editing ~/.xds-gui . | ||
* The first item of the left panel ("Show frame with predicted spots") generates the predicted pattern of reflections for a user-specified frame, overlaid on the frame, for display with [[XDS-viewer]]. The file FRAME.cbf (produced by INTEGRATE) is renamed to FRAME_$X.cbf (where X is the user-specified frame number) and remains in the temp subdirectory. It may of course be opened in the FRAME tab, but starting XDS-viewer automatically has the advantage that several frames with predictions may be inspected on the screen, at the same time. Please note: if the XDS directory resides in a FAT32 filesystem (which is often the case on a USB stick or disk), then "ln -s" (of the script line) should be replaced by "cp -p" since FAT32 does not support symlinks. Also note: for the script to work correctly, NAME_TEMPLATE_OF_DATA_FRAMES in XDS.INP has to specify an absolute, not a relative path. | * The first item of the left panel ("Show frame with predicted spots") generates the predicted pattern of reflections for a user-specified frame, overlaid on the frame, for display with [[XDS-viewer]]. The file FRAME.cbf (produced by INTEGRATE) is renamed to FRAME_$X.cbf (where X is the user-specified frame number) and remains in the temp subdirectory. It may of course be opened in the FRAME tab, but starting XDS-viewer automatically has the advantage that several frames with predictions may be inspected on the screen, at the same time. Please note: if the XDS directory resides in a FAT32 filesystem (which is often the case on a USB stick or disk), then "ln -s" (of the script line) should be replaced by "cp -p" since FAT32 does not support symlinks. Also note: for the script to work correctly, NAME_TEMPLATE_OF_DATA_FRAMES in XDS.INP has to specify an absolute, not a relative path. In the new release, predictions can also visualized in the Frame tab. | ||
* The second item ("Saving and comparing good results") offers commands to save/restore the current data processing files to/from a "save" directory. Make sure to replace "xdiff" with "xxdiff" or "tkdiff", if one of the latter is available. If [http://www.globalphasing.com/autoproc autoPROC] is installed, I suggest to use the "User defined command 2" for <code>mkdir staraniso; cd staraniso; aP_scale -hkl ../XDS_ASCII.HKL; echo anisotropy-corrected files are in staraniso subdirectory</code>. | * The second item ("Saving and comparing good results") offers commands to save/restore the current data processing files to/from a "save" directory. Make sure to replace "xdiff" with "xxdiff" or "tkdiff", if one of the latter is available. If [http://www.globalphasing.com/autoproc autoPROC] is installed, I suggest to use the "User defined command 2" for <code>mkdir staraniso; cd staraniso; aP_scale -hkl ../XDS_ASCII.HKL; echo anisotropy-corrected files are in staraniso subdirectory</code>. | ||
* The third item ("Optimizing data quality") offers commands that manipulate [[XDS.INP]] in several ways. Please note: the popup "XDS.INP has been changed externally" is emitted by the Qt system and cannot be switched off. It appears if one of the scripts changes XDS.INP while it is opened by XDSGUI (which is always the case) . Thus, one should simply press the "Reload" button | * The third item ("Optimizing data quality") offers commands that manipulate [[XDS.INP]] in several ways. Please note: the popup "XDS.INP has been changed externally" is emitted by the Qt system and cannot be switched off. It appears if one of the scripts changes XDS.INP while it is opened by XDSGUI (which is always the case) . Thus, one should simply press the "Reload" button. The user-definable command may be used e.g. for <pre>echo RELRAD=7 ! default is 5 >x; grep -v BEAM_DIVERGENCE XDS.INP >>x; cat x > XDS.INP; rm x</pre> This provides more pixels to the background estimation (for another INTEGRATE), and is useful for weakly exposed data sets which have mostly pixels with zero counts. | ||
* The last item ("Further analyses") | * The last item ("Further analyses") offers useful commands. "determine spacegroup with pointless" runs [[pointless]] against the XDS_ASCII.HKL file. The button "show spots in reciprocal space" runs [[spot2pdb]] to show you the reflections of SPOT.XDS in reciprocal space, with coot. You see the reflections that could be indexed in IDXREF in yellow, the not-indexed ones in pink, the rotation axis in blue and the origin as a blue cross. Ice rings and additional lattices can be easily identified - see [[spot2pdb]] for examples! | ||
* Useful scripts for user tools are shared in [[XDSGUI User tools]] | |||
=== statistics === | === statistics === | ||
| Line 108: | Line 109: | ||
=== XDSCONV === | === XDSCONV === | ||
A simple XDSCONV.INP is provided by default. It may be edited by the user, and saved. Upon clicking a button, [[xdsconv]] is run and if necessary a MTZ file is produced. XDSCONV.LP may be displayed. Note that you do not need to run XSCALE first. Reflections are already scaled in XDS_ASCII.HKL by CORRECT. | |||
=== XSCALE === | === XSCALE === | ||
A simple XSCALE.INP is provided by default. It may be edited by the user, and saved. Upon clicking a button, [[xscale]] is run. XSCALE.LP may be displayed. | |||
=== SHELX === | === SHELX === | ||
| Line 124: | Line 125: | ||
== Availability == | == Availability == | ||
The [https://sourceforge.net/u/joseptrivino/xdsgui/ci/master/tree/ program's source code] is released under the terms of the [https://gnu.org/licenses/old-licenses/gpl-2.0.txt | The [https://sourceforge.net/u/joseptrivino/xdsgui/ci/master/tree/ program's source code] is released under the terms of the [https://gnu.org/licenses/old-licenses/gpl-2.0.txt GPL]; it uses the [http://www.qt.io/ Qt library] which was released under the terms of the [https://gnu.org/licenses/lgpl.txt LGPL]. The program is under development and probably has bugs. If it crashes, it should simply be restarted. A crash of the program ''does not interfere'' with the operation of [[XDS]]; likewise, closing the program window does not influence any XDS run started from XDSGUI. | ||
=== Dependencies === | === Dependencies === | ||
| Line 140: | Line 141: | ||
=== Installation === | === Installation === | ||
The program can be downloaded, by academic users, as | The program can be downloaded, by academic users, as binary for Linux [https://{{SERVERNAME}}/pub/linux_bin/xdsgui 64bit] (compiled on RHEL7) and for Mac ([https://{{SERVERNAME}}/pub/mac_bin/xdsgui_mac_intel.dmg Intel] and [https://{{SERVERNAME}}/pub/mac_bin/xdsgui_mac_silicon.dmg Apple Silicon], respectively). | ||
The 2020 version is available in the same folders with name starting with xdsgui.2020 . | |||
Industrial users: pls contact me directly. | Industrial users: pls contact me directly. | ||
| Line 146: | Line 149: | ||
The easiest way to install XDSGUI, and all XDSGUI-related programs is documented in [[Installation]]. | The easiest way to install XDSGUI, and all XDSGUI-related programs is documented in [[Installation]]. | ||
If you want to install manually, in case of Linux, after downloading the binary file, its name should be changed to "xdsgui" and then be made executable ("chmod a+x xdsgui"). The binary may then be copied into e.g. the directory /usr/local/bin by the administrator, or to the $HOME/bin directory by the user. Any other directory should be suitable as long as it is in your $PATH. In case of MacOS, the DMG file may be installed in the usual way, by double-clicking it and dragging the icon into the Applications folder. It is advantageous to place a symbolic link, like | |||
sudo ln -s /Applications/xdsgui.app/Contents/MacOS/xdsgui /usr/local/bin/xdsgui | sudo ln -s /Applications/xdsgui.app/Contents/MacOS/xdsgui /usr/local/bin/xdsgui | ||
'''and to start the GUI from a terminal window, by typing xdsgui'''. In a similar way, XDS-viewer should be put into the $PATH under the name xds-viewer, e.g. using | '''and to start the GUI from a terminal window, by typing xdsgui'''. In a similar way, XDS-viewer should be put into the $PATH under the name xds-viewer, e.g. using | ||
sudo ln -s /Applications/XDS-Viewer.app/Contents/MacOS/ | sudo ln -s /Applications/XDS-Viewer.app/Contents/MacOS/XDS-Viewer /usr/local/bin/xds-viewer | ||
==== Libraries and software that the program depends on ==== | ==== Libraries and software that the program depends on ==== | ||
Procedures for identifying and installing missing libraries are given in the [[Installation]] article. | |||
''' | On the '''Mac''' (e.g. Yosemite) one needs the (free) [https://itunes.apple.com/us/app/xcode/id497799835?ls=1&mt=12 Xcode] command line tools installed. | ||
If on '''Linux''' you get an error message like | |||
xdsgui: error while loading shared libraries: libQtGui.so.4: cannot open shared object file: No such file or directory | |||
or similar for libQtCore.so.4 or libQtOpenGL.so.4 , you should install the Qt5 version which is now the default on Linux. | |||
== Known bugs, problems and workarounds == | == Known bugs, problems and workarounds == | ||
| Line 171: | Line 170: | ||
# ''Problem description: On the Mac, after loading frames, by clicking “generate XDS.INP”, the program gives some strange symbol “Ô in XDS.INP. And the more you click “save” button, the more “Ô appeared. This looks like <br>SPACE_GROUP_NUMBER=0 à ! 0 if unknown <br>UNIT_CELL_CONSTANTS= 70 80 90 90 90 90 à .''<br> The problem is due to the “Rich text” format in TextEdit when saving "generate_XDS.INP". It is solved by re-downloading the script, and changing format to Plain - everything is working now. | # ''Problem description: On the Mac, after loading frames, by clicking “generate XDS.INP”, the program gives some strange symbol “Ô in XDS.INP. And the more you click “save” button, the more “Ô appeared. This looks like <br>SPACE_GROUP_NUMBER=0 à ! 0 if unknown <br>UNIT_CELL_CONSTANTS= 70 80 90 90 90 90 à .''<br> The problem is due to the “Rich text” format in TextEdit when saving "generate_XDS.INP". It is solved by re-downloading the script, and changing format to Plain - everything is working now. | ||
# on a Mac, you may find a stream of the following error messages in the console window: <code>Jan 31 14:35:17 xdsgui[20939] <Error>: Error: this application, or a library it uses, has passed an invalid numeric value (NaN, or not-a-number) to CoreGraphics API and this value is being ignored. Please fix this problem. Jan 31 14:35:17 xdsgui[20939] <Error>: If you want to see the backtrace, please set CG_NUMERICS_SHOW_BACKTRACE environmental variable.</code> This sounds terrible, but is actually harmless. Since it is a message from Qt (which XDSGUI uses for graphics), ''it does not compromise the XDS data processing in any way''. If it bothers you, just suppress the printout of these messages, by <code>xdsgui 2>&1 | egrep -v 'CoreGraphics|CG_NUMERICS'</code> | # on a Mac, you may find a stream of the following error messages in the console window: <code>Jan 31 14:35:17 xdsgui[20939] <Error>: Error: this application, or a library it uses, has passed an invalid numeric value (NaN, or not-a-number) to CoreGraphics API and this value is being ignored. Please fix this problem. Jan 31 14:35:17 xdsgui[20939] <Error>: If you want to see the backtrace, please set CG_NUMERICS_SHOW_BACKTRACE environmental variable.</code> This sounds terrible, but is actually harmless. Since it is a message from Qt (which XDSGUI uses for graphics), ''it does not compromise the XDS data processing in any way''. If it bothers you, just suppress the printout of these messages, by <code>xdsgui 2>&1 | egrep -v 'CoreGraphics|CG_NUMERICS'</code> | ||
# | # Frames from HDF5-files (*master.h5) are not being displayed: the symptom is that the frames appear to be empty/blank. This means you have not specified the correct "generic library" - see [[XDSGUI#Technical aspects of XDSGUI usage|Technical aspects of XDSGUI usage]]. The fix is: go to (upper left) "Menu"->"Settings"->insert the string "/usr/local/lib64/dectris-neggia.so" as "generic library", or try "/usr/local/lib64/durin-plugin.so" for DIAMOND data. Ask the system administrator for the exact path! If none of these works, leave "generic library" empty and install <code>H5ToXds</code> or <code>hdf2mini-cbf</code> as explained above. | ||
If you find a bug, please send email to Kay dot Diederichs at uni-konstanz dot de , with enough information/data to reproduce the bug. | If you find a bug, please send email to Kay dot Diederichs at uni-konstanz dot de , with enough information/data to reproduce the bug ("it does not work" is not enough!). | ||
== Limitations == | == Limitations == | ||
| Line 182: | Line 179: | ||
* The display of large frames in the FRAME tab may be slow over the network. | * The display of large frames in the FRAME tab may be slow over the network. | ||
* The green INCLUDE_RESOLUTION and red semi-transparent EXCLUDE_RESOLUTION_RANGE circles in the Frame tab as well as all resolution values in the Frame and IDXREF tabs are calculated from values in XDS.INP (rather than from the refined values in XPARM.XDS, which XDS uses). For the (curved) Pilatus 12M, the calculations in XDSGUI do not give the correct results. | * The green INCLUDE_RESOLUTION and red semi-transparent EXCLUDE_RESOLUTION_RANGE circles in the Frame tab as well as all resolution values in the Frame and IDXREF tabs are calculated from values in XDS.INP (rather than from the refined values in XPARM.XDS, which XDS uses). For the (curved) Pilatus 12M, the calculations in XDSGUI do not give the correct results. | ||
* The path (e.g. /usr/local/lib64/dectris-neggia.so) to the <code>dectris-neggia</code> library (available from Dectris) or the <code>durin-plugin</code> library (download from Diamond Light Source, DLS) should be entered into the "generic frame library" entry, in Menu->Settings, to enable visualization of raw data from Eiger detectors, but this mechanism | * The path (e.g. /usr/local/lib64/dectris-neggia.so) to the <code>dectris-neggia</code> library (available from Dectris) or the <code>durin-plugin</code> library (download from Diamond Light Source, DLS) should be entered into the "generic frame library" entry, in Menu->Settings, to enable visualization of raw data from Eiger detectors, but this mechanism may not work properly. Therefore, <code>H5ToXds</code> or (better because maintained by GlobalPhasing)<code>hdf2mini-cbf</code> may be installed as explained above. | ||
== News == | == News == | ||
| Line 217: | Line 214: | ||
update April 17, 2019: Arcimboldo programs are available through the ARCIMBOLDO tab. | update April 17, 2019: Arcimboldo programs are available through the ARCIMBOLDO tab. | ||
update October 22, 2020: Qt4 | update October 22, 2020: Qt4 for Mac and Qt5 for Linux versions with improved TOOLS, and bugfixes. | ||
update May 5, 2023: lots of bugfixes and enhancements mainly by Juno Krahn. Should evaluate the environment variable NEGGIA_PATH and DURIN_PATH, as does generate_XDS.INP from v1.14 onwards. Binaries for Linux, Intel Mac and Silicon Mac. | |||
== See also == | == See also == | ||