XDSGUI: Difference between revisions

XDSGUI (view source)

Revision as of 18:46, 10 September 2017

4,097 bytes added , 10 September 2017

→‎Known bugs, problems and workarounds: MarCCD frames not being displayed

Kay

Bureaucrats

2,652

edits

@@ Line 36: / Line 36: @@
 Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are shown in the XDS.INP tab. Step-by-step:
 * "Load" a raw frame or a CBF file (e.g. FRAME.cbf) to have it displayed. The pane can be "dragged" with the left mouse button; the mouse wheel zooms. The parameters in the XDS.INP tab are taken for resolution calculations (i.e. the frame header is not being interpreted in any way).
-* if XDS.INP does not yet exist, click "generate XDS.INP" (this ''will'' read the header). Check the XDS.INP tab afterwards but then go back to the Frame tab. Note that the current generate_XDS.INP works well for Pilatus, ADSC, Mar and some Rigaku detectors; for other kinds of detectors the values marked XXX in XDS.INP have to be filled in manually.
+* if XDS.INP does not yet exist, click "generate XDS.INP" (this ''will'' read the header). Check the XDS.INP tab afterwards but then go back to the Frame tab. Note that the current generate_XDS.INP works well for Pilatus, ADSC, Mar and some Rigaku detectors; for other kinds of detectors the values marked XXX in XDS.INP have to be filled in manually. The [[Eiger]] detector is not yet implemented in the script, but this will eventually come.
 * left-click on "Untrusted areas" -> a pulldown menu appears
 * left-click on (say) "Untrusted Rectangle (2 clicks)"
@@ Line 61: / Line 61: @@
 === [[IDXREF]] ===
-detailed explanation of output: see [[IDXREF.LP]]
+detailed explanation of output: see [[IDXREF.LP]].
+The right side of the tab shows the indexed (black) and not-indexed (red) reflections of [[SPOT.XDS]].
 === DEFPIX ===
@@ Line 77: / Line 78: @@
 The border between the left and right side of the window may be adjusted with the mouse.<br>
-=== TOOLS ===
+=== tools ===
 offers possibilities for running short scripts.
@@ Line 85: / Line 86: @@
 * The first item of the left panel ("Show frame with predicted spots") generates the predicted pattern of reflections for a user-specified frame, overlaid on the frame, for display with [[XDS-viewer]]. The file FRAME.cbf (produced by INTEGRATE) is renamed to FRAME_$X.cbf (where X is the user-specified frame number) and remains in the temp subdirectory. It may of course be opened in the FRAME tab, but starting XDS-viewer automatically has the advantage that several frames with predictions may be inspected on the screen, at the same time. Please note: if the XDS directory resides in a FAT32 filesystem (which is often the case on a USB stick or disk), then "ln -s" (of the script line) should be replaced by "cp -p" since FAT32 does not support symlinks. Also note: for the script to work correctly, NAME_TEMPLATE_OF_DATA_FRAMES in XDS.INP has to specify an absolute, not a relative path.
-* The second item ("Optimizing data quality") offers commands that manipulate [[XDS.INP]] in several ways. Please note: the popup "XDS.INP has been changed externally" is emitted by the Qt system and cannot be switched off. It appears if one of the scripts changes XDS.INP while it is opened by XDSGUI (which is always the case) . Thus, one should simply press the "Reload" button. The user-definable commandline may be used e.g. for
+* The second item ("Optimizing data quality") offers commands that manipulate [[XDS.INP]] in several ways. Please note: the popup "XDS.INP has been changed externally" is emitted by the Qt system and cannot be switched off. It appears if one of the scripts changes XDS.INP while it is opened by XDSGUI (which is always the case) . Thus, one should simply press the "Reload" button. The user-definable command may be used e.g. for
-  grep -v "JOB=" XDS.INP > x; echo "JOB= DEFPIX INTEGRATE CORRECT" > XDS.INP; cat x >> XDS.INP
+  awk '/BEAM_DIVERGENCE=/{print "",$1,2*$2,$3,$4}' INTEGRATE.LP>x; grep -v BEAM_DIVERGENCE XDS.INP>>x; mv x XDS.INP
+This multiplies the estimate of BEAM_DIVERGENCE in INTEGRATE.LP by 2 and thus provides more pixels to the background estimation (for another INTEGRATE). This is useful for weakly exposed data sets which have mostly pixels with zero counts. Of course, any other factor (other than 2) may be tried.
 * The third item ("Saving and comparing good results") offers commands to save/restore the current data processing files to/from a "save" directory. Make sure to replace "xdiff" with "xxdiff" or "tkdiff", if one of the latter is available.
@@ Line 93: / Line 95: @@
   phenix.xtriage XDS_ASCII.HKL
-=== XDSSTAT ===
+=== statistics ===
-This tab allows to run [[XDSSTAT]] with a mouseclick. As soon as XDSSTAT.LP exists, the main plots derived from it are displayed. Another button allows to run [[XDS-viewer]] to visualize the control images that XDSSTAT generates, namely anom.pck, scales.pck, rf.pck, misfits.pck, corr.pck, rlps.pck, peaks.pck and nobs.pck .
+This tab allows to run [[XDSSTAT]] and [[XDSCC12]] with a mouseclick.
+As soon as XDSSTAT.LP exists, the main plots derived from it are displayed (second and further panes). The "view" button allows to run [[XDS-viewer]] to visualize the control images that XDSSTAT generates, namely anom.pck, scales.pck, rf.pck, misfits.pck, corr.pck, rlps.pck, peaks.pck and nobs.pck .
+Likewise, as soon as the output file XDSCC12.LP exists, a plot derived from those lines starting with "a:" and "b:" will be generated. In black, this shows the overall contribution of a frame, or of several frames pooled, to the CC<sub>1/2</sub> value of those reflections that this frame (or these frames) contributes to. The rainbow-coloured curves divide the total resolution range into a user-selectable number (chosen with the -nbin option of XDSCC12) of resolution ranges; blue is low resolution and red is high resolution.
+Fine-sliced frames (0.1°) can be pooled into 1°-pseudoframes by using the -t option, to increase the number of reflections contributing to the plot and to make it less noisy.
 === XDSCONV ===
@@ Line 111: / Line 118: @@
 === Dependencies ===
-XDSGUI depends on [[generate_XDS.INP]], [[XDS-viewer]], and of course [[XDS]] (both the xds and xds_par binaries!), [[XDSCONV]], and [[2cbf]]. The XDSSTAT tab requires [[XDSSTAT]]. One of the items in the TOOLS tab calls a graphical file comparison program; if the default (xdiff) is not available, [http://furius.ca/xxdiff xxdiff] or [http://sourceforge.net/projects/tkdiff tkdiff] or [http://kdiff3.sourceforge.net/ kdiff3] or [http://meldmerge.org/ meld] may be used - please adjust the commandline below the button accordingly. See also [[Installation]].
+XDSGUI depends on [[generate_XDS.INP]], [[XDS-viewer]], and of course [[XDS]] (both the xds and xds_par binaries!), [[XDSCONV]], and [[2cbf]]. The ''statistics'' tab requires [[XDSSTAT]] and [[XDSCC12]]. [[Eiger]] data processing requires <code>h5dump</code> and <code>H5ToXds</code> (or equivalent). One of the items in the <code>tools</code> tab calls a graphical file comparison program; if the default (xdiff) is not available, [http://furius.ca/xxdiff xxdiff] or [http://sourceforge.net/projects/tkdiff tkdiff] or [http://kdiff3.sourceforge.net/ kdiff3] or [http://meldmerge.org/ meld] may be used - please adjust the commandline below the button accordingly. See also [[Installation]].
-Technically, the word "dependency" means that these scripts/programs should be in your $PATH - an 'alias' is often not good enough because the TOOLS commands use bash as a shell, and that shell does not inherit the aliases from e.g. (t)csh.
+Technically, the word "dependency" means that these scripts/programs should be in your $PATH - an 'alias' is often not good enough because the ''TOOLS'' commands use <code>bash</code> as a shell, and that shell does not inherit the aliases from e.g. (t)csh.
 === Installation ===
@@ Line 140: / Line 147: @@
 if you get the error message
   xdsgui: error while loading shared libraries: libmng.so.1: cannot open shared object file: No such file or directory
+On the '''Mac''' (e.g. Yosemite) one needs [https://itunes.apple.com/us/app/xcode/id497799835?ls=1&mt=12 Xcode] installed, seemingly before XDSGUI is installed. Otherwise one may obtain a popup error “You can’t open the application because it is not supported on this type of Mac.”
 == Known bugs, problems and workarounds ==
 # ''Question concerning TOOLS: "Show frame..."  on Mac: it creates everything shown in the commandline but it seem to us that everything runs in the background. Therefore xds-viewer does not open anywhere as it is not seen. Also, it would be great if this image together with the predictions would be re-directed to the FRAME tab.'' <br> Answer: On Linux, the xds-viewer window is brought to the foreground automatically, whereas on the Mac this does not seem to happen. However, an icon for xds-viewer appears in the dock (on the right) and I can double-click it to see the xds-viewer window. I have no idea how to bring xds-viewer to the foreground automatically, and I need input from people who know Macs and tell me how to do it. The reason why we do not open automatically in the FRAME tab is that one can have several xds-viewer windows open at the same time, and compare the patterns. As a '''workaround''', you can manually open the resulting temp/FRAME_$X.cbf in the FRAME tab.
 # ''Another question concerning TOOLS: "Further analyses" on Mac: again pointless runs nicely but there is no output for the user.'' <br> Answer: Could it be that you started xdsgui by double-clicking its icon in the Finder? In that case, there is no output visible, because there is no console. Unfortunately, for all the Tools, the output is in the console window where xdsgui was started. Thus, you should really start xdsgui in a console window.
-# both generate_XDS.INP and [[XDS]] do not seem to like spaces ("blanks") in file/directory names. While frame names rarely have a space in them, directories might, e.g. if an external disk has the label "John's USB disk" then this would be mounted on Linux as /media/John's USB disk/ and that would result in a error message from generate_XDS.INP and/or XDS. The '''workaround''' is to use a symlink on a different hard disk.
+# both generate_XDS.INP and [[XDS]] do not tolerate spaces ("blanks") in file/directory names. While frame names rarely have a space in them, directories might, e.g. if an external disk has the label "John's USB disk" then this would be mounted on Linux as /media/John's USB disk/ and that would result in a error message from generate_XDS.INP and/or XDS. The '''workaround''' is to use a symlink on a different hard disk.
 # ''Problem description: On the Mac, after loading frames, by clicking “generate XDS.INP”, the program gives some strange symbol “Ã” in XDS.INP. And the more you click “save” button, the more “Ã” appeared. This looks like <br>SPACE_GROUP_NUMBER=0 Ã ! 0 if unknown <br>UNIT_CELL_CONSTANTS= 70 80 90 90 90 90 Ã .''<br> The problem is due to the “Rich text” format in TextEdit when saving "generate_XDS.INP". It is solved by re-downloading the script, and changing format to Plain - everything is working now.
+# on a Mac, you may find a stream of the following error messages in the console window: <code>Jan 31 14:35:17  xdsgui[20939] <Error>: Error: this application, or a library it uses, has passed an invalid numeric value (NaN, or not-a-number) to CoreGraphics API and this value is being ignored. Please fix this problem. Jan 31 14:35:17  xdsgui[20939] <Error>: If you want to see the backtrace, please set CG_NUMERICS_SHOW_BACKTRACE environmental variable.</code> This sounds terrible, but is actually harmless. Since it is a message from Qt (which XDSGUI uses for graphics), ''it does not compromise the XDS data processing in any way''. If it bothers you, just suppress the printout of these messages, by <code>xdsgui 2>&1 | egrep -v 'CoreGraphics|CG_NUMERICS'</code>
+# there is a bug in the 2017 version of XDSGUI that prevents it from displaying non-CBF files. The symptom is that the frames appear to be blank. The workaround is: go to (upper left) "Menu"->"Settings"->remove the string "/usr/local/lib64/dectris-neggia.so"
-If you find a bug, please send email to Kay dot Diederichs at uni-konstanz dot de , ideally with enough information/data to reproduce the bug.
+If you find a bug, please send email to Kay dot Diederichs at uni-konstanz dot de , with enough information/data to reproduce the bug.
 == Limitations ==
-* The program depends on [[generate_XDS.INP]] to interpret the header of the frames, so is currently limited to Dectris (Pilatus), ADSC (Quantum), Rigaku (several types) and MAR (CCD and image plate) detectors. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like oscillation range, wavelength, distance, and direct beam position (or rather: point of detector that is closest to the crystal). For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates].
+* The program depends on [[generate_XDS.INP]] to interpret the header of the frames, so is currently limited to Dectris (Pilatus), ADSC (Quantum), Rigaku (several types), MAR (CCD and image plate) detectors, and the Bruker PHOTON II. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like oscillation range, wavelength, distance, and direct beam position (or rather: point of detector that is closest to the crystal). For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates].
-* The authors of [[generate_XDS.INP]] have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has a anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing.
+* The authors of [[generate_XDS.INP]] have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has a significant anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing.
-* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. We have no exhaustive list of these beamlines, and the frame headers do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction.
+* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs, or even 0 -1 0 (vertical and backwards) like for some measurements at Diamond's ID24. We have no exhaustive list of these beamlines, and the frame headers ususally do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction.
 * The display of large frames in the FRAME tab may be slow over the network.
+* Eiger data may be processed like this: a) run <code>generate_XDS.INP /path/to/XXX_master.h5</code> in an empty directory, to produce XDS.INP. b) start XDSGUI in the same directory (or navigate there, from its ''Projects'' tab) c) you will not be able to use the ''Frame'' tab for looking at the raw data, but you could use the tab to look at FRAME.cbf or other XDS-produced images. d) you should use/check/modify XDS.INP as usual; all the other tabs work normally.
+* The green INCLUDE_RESOLUTION and red semi-transparent EXCLUDE_RESOLUTION_RANGE circles in the Frame tab as well as all resolution values in the Frame and IDXREF tabs are calculated from values in XDS.INP (rather than from the refined values in XPARM.XDS, which XDS uses). For the (curved) Pilatus 12M, the calculations in XDSGUI do not give the correct results.
 == News ==
@@ Line 174: / Line 187: @@
 update December 11, 2014: fix bugs; SETTING SYMMETRY-BASED for [[pointless]] (in the TOOLS tab); radiation damage (R_d) plot with recommended limit shown in the XDSSTAT tab
+update Sep 17, 2015: belatedly updated xdsgui.rhel6.* - this version correctly visualizes the tables of INTEGRATE.LP which have a different format since XDS version June-2015
+update May 20, 2016: many bug fixes, rename XDSSTAT tab to <code>statistics</code> tab, include delta-CC<sub>1/2</sub> plot.
+update Jan 31, 2017: many bug fixes. Now works with Eiger HDF5 - select the *master.h5 file in the FRAME tab.
 == See also ==
 [[Installation]]

XDSGUI: Difference between revisions

XDSGUI (view source)

Revision as of 18:46, 10 September 2017

Navigation menu

Search