2,652
edits
Eiger: Difference between revisions
Jump to navigation
Jump to search
→Less efficient way of processing Eiger data, using conversion to CBF
(→Benchmark using H5ToXds: remove - too old) |
|||
| (One intermediate revision by the same user not shown) | |||
| Line 18: | Line 18: | ||
* make sure that master.h5 and the corresponding data.h5 files remain together as collected, and '''don't rename the data.h5 files''' - they are referred to from master.h5. If you change the names of the data.h5 files or copy them somewhere else, that link is broken unless you fix master.h5. | * make sure that master.h5 and the corresponding data.h5 files remain together as collected, and '''don't rename the data.h5 files''' - they are referred to from master.h5. If you change the names of the data.h5 files or copy them somewhere else, that link is broken unless you fix master.h5. | ||
== Script for generating XDS.INP from master.h5 == | == Script by Andreas Förster (Dectris) for generating XDS.INP from master.h5 == | ||
<div class="mw-collapsible mw-collapsed"> | <div class="mw-collapsible mw-collapsed"> | ||
Expand code section below (i.e. click on blue <code>[Expand]</code> at the end of this line if there is no code visible), download it and save as XDS_from_H5.py . | Expand code section below (i.e. click on blue <code>[Expand]</code> at the end of this line if there is no code visible), download it and save as XDS_from_H5.py . | ||
| Line 623: | Line 623: | ||
* Set MAXIMUM_NUMBER_OF_JOBS= and MAXIMUM_NUMBER_OF_PROCESSORS= to similar values whose product is slightly smaller than the total number of threads on your system. | * Set MAXIMUM_NUMBER_OF_JOBS= and MAXIMUM_NUMBER_OF_PROCESSORS= to similar values whose product is slightly smaller than the total number of threads on your system. | ||
= Less efficient way of processing Eiger data, using conversion to CBF= | = Less efficient way of processing Eiger data, using conversion to CBF = | ||
Since the release of Neggia, a plugin for XDS that parallelizes the reading of images from HDF5 data, conversion by H5ToXds should no longer be required in most usage scenarios. The sections below nevertheless describe this possibility, since preliminary experience with some less common network file systems (apparently GPFS, but not NFS) seems to indicate low performance of Neggia. | Since the release of Neggia, a plugin for XDS that parallelizes the reading of images from HDF5 data, conversion by H5ToXds should no longer be required in most usage scenarios. The sections below nevertheless describe this possibility, since preliminary experience with some less common network file systems (apparently GPFS, but not NFS) seems to indicate low performance of Neggia. | ||
Conversion program options: Dectris provides [https://www.dectris.com/news.html?page=2 H5ToXds] (Linux only!). That program converts (as the name indicates) the HDF5 files to CBF files; however, it does not write the geometry and other information into the CBF header (therefore, [[generate_XDS.INP]] or MOSFLM does not work with these files). Alternatives are GlobalPhasing's hdf2mini-cbf program ( | Conversion program options: Dectris provides [https://www.dectris.com/news.html?page=2 H5ToXds] (Linux only!). That program converts (as the name indicates) the HDF5 files to CBF files; however, it does not write the geometry and other information into the CBF header (therefore, [[generate_XDS.INP]] or MOSFLM does not work with these files). Alternatives are GlobalPhasing's hdf2mini-cbf program (does ''not'' need autoPROC license) or, from http://www.mrc-lmb.cam.ac.uk/harry/imosflm/ver721/downloads, the eiger2cbf-osx or eiger2cbf-linux program written by T. Nakane. The latter programs do write a useful CBF header. | ||
H5ToXds and eiger2cbf-osx / eiger2cbf-linux do not work with files produced at Diamond Light Source. | |||
== A script for faster XDS processing of CBF-converted Eiger data (this is only shown out of historic interest) == | |||
For faster processing, the [[Eiger#A_script_for_faster_XDS_processing_of_CBF-converted Eiger data|shell script]] below should be copied to /usr/local/bin/H5ToXds and made executable (<code>chmod a+rx /usr/local/bin/H5ToXds*</code>). The binary H5ToXds then should be named e.g. /usr/local/bin/H5ToXds.bin - note the .bin filename extension! The script ''also'' uses RAM to speed up processing; it uses it for fast storage of the temporary CBF file that H5ToXds/eiger2cbf/hdf2mini-cbf writes, and that each parallel thread ("processor") of XDS reads. The amount of additional RAM this requires is modest (about (number of pixels)*(number of threads) bytes). | For faster processing, the [[Eiger#A_script_for_faster_XDS_processing_of_CBF-converted Eiger data|shell script]] below should be copied to /usr/local/bin/H5ToXds and made executable (<code>chmod a+rx /usr/local/bin/H5ToXds*</code>). The binary H5ToXds then should be named e.g. /usr/local/bin/H5ToXds.bin - note the .bin filename extension! The script ''also'' uses RAM to speed up processing; it uses it for fast storage of the temporary CBF file that H5ToXds/eiger2cbf/hdf2mini-cbf writes, and that each parallel thread ("processor") of XDS reads. The amount of additional RAM this requires is modest (about (number of pixels)*(number of threads) bytes). | ||
<pre> | <pre> | ||
#!/bin/bash | #!/bin/bash | ||