2,684
edits
Eiger: Difference between revisions
no edit summary
No edit summary |
No edit summary |
||
| Line 8: | Line 8: | ||
The number of pixels of the Eiger 16M is three times higher than that of the Pilatus 6M, but since the Eiger firmware update in November 2015, the ("bit shufflle LZ4") compression of the .h5 files containing data is better than that of CBF files, which mostly compensates for the increased number of pixels. However, the size of the *master.h5 file from a Eiger 16M experiment at SLS X06SA is more than 300MB, ''no matter how many frames are collected''. It is therefore advisable to compress (by ~75%) the *master.h5 files on-site, before transferring them home using disk or internet. The fastest (parallel) program with the best compression that I found is [http://lbzip2.org lbzip2] (available from the EPEL repository for RHEL clones). It is supposedly fully compatible with bzip2. | The number of pixels of the Eiger 16M is three times higher than that of the Pilatus 6M, but since the Eiger firmware update in November 2015, the ("bit shufflle LZ4") compression of the .h5 files containing data is better than that of CBF files, which mostly compensates for the increased number of pixels. However, the size of the *master.h5 file from a Eiger 16M experiment at SLS X06SA is more than 300MB, ''no matter how many frames are collected''. It is therefore advisable to compress (by ~75%) the *master.h5 files on-site, before transferring them home using disk or internet. The fastest (parallel) program with the best compression that I found is [http://lbzip2.org lbzip2] (available from the EPEL repository for RHEL clones). It is supposedly fully compatible with bzip2. | ||
== A benchmark == | |||
Any comparisons should be based on a common dataset. I downloaded from https://www.dectris.com/datasets.html their latest dataset | |||
ftp://dectris.com/EIGER_16M_Nov2015.tar.bz2 (900 frames) and processed it on a single unloaded CentOS7.2 64bit machine with dual Intel(R) Xeon(R) CPU E5-2667 v2 @ 3.30GHz , HT enabled (showing 32 processors in /proc/cpuinfo), on a local XFS filesystem (all defaults), with four JOBs and 12 PROCESSORS (the XDS.INP that Dectris provides suggests 8 JOBs of 12 PROCESSORS, but I changed that). | |||
The timing, using the latest XDS (BUILT=20151231), is on the first run | |||
INIT: elapsed wall-clock time 12.0 sec | |||
COLSPOT: elapsed wall-clock time 44.9 sec | |||
INTEGRATE: total elapsed wall-clock time 65.1 sec | |||
CORRECT: elapsed wall-clock time 2.9 sec | |||
Total elapsed wall-clock time for XDS 133.6 sec | |||
When I repeat this, I get | |||
Total elapsed wall-clock time for XDS 128.3 sec | |||
Once again: | |||
Total elapsed wall-clock time for XDS 129.3 sec | |||
So a bit of cache-warming helps, but not much. This machine has 64GB RAM. From the output of "top", the highest memory usage occurs during INTEGRATE, when each of the mintegrate_par processes consumes up to 7.4% of the memory. In other words, in this way less than 20GB of total memory are used. "top" shows a CPU consumption around (on average) 4 times 650%. | |||
The number of JOBs and PROCESSORs could be optimized. I tried 6 JOBs and get | |||
Total elapsed wall-clock time for XDS 120.1 sec | |||
so there's still some room for improvement. | |||
== A script for faster XDS processing of Eiger data == | == A script for faster XDS processing of Eiger data == | ||