2,652
edits
LIB: Difference between revisions
Jump to navigation
Jump to search
→Existing implementations
(first version) |
|||
| (22 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
The possibility of using external libraries (that are loaded at runtime) has been available in C/C++ | The possibility of using external libraries (that are loaded at runtime) has been available in C/C++ for a long time, but in Fortran became only available as of Fortran2003. | ||
In the case of XDS, frame-reading and computation can be separated starting with version November-2016. This allows users/companies to develop their own specialized frame-reading libraries, and relieves the XDS maintainers from implementing even more file formats. | In the case of XDS, frame-reading and computation can be separated starting with version November-2016. This allows users/companies to develop their own specialized frame-reading libraries, and relieves the XDS maintainers from implementing even more file formats. The feature was developed in order to be able to natively (i.e. without temporary intermediates) read the HDF5 files written for data from the [[Eiger]] detector. | ||
In the following, small examples are given for | In the following, small examples are given for | ||
* how a program ("'''host'''") may use an existing external library, e.g. the dectris-neggia library ([https://github.com/dectris/neggia source]; [https://www.dectris.com/ | * how a program ("'''host'''") may use an existing external library, e.g. the dectris-neggia library ([https://github.com/dectris/neggia source]; [https://www.dectris.com/company/news/newsroom/news-details/process-eiger-data-with-xds-fast pre-compiled]) | ||
* how an external library ("'''client'''") may be implemented that XDS can use | * how an external library ("'''client'''") may be implemented that XDS can use | ||
The glue between host and client is based on http://cims.nyu.edu/~donev/Fortran/DLL/DLL.Forum.txt . There should be no need to change this, unless the interface design changes. | The [[LIB#Glue_code|glue code]] between host and client is based on http://cims.nyu.edu/~donev/Fortran/DLL/DLL.Forum.txt . There should be no need to change this, unless the interface design changes. | ||
The interface was designed by Markus | The interface was designed by Markus Mathes (Dectris), Vittorio Boccone (Dectris) and Kay Diederichs. It is supposed to be generic, i.e. useful beyond XDS. In particular, the 4096 bytes of the info_array can be utilized to obtain and use header information (e.g. wavelength, distance, axes specifications and other metadata). | ||
== | The interface has several [[LIB#Existing_implementations|implementations]]. | ||
== Host code example == | |||
<pre> | <pre> | ||
! | ! Example test program for existing external library | ||
! This should be saved in a file called test_generic_host.f90 | |||
! Kay Diederichs 4/2017 | ! Kay Diederichs 4/2017 | ||
! | ! | ||
| Line 19: | Line 22: | ||
! ifort -qopenmp generic_data_plugin.f90 test_generic_host.f90 -o test_generic_host | ! ifort -qopenmp generic_data_plugin.f90 test_generic_host.f90 -o test_generic_host | ||
! or | ! or | ||
! gfortran -O -fopenmp -ldl generic_data_plugin.f90 | ! gfortran -O -fopenmp -ldl generic_data_plugin.f90 test_generic_host.f90 -o test_generic_host | ||
! run with | ! run with | ||
! ./test_generic_host < test.in | ! ./test_generic_host < test.in | ||
| Line 30: | Line 33: | ||
PROGRAM | PROGRAM test_generic_host | ||
USE generic_data_plugin, ONLY: library, firstqm, lastqm, nx, ny, is_open, & | USE generic_data_plugin, ONLY: library, firstqm, lastqm, nx, ny, is_open, & | ||
generic_open, generic_get_header, generic_get_data, generic_close | generic_open, generic_get_header, generic_get_data, generic_close | ||
| Line 51: | Line 54: | ||
! set some more module variables | ! set some more module variables | ||
firstqm=INDEX(actnam,'?') | firstqm=INDEX(actnam,'?') ! qm means question mark | ||
lastqm =INDEX(actnam,'?',BACK=.TRUE.) | lastqm =INDEX(actnam,'?',BACK=.TRUE.) | ||
len =LEN_TRIM(actnam) | len =LEN_TRIM(actnam) | ||
| Line 114: | Line 117: | ||
</pre> | </pre> | ||
== | == Client code example == | ||
The following code should be saved as file test_generic_client.f90 : | |||
<pre> | <pre> | ||
! This reads single data files which have a header of 7680 bytes | ! This reads single data files which have a header of 7680 bytes | ||
| Line 122: | Line 126: | ||
! or | ! or | ||
! gfortran -c -fpic test_generic_client.f90 && ld -shared test_generic_client.o -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/ -lgfortran -o libtest_generic_client.so | ! gfortran -c -fpic test_generic_client.f90 && ld -shared test_generic_client.o -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/ -lgfortran -o libtest_generic_client.so | ||
! (attention: the above - from "gfortran" to "libtest_generic_client.so" - is one looong line) | |||
! The resulting file can be used with a LIB=./libtest_generic_client.so line in XDS.INP, and enables | |||
! reading of data files with a 7680 bytes header plus 1024*1024 pixels of integer data, without any record structure. | |||
MODULE plugin_test_mod | MODULE plugin_test_mod | ||
CHARACTER :: fn_template*132='' | CHARACTER :: fn_template*132='',cformat*6='(i4.4)' | ||
INTEGER :: lenfn,firstqm,lastqm | INTEGER :: lenfn,firstqm,lastqm | ||
END MODULE | END MODULE | ||
| Line 147: | Line 154: | ||
firstqm=INDEX(fn_template,'?') | firstqm=INDEX(fn_template,'?') | ||
lastqm =INDEX(fn_template,'?',BACK=.TRUE.) | lastqm =INDEX(fn_template,'?',BACK=.TRUE.) | ||
WRITE(cformat(3:5),'(i1,a1,i1)')lastqm-firstqm+1,'.',lastqm-firstqm+1 | |||
END SUBROUTINE plugin_open | END SUBROUTINE plugin_open | ||
! | ! | ||
| Line 177: | Line 185: | ||
! local variables | ! local variables | ||
INTEGER k,i,dummy | INTEGER k,i,dummy | ||
CHARACTER :: fn*132 | CHARACTER :: fn*132 | ||
fn=fn_template | fn=fn_template | ||
IF (frame_number>0) WRITE(fn(firstqm:lastqm),cformat) frame_number | IF (frame_number>0) WRITE(fn(firstqm:lastqm),cformat) frame_number | ||
! -qopenmp compile option needs to be used otherwise race in writing fn | ! -qopenmp compile option needs to be used otherwise race in writing fn | ||
| Line 198: | Line 205: | ||
</pre> | </pre> | ||
== Glue code == | |||
== | |||
The following code should be saved as generic_data_plugin.f90 . | The following code should be saved as generic_data_plugin.f90 . | ||
<pre> | <pre> | ||
| Line 220: | Line 226: | ||
! THE USE OR OTHER DEALINGS IN THE SOFTWARE. | ! THE USE OR OTHER DEALINGS IN THE SOFTWARE. | ||
! | ! | ||
! | ! For more information, please refer to <http://unlicense.org/> | ||
! | ! | ||
! | ! | ||
| Line 654: | Line 660: | ||
end module generic_data_plugin | end module generic_data_plugin | ||
</pre> | </pre> | ||
== Problems == | |||
Programs using dlclose (which is what the glue code does) may display | |||
Program received signal SIGSEGV, Segmentation fault. | |||
at termination. Using gdb, this reveals | |||
<pre> | |||
#0 0x00000000410c08b0 in ?? () | |||
#1 0x00000000406acbc2 in __nptl_deallocate_tsd () from /lib64/libpthread.so.0 | |||
#2 0x00000000406acdd3 in start_thread () from /lib64/libpthread.so.0 | |||
#3 0x00000000409b873d in clone () from /lib64/libc.so.6 | |||
</pre> | |||
Google returns https://github.com/apple/cups/issues/4410 and https://bugzilla.redhat.com/show_bug.cgi?id=1065695 when searching for similar problems. Overall, this appears to be harmless and in fact I don't know how to change the code to make the segfault disappear - I'd appreciate a patch! | |||
== Existing implementations == | |||
# [https://github.com/dectris/neggia Dectris Neggia-plugin] to read HDF5 written by Dectris-supplied software of Eiger detectors | |||
# [https://github.com/DiamondLightSource/durin Diamond's Durin-plugin] to read HDF5 written by Eiger detectors at Diamond (and presumably elsewhere); latest binaries for MacOS and Linux (RHEL6) as well as example XDS.INP and source at https://github.com/DiamondLightSource/durin/releases/latest | |||
# [https://git.embl.de/nikolova/xds-zcbf/ EMBL-Hamburg's zcbf-plugin] to read gzip-compressed CBF files without intermediate file | |||
== See also == | |||
# https://rosettacode.org/wiki/Call_a_function_in_a_shared_library#GNU_Fortran_on_Linux | |||