Installation: Difference between revisions
(47 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
This article has some little script snippets that should ease the installation of XDS and related programs ([[XDS-viewer]], [[xdsstat]], [[XDSGUI|xdsgui]]). It is assumed that binaries (or links to binaries) should go to /usr/local/bin . | This article has some little script snippets that should ease the installation of XDS and related programs like ([[XDS-viewer]], [[xdsstat]], [[XDSGUI|xdsgui]]). It is assumed that binaries (or links to binaries) should go to /usr/local/bin . This means that you should do this as the administrator (root), or have sudo rights (Mac, and some Linux distros). | ||
== Linux == | == Linux == | ||
Log in as root - we need write permission for /usr/local/bin . | '''Log in as root''' - we need write permission for /usr/local/bin . | ||
=== [[XDS]] package === | |||
If you are an academic user, as root | |||
<pre> | <pre> | ||
cd /usr/local/bin | cd /usr/local/bin | ||
Line 11: | Line 12: | ||
ln -sf XDS-INTEL64_Linux_x86_64/* . | ln -sf XDS-INTEL64_Linux_x86_64/* . | ||
</pre> | </pre> | ||
In case the [ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch above site] is unavailable, there is a [https://strucbio.biologie.uni-konstanz.de/pub/xds backup site]. | |||
=== other programs: [[XDS-Viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]] === | |||
... can be downloaded from https://{{SERVERNAME}}/pub/linux_bin/ . A simple way to obtain them is to create an empty folder, "cd" to that folder and then (as root) | |||
wget https://{{SERVERNAME}}/pub/linux_bin/get_folder.sh | |||
chmod +x get_folder.sh | |||
chmod +x | ./get_folder.sh | ||
You may repeat this, as root, every now and then, to update the programs. | |||
=== tools and helper programs for [[generate_XDS.INP]] and [[XDSGUI]], and libraries === | |||
* check and installation of Unix tools for [[generate_XDS.INP]] is shown at [[Generate_XDS.INP#Dependencies]] | |||
* helper programs for [[XDSGUI]] are listed and discussed at [[XDSGUI#Dependencies]]. Nota bene: XDSGUI needs [[generate_XDS.INP]] unless you already have a working XDS.INP. | |||
Since XDSGUI depends on graphics packages that may need to be installed, you can check the xdsgui binary with | Since XDSGUI depends on graphics packages that may need to be installed, you can check the xdsgui binary with | ||
ldd `which xdsgui` | ldd `which xdsgui` | grep found | ||
and that should show you the libraries it | and that should show you the libraries it it didn't find. Your friendly system administrator will then work out the specific commands to install those libraries. | ||
For RedHat-type distributions, that would typically be e.g. | |||
yum | yum provides libXfixes.so.3 | ||
the output of which will tell you that this is in the libXfixes RPM. | |||
If in the case of Ubuntu you don't know the name of the package that provides a certain library, the | |||
sudo apt-file search <name-of-library> | |||
command should find it for you (where <code><name-of-library></code> could e.g. be <code>libGLU.so.1</code>). This needs a one-time installation by | |||
sudo apt install apt-file | |||
sudo apt-file update | |||
Ubuntu 18.04: if libQtOpenGL.so.4 is missing, use <code>sudo apt install libqt4-opengl</code>. | |||
Ubuntu 20.04: if libQtOpenGL.so.4 is missing, use | |||
sudo add-apt-repository -y ppa:rock-core/qt4 | |||
sudo apt update | |||
sudo apt install libqt4-opengl | |||
But it would be better to use the [https://strucbio.biologie.uni-konstanz.de/pub/linux_bin/xdsgui_qt5 Qt5 binary of XDSGUI]. | |||
<code>xxdiff</code> is likely available for your distribution - google for it, or try e.g. | |||
yum -y install xxdiff | |||
on RHEL6/CentOS6/SL6 systems, or | |||
apt install xxdiff | |||
on Ubuntu. If Ubuntu 20.04 doesn't find it, use the latest from http://archive.ubuntu.com/ubuntu/pool/universe/x/xxdiff/ | |||
An <code>xxdiff</code> binary that runs on CentOS7 (RHEL7, SL7) is provided by the <code>get_folder.sh</code> script. | |||
Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed. | |||
For [[Eiger]] data processing, <code>H5ToXDS</code> should be installed as [[Eiger|explained]], and <code>h5dump</code> should be installed by e.g. <code>yum -y install hdf5</code>. Furthermore, you want the [https://www.dectris.com/company/news/newsroom/news-details/process-eiger-data-with-xds-fast Neggia] library or the [https://github.com/DiamondLightSource/durin Durin plugin]. | |||
== | == Mac OSX == | ||
Attention: [[generate_XDS.INP]] requires the [https://developer.apple.com/download/more/ Xcode command line tools] to be installed (free of charge). In my experience, you have to explicitly agree to the License terms when running a Command Line Tool (e.g. <code>strings</code>) for the first time. | |||
< | |||
</ | |||
Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin. (As an alternative that does not require root, one may create a directory $HOME/bin and use that for the programs and links. That would also require modification of the $PATH, by a one-time <code>echo 'export PATH=$PATH:$HOME/bin' >> ~/.profile</code>) | |||
So, prepare the following steps by opening the Terminal window, and then | |||
sudo su # this will make you root, and ask for your password | |||
mkdir /usr/local/bin # only if /usr/local/bin was not created before | |||
=== | === [[XDS]] package === | ||
If you are an academic user, | |||
echo you may have to "sudo su" first, to obtain administrator permissions! | |||
cd /usr/local/bin | cd /usr/local/bin | ||
curl -L -o - ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-OSX_64.tar.gz | tar xzvf - | |||
ln -sf XDS-OSX_64/* . | |||
Starting with macOS Catalina, you also need | |||
xattr -dr XDS-OSX_64/* | |||
if you get a security error popup (google "xattr com.apple.quarantine catalina"). | |||
=== other programs: [[XDS-Viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]] === | |||
sudo | ... can be downloaded from https://{{SERVERNAME}}/pub/mac_bin/ . A simple way to obtain them is is to open a Terminal, and type: | ||
sudo su | |||
which asks for the password and gives you administrator permissions. Then, | |||
# any folder would do; here we use /usr/local/bin/mac_bin | |||
mkdir -p /usr/local/bin/mac_bin | |||
cd /usr/local/bin/mac_bin | |||
curl -O -R https://{{SERVERNAME}}/pub/mac_bin/get_folder.sh | |||
chmod +x get_folder.sh | |||
./get_folder.sh | |||
If you want to update the programs some time later, just run these commands again. | |||
In order to enable [[Eiger]] data processing, the <code>get_folder.sh</code> script | |||
# copies Dectris' Neggia library into /usr/local/lib64 | |||
# copies [[Eiger#General_aspects|eiger2cbf-osx]] into /usr/local/bin, but under the name <code>H5ToXds</code> | |||
# copies [https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8/hdf5-1.8.7/bin/mac-intel-x86_64/utilities/h5dump version 1.8.7] of <code>h5dump</code> into /usr/local/bin | |||
Lastly, it is useful (for e.g. XDSGUI) to install <code>xxdiff</code> from http://furius.ca/downloads/xxdiff/releases/macosx/ | |||
=== Troubleshooting === | === Troubleshooting === | ||
Line 190: | Line 107: | ||
ls -l /usr/local/bin/xdsgui | ls -l /usr/local/bin/xdsgui | ||
ls -l /usr/local/bin/xds-viewer | ls -l /usr/local/bin/xds-viewer | ||
# in the same way, check the other programs that should be used | |||
</pre> | </pre> | ||
returns a message like <code>file or directory not found</code> then this means that you do not have the symlink. But even if these commands do not return errors, you should also check the targets of the symlink: | returns a message like <code>file or directory not found</code> then this means that you do not have the symlink. But even if these commands do not return errors, you should also check the targets of the symlink: | ||
Line 197: | Line 115: | ||
</pre> | </pre> | ||
Again, these commands should not return an error message. If they do, the programs are not installed in the location where the symlink points to. You'll have to either install the programs properly (in /Applications) or make the symlink point to the correct location. | Again, these commands should not return an error message. If they do, the programs are not installed in the location where the symlink points to. You'll have to either install the programs properly (in /Applications) or make the symlink point to the correct location. | ||
A quick way is also | |||
which generate_XDS.INP xdsgui xds xds-viewer xdsstat xdscc12 | |||
and this should return a line for each of the programs asked for. | |||
== Windows == | == Windows == | ||
The programs of the XDS package as well as XDSGUI, XDSSTAT and so on may be run within the [ | The programs of the [[XDS]] package as well as [[XDSGUI]], [[XDSSTAT]], [[XDSCC12]] and so on may be run on 64bit Windows 10 within the [https://docs.microsoft.com/windows/wsl/install-win10 Linux Subsystem for Windows] (WSL). This (easily!) installs a Ubuntu environment, which supports the apt package manager, so in principle any software available for Ubuntu may be installed (or other distros, see [https://blogs.msdn.microsoft.com/commandline/2017/05/11/new-distros-coming-to-bashwsl-via-windows-store/]). | ||
Graphical Linux programs require installation of a X server like [https://sourceforge.net/projects/xming/files/latest/download Xming] on the Windows host. To use | Graphical Linux programs require installation of a X server like [https://sourceforge.net/projects/xming/files/latest/download Xming] or [https://sourceforge.net/projects/vcxsrv VcXsvr] on the Windows host. There are helpful writeups (e.g. [http://wsl-guide.org/en/latest/]). To use the X server, one has to say in the shell window | ||
export DISPLAY=:0 | export DISPLAY=:0 | ||
XDSGUI | before running the program. Running XDSGUI under WSL's Ubuntu may require installation of a few packages with e.g. | ||
sudo apt-get install | sudo apt-get install libgomp1 libqtgui4 | ||
and [[generate_XDS.INP]] requires a number of packages (some of which only for specific detectors!); I'd start with | (see [[XDSGUI#Libraries_and_software_that_the_program_depends_on|XDSGUI]] and above). | ||
[[generate_XDS.INP]] requires a number of packages (some of which are only for specific detectors!); I'd start with | |||
sudo apt-get install coreutils binutils gawk sed bc grep | sudo apt-get install coreutils binutils gawk sed bc grep | ||
and if required, also install python and hdf5-tools. | and if required, also install python and hdf5-tools. | ||
XDSSTAT and the conversion to MTZ files by XDSCONV require a CCP4 installation accessible by WSL. In principle, CCP4 may be installed within WSL (have not tested this), or on the Windows host. | |||
[[File:Xds_win.PNG|1000px]] | |||
(Screenshot provided by Gustavo Lima) | |||
A detailed [https://strucbio.biologie.uni-konstanz.de/pub/xds%20%20prog%20installation%20in%20WSL%20(Lata%20Panicker).pdf writeup for Ubuntu 20.04 LTS] was provided by Dr Lata Panicker, SO(G), BARC, India. | |||
== See also == | |||
[[Cluster Installation]] |
Revision as of 17:20, 22 October 2020
This article has some little script snippets that should ease the installation of XDS and related programs like (XDS-viewer, xdsstat, xdsgui). It is assumed that binaries (or links to binaries) should go to /usr/local/bin . This means that you should do this as the administrator (root), or have sudo rights (Mac, and some Linux distros).
Linux
Log in as root - we need write permission for /usr/local/bin .
XDS package
If you are an academic user, as root
cd /usr/local/bin wget -O- ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-INTEL64_Linux_x86_64.tar.gz | tar xzvf - ln -sf XDS-INTEL64_Linux_x86_64/* .
In case the above site is unavailable, there is a backup site.
other programs: XDS-Viewer, XDSSTAT, XDSCC12, XDSGUI, XSCALE_ISOCLUSTER
... can be downloaded from https://wiki.uni-konstanz.de/pub/linux_bin/ . A simple way to obtain them is to create an empty folder, "cd" to that folder and then (as root)
wget https://wiki.uni-konstanz.de/pub/linux_bin/get_folder.sh chmod +x get_folder.sh ./get_folder.sh
You may repeat this, as root, every now and then, to update the programs.
tools and helper programs for generate_XDS.INP and XDSGUI, and libraries
- check and installation of Unix tools for generate_XDS.INP is shown at Generate_XDS.INP#Dependencies
- helper programs for XDSGUI are listed and discussed at XDSGUI#Dependencies. Nota bene: XDSGUI needs generate_XDS.INP unless you already have a working XDS.INP.
Since XDSGUI depends on graphics packages that may need to be installed, you can check the xdsgui binary with
ldd `which xdsgui` | grep found
and that should show you the libraries it it didn't find. Your friendly system administrator will then work out the specific commands to install those libraries.
For RedHat-type distributions, that would typically be e.g.
yum provides libXfixes.so.3
the output of which will tell you that this is in the libXfixes RPM.
If in the case of Ubuntu you don't know the name of the package that provides a certain library, the
sudo apt-file search <name-of-library>
command should find it for you (where <name-of-library>
could e.g. be libGLU.so.1
). This needs a one-time installation by
sudo apt install apt-file sudo apt-file update
Ubuntu 18.04: if libQtOpenGL.so.4 is missing, use sudo apt install libqt4-opengl
.
Ubuntu 20.04: if libQtOpenGL.so.4 is missing, use
sudo add-apt-repository -y ppa:rock-core/qt4 sudo apt update sudo apt install libqt4-opengl
But it would be better to use the Qt5 binary of XDSGUI.
xxdiff
is likely available for your distribution - google for it, or try e.g.
yum -y install xxdiff
on RHEL6/CentOS6/SL6 systems, or
apt install xxdiff
on Ubuntu. If Ubuntu 20.04 doesn't find it, use the latest from http://archive.ubuntu.com/ubuntu/pool/universe/x/xxdiff/
An xxdiff
binary that runs on CentOS7 (RHEL7, SL7) is provided by the get_folder.sh
script.
Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed.
For Eiger data processing, H5ToXDS
should be installed as explained, and h5dump
should be installed by e.g. yum -y install hdf5
. Furthermore, you want the Neggia library or the Durin plugin.
Mac OSX
Attention: generate_XDS.INP requires the Xcode command line tools to be installed (free of charge). In my experience, you have to explicitly agree to the License terms when running a Command Line Tool (e.g. strings
) for the first time.
Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin. (As an alternative that does not require root, one may create a directory $HOME/bin and use that for the programs and links. That would also require modification of the $PATH, by a one-time echo 'export PATH=$PATH:$HOME/bin' >> ~/.profile
)
So, prepare the following steps by opening the Terminal window, and then
sudo su # this will make you root, and ask for your password mkdir /usr/local/bin # only if /usr/local/bin was not created before
XDS package
If you are an academic user,
echo you may have to "sudo su" first, to obtain administrator permissions! cd /usr/local/bin curl -L -o - ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-OSX_64.tar.gz | tar xzvf - ln -sf XDS-OSX_64/* .
Starting with macOS Catalina, you also need
xattr -dr XDS-OSX_64/*
if you get a security error popup (google "xattr com.apple.quarantine catalina").
other programs: XDS-Viewer, XDSSTAT, XDSCC12, XDSGUI, XSCALE_ISOCLUSTER
... can be downloaded from https://wiki.uni-konstanz.de/pub/mac_bin/ . A simple way to obtain them is is to open a Terminal, and type:
sudo su
which asks for the password and gives you administrator permissions. Then,
# any folder would do; here we use /usr/local/bin/mac_bin mkdir -p /usr/local/bin/mac_bin cd /usr/local/bin/mac_bin curl -O -R https://wiki.uni-konstanz.de/pub/mac_bin/get_folder.sh chmod +x get_folder.sh ./get_folder.sh
If you want to update the programs some time later, just run these commands again.
In order to enable Eiger data processing, the get_folder.sh
script
- copies Dectris' Neggia library into /usr/local/lib64
- copies eiger2cbf-osx into /usr/local/bin, but under the name
H5ToXds
- copies version 1.8.7 of
h5dump
into /usr/local/bin
Lastly, it is useful (for e.g. XDSGUI) to install xxdiff
from http://furius.ca/downloads/xxdiff/releases/macosx/
Troubleshooting
If any of the commands
ls -l /usr/local/bin/generate_XDS.INP ls -l /usr/local/bin/xdsgui ls -l /usr/local/bin/xds-viewer # in the same way, check the other programs that should be used
returns a message like file or directory not found
then this means that you do not have the symlink. But even if these commands do not return errors, you should also check the targets of the symlink:
ls -l /Applications/xdsgui.app/Contents/MacOS/xdsgui ls -l /Applications/XDS-Viewer.app/Contents/MacOS/xds-viewer-bin
Again, these commands should not return an error message. If they do, the programs are not installed in the location where the symlink points to. You'll have to either install the programs properly (in /Applications) or make the symlink point to the correct location.
A quick way is also
which generate_XDS.INP xdsgui xds xds-viewer xdsstat xdscc12
and this should return a line for each of the programs asked for.
Windows
The programs of the XDS package as well as XDSGUI, XDSSTAT, XDSCC12 and so on may be run on 64bit Windows 10 within the Linux Subsystem for Windows (WSL). This (easily!) installs a Ubuntu environment, which supports the apt package manager, so in principle any software available for Ubuntu may be installed (or other distros, see [1]).
Graphical Linux programs require installation of a X server like Xming or VcXsvr on the Windows host. There are helpful writeups (e.g. [2]). To use the X server, one has to say in the shell window
export DISPLAY=:0
before running the program. Running XDSGUI under WSL's Ubuntu may require installation of a few packages with e.g.
sudo apt-get install libgomp1 libqtgui4
(see XDSGUI and above).
generate_XDS.INP requires a number of packages (some of which are only for specific detectors!); I'd start with
sudo apt-get install coreutils binutils gawk sed bc grep
and if required, also install python and hdf5-tools.
XDSSTAT and the conversion to MTZ files by XDSCONV require a CCP4 installation accessible by WSL. In principle, CCP4 may be installed within WSL (have not tested this), or on the Windows host.
(Screenshot provided by Gustavo Lima)
A detailed writeup for Ubuntu 20.04 LTS was provided by Dr Lata Panicker, SO(G), BARC, India.