Installation: Difference between revisions
(10 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
Line 25: | Line 26: | ||
=== other programs: [[XDS-Viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]], [[spot2pdb]], [[generate_XDS.INP]] === | === other programs: [[XDS-Viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]], [[spot2pdb]], [[generate_XDS.INP]] === | ||
... can be downloaded from https://{{SERVERNAME}}/pub/linux_bin/ . A simple way to obtain them is to create an empty folder, "cd" to that folder and then | ... can be downloaded from https://{{SERVERNAME}}/pub/linux_bin/ . A simple way to obtain them is to create an empty folder, "cd" to that folder and then | ||
wget https://{{SERVERNAME}}/pub/linux_bin/get_folder.sh | wget -N https://{{SERVERNAME}}/pub/linux_bin/get_folder.sh | ||
chmod +x get_folder.sh | chmod +x get_folder.sh | ||
./get_folder.sh | ./get_folder.sh | ||
Line 72: | Line 73: | ||
Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed. | Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed. | ||
For [[Eiger]] data processing, <code>H5ToXDS</code> should be installed as [[Eiger|explained]], and <code>h5dump</code> should be installed by e.g. <code>yum -y install hdf5</code> (Ubuntu: <code>apt install hdf5-tools</code>). Furthermore, you want the [https://www.dectris.com/company/news/newsroom/news-details/process-eiger-data-with-xds-fast Neggia] library or the [https://github.com/DiamondLightSource/durin Durin plugin]. | For [[Eiger]] data processing, a h5dump binary (version 1.10 req'd for HDF5 data from DLS) and <code>H5ToXDS</code> is installed by the <code>get_folder.sh</code> script into /usr/local/bin. If the script is not used, <code>H5ToXDS</code> should be installed as [[Eiger|explained]], and <code>h5dump</code> should be installed by e.g. <code>yum -y install hdf5</code> (Ubuntu: <code>apt install hdf5-tools</code>) - but make sure that h5dump is version 1.10 or higher when processing HDF5 data from DLS. | ||
Furthermore, you want the [https://www.dectris.com/company/news/newsroom/news-details/process-eiger-data-with-xds-fast Neggia] library (does not work for HDF5 data from DLS) or the [https://github.com/DiamondLightSource/durin Durin plugin]. These may be installed through the <code>get_folder.sh</code> script, in <code>/usr/local/lib64</code>. | |||
== Mac OSX == | == Mac OSX == | ||
Attention: [[generate_XDS.INP]] requires the [https://developer.apple.com/download/more/ Xcode command line tools] to be installed (free of charge): open a Terminal window, and enter <code>sudo xcode-select --install</code>. | Attention: [[generate_XDS.INP]] requires the [https://developer.apple.com/download/more/ Xcode command line tools] to be installed (free of charge): open a Terminal window, and enter <code>sudo xcode-select --install</code> (for Big Sur, the download has a size of about half a GB; the downloader is much too pessimistic about the time it takes). | ||
Although CCP4 is not required by XDS itself, some of the programs (XDSSTAT, and MTZ file creation in XDSCONV) use CCP4 tools or libraries. Therefore, CCP4 should be installed. Open a Terminal and check the existence of the file .zprofile , and its content:<pre> | |||
kay@Kays-MacBook-Air-2018 ~ % ls -l .zprofile | |||
-rw-r--r-- 1 kay staff 107 7 Mär 09:41 .zprofile | |||
kay@Kays-MacBook-Air-2018 ~ % cat .zprofile | |||
source /Users/kay/.profile | |||
# Added by CCP4 package manager: | |||
. '/Applications/ccp4-7.1/bin/ccp4.setup-sh' | |||
</pre>If the file does not exist, create it with the contents indicated above; you have to adapt it to your user name and the CCP4 version. | |||
Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the installation tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin. (As an alternative that does not require root for installation, one may create a directory $HOME/bin and use that for the programs and links. That would also require modification of the $PATH, by a one-time <code>echo 'export PATH=$PATH:$HOME/bin' >> ~/.profile</code>) | Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the installation tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin. (As an alternative that does not require root for installation, one may create a directory $HOME/bin and use that for the programs and links. That would also require modification of the $PATH, by a one-time <code>echo 'export PATH=$PATH:$HOME/bin' >> ~/.profile</code>) | ||
Line 109: | Line 121: | ||
In order to enable [[Eiger]] data processing, the <code>get_folder.sh</code> script | In order to enable [[Eiger]] data processing, the <code>get_folder.sh</code> script | ||
# copies Dectris' Neggia library into /usr/local/lib64 | # copies Dectris' Neggia library and DLS's Durin library into <code>/usr/local/lib64</code> | ||
# copies [ | # copies [https://www.globalphasing.com GlobalPhasing's] <code>hdf2mini-cbf</code> into <code>/usr/local/bin</code>, but under the name <code>H5ToXds</code> (consider getting a [https://www.globalphasing.com/autoproc/ autoPROC] license!) | ||
# copies [https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1. | # copies <code>h5dump</code> [https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.1/bin/unix/hdf5-1.12.1-Std-macos11_64-clang.tar.gz] into <code>/usr/local/bin</code> | ||
A [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-macOS10.13.dmg 10.13] Qt5 version of XDS-viewer (compiled by T. Hauß) works on macOS Catalina, and is installed by <code>get_folder.sh</code>. Older Macs need [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-0.6 this version]; newer macOS should install [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-macOS10.15.dmg 10.15]. For XDSGUI, the [https://{{SERVERNAME}}/pub/mac_bin/xdsgui_qt4.dmg Qt4 version] is still the default, but a [https://{{SERVERNAME}}/pub/mac_bin/xdsgui_qt5.dmg Qt5 version] is available. | A [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-macOS10.13.dmg 10.13] Qt5 version of XDS-viewer (compiled by T. Hauß) works on macOS Catalina, and is installed by <code>get_folder.sh</code>. Older Macs need [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-0.6 this version]; newer macOS should install [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-macOS10.15.dmg 10.15]. For XDSGUI, the [https://{{SERVERNAME}}/pub/mac_bin/xdsgui_qt4.dmg Qt4 version] is still the default, but a [https://{{SERVERNAME}}/pub/mac_bin/xdsgui_qt5.dmg Qt5 version] is available. | ||
Lastly, it is useful (for e.g. XDSGUI) to install <code>xxdiff</code> from http://furius.ca/downloads/xxdiff/releases/macosx/ . | Lastly, it is useful (for e.g. XDSGUI) to install <code>xxdiff</code> from http://furius.ca/downloads/xxdiff/releases/macosx/ . I had to go to System Preferences > Security & Privacy > Privacy and add <code>xxdiff</code> to the programs with full disk access. | ||
=== Troubleshooting === | === Troubleshooting === | ||
Line 143: | Line 155: | ||
Graphical Linux programs require installation of a X server like [https://sourceforge.net/projects/xming/files/latest/download Xming] or [https://sourceforge.net/projects/vcxsrv VcXsvr] or [https://mobaxterm.mobatek.net/ Mobaxterm] on the Windows host. There are helpful writeups (e.g. [https://wiki.ubuntu.com/WSL#Running_Graphical_Applications] and [http://wsl-guide.org/en/latest/]). VcXsrc seems to have a problem with coot for some people (see [https://www.jiscmail.ac.uk/cgi-bin/wa-jisc.exe?A2=COOT;6ef4a967.2011 WSL2 installation] and responses). To use the X server with WSL, one has to say in the shell window | Graphical Linux programs require installation of a X server like [https://sourceforge.net/projects/xming/files/latest/download Xming] or [https://sourceforge.net/projects/vcxsrv VcXsvr] or [https://mobaxterm.mobatek.net/ Mobaxterm] on the Windows host. There are helpful writeups (e.g. [https://wiki.ubuntu.com/WSL#Running_Graphical_Applications] and [http://wsl-guide.org/en/latest/]). VcXsrc seems to have a problem with coot for some people (see [https://www.jiscmail.ac.uk/cgi-bin/wa-jisc.exe?A2=COOT;6ef4a967.2011 WSL2 installation] and responses). To use the X server with WSL, one has to say in the shell window | ||
export DISPLAY=:0 | export DISPLAY=:0 | ||
before running the program. Running XDSGUI under WSL's Ubuntu may require installation of a few packages with e.g. | before running the program. Running [[XDSGUI]] under WSL's Ubuntu 20.04 may require installation of a few packages with e.g. | ||
sudo apt-get install libgomp1 libqtgui4 | sudo apt-get install libgomp1 libqtgui4 binutils | ||
(see [[XDSGUI#Libraries_and_software_that_the_program_depends_on|XDSGUI]] and above). | (see [[XDSGUI#Libraries_and_software_that_the_program_depends_on|XDSGUI]] and above). If required, also install python and hdf5-tools. | ||
XDSSTAT and the conversion to MTZ files by XDSCONV require a CCP4 installation accessible by WSL. In principle, CCP4 may be installed within WSL (have not tested this), or on the Windows host. | XDSSTAT and the conversion to MTZ files by XDSCONV require a CCP4 installation accessible by WSL. In principle, CCP4 may be installed within WSL (have not tested this), or on the Windows host. | ||
Line 170: | Line 178: | ||
sudo chown MYWSLNAME /opt/xtal | sudo chown MYWSLNAME /opt/xtal | ||
A few libraries are needed before CCP4 installation (tcsh is needed by CCP4 | A few libraries are needed before CCP4 installation (tcsh is needed by CCP4; most of the others are graphics libraries; binutils provides <code>strings</code>): | ||
sudo apt install tcsh libqt5opengl5 libqt5printsupport5 libqt5gui5 libxcb-render0 libxcb-shm0 libglu1-mesa libgomp1 | sudo apt install tcsh libqt5opengl5 libqt5printsupport5 libqt5gui5 libxcb-render0 libxcb-shm0 libglu1-mesa libgomp1 binutils | ||
When installing CCP4, let it "modify | When installing CCP4, let it "modify command line environment" for you (on one of the very first screens of the installation GUI). If you forgot this, insert a line into your ~/.bashrc : | ||
source /opt/xtal/ccp4-7.1/bin/ccp4.setup-sh | source /opt/xtal/ccp4-7.1/bin/ccp4.setup-sh | ||
afterwards (this is not WSL specific, but is just Linux). I created a file in the Windows filesystem (replace MYUSERNAME with your Windows user name): | afterwards (this is not WSL specific, but is just Linux). I created a file in the Windows filesystem (replace MYUSERNAME with your Windows user name): |
Revision as of 11:14, 27 July 2021
This article has some little script snippets that should ease the installation of XDS and related programs like (XDS-viewer, xdsstat, xdsgui). It is assumed that binaries (or links to binaries) should go to /usr/local/bin . This means that you should do this as the administrator (root), or have sudo rights (Mac, and some Linux distros).
Linux
Log in as root - we need write permission for /usr/local/bin .
XDS package
If you are an academic user, as root (on Ubuntu, use "sudo -i" to become root)
cd /usr/local/bin wget -O- https://xds.mr.mpg.de/XDS-INTEL64_Linux_x86_64.tar.gz | tar xzvf - ln -sf XDS-INTEL64_Linux_x86_64/* .
In case the above website is unavailable, there is a backup site.
other programs: XDS-Viewer, XDSSTAT, XDSCC12, XDSGUI, XSCALE_ISOCLUSTER, spot2pdb, generate_XDS.INP
... can be downloaded from https://wiki.uni-konstanz.de/pub/linux_bin/ . A simple way to obtain them is to create an empty folder, "cd" to that folder and then
wget -N https://wiki.uni-konstanz.de/pub/linux_bin/get_folder.sh chmod +x get_folder.sh ./get_folder.sh
You may repeat this every now and then to update the programs.
As root, the script copies the programs into /usr/local/bin/linux_bin, and links them to /usr/local/bin. As a normal user, they are downloaded to $HOME/bin/linux_bin, and linked to your $HOME/bin. In the latter case, make sure that $HOME/bin is in your $PATH.
The Qt5 graphics library is now the default for xdsgui and xds-viewer. Qt4 versions are available as xdsgui.qt4 and xds-viewer.qt4, for old Linux distributions (and for use with x2goclient, which would otherwise need special action for Qt5 binaries).
tools and helper programs for generate_XDS.INP and XDSGUI, and libraries
- check and installation of Unix tools for generate_XDS.INP is shown at Generate_XDS.INP#Dependencies
- helper programs for XDSGUI are listed and discussed at XDSGUI#Dependencies. Nota bene: XDSGUI needs generate_XDS.INP unless you already have a working XDS.INP.
Since XDSGUI depends on graphics packages that may need to be installed, you can check the xdsgui binary with
ldd `which xdsgui` | grep found
and that should show you the libraries it it didn't find. Your friendly system administrator will then work out the specific commands to install those libraries.
For RedHat-type distributions, that would typically be e.g.
yum provides libXfixes.so.3
the output of which will tell you that this is in the libXfixes RPM.
If in the case of Ubuntu you don't know the name of the package that provides a certain library, the
sudo apt-file search <name-of-library>
command should find it for you (where <name-of-library>
could e.g. be libGLU.so.1
). This needs a one-time installation by
sudo apt install apt-file sudo apt-file update
Ubuntu 18.04: if libQtOpenGL.so.4 is missing, use sudo apt install libqt4-opengl
.
Ubuntu 20.04: if libQtOpenGL.so.4 is missing (for the Qt4 binary of XDSGUI), use
sudo add-apt-repository -y ppa:rock-core/qt4 sudo apt update sudo apt install libqt4-opengl
If the Qt5 libraries are missing, use sudo apt install libqt5gui5 libqt5printsupport5 libqt5opengl5
.
On CentOS7, this would be yum -y install qt5-qtbase-gui qt5-qtbase
.
xxdiff
is likely available for your distribution - google for it, or try e.g.
yum -y install xxdiff
on RHEL6/CentOS6/SL6 systems, or
apt install xxdiff
on Ubuntu. If Ubuntu 20.04 doesn't find it, use the latest from http://archive.ubuntu.com/ubuntu/pool/universe/x/xxdiff/
An xxdiff
binary that runs on CentOS7 (RHEL7, SL7) is provided by the get_folder.sh
script.
Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed.
For Eiger data processing, a h5dump binary (version 1.10 req'd for HDF5 data from DLS) and H5ToXDS
is installed by the get_folder.sh
script into /usr/local/bin. If the script is not used, H5ToXDS
should be installed as explained, and h5dump
should be installed by e.g. yum -y install hdf5
(Ubuntu: apt install hdf5-tools
) - but make sure that h5dump is version 1.10 or higher when processing HDF5 data from DLS.
Furthermore, you want the Neggia library (does not work for HDF5 data from DLS) or the Durin plugin. These may be installed through the get_folder.sh
script, in /usr/local/lib64
.
Mac OSX
Attention: generate_XDS.INP requires the Xcode command line tools to be installed (free of charge): open a Terminal window, and enter sudo xcode-select --install
(for Big Sur, the download has a size of about half a GB; the downloader is much too pessimistic about the time it takes).
Although CCP4 is not required by XDS itself, some of the programs (XDSSTAT, and MTZ file creation in XDSCONV) use CCP4 tools or libraries. Therefore, CCP4 should be installed. Open a Terminal and check the existence of the file .zprofile , and its content:
kay@Kays-MacBook-Air-2018 ~ % ls -l .zprofile -rw-r--r-- 1 kay staff 107 7 Mär 09:41 .zprofile kay@Kays-MacBook-Air-2018 ~ % cat .zprofile source /Users/kay/.profile # Added by CCP4 package manager: . '/Applications/ccp4-7.1/bin/ccp4.setup-sh'
If the file does not exist, create it with the contents indicated above; you have to adapt it to your user name and the CCP4 version.
Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the installation tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin. (As an alternative that does not require root for installation, one may create a directory $HOME/bin and use that for the programs and links. That would also require modification of the $PATH, by a one-time echo 'export PATH=$PATH:$HOME/bin' >> ~/.profile
)
So, prepare the following steps by opening the Terminal window, and then
sudo su # this will make you root, and ask for your password mkdir /usr/local/bin # only if /usr/local/bin was not created before
XDS package
If you are an academic user,
echo you may have to "sudo su" first, to obtain administrator permissions! cd /usr/local/bin curl -L -o - https://xds.mr.mpg.de/XDS-OSX_64.tar.gz | tar xzvf - ln -sf XDS-OSX_64/* .
Starting with macOS Catalina, you also need
xattr -dr XDS-OSX_64/*
if you get a security error popup (google "xattr com.apple.quarantine catalina").
other programs: XDS-viewer, XDSSTAT, XDSCC12, XDSGUI, XSCALE_ISOCLUSTER, generate_XDS.INP
... can be downloaded from https://wiki.uni-konstanz.de/pub/mac_bin/ . A simple way to obtain them is is to open a Terminal, and type:
sudo su
which asks for the password and gives you administrator permissions. Then,
# any folder would do; here we use /usr/local/bin/mac_bin mkdir -p /usr/local/bin/mac_bin cd /usr/local/bin/mac_bin curl -O -R https://wiki.uni-konstanz.de/pub/mac_bin/get_folder.sh chmod +x get_folder.sh ./get_folder.sh
If you want to update the programs some time later, just run these commands again.
In order to enable Eiger data processing, the get_folder.sh
script
- copies Dectris' Neggia library and DLS's Durin library into
/usr/local/lib64
- copies GlobalPhasing's
hdf2mini-cbf
into/usr/local/bin
, but under the nameH5ToXds
(consider getting a autoPROC license!) - copies
h5dump
[1] into/usr/local/bin
A 10.13 Qt5 version of XDS-viewer (compiled by T. Hauß) works on macOS Catalina, and is installed by get_folder.sh
. Older Macs need this version; newer macOS should install 10.15. For XDSGUI, the Qt4 version is still the default, but a Qt5 version is available.
Lastly, it is useful (for e.g. XDSGUI) to install xxdiff
from http://furius.ca/downloads/xxdiff/releases/macosx/ . I had to go to System Preferences > Security & Privacy > Privacy and add xxdiff
to the programs with full disk access.
Troubleshooting
If any of the commands
ls -l /usr/local/bin/generate_XDS.INP ls -l /usr/local/bin/xdsgui ls -l /usr/local/bin/xds-viewer # in the same way, check the other programs that should be used
returns a message like file or directory not found
then this means that you do not have the symlink. But even if these commands do not return errors, you should also check the targets of the symlink, as shown in the output of the ls -l
commands.
Again, these commands should not return an error message. If they do, the programs are not installed in the location where the symlink points to. You'll have to install the programs properly (in /Applications) and make the symlink point to the correct location.
A quick way is also
which generate_XDS.INP xdsgui xds xds-viewer xdsstat xdscc12
and this should return a line for each of the programs asked for.
Windows
The programs of the XDS package as well as XDSGUI, XDSSTAT, XDSCC12 and so on may be run on 64bit Windows 10 within the Linux Subsystem for Windows (WSL, or preferably WSL2). This (easily!) installs e.g. an Ubuntu environment, which supports the apt package manager, so in principle any software available for Ubuntu may be installed (or other distros, see [2]).
A lot of technical detail is at [3] .
CCP4 supports WSL-1 ; see info at http://www.ccp4.ac.uk/download/#os=wsl .
Graphical Linux programs require installation of a X server like Xming or VcXsvr or Mobaxterm on the Windows host. There are helpful writeups (e.g. [4] and [5]). VcXsrc seems to have a problem with coot for some people (see WSL2 installation and responses). To use the X server with WSL, one has to say in the shell window
export DISPLAY=:0
before running the program. Running XDSGUI under WSL's Ubuntu 20.04 may require installation of a few packages with e.g.
sudo apt-get install libgomp1 libqtgui4 binutils
(see XDSGUI and above). If required, also install python and hdf5-tools.
XDSSTAT and the conversion to MTZ files by XDSCONV require a CCP4 installation accessible by WSL. In principle, CCP4 may be installed within WSL (have not tested this), or on the Windows host.
(Screenshot provided by Gustavo Lima)
A detailed writeup for Ubuntu 20.04 LTS was provided by Dr Lata Panicker, SO(G), BARC, India.
WSL2
CCP4 7.1 including coot-0.9.5 works in a Ubuntu 20.04 LTS machine under WSL2 on Windows 10 version 20H2.
After installation of WSL2, I installed MobaXterm as the Xserver. When the Windows firewall asked whether it should allow MobaXterm communication, I answered "yes" for both private and public networks.
If you want to install CCP4 as a regular user, you need (as this is Linux! And replace MYWSLNAME with your WSL user name):
sudo mkdir /opt/xtal sudo chown MYWSLNAME /opt/xtal
A few libraries are needed before CCP4 installation (tcsh is needed by CCP4; most of the others are graphics libraries; binutils provides strings
):
sudo apt install tcsh libqt5opengl5 libqt5printsupport5 libqt5gui5 libxcb-render0 libxcb-shm0 libglu1-mesa libgomp1 binutils
When installing CCP4, let it "modify command line environment" for you (on one of the very first screens of the installation GUI). If you forgot this, insert a line into your ~/.bashrc :
source /opt/xtal/ccp4-7.1/bin/ccp4.setup-sh
afterwards (this is not WSL specific, but is just Linux). I created a file in the Windows filesystem (replace MYUSERNAME with your Windows user name):
cd /mnt/c/Users/MYUSERNAME/ echo "[wsl2]" > .wslconfig echo "kernelCommandLine = vsyscall=emulate" >> .wslconfig
to make the shelx* programs work (see https://github.com/microsoft/WSL/issues/4694 ), and to make "vsyscall=emulate" appear in /proc/cmdline. After creating the file, WSL2 must be restarted with "wsl.exe --shutdown" (or the machine booted).
Performance of coot is good enough for occasional work. XDSGUI and multi-threaded XDS work as expected. For reading .h5 files, the NEGGIA plugin works correctly, but the DURIN plugin currently crashes (a bug has been reported).