Installation: Difference between revisions

From XDSwiki
Jump to navigation Jump to search
No edit summary
Line 1: Line 1:
This article has some little script snippets that should ease the installation of XDS and related programs ([[XDS-viewer]], [[xdsstat]], [[XDSGUI|xdsgui]]). It is assumed that binaries (or links to binaries) should go to /usr/local/bin .
This article has some little script snippets that should ease the installation of XDS and related programs like ([[XDS-viewer]], [[xdsstat]], [[XDSGUI|xdsgui]]). It is assumed that binaries (or links to binaries) should go to /usr/local/bin .




Line 11: Line 11:
  ln -sf XDS-INTEL64_Linux_x86_64/* .
  ln -sf XDS-INTEL64_Linux_x86_64/* .
</pre>
</pre>
To get the script [[generate_XDS.INP]] from its XDSwiki article,
 
<pre>
=== other programs: [[XDS-Viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]] ===
  cd /usr/local/bin
... can be downloaded from ftp://turn5.biologie.uni-konstanz.de/pub/linux_bin/ . A simple way to obtain them is
wget http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/generate_XDS.INP -O - | \
wget ftp://turn5.biologie.uni-konstanz.de/pub/linux_bin/get_folder.sh
  sed -e s/\&nbsp\;/\ /g -e s/\&gt\;/\>/g -e s/\&lt\;/\</g -e s/amp\;//g -e s/\&quot\;/\"/g -e s/\&\#\1\6\0\;/\ /g | \
  chmod +x get_folder.sh
  sed '/# end of generate_XDS.INP/,$d' | awk '/^#/,/rm -f tmp1 tmp2/' > generate_XDS.INP
./get_folder.sh
  chmod +x generate_XDS.INP
 
</pre>
Please note that to get the <code>strings</code> command, which is used by [[generate_XDS.INP]], on some Linux distributions (e.g. FC23) you need to install the <code>binutils</code> RPM package.
Please note that to get the <code>strings</code> command, which is used by [[generate_XDS.INP]], on some Linux distributions (e.g. FC23) you need to install the <code>binutils</code> RPM package.
To get [[XDS-Viewer]],
<pre>
cd /usr/local/bin
wget ftp://turn5.biologie.uni-konstanz.de/pub/xds-viewer-0.6.64bit -O xds-viewer
chmod a+x xds-viewer 
ln -sf xds-viewer xdsviewer
</pre>
To get [[XDSSTAT]],
<pre>
cd /usr/local/bin
wget ftp://turn5.biologie.uni-konstanz.de/pub/xdsstat-linux64.bz2 -O xdsstat.bz2
bunzip2 -f xdsstat.bz2
chmod a+x xdsstat
</pre>
To get [[XDSCC12]],
<pre>
cd /usr/local/bin
wget ftp://turn5.biologie.uni-konstanz.de/pub/xdscc12.rhel6.64 -O xdscc12
chmod a+x xdscc12
</pre>
To get [[XDSGUI]],
<pre>
cd /usr/local/bin
wget ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.rhel6.64 -O xdsgui
chmod a+x xdsgui
</pre>


Since XDSGUI depends on graphics packages that may need to be installed, you can check the xdsgui binary with
Since XDSGUI depends on graphics packages that may need to be installed, you can check the xdsgui binary with
Line 53: Line 25:
Ubuntu 14 users: please read http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/XDSGUI#Installation if you have a problem with missing libmng2.
Ubuntu 14 users: please read http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/XDSGUI#Installation if you have a problem with missing libmng2.


It is useful to install xxdiff or one of its alternatives (see below). The package is likely available for your distribution - google for it, or try e.g.
Of course, this still requires installation of <code>xxdiff</code> (or alternative), and for .h5 data: HDF5 package, and <code>H5ToXds</code> and/or [https://www.dectris.com/company/news/newsroom/news-details/process-eiger-data-with-xds-fast Neggia] library.
 
<code>xxdiff</code> is likely available for your distribution - google for it, or try e.g.
  yum -y install xxdiff
  yum -y install xxdiff
on RHEL6/CentOS6/SL6 systems, or
on RHEL6/CentOS6/SL6 systems, or
Line 59: Line 33:
on Ubuntu.
on Ubuntu.


On Centos7 (RHEL7, SL7) I went to http://rpmfind.net/linux/rpm2html/search.php?query=xxdiff and found that the Fedora 20 RPM can be installed using
An <code>xxdiff</code> binary that runs on CentOS7 (RHEL7, SL7) is provided by the <code>get_folder.sh</code> script.
yum install ftp://rpmfind.net/linux/fedora/linux/releases/20/Everything/x86_64/os/Packages/x/xxdiff-3.2-20.fc20.x86_64.rpm
Perhaps xxdiff will be available on EPEL some day - maybe it is available on RPMforge; I didn't check.


Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed.
Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed.


For [[Eiger]] data processing, H5ToXDS should be installed as [[Eiger|explained]], and <code>h5dump</code> should be installed by e.g. <code>yum -y install hdf5</code>.
For [[Eiger]] data processing, <code>H5ToXDS</code> should be installed as [[Eiger|explained]], and <code>h5dump</code> should be installed by e.g. <code>yum -y install hdf5</code>.


=== current versions of programs outside the official XDS distribution ===
... can be downloaded from ftp://turn5.biologie.uni-konstanz.de/pub/linux_bin/ . A simple way is
wget -r ftp://turn5.biologie.uni-konstanz.de/pub/linux_bin/
chmod a+x turn5.biologie.uni-konstanz.de/pub/linux_bin/*
# next, move the binaries to the destination on your machine
Of course, this still requires installation of xxdiff (or alternative), and for .h5 data: HDF5 package, and H5ToXds and/or Neggia library.


== Mac (Intel) ==
== Mac (Intel) ==


Attention: [[generate_XDS.INP]] requires the [https://developer.apple.com/download/more/ Xcode command line tools] to be installed (free of charge).  
Attention: [[generate_XDS.INP]] requires the [https://developer.apple.com/download/more/ Xcode command line tools] to be installed (free of charge). In my experience, you have to explicitly agree to the License terms when running a Command Line Tool (e.g. <code>strings</code>) for the first time.


You also have to explicitly agree to the License terms when running a Command Line Tool (e.g. <code>strings</code>) for the first time.
Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin.


=== personal installation (experimental writeup) ===
Thus, prepare the steps by opening the Terminal window, and then
su                    # this will ask for password
mkdir /usr/local/bin  # only if /usr/local/bin was not created before


'''Preparation''':
* make sure your $PATH includes your $HOME/bin - the programs will be put into that directory. To achieve this, open a Terminal window, and then
<pre>
cd $HOME
echo 'export PATH=$PATH:$HOME/bin' >> ~/.profile
</pre>
</pre>
(I'm not sure if it should be .profile or .bashrc; I have seen it work with .profile but not with .bashrc on a Leopard system)
'''To install [[XDS]]''',  
* create the $HOME/bin and $HOME/Applications directories. Still in the Terminal window, type
<pre>
<pre>
mkdir $HOME/bin
  cd /usr/local/bin
mkdir $HOME/Applications
</pre>
'''To install [[generate_XDS.INP]]''':
<pre>
  cd $HOME/bin
curl -L -o - http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/generate_XDS.INP | \
  sed -e s/\&nbsp\;/\ /g -e s/\&gt\;/\>/g -e s/\&lt\;/\</g -e s/amp\;//g -e s/\&quot\;/\"/g -e s/\&\#\1\6\0\;/\ /g | \
  sed '/# end of generate_XDS.INP/,$d' | awk '/^#/,/rm -f tmp1 tmp2/' > generate_XDS.INP
chmod +x generate_XDS.INP
</pre>
'''To install [[XDS]]''':
<pre>
cd $HOME/bin
  curl -L -o - ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-OSX_64.tar.gz | tar xvf -
  curl -L -o - ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-OSX_64.tar.gz | tar xvf -
  ln -sf XDS-OSX_64/* .
  ln -sf XDS-OSX_64/* .
</pre>
</pre>


'''To install [[XDS-Viewer]]''', click https://sourceforge.net/project/platformdownload.php?group_id=239755 , open up the downloaded XDS-Viewer.app by clicking, and then (once you see it in the Finder), go to the Terminal and
=== other programs: [[XDS-Viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]]  ===
cp -pr /Volumes/XDS-Viewer/XDS-Viewer.app/ $HOME/Applications
... can be downloaded from ftp://turn5.biologie.uni-konstanz.de/pub/mac_bin/ . A simple way to obtain them is
ln -s $HOME/Applications/XDS-Viewer.app/Contents/MacOS/xds-viewer-bin $HOME/bin/xds-viewer
   
   
'''To install [[XDSSTAT]]''',
  curl -O -R ftp://turn5.biologie.uni-konstanz.de/pub/mac_bin/get_folder.sh
<pre>
  chmod +x get_folder.sh
cd $HOME/bin
  ./get_folder.sh
  curl -L -o xdsstat.bz2 ftp://turn5.biologie.uni-konstanz.de/pub/xdsstat-i386-mac.bz2
bunzip2 -f xdsstat.bz2
chmod a+x xdsstat
</pre>
'''To install [[XDSCC12]]''',
<pre>
cd $HOME/bin
curl -L -o xdscc12 ftp://turn5.biologie.uni-konstanz.de/pub/xdscc12-mac
  chmod a+x xdscc12
 
</pre>
'''To install [[XDSGUI]]''', please download from ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.dmg, open up the downloaded xdsgui.app by clicking, and then (once you see it in the Finder), go to the Terminal and
cp -pr /Volumes/xdsgui/xdsgui.app/ $HOME/Applications
ln -s $HOME/Applications/xdsgui.app/Contents/MacOS/xdsgui $HOME/bin/xdsgui
Please also check [[XDSGUI#Installation]].
 
It is useful to install xxdiff from http://furius.ca/downloads/xxdiff/releases/macosx/xxdiff-4.0b1.osx.dmg (or newer). After downloading, open up the downloaded xxdiff.app by clicking, and then (once you see it in the Finder), go to the Terminal and
  cp -pr /Volumes/xxdiff/xxdiff.app/ $HOME/Applications
ln -s $HOME/Applications/xxdiff.app/Contents/MacOS/xxdiff $HOME/bin/xxdiff
 
(the names of the files may not be entirely correct in these two lines)
 
=== system-wide installation (requires root permission) ===


To get [[generate_XDS.INP]]:
Using the Finder window which should have opened automagically, install XDSGUI and XDS-Viewer graphically, by pulling them into the Applications folder.  
<pre>
cd /usr/local/bin
sudo sh -c 'curl -L -o - http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/generate_XDS.INP | \
  sed -e s/\&nbsp\;/\ /g -e s/\&gt\;/\>/g -e s/\&lt\;/\</g -e s/amp\;//g -e s/\&quot\;/\"/g -e s/\&\#\1\6\0\;/\ /g | \
  sed '/# end of generate_XDS.INP/,$d' | awk '/^#/,/rm -f tmp1 tmp2/' > generate_XDS.INP'
sudo chmod +x generate_XDS.INP
</pre>
 
To get [[XDS]]:
<pre>
cd /usr/local/bin
sudo sh -c 'curl -L -o - ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-OSX_64.tar.gz | tar xvf -'
sudo ln -sf XDS-OSX_64/* .
</pre>
 
To get [[XDS-Viewer]], click https://sourceforge.net/project/platformdownload.php?group_id=239755 , install in the usual graphical way to your /Applications and then
sudo ln -s /Applications/XDS-Viewer.app/Contents/MacOS/xds-viewer-bin /usr/local/bin/xds-viewer
To get [[XDSSTAT]],
<pre>
cd /usr/local/bin
sudo curl -L -o xdsstat.bz2 ftp://turn5.biologie.uni-konstanz.de/pub/xdsstat-i386-mac.bz2
sudo bunzip2 -f xdsstat.bz2
sudo chmod a+x xdsstat
</pre>
 
To get [[XDSCC12]],
<pre>
cd /usr/local/bin
sudo curl -L -o xdscc12 ftp://turn5.biologie.uni-konstanz.de/pub/xdscc12-mac
sudo chmod a+x xdscc12
</pre>
 
To get [[XDSGUI]], please download from ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.dmg and then
sudo ln -s /Applications/xdsgui.app/Contents/MacOS/xdsgui /usr/local/bin/xdsgui
Please also check [[XDSGUI#Installation]].


It is useful to install xxdiff from http://furius.ca/downloads/xxdiff/releases/macosx/
It is useful to install xxdiff from http://furius.ca/downloads/xxdiff/releases/macosx/

Revision as of 13:41, 23 July 2018

This article has some little script snippets that should ease the installation of XDS and related programs like (XDS-viewer, xdsstat, xdsgui). It is assumed that binaries (or links to binaries) should go to /usr/local/bin .


Linux

Log in as root - we need write permission for /usr/local/bin .

To get the XDS package if you are an academic user,

 cd /usr/local/bin
 wget -O- ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-INTEL64_Linux_x86_64.tar.gz | tar xzvf -
 ln -sf XDS-INTEL64_Linux_x86_64/* .

other programs: XDS-Viewer, XDSSTAT, XDSCC12, XDSGUI, XSCALE_ISOCLUSTER

... can be downloaded from ftp://turn5.biologie.uni-konstanz.de/pub/linux_bin/ . A simple way to obtain them is

wget ftp://turn5.biologie.uni-konstanz.de/pub/linux_bin/get_folder.sh
chmod +x get_folder.sh
./get_folder.sh

Please note that to get the strings command, which is used by generate_XDS.INP, on some Linux distributions (e.g. FC23) you need to install the binutils RPM package.

Since XDSGUI depends on graphics packages that may need to be installed, you can check the xdsgui binary with

ldd `which xdsgui`

and that should show you the libraries it found, and more importantly, those that it didn't find. Your friendly system administrator will then work out the specific commands to install those libraries; on RedHat-type distributions that would typically be e.g. yum provides libXfixes.so.3 the output of which will tell you that this is in the libXfixes RPM. Ubuntu 14 users: please read http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/XDSGUI#Installation if you have a problem with missing libmng2.

Of course, this still requires installation of xxdiff (or alternative), and for .h5 data: HDF5 package, and H5ToXds and/or Neggia library.

xxdiff is likely available for your distribution - google for it, or try e.g.

yum -y install xxdiff

on RHEL6/CentOS6/SL6 systems, or

apt-get install xxdiff

on Ubuntu.

An xxdiff binary that runs on CentOS7 (RHEL7, SL7) is provided by the get_folder.sh script.

Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed.

For Eiger data processing, H5ToXDS should be installed as explained, and h5dump should be installed by e.g. yum -y install hdf5.


Mac (Intel)

Attention: generate_XDS.INP requires the Xcode command line tools to be installed (free of charge). In my experience, you have to explicitly agree to the License terms when running a Command Line Tool (e.g. strings) for the first time.

Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin.

Thus, prepare the steps by opening the Terminal window, and then

su                    # this will ask for password
mkdir /usr/local/bin  # only if /usr/local/bin was not created before

To install XDS,

 cd /usr/local/bin
 curl -L -o - ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-OSX_64.tar.gz | tar xvf -
 ln -sf XDS-OSX_64/* .

other programs: XDS-Viewer, XDSSTAT, XDSCC12, XDSGUI, XSCALE_ISOCLUSTER

... can be downloaded from ftp://turn5.biologie.uni-konstanz.de/pub/mac_bin/ . A simple way to obtain them is

curl -O -R ftp://turn5.biologie.uni-konstanz.de/pub/mac_bin/get_folder.sh
chmod +x get_folder.sh
./get_folder.sh

Using the Finder window which should have opened automagically, install XDSGUI and XDS-Viewer graphically, by pulling them into the Applications folder.

It is useful to install xxdiff from http://furius.ca/downloads/xxdiff/releases/macosx/

For Eiger data processing, eiger2cbf-osx should be sym-linked into a directory in your $PATH, but under the name H5ToXds, and h5dump should be installed from a .dmg file.

Troubleshooting

If any of the commands

ls -l /usr/local/bin/generate_XDS.INP
ls -l /usr/local/bin/xdsgui
ls -l /usr/local/bin/xds-viewer
# in the same way, check the other programs that should be used

returns a message like file or directory not found then this means that you do not have the symlink. But even if these commands do not return errors, you should also check the targets of the symlink:

ls -l /Applications/xdsgui.app/Contents/MacOS/xdsgui
ls -l /Applications/XDS-Viewer.app/Contents/MacOS/xds-viewer-bin 

Again, these commands should not return an error message. If they do, the programs are not installed in the location where the symlink points to. You'll have to either install the programs properly (in /Applications) or make the symlink point to the correct location.

A quick way is also

which generate_XDS.INP xdsgui xds xds-viewer xdsstat xdscc12

and this should return a line for each of the programs asked for.

Windows

The programs of the XDS package as well as XDSGUI, XDSSTAT, XDSCC12 and so on may be run on 64bit Windows 10 within the Linux Subsystem for Windows (WSL). This (easily!) installs a Ubuntu 14.04.5 LTS environment, which supports the apt package manager, so in principle any software available for Ubuntu may be installed. Windows 10 Creators Update (CU) allows to install (or update to) Ubuntu 16.04 (or other distros, see [1]), and fixes many bugs.

Graphical Linux programs require installation of a X server like Xming or VcXsvr on the Windows host. There are helpful writeups (e.g. [2],[3]). To use the X server, one has to say in the shell window

export DISPLAY=:0

before running the program. Running XDSGUI under WSL's Ubuntu 14.04 requires installation of a few packages with

sudo apt-get install libgomp1 libqtgui4

and generate_XDS.INP requires a number of packages (some of which are only for specific detectors!); I'd start with

sudo apt-get install coreutils binutils gawk sed bc grep 

and if required, also install python and hdf5-tools.

The CU fixes many aspects of the interoperability between Windows and Linux, so is recommended. Without it, only single-processor xds runs smoothly, whereas the parallel xds_par stops with an error message.

XDSSTAT and the conversion to MTZ files by XDSCONV require a CCP4 installation accessible by WSL. In principle, CCP4 may be installed within WSL (have not tested this), or on the Windows host.

Xds win.PNG

(Screenshot provided by Gustavo Lima)

See also

Cluster Installation