Installing LOFAR software on Ubuntu 12.04

Installing LOFAR software on Ubuntu 12.04

These are some notes on how I installed some LOFAR software on Ubuntu 12.04. They are based on some notes I found on the LOFAR wiki at http://www.lofar.org/operations/doku.php?id=engineering:user_software:ubuntu_12_4.

Install main LOFAR suite
===============================================
N.B. Notes below install to strange directory. Build in one directory, and install everything else into the CASACORE directory (binary)


Dependencies:
libgtkmm-2.4-dev python-matplotlib python-pyfits libatlas-base-dev
mpi-default-bin mpi-default-dev libfreetype6-dev python-setuptools
libxml2-dev libpng12-dev libcfitsio3 libcfitsio3-dev libboost-all-dev
autoconf autoconf-archive autogen automake binutils-dev cmake cmake-curses-gui
cvs doxygen flex gfortran git guile-1.8-dev ipython libblas-dev libblitz0-dev
libboost-all-dev libboost-dev libfftw3-dev libfftw3-doc libgfortran3
libglib2.0-dev libgsl0-dev liblapack-dev liblog4cxx10 liblog4cxx10-dev
libopenmpi-dev libpqxx3-dev libx11-dev mgdiff mpi-default-dev patch pgplot5
python-dev python-numeric python-numpy python-scipy scons subversion-tools
swig bison libbison-dev
tcl tcl-dev tk tk-dev tk8.5-dev tcl8.5-dev
libhdf5-dev (oder: libhdf5-serial-1.8.4 libhdf5-serial-dev)
### "libhdf5-serial" is needed for DAL, it doesn't work with "libhdf5-openmpi"
wcslib-dev liblog4cplus-dev liblog4cplus-1.0-4 cython
###for parmdbplot:
python-sip python-qt4




########## LOFAR software:

Download packages:
##################
  mkdir Downloads
  cd Downloads
  wget ftp://ftp.atnf.csiro.au/pub/software/asap/data/asap_data.tar.bz2

Download and Build Casacore:
############################
  tar -xjvf ../Downloads/asap_data.tar.bz2
(This creates the "data" subdirectory)
  mkdir BuildDir/casacore
  cd BuildDir/casacore
  svn co http://casacore.googlecode.com/svn/trunk source
  mkdir -p build/opt
  cd build/opt
  cmake -DBUILD_TESTING=NO -DCMAKE_INSTALL_PREFIX=/opt/lofar-stuff -DUSE_FFTW3=Yes -DUSE_THREADS=YES  -DUSE_HDF5=YES -DUSE_OPENMP=YES -DDATA_DIR=/opt/lofar-stuff/data  ../../source
make -j12
make install

Install Casacore data:
############################
  cd BuildDir/..
  tar -xjvf Download/asap_data.tar.bz2

Download and Build pyrap
############################
  mkdir /cluster/lofar/BuildDir/pyrap
  cd /cluster/lofar/BuildDir/pyrap
  svn co http://pyrap.googlecode.com/svn/trunk dev-source
  export LOFAR_STUFF_ROOT="/opt/lofar-stuff"
  export PATH="${PATH}:${LOFAR_STUFF_ROOT}/bin/"
  export LD_LIBRARY_PATH="${LOFAR_STUFF_ROOT}/lib/"
  export PYTHONPATH="${LOFAR_STUFF_ROOT}/lib/python/"
  ln -s /opt/soft/lofar-stuff/lib/ /opt/soft/lofar-stuff/lib64
  cd dev-source/
  ./batchbuild-trunk.py --casacore-root=/opt/lofar-stuff --prefix=/opt/lofar-stuff --python-prefix=/opt/lofar-stuff/lib/python

Download and Build casarest
###########################
  mkdir /cluster/lofar/BuildDir/casarest
  cd /cluster/lofar/BuildDir/casarest
  svn co https://svn.astron.nl/casarest/trunk/casarest source
  mkdir build
  cd build
  cmake -DCASACORE_ROOT_DIR=/opt/lofar-stuff -DBUILD_ALL=1 -DCMAKE_INSTALL_PREFIX:PATH=/opt/lofar-stuff ../source
  make -j12
  make install


Download and Build the LOFAR Software
#####################################
Download latest copy of LOFAR software
 mkdir -p build/gnu_opt
 cd build/gnu_opt
 cmake -DCASACORE_ROOT_DIR=/opt/lofar-stuff/ -DBUILD_SHARED_LIBS=ON -DUSE_OPENMP=ON -DBUILD_PACKAGES="ParmDB Calibration DP3 Pipeline MSLofar LofarFT GSM" -DCMAKE_INSTALL_PREFIX:PATH=/opt/lofar-stuff ../../LOFAR/
  make -j12
  make install



Note: I found that the above instructions installed the pyrap python libraries in a separate folder to where LOFAR was looking, so I just found them in lofar-software/lib/python and manually pasted them into the right directory (the one LOFAR regards as PYTHONPATH)

Harvard-style referencing using Latex and Mendeley

I needed to use Harvard-style referencing for my thesis and ran into some trouble as I was using Mendeley to generate my bibliography. The way that worked for me in the end was to use the natbib latex package with the \bibliographystyle{agsm} command. I copied the local version to my working directory and edited the write URL function to

FUNCTION {write.url}
{
    { skip$ }
}

to get rid of the links to webpages that were appearing in my output. I had a lot of unusual (Russian) names in my references which were giving errors, so I edited the final .bbl file (not the .bib created by mendeley) manually to replace troublesome characters and errors.

Installing AIPS on Ubuntu 14.04

The instructions for installing AIPS on Ubuntu 13.04 (http://astronomicalproblems.blogspot.co.uk/2013/05/installing-aips-on-ubuntu-1304.html) work well for 14.04 too. Warning! see the full edit below.

[EDIT]

Some intel processors were giving a little trouble even when following the method linked above. The single offending file was AU7B.FOR, located in 31DEC14/AIPS/SUB (or the equivalent) and the issue was the MONTH parameter declared at the top of the file.

By deleting the MONTH declaration in the FORTAN code and the initialization of MONTH  to JAN, FEB etc. a little further down and replacing it with

character (len=3), dimension(12) :: MONTH
MONTH(1) = 'JAN'
MONTH(2) = 'FEB'
MONTH(3) = 'MAR'
MONTH(4) = 'APR'
MONTH(5) = 'MAY'
MONTH(6) = 'JUN'
MONTH(7) = 'JUL'
MONTH(8) = 'AUG'
MONTH(9) = 'SEP'
MONTH(10) = 'OCT'
MONTH(11) = 'NOV'
MONTH(12) = 'DEC'

you can get AIPS to compile with gfortan as usual. I think the problem relates to old fashioned fortran code, modern gfortran and intel processors (I don't seem to have the same trouble with AMD processors, though they are of different vintages).

The resulting aips installation does compile - but there is a serious problem with IMAGR, and likely other tasks too. The best bet is to download an older version of gfortran compatible with AIPS and use that to compile for a fully functional AIPS installation.

SWAP memory being used handling large datasets (Ubuntu 12.04)

I ran into an issue where SWAP memory was being used in moving around large files in Ubuntu 12.04. This slowed down my computer and I had to restart a couple of times a time. I found the solution at http://askubuntu.com/questions/157793/why-is-swap-being-used-even-though-i-have-plenty-of-free-ram. A similar solution probably works for other distributions.

Installing AIPS on ubuntu 13.04

Just as installing AIPS earlier - you need

        libx11-dev and its dependancies

        x11proto-xext-dev

        libxext-dev

        libxpm-dev

        libncurses5-dev

        libncursesw5-dev

        libbsd-dev

        libedit-dev

I use gcc and gfortran to compile AIPS - you need to specify their location in the AIPS installer (/usr/bin/gfortran is both the fortran compiler and the linker for me, while /usr/bin/gcc compiles for C). I also have trouble with the fortran compiler flags -axWPT and -ip options, so I turn them off. XAS isn't built using these directives however, so you need to go to Y/SERVERS and edit XAS.SHR to remove the use of the axWPT and -ip flags when compiling the TV server.

The above was enough to install AIPS on Ubuntu 12.04 LTS, but for 13.04 you also need to specify the additional C compiler option -DHAVE_LINUX_GLIBC.

I doubt this is an optimal installation - it might be possible to compile with those flags on a different system (I use a sandybridge based laptop with 64-bit ubuntu 13.04 at the moment) and some of the dependencies might not be necessary (especially if you use the -DHAVE_LINUX_GLIBC the bsd dependencies may drop out). It does result in a working AIPS however!

Make sure to add

        sssin        5000/tcp    SSSIN        # AIPS TV
        ssslock        5002/tcp    SSSLOCK        # AIPS TV Lock
        msgserv        5008/tcp    MSGSERV        # AIPS Message Server
        tekserv        5009/tcp    TEKSERV        # AIPS TekServer
        aipsmt0        5010/tcp    AIPSMT0
        aipsmt1        5011/tcp    AIPSMT1
        aipsmt2        5012/tcp    AIPSMT2
        aipsmt3        5013/tcp    AIPSMT3
        aipsmt4        5014/tcp    AIPSMT4
        aipsmt5        5015/tcp    AIPSMT5
        aipsmt6        5016/tcp    AIPSMT6
        aipsmt7        5017/tcp    AIPSMT7

to /etc/services to get AIPS to run properly. To do this you may need to open the file as root.

One last thing - add the aips path to your profile so you can run it straight from the terminal. For me the line was ". /home/colm/aips/LOGIN.SH; $CDTST". The CDTST bit at the end is needed if you intend to compile custom AIPS tasks.

Working on a website for my Dad - not sure if posting it here will help it get higher on google, but check out www.corkhomemaintenance.ie

AIPS on Ubuntu 11.10

Contrary to what was said in the bananas mailing list - AIPS does work fine on newer versions of Ubuntu. There are a few hacks I needed to get it working on my new laptop though.

You need the dev files for the bds library - libbsd-dev and libedit-dev seemed to work for this.

The first thing was getting XAS - the tv server - to compile okay. There are a few options that seem to give my compiler trouble (I use gcc and gfortran). These are the -ip and -axWPT (or something like that) options. I needed to remove these from the options list when specifying the compilers to use.

But XAS is build from Y/SERVERS and has it's own build file with more options in it. Search for XAS.SHR and look for local options (or search for the link options that cause trouble). I had to remove the -axWPT option.

XAS should then build successfully, and the rest of the installation should work okay.