Installing LOFAR software on a mac (Mavericks)

I've installed some of the LOFAR data reduction software on a macbook pro running OSX 10.9 (Mavericks). This is probably not the best method in the world to install the LOFAR software, but it works reasonably well and I haven't seen any other documentation for a mac installation so it might be useful to someone.

I roughly followed the same procedure as for Ubuntu, but with a few differences.

First I used macports to download as many of the dependencies I listed here as I could find. The full list of macports I installed is

  binutils @2.24_0 (active)

  boost @1.56.0_1+no_single+no_static+python27 (active)

  bzip2 @1.0.6_0 (active)

  cctools @855_1+llvm35 (active)

  cctools-headers @855_0 (active)

  cfitsio @3.340_0 (active)

  cloog @0.18.2_0 (active)

  cmake @3.0.2_0 (active)

  curl @7.38.0_0+ssl (active)

  curl-ca-bundle @7.38.0_0 (active)

  cython_select @0.1_0 (active)

  db48 @4.8.30_3 (active)

  db_select @0.1_2 (active)

  expat @2.1.0_0 (active)

  fftw-3 @3.3.4_0 (active)

  gcc49 @4.9.1_0 (active)

  gcc_select @0.1_8 (active)

  gettext @0.19.2_0 (active)

  gmp @6.0.0_1 (active)

  hdf5 @1.8.13_0+cxx (active)

  icu @53.1_1 (active)

  isl @0.13_0 (active)

  ld64 @236.3_1+llvm35 (active)

  libarchive @3.1.2_0 (active)

  libcxx @183506_1 (active)

  libedit @20121213-3.0_0 (active)

  libffi @3.1_4 (active)

  libgcc @4.9.1_0 (active)

  libiconv @1.14_0 (active)

  libidn @1.29_0 (active)

  libmpc @1.0.2_0 (active)

  libxml2 @2.9.1_0 (active)

  llvm-3.5 @3.5-r216817_0+assertions (active)

  llvm_select @1.0_0 (active)

  lzo2 @2.06_0 (active).

  mpfr @3.1.2-p10_0 (active)

  ncurses @5.9_2 (active)

  nosetests_select @0.1_0 (active)

  openssl @1.0.1i_0 (active)

  pcre @8.35_0 (active)

  py27-cython @0.21_0 (active)

  py27-nose @1.3.1_0 (active)

  py27-numpy @1.9.0_0 (active)

  py27-pyfits @3.3_0 (active)

  py27-pywcs @1.11-4.8.2_1 (active)

  py27-scipy @0.14.0_0+gcc48 (active)

  py27-setuptools @6.0.2_0 (active)

  python27 @2.7.8_2 (active)

  python_select @0.3_3 (active)

  sqlite3 @3.8.6_0 (active)

  SuiteSparse @4.2.1_3 (active)

  swig @3.0.2_0 (active)

  swig-python @3.0.2_0 (active)

  wcslib @4.23_1 (active)

  xz @5.0.7_0 (active)

  zlib @1.2.8_0 (active)

I then downloaded and compiled FFTW and boost manually. This was required for boost because macports has trouble building boost with the gnu compiler suite and the version of boost built with clang doesn't seem to work with the LOFAR software. I thought something similar was happening with FFTW, but it might have turned out to be a linking error on my behalf - so you can try to link the macports-provided installation of FFTW instead of building your own if you prefer.

The commands I used were:

Boost:

./bootstrap.sh --prefix=/Users/admin/boost --withpython=/opt/local/bin/python2.7 --with-toolset=gcc

./b2

./b2 install

FFTW

./configure --prefix=/Users/admin/fftw --enable-threads

make -j 8

make install

./configure --prefix=/Users/admin/fftwf --enable-float --enable-threads

make -j 8

make install

Issues

There were lots of failures the first time I tried to compile the LOFAR software. Refer to the Ubuntu instructions for the gist of what I was doing. I installed everything LOFAR-related into the /opt/lofar-software directory.

cmake -DBUILD_TESTING=NO -DCMAKE_INSTALL_PREFIX=/opt/lofar-software -DUSE_FFTW3=Yes -DUSE_THREADS=YES -DUSE_HDF5=YES -DUSE_OPENMP=YES -DDATA_DIR=/opt/lofar-software/data -DFFTW3_ROOT_DIR=/usr/local/lib ../..

Installed as usual

export LOFAR_ROOT="/opt/lofar-software"

export PATH="${PATH}:${LOFAR_ROOT}/bin/"

export LD_LIBRARY_PATH="${LOFAR_ROOT}/lib/"

export PYTHONPATH="{LOFAR_ROOT}/lib/python/"

Next is the installation of PYRAP. PYRAP has not been updated in a while and you need to apply the patch indicated here to make sure it installs correctly on a mac (the bug reporter links the patch, the plus lines in the diff file indicate new lines to be added, the minuses lines that need to be taken away). Basically you need to ensure it only tries to build for one architecture. I did this manually by going to the affected setup.py files (you'll spot them if you try to build), but you might be able to do this a lot faster if you know more about patching than me. Then build with

./batchbuild-trunk.py --casacore-root=/opt/lofar-software/ --prefix=/opt/lofar-software/ --python-prefix=/opt/lofar-software/lib/python --extra-root=/opt/local/ --boost-lib=boost_python-mt –universal=x86_64

(You might notice the boost_python-mt library here. That is from the macports boost install (clang) which caused no problems in this step.)

Compiled and installed CASAREST as usual

Added data folder as usual.

I needed to patch some LOFAR files to get them to compile successfully. I now suspect some of these patches might have been unnecessary as I was using incorrect settings.

Necessary patches:

1. LCS/pytools. tconvert.cc needs to have the python.h library included at the start (#include <python.h>.

2. Baselinefitting.cc needs to have sincosf replaced with separate calls to sin and cos (apparently sincosf is not available in OSX's version of the standard math library?).  I commented out the original function and replaced it as follows:

//sincosf(phase_ij_obs - phase_ij_model, &sin_dphase, &cos_dphase);
sin_dphase = sin(phase_ij_obs - phase_ij_model);
cos_dphase = cos(phase_ij_obs - phase_ij_model);

Probably unnecessary patches:

The first three patches are probably redundant if you remember to include the DUSE_SHMEM=OFF compiler option.

1. LCS/common. dlmalloc.c. Comment out reference around Have_usr_include_malloc_h, thus forcing the code to define its own.

2. shmem_alloc.cc comment out def of "union semum".

3. shmem_alloc.h comment out inclusion of malloc.h.

I don't know why these didn't re-occur:

4. image2d.cpp needs to have #include <cstring> added.

5. tickset.h needs to have exp10 functions replaced with pow(10,x) functions.

6. imagewidget.cpp same exp10 patch as above.

7. processcommander.cpp. Add definition for HOST_NAME_MAX (a large long).

Note that the CMAKE/variants/GNU.cmake file needs to be edited to point to the macports compilers as follows:

set(GNU_C         /opt/local/bin/gcc)      # GNU C compiler
set(GNU_CXX       /opt/local/bin/g++)      # GNU C++ compiler
set(GNU_Fortran   /opt/local/bin/gfortran) # GNU Fortran compiler
set(GNU_ASM       /opt/local/bin/gcc)      # GNU assembler

After applying the necessary patches above, the following compilation command should work when called from /build/gnu_opt.

cmake -DCASACORE_ROOT_DIR=/opt/lofar-software/ -DBUILD_SHARED_LIBS=ON -DUSE_OPENMP=ON -DBUILD_PACKAGES="ParmDB Calibration DP3 Pipeline MSLofar LofarFT GSM" -DCMAKE_INSTALL_PREFIX:PATH=/opt/lofar-software/ -DCMAKE_C_COMPILER=/opt/local/bin/gcc -DCMAKE_CXX_COMPILER=/opt/local/bin/g++ -DFFTW3_LIBRARY=/Users/admin/fftw/lib/libfftw3.a -DFFTW3F_LIBRARY=/Users/admin/fftwf/lib/libfftw3f.a -DFFTW3F_THREADS_LIBRARY=/Users/admin/fftwf/lib/libfftw3f_threads.a -DPYTHON_EXECUTABLE:FILEPATH=/opt/local/bin/python2.7 -DF2PY_EXECUTABLE=/opt/local/bin/f2py-2.7 -DUSE_LOG4CPLUS=OFF -DUSE_SHMEM=OFF -DBOOST_ROOT=/Users/admin/boost -Wno-dev ../..

Then run make and make install to finish up.

One final thing to do is to add the following lines to the .profile file in your home directory.

source /opt/lofar-software/lofarinit.sh

export DYLD_LIBRARY_PATH="$LD_LIBRARY_PATH"

This should allow LOFAR commands to be run from any terminal.