NumPy and SciPy rely on BLAS and LAPACK for basic linear algebra functionality like matrix-vector multiplication, linear system solves, or routines for eigenvalue computation. The Intel Math Kernel Library (MKL) is a mathematics library providing amongst other things fast and multithreaded implementations of BLAS and LAPACK. In this blog post, I describe how to compile NumPy and SciPy with the Intel compilers using Intel MKL on Linux. Note NumPy and SciPy can be linked to MKL without the Intel compilers by providing the proper linker options to, e.g., GCC, and I will briefly explain this as well.
Introduction
In this post, I describe how I built NumPy 1.10.4 and SciPy 0.17.0 for Python 2.7.10 with Intel compilers v16.0.0.109 and Intel MKL 11.3 Update 2 on 64bit Linux in Bash. There are (at least) two other recent guides describing how to build NumPy and SciPy using Intel software. One of these guides is from Intel itself and is called Numpy/Scipy with Intel MKL and Intel Compilers. The other guide was written by Jan-Philip Gehrcke and it is called Building numpy and scipy with Intel compilers and Intel MKL on a 64 bit machine. Both of these guides did not work for me because of issues with the SciPy build and below I elaborate on the fixes for the problems I encountered. The Intel article can be used to adapt this guide for builds on 32bit systems.
Note if you want to use Intel MKL, then you do need to link against the MKL library. You do not need to compile with the Intel compilers. The linker flags for the GCC compilers can be found with the aid of the Math Kernel Library Link Line Advisor.
In this guide, I assume the Intel compilers are already installed on the system. We will test if the builds work by running the test suites of NumPy and SciPy and to that end, I assume the Python test framework Nose is installed. The guide is structured as follows: first of all, we will need to set the correct environment flags so that the Intel compiler executables and libraries are found. Next, we will compile NumPy before installing a hack that allows us to compile SciPy at last.
Updating the Environment Variables
The binaries for the Intel compiler suite are installed in non-standard locations so whenever you want to run an Intel compiler you must make sure the executables are found by updating the environment variable PATH; whenever you want to run programs linked against Intel libraries you must make sure that the linker can find the libraries by updating the environment variable LD_LIBRARY_PATH. Throughout this guide I assume the Intel software can be found in /opt/intel. Intel provides a bash script updating the environment variables PATH and LD_LIBRARY_PATH appropriately. It can be found in /opt/intel/bin and is called iccvars.sh. Type the following command in Bash to use it:
source /opt/intel/bin/iccvars.sh -arch intel64 -platform linux
You can check if the variables were set correctly by trying to call the Intel C Compiler:
icc -v
Building NumPy
Download the NumPy source code, unpack it, and change into the folder containing the source code, e.g.,
cd /tmp # download numpy-1.10.4.tar.gz tar -zxf numpy-1.10.4.tar.gz cd numpy-1.10.4
We will customize our NumPy build by providing a file called site.cfg and by modifying the preset compiler flags. Create a file called site.cfg, paste the following text into it, and save the file afterwards:
[mkl] library_dirs = /opt/intel/mkl/lib/intel64/ include_dirs = /opt/intel/mkl/include/ mkl_libs = mkl_rt lapack_libs =
The compiler flags for the C compiler can be found in numpy/distutils/intelccompiler.py. Open this file and look for the Intel x86_64 compiler options in the Python class IntelEM64TCCompiler (EM64T stands for the Extended Memory 64 Technology and is nowadays called Intel 64). The string assigned to the variable self.cc_exe contains the most relevant compiler flags:
self.cc_exe = ('icc -m64 -fPIC -fp-model strict -O3 '
'-fomit-frame-pointer -openmp -xSSE4.2')
Replace the last flag:
self.cc_exe = ('icc -m64 -fPIC -fp-model strict -O3 '
'-fomit-frame-pointer -openmp -xHost')
The new flag ensures the code is optimized for the host machine. If you omit this step and if your CPU does not provide the SSE 4.2 instruction set, then NumPy or SciPy will crash at run time when an illegal instruction is encountered. Repeat this procedure for the Fortran compiler flags in numpy/distutils/fcompiler/intel.py; look for the IntelEM64TFCompiler and replace
def get_flags_arch(self):
return ['-xSSE4.2']
with
def get_flags_arch(self):
return ['-xHost']
You are now ready to compile NumPy:
python setup.py \
config --compiler=intelem --fcompiler=intelem \
build_clib --compiler=intelem --fcompiler=intelem \
build_ext --compiler=intelem --fcompiler=intelem
NumPy can be installed in a location of your choice with the --prefix flag, e.g., to install in your home directory, type
python setup.py install --prefix=~
If you install NumPy in a non-standard location, then you have to update the environment variable PYTHONPATH afterwards so that SciPy finds the existing NumPy installation. Here, I assume NumPy was installed in the user's home directory:
export PYTHONPATH=~/lib64/python2.7/site-packages export PATH=$PATH:~/bin
The update of the PATH variable is necessary because NumPy installed the program f2py. To check the build, run the NumPy test suite outside of the source directory, e.g.,
cd /tmp python -c 'import numpy; numpy.test()'
Building SciPy
SciPy is partially written in C++ and unfortunately, the build system calls c++ to compile the C++ code. By default, c++ is a symbolic link to the system's standard C++ compiler which causes linker errors when the test suite is run, e.g.,
Traceback (most recent call last):
File "solve_gep.py", line 9, in <module>
import scipy.sparse
File "/home/user/lib64/python2.7/site-packages/scipy/sparse/__init__.py", line 214, in <module>
from .csr import *
File "/home/user/lib64/python2.7/site-packages/scipy/sparse/csr.py", line 13, in <module>
from ._sparsetools import csr_tocsc, csr_tobsr, csr_count_blocks, \
ImportError: /home/user/lib64/python2.7/site-packages/scipy/sparse/_sparsetools.so: undefined symbol: _intel_fast_memset
The author of the blog Computing Star figured the problem out here and fixed it by manually recompiling the files in question. In this guide, we will use another approach. Whenever you call a program in Linux, the shell searches the program in the directories listed in the environment variable PATH in the order of appearance in this list. For the fix, the idea is to create a new directory containing an executable file called c++ calling the Intel C++ compiler and to put the path of this new directory at the beginning of the list of directories in PATH. Create a directory called, e.g., mycpp in /tmp:
mkdir /tmp/mycpp
Create the file c++ in it with the following contents:
$ cat /tmp/mycpp/c++ #!/bin/bash icpc $@
Make the file executable:
chmod u+x /tmp/mycpp/c++
Update PATH:
export PATH=/tmp/mycpp:$PATH
Tell Bash to update its cache:
hash -r
Test the result:
$ c++ -v icpc version 16.0.0 (gcc version 4.9.3 compatibility)
We can now build SciPy.
Download and upack the SciPy source code, change into the directory with the source code, and execute the same build commands as for Numpy:
cd /tmp
wget https://github.com/scipy/scipy/releases/download/v0.17.0/scipy-0.17.0.tar.gz
tar -zxf scipy-0.17.0.tar.gz
cd scipy-0.17.0
python setup.py \
config --compiler=intelem --fcompiler=intelem \
build_clib --compiler=intelem --fcompiler=intelem \
build_ext --compiler=intelem --fcompiler=intelem
python setup.py install --prefix=~
Leave the directory with the SciPy source code and start the test suite:
cd /tmp python -c 'import scipy; scipy.test()'