Background
The Cray compiler on Spock is based on Clang and uses the hipcc compiler wrapper.
Problem
Building MFIX on Spock with the Cray compiler and CSG-EB support enabled has problems, because:
- Building with the compiler
CXX=hipccfails to build. - Building with the compiler
CXX=$ROCM_PATH/llvm/bin/clang++builds successfully, but the resulting executable crashes.
Procedure
The following steps are intended to work around these difficulties.
Check conan packages [optional]
Running conan search shows what Conan packages are present on your system; should look something like:
❯ conan search
Existing package recipes:
b2/4.5.0
boost/1.75.0
bzip2/1.0.8
catch2/2.13.7
cgal/5.2.1
eigen/3.3.9
gmp/6.2.1
libbacktrace/cci.20210118
libiconv/1.16
m4/1.4.18
meson/0.60.0
mpfr/4.1.0
ninja/1.10.2
pkgconf/1.7.4
taocpp-pegtl/3.2.1
zlib/1.2.11first to see which conan packages are already built (and cached) on your system
Build Conan CSG-EB package
cd subprojects/csg-eb
env CC=$ROCM_PATH/llvm/bin/clang CXX=$ROCM_PATH/llvm/bin/clang++ conan create .Check conan packages [optional]
Running conan search again should now show a cached csg-eb package:
❯ conan search
Existing package recipes:
b2/4.5.0
boost/1.75.0
bzip2/1.0.8
catch2/2.13.7
cgal/5.2.1
csg-eb/d0139be5aa26d08455125c7eb3e8eac4c9cf6742
eigen/3.3.9
gmp/6.2.1
libbacktrace/cci.20210118
libiconv/1.16
m4/1.4.18
meson/0.60.0
mpfr/4.1.0
ninja/1.10.2
pkgconf/1.7.4
taocpp-pegtl/3.2.1
zlib/1.2.11Build MFIX
❯ mkdir builddir; cd builddir
❯ env CC=$(which hipcc) CXX=$(which hipcc) \
cmake ~/mfix/exa/ -DMFIX_MPI=YES -DMFIX_OMP=NO \
-DMFIX_CSG=YES \
-DMFIX_HYPRE=YES \
-DAMReX_TINY_PROFILE=YES \
-DMFIX_GPU_BACKEND=HIP -DAMReX_AMD_ARCH=gfx908 \
-DCMAKE_CXX_FLAGS='-std=c++17' \
-DMPI_COMPILER_FLAGS="--cray-print-opts=all"
❯ cmake --build .Building MFIX should use the (already cached) conan packages.