Skip to content
Snippets Groups Projects
Commit 44c6271c authored by Michele Rosso's avatar Michele Rosso
Browse files

Merge branch 'mr/cmake-fetch'

parents b7a992d7 2987934a
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
docs/source/BuildingCMake.rst
+ 147
− 126
View file @ 44c6271c
Building MFiX-Exa with CMake Building MFiX-Exa with CMake
============================ ============================
There are two options to building MFiX-Exa with cmake: CMake build is a two-step process. First ``cmake`` is invoked to create
configuration files and makefiles in a chosen directory (``builddir``).
Next, the actual build is performed by invoking ``make`` from within ``builddir``.
The CMake build process is summarized as follows:
o **SUPERBUILD (default):** Utilities download and build AMReX as part .. highlight:: console
::
mkdir /path/to/builddir
cd /path/to/builddir
cmake [options] -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel] /path/to/mfix
make
In the above snippet, ``[options]`` indicates one or more options for the
customization of the build, as described later in this section.
If the option ``CMAKE_BUILD_TYPE`` is omitted,
``CMAKE_BUILD_TYPE=Release`` is assumed.
There are two modes to build MFiX-Exa with cmake:
o **SUPERBUILD (recommended):** AMReX is downloaded and built as part
of the MFiX-Exa build process. This method is strongly encouraged as it of the MFiX-Exa build process. This method is strongly encouraged as it
ensures configure options for MFiX-Exa and AMReX are consistent. ensures that the configuration options for MFiX-Exa and AMReX are consistent.
o **Using an existing AMReX Library:** MFiX-Exa is linked to an existing o **STANDALONE:** MFiX-Exa source code is built separately and linked to an existing
AMReX installation. This is ideal for continuous integration severs (CI) AMReX installation. This is ideal for continuous integration severs (CI)
and regression testing applications. AMReX library version must meet and regression testing applications. AMReX library version and configuration options
MFiX-Exa requirements. must meet MFiX-Exa requirements.
.. note::
**MFiX-Exa requires CMake 3.14 or higher.**
SUPERBUILD Instructions (recommended) SUPERBUILD Instructions (recommended)
------------------------------------- -------------------------------------
When building in **SUPERBUILD** mode, MFiX-Exa build system will take By default MFiX-Exa CMake looks for an existing AMReX installation on the system. If none is found, it falls back to **SUPERBUILD** mode.
care of downloading, configuring and installing AMReX as part of the In this mode, MFiX-Exa CMake inherents AMReX CMake targets and configuration options, that is, MFiX-Exa and AMReX are configured and built as a single entity
MFiX-Exa build. The following commands build MFiX-Exa and AMReX in a
single step. Assuming no valid AMReX installation is present on the target system, and ``AMReX_ROOT`` is not set (see :ref:`sec:build:standalone`), the following code will build MFiX-Exa in **SUPERBUILD** mode:
.. code:: shell .. code:: shell
...@@ -26,117 +51,83 @@ single step. ...@@ -26,117 +51,83 @@ single step.
> cd mfix > cd mfix
> mkdir build > mkdir build
> cd build > cd build
> cmake CONFIG_OPTIONS .. > cmake [amrex options] -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel] ..
> make -j > make -j
AMReX is built the first time the ``make`` command is issued. No ``[amrex options]`` in the snippet above is a list of any of the AMReX configuration options listed in the `AMReX user guide <https://amrex-codes.github.io/amrex/docs_html/BuildingAMReX.html#building-with-cmake>`_
external installation of AMReX is needed. However, internet access to
the AMReX github repository is required. The optional string
``CONFIG_OPTIONS`` allows to customize the build of both AMReX and
MFiX-Exa. ``CONFIG_OPTIONS`` is a list of one or more configuration
options given in the form **-D**\ *OPTION=VALUE*\ \`.
The table below lists configuration options, possible values, and their
effect on the build. Options prefixed by ``AMREX_`` are specific to the
build of AMReX.
+-----------------+------------------------------+------------------+-------------+ Working with the AMReX submodule
| Option name | Description | Possible values | Default | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| | | | value |
+=================+==============================+==================+=============+
| DEBUG | Build in debug mode | ON/OFF | OFF |
+-----------------+------------------------------+------------------+-------------+
| CMAKE\_Fortran\ | User-defined Fortran flags | valid F90 | None |
| _FLAGS | for MFiX build | compiler flags | |
+-----------------+------------------------------+------------------+-------------+
| CMAKE\_CXX\_FLA | User-defined C++ flags for | valid C++ | None |
| GS | MFiX build | compiler flags | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_Fortran\ | User-defined Fortran flags | valid F90 | None |
| _FLAGS | for AMReX build | compiler flags | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_CXX\_FLA | User-defined C++ flags for | valid C++ | None |
| GS | AMReX build | compiler flags | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_MPI | Enable build with MPI | 0/1 | 1 |
| | | | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_OMP | Enable build with OpenMP | 0/1 | 0 |
| | | | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_FPE | Build with Floating-Point | 0/1 | 0 |
| | Exceptions checks | | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_PTESTS | Include tests for projection | 0/1 | 0 |
| | method in Ctest suite | | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_STESTS | Include tests for SIMPLE | 0/1 | 1 |
| | method in Ctest suite | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Enable build with MPI | 0/1 | 1 |
| MPI | | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Enable build with OpenMP | 0/1 | 0 |
| OMP | | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Enable double precision | 0/1 | 1 |
| DP | | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Enable double precision in | 0/1 | 1 |
| DP\_PARTICLES | particles classes | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Include profiling info | 0/1 | 0 |
| BASE\_PROFILE | | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Include tiny profiling info | 0/1 | 0 |
| TINY\_PROFILE | | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Include communicators | 0/1 | 0 |
| COMM\_PROFILE | profiling info | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Include trace profiling info | 0/1 | 0 |
| TRACE\_PROFILE | | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Include memory profiling | 0/1 | 0 |
| MEM\_PROFILE | info | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Include backtrace info | 0/1 | 0 |
| BACKTRACE | | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Include profile parser | 0/1 | 0 |
| PROFPARSER | | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Build position-independent | 0/1 | 0 |
| PIC | code | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_ENABLE\_ | Build position-independent | 0/1 | 0 |
| ASSERTION | code | | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_GIT\_COM | AMReX commit to be used in | valid git commit | None |
| MIT | the build | id/branch | |
+-----------------+------------------------------+------------------+-------------+
| AMREX\_INSTALL\ | Global path to AMReX install | valid global | None |
| _DIR | directory | path | (superbuild |
| | | | ) |
+-----------------+------------------------------+------------------+-------------+
``SUPERBUILD mode is enabled automatically when AMREX_INSTALL_DIR is not given.`` **SUPERBUILD** mode relies on a git submodule to checkout the AMReX git repository.
If the AMReX submodule is not initialized, **SUPERBUILD** mode will initialize it and checkout
the AMReX commit the submodule is pointing to.
Instead, if the AMReX submodule has already been manually initialized and a custom commit has been checked out,
**SUPERBUILD** mode will use that commit. For MFiX-Exa development or testing, you may need to build with a different
branch or release of AMReX.
Example: build mfix with custom fortran flags, AMReX profiling enabled The ``subprojects/amrex`` directory is a Git repo, so use all standard Git
and single precision particles: commands. Either ``cd subprojects/amrex`` to run Git commands in the ``amrex``
directory, or use ``git -C subprojects/amrex`` in the MFiX-Exa repo. For
instance, to build with the ``my-amrex-branch`` branch of the AMReX repo:
:: .. code:: shell
> git -C subprojects/amrex checkout my-amrex-branch
> git status
...
modified: subprojects/amrex (new commits)
The branch ``my-amrex-branch`` will then be used when building MFiX-Exa.
To revert to the default version of the AMReX submodule, run ``git submodule
update``:
.. code:: shell
> git submodule update
> git status
...
nothing to commit, working tree clean
You can edit, commit, pull, and push AMReX changes from ``subprojects/amrex``.
AMReX development is outside the scope of this document. Run ``git status`` in
the top-level MFix-Exa repo to see whether the AMReX submodule has new commits,
modified files, or untracked files.
The CMake variables ``AMREX_GIT_COMMIT_DEVELOP`` and
``AMREX_GIT_COMMIT_DEVELOP`` have been removed. Instead, to update the AMReX
submodule referenced by MFiX-Exa:
.. code:: shell
cmake -DMFiX_FFLAGS_OVERRIDES="custom flags" -DAMREX_ENABLE_BASE_PROFILE=1 -DAMREX_ENABLE_DP_PARTICLES=0 .. > git -C subprojects/amrex checkout UPDATED_AMREX_COMMIT_SHA1
> git add subprojects/amrex
> git commit -m 'Updating AMReX version'
Building MFiX-Exa using a separate AMReX installation (no superbuild) This will only update the AMReX SHA-1 referenced by MFiX-Exa. Uncommitted AMReX
changes and untracked AMReX files under ``subprojects/amrex`` are not added by
``git add subprojects/amrex``. (To commit to the AMReX repo, change directories
to ``subprojects/amrex`` and run Git commands there, before ``git add
subprojects/amrex``.)
.. note::
Only update the AMReX submodule reference in coordination with the other
MFiX-Exa developers!
.. _sec:build:standalone:
**STANDALONE** instructions
--------------------------------------------------------------------- ---------------------------------------------------------------------
Build AMReX Library Building AMReX
~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
Clone AMReX from the official Git repository and checkout the Clone AMReX from the official Git repository and checkout the
*development* branch. *development* branch:
.. code:: shell .. code:: shell
...@@ -148,37 +139,67 @@ Next, configure, build and install AMReX as follows: ...@@ -148,37 +139,67 @@ Next, configure, build and install AMReX as follows:
.. code:: shell .. code:: shell
> cmake . AMREX_CONFIG_OPTIONS \ > mkdir build
-DENABLE_PARTICLES=1 \ > cd build
-DENABLE_AMRDATA=1 \ > cmake -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel] -DENABLE_PARTICLES=yes -DENABLE_AMRDATA=yes -DENABLE_EB=yes [other amrex options] -DCMAKE_INSTALL_PREFIX:PATH=/absolute/path/to/installdir ..
-DENABLE_EB=1 \
-DENABLE_3D_NODAL_MLMG=1 \
-DCMAKE_INSTALL_PREFIX:PATH=/absolute/path/to/installdir
> make install > make install
Here,\ ``AMREX_CONFIG_OPTIONS`` are optional configure options for AMReX. Please The options **ENABLE\_PARTICLES=yes**, **ENABLE\_AMRDATA=yes**, and **ENABLE\_EB=yes** are required by MFiX-Exa. ``[other amrex options]`` in the snippet above refers to any other AMReX configuration option in addition to the required ones. Please refer to the `AMReX user guide <https://amrex-codes.github.io/amrex/docs_html/BuildingAMReX.html#building-with-cmake>`_ for more details on building AMReX with CMake.
refer to the AMReX user guide for a list of all the possible configuration
options. The only options required are **-DENABLE\_PARTICLES=1**,
**-DENABLE\_AMRDATA=1**, **-DENABLE\_EB=1**, and **-DENABLE_3D_NODAL_MLMG=1**.
Building MFiX-Exa Building MFiX-Exa
~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~
Clone and build MFiX-Exa. Clone and build MFiX-Exa:
.. code:: shell .. code:: shell
> git clone http://mfix.netl.doe.gov/gitlab/exa/mfix.git > git clone http://mfix.netl.doe.gov/gitlab/exa/mfix.git
> mkdir build > mkdir build
> cd build > cd build
> cmake CONFIG_OPTIONS -DAMREX_INSTALL_DIR=/absolute/path/to/amrex/installdir .. > cmake -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel] [mfix options] -DAMReX_ROOT=/absolute/path/to/amrex/installdir ..
> make -j > make -j
Here, ``CONFIG_OPTIONS`` are MFiX-Exa specific configuration options,
that is, any option NOT prefixed by ``AMREX_`` in the table above. Passing ``-DAMReX_ROOT=/absolute/path/to/amrex/installdir`` instructs CMake to search
Options prefixed by ``AMREX_`` are always ignored when using an external ``/absolute/path/to/amrex/installdir`` before searching system paths
AMReX installation. for an available AMReX installation.
``AMReX_ROOT`` can also be set as an environmental variable instead of passing it as a command line option.
``[mfix options]`` indicates any of the configuration option listed in the table below.
+-----------------+------------------------------+------------------+-------------+
| Option name | Description | Possible values | Default |
| | | | value |
+=================+==============================+==================+=============+
| CMAKE\_Fortran\ | User-defined Fortran flags | valid Fortran | None |
| _FLAGS | | compiler flags | |
+-----------------+------------------------------+------------------+-------------+
| CMAKE\_CXX\ | User-defined C++ flags | valid C++ | None |
| _FLAGS | | compiler flags | |
+-----------------+------------------------------+------------------+-------------+
| CMAKE\_CUDA\ | User-defined CUDA flags | valid CUDA | None |
| _FLAGS | | compiler flags | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_MPI | Enable build with MPI | no/yes | yes |
| | | | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_OMP | Enable build with OpenMP | no/yes | no |
| | | | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_CUDA | Enable build with CUDA | no/yes | no |
| | | | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_HYPRE | Enable HYPRE support | no/yes | no |
| | | | |
+-----------------+------------------------------+------------------+-------------+
| ENABLE\_FPE | Build with Floating-Point | no/yes | no |
| | Exceptions checks | | |
+-----------------+------------------------------+------------------+-------------+
Few more notes on building MFiX-Exa Few more notes on building MFiX-Exa
----------------------------------- -----------------------------------
...@@ -187,14 +208,14 @@ The system defaults compilers can be overwritten as follows: ...@@ -187,14 +208,14 @@ The system defaults compilers can be overwritten as follows:
.. code:: shell .. code:: shell
> cmake -DCMAKE_CXX_COMPILER=<c++-compiler> -DCMAKE_Fortran_COMPILER=<f90-compiler> CONFIG_OPTIONS .. > cmake -DCMAKE_CXX_COMPILER=<c++-compiler> -DCMAKE_Fortran_COMPILER=<f90-compiler> [options] ..
When building on a platform that uses the ``module`` utility, use either When building on a platform that uses the ``module`` utility, use either
the above command (with full path to the compilers) or the following: the above command (with full path to the compilers) or the following:
.. code:: shell .. code:: shell
> cmake -DCMAKE_CXX_COMPILER=CC -DCMAKE_Fortran_COMPILER=ftn CONFIG_OPTIONS .. > cmake -DCMAKE_CXX_COMPILER=CC -DCMAKE_Fortran_COMPILER=ftn [options] ..
MFiX-Exa uses the same compiler flags used to build AMReX, unless MFiX-Exa uses the same compiler flags used to build AMReX, unless
``CMAKE_Fortran_FLAGS``/``CMAKE_CXX_FLAGS`` is explicitly provided, or ``CMAKE_Fortran_FLAGS``/``CMAKE_CXX_FLAGS`` is explicitly provided, or
......
docs/source/InputsMonitors.rst
+ 4
− 7
View file @ 44c6271c
...@@ -4,16 +4,16 @@ Monitors ...@@ -4,16 +4,16 @@ Monitors
======== ========
The following inputs must be preceded by "amr" and control whether to compute The following inputs must be preceded by "amr" and control whether to compute
monitors, i.e., spatial averages, and how often to output the results. monitors, i.e., spatial averages, and how often to output the results.
n is the number of monitors implicitly defined by the size of avg_region_x_w. n is the number of monitors implicitly defined by the size of avg_region_x_w.
+------------------+-----------------------------------------------------------------------+-------------+-----------+ +------------------+-----------------------------------------------------------------------+-------------+-----------+
| | Description | Type | Default | | | Description | Type | Default |
+==================+=======================================================================+=============+===========+ +==================+=======================================================================+=============+===========+
| avg_int | Interval, in number of CFD dt's, to write output | Int | -1 | | avg_int | Interval, in number of CFD dt's, to write output | Int | -1 |
+------------------+-----------------------------------------------------------------------+-------------+-----------+ +------------------+-----------------------------------------------------------------------+-------------+-----------+
| avg_file | Base file name which is appended with the data type (vel_p_, p_g_, | String | avg_region| | avg_file | Base file name which is appended with the data type (vel_p, p_g, | String | avg_region|
| | ep_g_ or vel_g_), the number of this type of averaging, and the .csv | | | | | ep_g or vel_g), the number of this type of averaging, and the .csv | | |
| | file extension | | | | | file extension | | |
+------------------+-----------------------------------------------------------------------+-------------+-----------+ +------------------+-----------------------------------------------------------------------+-------------+-----------+
| avg_vel_p | Average and save particle velocity (if 1) | n*Int | 0 | | avg_vel_p | Average and save particle velocity (if 1) | n*Int | 0 |
...@@ -36,6 +36,3 @@ n is the number of monitors implicitly defined by the size of avg_region_x_w. ...@@ -36,6 +36,3 @@ n is the number of monitors implicitly defined by the size of avg_region_x_w.
+------------------+-----------------------------------------------------------------------+-------------+-----------+ +------------------+-----------------------------------------------------------------------+-------------+-----------+
| avg_region_z_t | Upper bound of averaging region in z-direction | n*Real | None | | avg_region_z_t | Upper bound of averaging region in z-direction | n*Real | None |
+------------------+-----------------------------------------------------------------------+-------------+-----------+ +------------------+-----------------------------------------------------------------------+-------------+-----------+
docs/source/InputsProblemDefinition.rst
+ 2
− 2
View file @ 44c6271c
...@@ -27,12 +27,12 @@ The following inputs must be preceded by "geometry." ...@@ -27,12 +27,12 @@ The following inputs must be preceded by "geometry."
The following inputs must be preceded by "mfix." The following inputs must be preceded by "mfix."
+----------------------+-------------------------------------------------------------------------+-------------+--------+ +----------------------+-------------------------------------------------------------------------+----------+-----------+
| | Description | Type | Default | | | Description | Type | Default |
+======================+=========================================================================+==========+===========+ +======================+=========================================================================+==========+===========+
| geometry | Which type of EB geometry are we using? | String | | | geometry | Which type of EB geometry are we using? | String | |
+----------------------+-------------------------------------------------------------------------+----------+-----------+ +----------------------+-------------------------------------------------------------------------+----------+-----------+
| levelset__refinement | Refinement factor of levelset resolution relative to level 0 resolution | Int | 1 ! | levelset__refinement | Refinement factor of levelset resolution relative to level 0 resolution | Int | 1 |
+----------------------+-------------------------------------------------------------------------+----------+-----------+ +----------------------+-------------------------------------------------------------------------+----------+-----------+
Setting basic EB walls can be specified by inputs preceded by "xlo", "xhi", "ylo", "yhi", "zlo", and "zhi" Setting basic EB walls can be specified by inputs preceded by "xlo", "xhi", "ylo", "yhi", "zlo", and "zhi"
......
docs/source/LoadBalancing.rst
+ 8
− 8
View file @ 44c6271c
...@@ -9,23 +9,23 @@ ...@@ -9,23 +9,23 @@
Load Balancing Load Balancing
-------------- --------------
The process of load balancing is typically independent of the process of grid creation; The process of load balancing is typically independent of the process of grid creation;
the inputs to load balancing are a given set of grids with a set of weights the inputs to load balancing are a given set of grids with a set of weights
assigned to each grid. (The exception to this is the KD-tree approach in which the assigned to each grid. (The exception to this is the KD-tree approach in which the
grid creation process is governed by trying to balance the work in each grid.) grid creation process is governed by trying to balance the work in each grid.)
Single-level load balancing algorithms are sequentially applied to each AMR level independently, Single-level load balancing algorithms are sequentially applied to each AMR level independently,
and the resulting distributions are mapped onto the ranks taking into account the weights and the resulting distributions are mapped onto the ranks taking into account the weights
already assigned to them (assign heaviest set of grids to the least loaded rank) already assigned to them (assign heaviest set of grids to the least loaded rank)
Options supported by AMReX include: Options supported by AMReX include:
- Knapsack: the default weight of a grid in the knapsack algorithm is the number of grid cells, - Knapsack: the default weight of a grid in the knapsack algorithm is the number of grid cells,
but AMReX supports the option to pass an array of weights – one per grid – or alternatively but AMReX supports the option to pass an array of weights – one per grid – or alternatively
to pass in a MultiFab of weights per cell which is used to compute the weight per grid to pass in a MultiFab of weights per cell which is used to compute the weight per grid
- SFC: enumerate grids with a space-filling Z-morton curve, then partition the - SFC: enumerate grids with a space-filling Z-morton curve, then partition the
resulting ordering across ranks in a way that balances the load resulting ordering across ranks in a way that balances the load
- Round-robin: sort grids and assign them to ranks in round-robin fashion -- specifically - Round-robin: sort grids and assign them to ranks in round-robin fashion -- specifically
FAB *i* is owned by CPU *i*%N where N is the total number of MPI ranks. FAB ``i`` is owned by CPU ``i % N`` where N is the total number of MPI ranks.
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment