Commit 0b28c583 authored by Jordan Musser's avatar Jordan Musser Committed by William D. Fullmer
Browse files

Expand mesh grids and tiles (exa/docs!62)

parent beb67141
Loading
Loading
Loading
Loading
+90 −7
Original line number Diff line number Diff line
@@ -37,12 +37,95 @@ GPU memory

The following inputs must be preceded by "amrex."

+------------------------+-----------------------------------------------------------------------+-------------+--------------+
+----------------------------+-----------------------------------------------------------------------+-------------+--------------+
|                            | Description                                                           |   Type      | Default      |
+========================+=======================================================================+=============+==============+
+============================+=======================================================================+=============+==============+
| the_arena_is_managed       | Abort if an invalid floating point exception is encountered.          |  Int        | 0            |
+------------------------+-----------------------------------------------------------------------+-------------+--------------+
| abort_on_out_of_gpu_memory | rt if a division by zero is computed.                             |  Int        | 0            |
+------------------------+-----------------------------------------------------------------------+-------------+--------------+
| the_area_init_size     |  Abort if and overflow is detected.                                   |  Int        | 0            |
+------------------------+-----------------------------------------------------------------------+-------------+--------------+
+----------------------------+-----------------------------------------------------------------------+-------------+--------------+
| the_arena_init_size        | Abort if and overflow is detected.                                    |  Int        | 0            |
+----------------------------+-----------------------------------------------------------------------+-------------+--------------+
| abort_on_out_of_gpu_memory | Abort if a division by zero is computed.                              |  Int        | 0            |
+----------------------------+-----------------------------------------------------------------------+-------------+--------------+


Load balancing
--------------

Note that when running a granular simulation, i.e., no fluid phase, :cpp:`mfix.dual_grid` must be 0. Hence,
the :cpp:`particles.max_grid_size` (in each direction) have no meaning. Therefore the fluid grid and tile
sizes should be set for particle load balancing. It may also be necessary to set the blocking factors to 1.


The following inputs must be preceded by "mfix." and determine how we load balance:

+----------------------+-----------------------------------------------------------------------+-------------+--------------+
|                      | Description                                                           |   Type      | Default      |
+======================+=======================================================================+=============+==============+
| dual_grid            | If true then use the "dual_grid" approach for load balancing          |  Bool       | False        |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| regrid_int           | How often to regrid (in number of steps at level 0)                   |   Int       |    -1        |
|                      | if regrid_int = -1 then no regridding will occur                      |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| load_balance_fluid   | Only relevant if (dual_grid); if so do we also regrid mesh data       |  Int        | 0            |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| load_balance_type    | What strategy to use for load balancing                               |  String     | KnapSack     |
|                      | Options are "KnapSack", "SFC", or "Greedy"                            |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| knapsack_weight_type | What weighting function to use if using Knapsack load balancing       |  String     | RunTimeCosts |
|                      | Options are "RunTimeCosts" or "NumParticles""                         |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| knapsack_nmax        | Maximum number of grids per MPI process if using knapsack algorithm   |  Int        | 128          |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| greedy_dir           | The direction in which the greedy algorithm cuts overloaded boxes     |  Int        | 0            |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| greedy_min_grid_size | The minimum particle grid size in the greedy load balance algorithm   |  Int        | 2            |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| greedy_3d            | Partition particle grids in 3D with the greedy algorithhm             |  Bool       | False        |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| overload_tolerance*  | The ratio between the maximum workload and the average workload       |  Real       | 1.2          |
|                      | in the greedy algorithm                                               |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| underload_tolerance* | The ratio between the minimum workload and the average workload       |  Real       | 0.8          |
|                      | in the greedy algorithm                                               |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| grid_pruning         | Remove all covered grids from the base mesh; this may result in       |  Bool       | False        |
|                      | disjoined grids                                                       |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+

\* The greedy partitioning algorithm uses the tolerances to set the expected
workload range for each rank but doesn't strictly enforce them. The algorithm
creates a partition as balance as possible even if the partition doesn't
meet the tolerances.

To allow a user to verify the breakdown of fluid grids created before running a full simulation, an input option,
:cpp:`mfix.only_print_grid_report` is supported. By default, it is :cpp:`False`. When set to :cpp:`True`, the run uses
minimal memory to print the grid coverage report and exits immediately after that.


The following inputs are defined using the ``particles`` prefix.

+----------------------+-----------------------------------------------------------------------+-------------+--------------+
|                      | Description                                                           |   Type      | Default      |
+======================+=======================================================================+=============+==============+
| max_grid_size_x      | Maximum number of cells at level 0 in each grid in x-direction        |    Int      | 32           |
|                      | for grids in the ParticleBoxArray if dual_grid is true                |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| max_grid_size_y      | Maximum number of cells at level 0 in each grid in y-direction        |    Int      | 32           |
|                      | for grids in the ParticleBoxArray if dual_grid is true                |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+
| max_grid_size_z      | Maximum number of cells at level 0 in each grid in z-direction        |    Int      | 32           |
|                      | for grids in the ParticleBoxArray if dual_grid is true.               |             |              |
+----------------------+-----------------------------------------------------------------------+-------------+--------------+


.. todo:: Is this needed anymore?

   The following inputs must be preceded by "particles."

   +----------------------+-----------------------------------------------------------------------+-------------+--------------+
   |                      | Description                                                           |   Type      | Default      |
   +======================+=======================================================================+=============+==============+
   | reduceGhostParticles | Remove unused ghost particles from communication,                     |    Bool     | True         |
   |                      | only supported by GPU build                                           |             |              |
   +----------------------+-----------------------------------------------------------------------+-------------+--------------+
+2 −0
Original line number Diff line number Diff line
@@ -8,6 +8,8 @@ low and high corners of the cuboid are defined by the ``prob_lo`` and ``prob_hi`

The following inputs are defined using the ``geometry`` prefix.

.. _InputsTable_domain:

+-----------------+-----------------------------------------------------------------------+-------------+-----------+
|                 | Description                                                           |   Type      | Default   |
+=================+=======================================================================+=============+===========+
+4.98 KiB
Loading image diff...
+114 −105

File changed.

Preview size limit exceeded, changes collapsed.