Loading docs/source_docs/user_guide/inputs/images/geometry/mesh_lev0_ex.png 0 → 100644 +4.98 KiB Viewed Loading image diff... docs/source_docs/user_guide/inputs/advanced.rst +90 −7 Viewed Changes for docs/source_docs/user_guide/inputs/advanced.rst: 90 added lines, 7 removed lines. Original line number Diff line number Diff line Loading @@ -37,12 +37,95 @@ GPU memory The following inputs must be preceded by "amrex." +------------------------+-----------------------------------------------------------------------+-------------+--------------+ +----------------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +========================+=======================================================================+=============+==============+ +============================+=======================================================================+=============+==============+ | the_arena_is_managed | Abort if an invalid floating point exception is encountered. | Int | 0 | +------------------------+-----------------------------------------------------------------------+-------------+--------------+ | abort_on_out_of_gpu_memory | rt if a division by zero is computed. | Int | 0 | +------------------------+-----------------------------------------------------------------------+-------------+--------------+ | the_area_init_size | Abort if and overflow is detected. | Int | 0 | +------------------------+-----------------------------------------------------------------------+-------------+--------------+ +----------------------------+-----------------------------------------------------------------------+-------------+--------------+ | the_arena_init_size | Abort if and overflow is detected. | Int | 0 | +----------------------------+-----------------------------------------------------------------------+-------------+--------------+ | abort_on_out_of_gpu_memory | Abort if a division by zero is computed. | Int | 0 | +----------------------------+-----------------------------------------------------------------------+-------------+--------------+ Load balancing -------------- Note that when running a granular simulation, i.e., no fluid phase, :cpp:`mfix.dual_grid` must be 0. Hence, the :cpp:`particles.max_grid_size` (in each direction) have no meaning. Therefore the fluid grid and tile sizes should be set for particle load balancing. It may also be necessary to set the blocking factors to 1. The following inputs must be preceded by "mfix." and determine how we load balance: +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | dual_grid | If true then use the "dual_grid" approach for load balancing | Bool | False | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | regrid_int | How often to regrid (in number of steps at level 0) | Int | -1 | | | if regrid_int = -1 then no regridding will occur | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | load_balance_fluid | Only relevant if (dual_grid); if so do we also regrid mesh data | Int | 0 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | load_balance_type | What strategy to use for load balancing | String | KnapSack | | | Options are "KnapSack", "SFC", or "Greedy" | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | knapsack_weight_type | What weighting function to use if using Knapsack load balancing | String | RunTimeCosts | | | Options are "RunTimeCosts" or "NumParticles"" | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | knapsack_nmax | Maximum number of grids per MPI process if using knapsack algorithm | Int | 128 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_dir | The direction in which the greedy algorithm cuts overloaded boxes | Int | 0 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_min_grid_size | The minimum particle grid size in the greedy load balance algorithm | Int | 2 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_3d | Partition particle grids in 3D with the greedy algorithhm | Bool | False | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | overload_tolerance* | The ratio between the maximum workload and the average workload | Real | 1.2 | | | in the greedy algorithm | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | underload_tolerance* | The ratio between the minimum workload and the average workload | Real | 0.8 | | | in the greedy algorithm | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | grid_pruning | Remove all covered grids from the base mesh; this may result in | Bool | False | | | disjoined grids | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ \* The greedy partitioning algorithm uses the tolerances to set the expected workload range for each rank but doesn't strictly enforce them. The algorithm creates a partition as balance as possible even if the partition doesn't meet the tolerances. To allow a user to verify the breakdown of fluid grids created before running a full simulation, an input option, :cpp:`mfix.only_print_grid_report` is supported. By default, it is :cpp:`False`. When set to :cpp:`True`, the run uses minimal memory to print the grid coverage report and exits immediately after that. The following inputs are defined using the ``particles`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true. | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ .. todo:: Is this needed anymore? The following inputs must be preceded by "particles." +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | reduceGhostParticles | Remove unused ghost particles from communication, | Bool | True | | | only supported by GPU build | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ docs/source_docs/user_guide/inputs/domain.rst +2 −0 Viewed Changes for docs/source_docs/user_guide/inputs/domain.rst: 2 added lines, 0 removed lines. Original line number Diff line number Diff line Loading @@ -8,6 +8,8 @@ low and high corners of the cuboid are defined by the ``prob_lo`` and ``prob_hi` The following inputs are defined using the ``geometry`` prefix. .. _InputsTable_domain: +-----------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +=================+=======================================================================+=============+===========+ Loading docs/source_docs/user_guide/inputs/mesh_and_gridding.rst +114 −105 Viewed Changes for docs/source_docs/user_guide/inputs/mesh_and_gridding.rst: 114 added lines, 105 removed lines. Original line number Diff line number Diff line .. sec:InputsMeshAndGridding: Mesh and gridding ================= Mesh, grids and tiles --------------------- Mesh, grids, and tiles ====================== Mesh ^^^^ ---- .. rubric:: Level-0 mesh The following inputs are defined using the ``amr`` prefix. There are some settings we can specify for the mesh and the automatic mesh refinement algorithm. These settings must be preceded by "amr." in the input file. .. _InputsTable_mesh: +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | | | | | | +======================+=======================================================================+=============+===========+ | max_level | Maximum level of refinement allowed (0 when single-level) | Int | 0 | | max_level | Maximum level of refinement. The default value, 0, means that there | int | 0 | | | is one cell size covering the entire domain (i.e., single-level). | | | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | n_cell | Number of cells at level 0 in each coordinate direction | ints | 0 0 0 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ The level 0 mesh spacing is computed for each direction by dividing the :ref:`domain length<InputsTable_domain>` by the number of cells. The mesh spacing is **required** to be the same in all directions: .. math:: \frac{\text{prob_hi[0] - prob_lo[0]}}{\text{n_cell[0]}} = \frac{\text{prob_hi[1] - prob_lo[1]}}{\text{n_cell[1]}} = \frac{\text{prob_hi[2] - prob_lo[2]}}{\text{n_cell[2]}} The inputs for defining the mesh for a single-level simulation are demonstrated in the :ref:`following example<inputs_mesh_ex>` and illustrated in :numref:`fig_basic_mesh_ex`. In this example, the domain is a :math:`4 \times 1 \times 1` cuboid, and there are :math:`32 \times 8 \times 8` cells in the *X*, *Y*, and *Z* directions, respectively. The result is a uniform mesh spacing of :math:`0.125` *m* in all three directions. .. _inputs_mesh_ex: .. code-block:: bash :caption: Snippet of intpus for mesh example. This is not a complete input file. # Define periodicity and domain extents # ------------------------------------------------------------- geometry.coord_sys = 0 # Cartesian coordinates geometry.is_periodic = 0 0 0 # periodicity for each direction geometry.prob_lo = 0. 0. 0 # lo corner of physical domain. geometry.prob_hi = 4. 1. 1. # hi corner of physical domain # Define the maximum level of refinement and number of cells # ------------------------------------------------------------- arm.max_level = 0 arm.n_cell = 32 8 8 .. _fig_basic_mesh_ex: .. figure:: ./images/geometry/mesh_lev0_ex.png :width: 100% :align: center :alt: domain used with box embedded boundary Example of a single-level mesh. .. warning:: MFIX-Exa simulations with a non-uniform mesh will not run. ----- .. rubric:: Mesh refinement The following inputs are defined using the ``amr`` prefix. These inputs control the automatic mesh refinement algorithm and are only applicable when ``amr.max_level > 0``. +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +======================+=======================================================================+=============+===========+ | grid_eff | Threshold value to ensure grids do not contain too large a fraction | Real | 0.7 | | | of un-tagged cells. | | | | | Applicable only when mesh refinement is enabled ("max_level" > 0). | | | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | n_error_buf | Controls the number of tagged cells before grids are defined. Used | Int | 1 | | n_error_buf | Controls the number of tagged cells before grids are defined. Used | int | 1 | | | to ensure coarse/fine boundaries are not too close to tagged cells. | | | | | Applicable only when mesh refinement is enabled ("max_level" > 0). | | | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | n_cell | Number of cells at level 0 in each coordinate direction | Ints | 0 0 0 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ .. caution:: **Mesh refinement limitations:** * MFIX-Exa simulations with particles currently do not support mesh refinement. See the `AMReX documentation <https://amrex-codes.github.io/amrex/docs_html/index.html>`_ for details on the adaptive mesh refinement algorithms. ----- Grids ^^^^^ ----- The following inputs must be preceded by "amr." and determine how we create the grids. The following inputs are defined using the ``amr`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +======================+=======================================================================+=============+===========+ | max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | | max_grid_size_x | Maximum number of cells at level 0 in each grid in *X* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | | max_grid_size_y | Maximum number of cells at level 0 in each grid in *Y* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | | max_grid_size_z | Maximum number of cells at level 0 in each grid in *Z* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | blocking_factor_x | Each grid must be divisible by blocking_factor_x in x-direction | Int | 8 | | blocking_factor_x | Each grid in *X* must be divisible by ``blocking_factor_x`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | blocking_factor_y | Each grid must be divisible by blocking_factor_y in y-direction | Int | 8 | | blocking_factor_y | Each grid in *Y* must be divisible by ``blocking_factor_y`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | blocking_factor_z | Each grid must be divisible by blocking_factor_z in z-direction | Int | 8 | | blocking_factor_z | Each grid in *Z* must be divisible by ``blocking_factor_z`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ The domain is decomposed into *grids* by dividing the number of cells by the max grid size for each direction (e.g., ``n_cells[0]/max_grid_size_x``). The blocking factor ensures that the grids will be sufficiently coarsenable for good multigrid performance; therefore, the ``max_grid_size`` must be divisible by the corresponding ``blocking_factor``. The following inputs must be preceded by "particles." .. note:: The `AMReX documentation <https://amrex-codes.github.io/amrex/docs_html/index.html>`_ contains a significant amount of information on grid creation and load balancing. Users are **strongly** encouraged to read the relevant sections. ----- +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true. | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ Tiles ^^^^^ ----- The following inputs must be preceded by "fabarray_mfiter." and determine how we create the logical tiles: The following inputs are defined using the ``fabarray`` prefix. +----------------------+-----------------------------------------------------------------------+----------+-------------+ | | Description | Type | Default | +======================+=======================================================================+==========+=============+ | tile_size | Maximum number of cells in each direction for (logical) tiles | IntVect | 1024000 | | | (3D CPU-only) | | 1024000,8,8 | | mfiter_tile_size | Maximum number of cells in each direction for (logical) tiles | IntVect | 1024000,8,8 | +----------------------+-----------------------------------------------------------------------+----------+-------------+ The following inputs must be preceded by "particles." The following inputs are defined using the ``particles`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | Loading @@ -87,69 +156,9 @@ The following inputs must be preceded by "particles." | | in the ParticleBoxArray if dual_grid is true. | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ When running on shared memory machines using an OpenMP enabled executable, *grids* are subdivided into *tiles* and iterated over to improve data locality by cache blocking. Load balancing -------------- Note that when running a granular simulation, i.e., no fluid phase, :cpp:`mfix.dual_grid` must be 0. Hence, the :cpp:`particles.max_grid_size` (in each direction) have no meaning. Therefore the fluid grid and tile sizes should be set for particle load balancing. It may also be necessary to set the blocking factors to 1. The following inputs must be preceded by "mfix." and determine how we load balance: +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | dual_grid | If true then use the "dual_grid" approach for load balancing | Bool | False | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | regrid_int | How often to regrid (in number of steps at level 0) | Int | -1 | | | if regrid_int = -1 then no regridding will occur | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | load_balance_fluid | Only relevant if (dual_grid); if so do we also regrid mesh data | Int | 0 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | load_balance_type | What strategy to use for load balancing | String | KnapSack | | | Options are "KnapSack", "SFC", or "Greedy" | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | knapsack_weight_type | What weighting function to use if using Knapsack load balancing | String | RunTimeCosts | | | Options are "RunTimeCosts" or "NumParticles"" | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | knapsack_nmax | Maximum number of grids per MPI process if using knapsack algorithm | Int | 128 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_dir | The direction in which the greedy algorithm cuts overloaded boxes | Int | 0 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_min_grid_size | The minimum particle grid size in the greedy load balance algorithm | Int | 2 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_3d | Partition particle grids in 3D with the greedy algorithhm | Bool | False | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | overload_tolerance* | The ratio between the maximum workload and the average workload | Real | 1.2 | | | in the greedy algorithm | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | underload_tolerance* | The ratio between the minimum workload and the average workload | Real | 0.8 | | | in the greedy algorithm | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | grid_pruning | Remove all covered grids from the base mesh; this may result in | Bool | False | | | disjoined grids | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ \* The greedy partitioning algorithm uses the tolerances to set the expected workload range for each rank but doesn't strictly enforce them. The algorithm creates a partition as balance as possible even if the partition doesn't meet the tolerances. To allow a user to verify the breakdown of fluid grids created before running a full simulation, an input option, :cpp:`mfix.only_print_grid_report` is supported. By default, it is :cpp:`False`. When set to :cpp:`True`, the run uses minimal memory to print the grid coverage report and exits immediately after that. .. todo:: Is this needed anymore? The following inputs must be preceded by "particles." +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | reduceGhostParticles | Remove unused ghost particles from communication, | Bool | True | | | only supported by GPU build | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ .. note:: MFIX-Exa disables tiling when using GPU accelerators. Loading
docs/source_docs/user_guide/inputs/images/geometry/mesh_lev0_ex.png 0 → 100644 +4.98 KiB Viewed Loading image diff...
docs/source_docs/user_guide/inputs/advanced.rst +90 −7 Viewed Changes for docs/source_docs/user_guide/inputs/advanced.rst: 90 added lines, 7 removed lines. Original line number Diff line number Diff line Loading @@ -37,12 +37,95 @@ GPU memory The following inputs must be preceded by "amrex." +------------------------+-----------------------------------------------------------------------+-------------+--------------+ +----------------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +========================+=======================================================================+=============+==============+ +============================+=======================================================================+=============+==============+ | the_arena_is_managed | Abort if an invalid floating point exception is encountered. | Int | 0 | +------------------------+-----------------------------------------------------------------------+-------------+--------------+ | abort_on_out_of_gpu_memory | rt if a division by zero is computed. | Int | 0 | +------------------------+-----------------------------------------------------------------------+-------------+--------------+ | the_area_init_size | Abort if and overflow is detected. | Int | 0 | +------------------------+-----------------------------------------------------------------------+-------------+--------------+ +----------------------------+-----------------------------------------------------------------------+-------------+--------------+ | the_arena_init_size | Abort if and overflow is detected. | Int | 0 | +----------------------------+-----------------------------------------------------------------------+-------------+--------------+ | abort_on_out_of_gpu_memory | Abort if a division by zero is computed. | Int | 0 | +----------------------------+-----------------------------------------------------------------------+-------------+--------------+ Load balancing -------------- Note that when running a granular simulation, i.e., no fluid phase, :cpp:`mfix.dual_grid` must be 0. Hence, the :cpp:`particles.max_grid_size` (in each direction) have no meaning. Therefore the fluid grid and tile sizes should be set for particle load balancing. It may also be necessary to set the blocking factors to 1. The following inputs must be preceded by "mfix." and determine how we load balance: +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | dual_grid | If true then use the "dual_grid" approach for load balancing | Bool | False | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | regrid_int | How often to regrid (in number of steps at level 0) | Int | -1 | | | if regrid_int = -1 then no regridding will occur | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | load_balance_fluid | Only relevant if (dual_grid); if so do we also regrid mesh data | Int | 0 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | load_balance_type | What strategy to use for load balancing | String | KnapSack | | | Options are "KnapSack", "SFC", or "Greedy" | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | knapsack_weight_type | What weighting function to use if using Knapsack load balancing | String | RunTimeCosts | | | Options are "RunTimeCosts" or "NumParticles"" | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | knapsack_nmax | Maximum number of grids per MPI process if using knapsack algorithm | Int | 128 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_dir | The direction in which the greedy algorithm cuts overloaded boxes | Int | 0 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_min_grid_size | The minimum particle grid size in the greedy load balance algorithm | Int | 2 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_3d | Partition particle grids in 3D with the greedy algorithhm | Bool | False | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | overload_tolerance* | The ratio between the maximum workload and the average workload | Real | 1.2 | | | in the greedy algorithm | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | underload_tolerance* | The ratio between the minimum workload and the average workload | Real | 0.8 | | | in the greedy algorithm | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | grid_pruning | Remove all covered grids from the base mesh; this may result in | Bool | False | | | disjoined grids | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ \* The greedy partitioning algorithm uses the tolerances to set the expected workload range for each rank but doesn't strictly enforce them. The algorithm creates a partition as balance as possible even if the partition doesn't meet the tolerances. To allow a user to verify the breakdown of fluid grids created before running a full simulation, an input option, :cpp:`mfix.only_print_grid_report` is supported. By default, it is :cpp:`False`. When set to :cpp:`True`, the run uses minimal memory to print the grid coverage report and exits immediately after that. The following inputs are defined using the ``particles`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true. | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ .. todo:: Is this needed anymore? The following inputs must be preceded by "particles." +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | reduceGhostParticles | Remove unused ghost particles from communication, | Bool | True | | | only supported by GPU build | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+
docs/source_docs/user_guide/inputs/domain.rst +2 −0 Viewed Changes for docs/source_docs/user_guide/inputs/domain.rst: 2 added lines, 0 removed lines. Original line number Diff line number Diff line Loading @@ -8,6 +8,8 @@ low and high corners of the cuboid are defined by the ``prob_lo`` and ``prob_hi` The following inputs are defined using the ``geometry`` prefix. .. _InputsTable_domain: +-----------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +=================+=======================================================================+=============+===========+ Loading
docs/source_docs/user_guide/inputs/mesh_and_gridding.rst +114 −105 Viewed Changes for docs/source_docs/user_guide/inputs/mesh_and_gridding.rst: 114 added lines, 105 removed lines. Original line number Diff line number Diff line .. sec:InputsMeshAndGridding: Mesh and gridding ================= Mesh, grids and tiles --------------------- Mesh, grids, and tiles ====================== Mesh ^^^^ ---- .. rubric:: Level-0 mesh The following inputs are defined using the ``amr`` prefix. There are some settings we can specify for the mesh and the automatic mesh refinement algorithm. These settings must be preceded by "amr." in the input file. .. _InputsTable_mesh: +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | | | | | | +======================+=======================================================================+=============+===========+ | max_level | Maximum level of refinement allowed (0 when single-level) | Int | 0 | | max_level | Maximum level of refinement. The default value, 0, means that there | int | 0 | | | is one cell size covering the entire domain (i.e., single-level). | | | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | n_cell | Number of cells at level 0 in each coordinate direction | ints | 0 0 0 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ The level 0 mesh spacing is computed for each direction by dividing the :ref:`domain length<InputsTable_domain>` by the number of cells. The mesh spacing is **required** to be the same in all directions: .. math:: \frac{\text{prob_hi[0] - prob_lo[0]}}{\text{n_cell[0]}} = \frac{\text{prob_hi[1] - prob_lo[1]}}{\text{n_cell[1]}} = \frac{\text{prob_hi[2] - prob_lo[2]}}{\text{n_cell[2]}} The inputs for defining the mesh for a single-level simulation are demonstrated in the :ref:`following example<inputs_mesh_ex>` and illustrated in :numref:`fig_basic_mesh_ex`. In this example, the domain is a :math:`4 \times 1 \times 1` cuboid, and there are :math:`32 \times 8 \times 8` cells in the *X*, *Y*, and *Z* directions, respectively. The result is a uniform mesh spacing of :math:`0.125` *m* in all three directions. .. _inputs_mesh_ex: .. code-block:: bash :caption: Snippet of intpus for mesh example. This is not a complete input file. # Define periodicity and domain extents # ------------------------------------------------------------- geometry.coord_sys = 0 # Cartesian coordinates geometry.is_periodic = 0 0 0 # periodicity for each direction geometry.prob_lo = 0. 0. 0 # lo corner of physical domain. geometry.prob_hi = 4. 1. 1. # hi corner of physical domain # Define the maximum level of refinement and number of cells # ------------------------------------------------------------- arm.max_level = 0 arm.n_cell = 32 8 8 .. _fig_basic_mesh_ex: .. figure:: ./images/geometry/mesh_lev0_ex.png :width: 100% :align: center :alt: domain used with box embedded boundary Example of a single-level mesh. .. warning:: MFIX-Exa simulations with a non-uniform mesh will not run. ----- .. rubric:: Mesh refinement The following inputs are defined using the ``amr`` prefix. These inputs control the automatic mesh refinement algorithm and are only applicable when ``amr.max_level > 0``. +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +======================+=======================================================================+=============+===========+ | grid_eff | Threshold value to ensure grids do not contain too large a fraction | Real | 0.7 | | | of un-tagged cells. | | | | | Applicable only when mesh refinement is enabled ("max_level" > 0). | | | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | n_error_buf | Controls the number of tagged cells before grids are defined. Used | Int | 1 | | n_error_buf | Controls the number of tagged cells before grids are defined. Used | int | 1 | | | to ensure coarse/fine boundaries are not too close to tagged cells. | | | | | Applicable only when mesh refinement is enabled ("max_level" > 0). | | | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | n_cell | Number of cells at level 0 in each coordinate direction | Ints | 0 0 0 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ .. caution:: **Mesh refinement limitations:** * MFIX-Exa simulations with particles currently do not support mesh refinement. See the `AMReX documentation <https://amrex-codes.github.io/amrex/docs_html/index.html>`_ for details on the adaptive mesh refinement algorithms. ----- Grids ^^^^^ ----- The following inputs must be preceded by "amr." and determine how we create the grids. The following inputs are defined using the ``amr`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +======================+=======================================================================+=============+===========+ | max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | | max_grid_size_x | Maximum number of cells at level 0 in each grid in *X* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | | max_grid_size_y | Maximum number of cells at level 0 in each grid in *Y* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | | max_grid_size_z | Maximum number of cells at level 0 in each grid in *Z* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | blocking_factor_x | Each grid must be divisible by blocking_factor_x in x-direction | Int | 8 | | blocking_factor_x | Each grid in *X* must be divisible by ``blocking_factor_x`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | blocking_factor_y | Each grid must be divisible by blocking_factor_y in y-direction | Int | 8 | | blocking_factor_y | Each grid in *Y* must be divisible by ``blocking_factor_y`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | blocking_factor_z | Each grid must be divisible by blocking_factor_z in z-direction | Int | 8 | | blocking_factor_z | Each grid in *Z* must be divisible by ``blocking_factor_z`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ The domain is decomposed into *grids* by dividing the number of cells by the max grid size for each direction (e.g., ``n_cells[0]/max_grid_size_x``). The blocking factor ensures that the grids will be sufficiently coarsenable for good multigrid performance; therefore, the ``max_grid_size`` must be divisible by the corresponding ``blocking_factor``. The following inputs must be preceded by "particles." .. note:: The `AMReX documentation <https://amrex-codes.github.io/amrex/docs_html/index.html>`_ contains a significant amount of information on grid creation and load balancing. Users are **strongly** encouraged to read the relevant sections. ----- +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | | | for grids in the ParticleBoxArray if dual_grid is true. | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ Tiles ^^^^^ ----- The following inputs must be preceded by "fabarray_mfiter." and determine how we create the logical tiles: The following inputs are defined using the ``fabarray`` prefix. +----------------------+-----------------------------------------------------------------------+----------+-------------+ | | Description | Type | Default | +======================+=======================================================================+==========+=============+ | tile_size | Maximum number of cells in each direction for (logical) tiles | IntVect | 1024000 | | | (3D CPU-only) | | 1024000,8,8 | | mfiter_tile_size | Maximum number of cells in each direction for (logical) tiles | IntVect | 1024000,8,8 | +----------------------+-----------------------------------------------------------------------+----------+-------------+ The following inputs must be preceded by "particles." The following inputs are defined using the ``particles`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | Loading @@ -87,69 +156,9 @@ The following inputs must be preceded by "particles." | | in the ParticleBoxArray if dual_grid is true. | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ When running on shared memory machines using an OpenMP enabled executable, *grids* are subdivided into *tiles* and iterated over to improve data locality by cache blocking. Load balancing -------------- Note that when running a granular simulation, i.e., no fluid phase, :cpp:`mfix.dual_grid` must be 0. Hence, the :cpp:`particles.max_grid_size` (in each direction) have no meaning. Therefore the fluid grid and tile sizes should be set for particle load balancing. It may also be necessary to set the blocking factors to 1. The following inputs must be preceded by "mfix." and determine how we load balance: +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | dual_grid | If true then use the "dual_grid" approach for load balancing | Bool | False | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | regrid_int | How often to regrid (in number of steps at level 0) | Int | -1 | | | if regrid_int = -1 then no regridding will occur | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | load_balance_fluid | Only relevant if (dual_grid); if so do we also regrid mesh data | Int | 0 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | load_balance_type | What strategy to use for load balancing | String | KnapSack | | | Options are "KnapSack", "SFC", or "Greedy" | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | knapsack_weight_type | What weighting function to use if using Knapsack load balancing | String | RunTimeCosts | | | Options are "RunTimeCosts" or "NumParticles"" | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | knapsack_nmax | Maximum number of grids per MPI process if using knapsack algorithm | Int | 128 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_dir | The direction in which the greedy algorithm cuts overloaded boxes | Int | 0 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_min_grid_size | The minimum particle grid size in the greedy load balance algorithm | Int | 2 | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | greedy_3d | Partition particle grids in 3D with the greedy algorithhm | Bool | False | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | overload_tolerance* | The ratio between the maximum workload and the average workload | Real | 1.2 | | | in the greedy algorithm | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | underload_tolerance* | The ratio between the minimum workload and the average workload | Real | 0.8 | | | in the greedy algorithm | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | grid_pruning | Remove all covered grids from the base mesh; this may result in | Bool | False | | | disjoined grids | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ \* The greedy partitioning algorithm uses the tolerances to set the expected workload range for each rank but doesn't strictly enforce them. The algorithm creates a partition as balance as possible even if the partition doesn't meet the tolerances. To allow a user to verify the breakdown of fluid grids created before running a full simulation, an input option, :cpp:`mfix.only_print_grid_report` is supported. By default, it is :cpp:`False`. When set to :cpp:`True`, the run uses minimal memory to print the grid coverage report and exits immediately after that. .. todo:: Is this needed anymore? The following inputs must be preceded by "particles." +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | +======================+=======================================================================+=============+==============+ | reduceGhostParticles | Remove unused ghost particles from communication, | Bool | True | | | only supported by GPU build | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ .. note:: MFIX-Exa disables tiling when using GPU accelerators.