From 243c896b56d4e37ed6d6ca13b057f3070158c6fb Mon Sep 17 00:00:00 2001 From: Jordan Musser Date: Fri, 26 Jan 2024 16:36:34 -0500 Subject: [PATCH 1/2] Add changes to MR --- .../user_guide/inputs/advanced.rst | 83 +++++++ docs/source_docs/user_guide/inputs/domain.rst | 2 + .../inputs/images/geometry/mesh_lev0_ex.png | Bin 0 -> 5097 bytes .../user_guide/inputs/mesh_and_gridding.rst | 219 +++++++++--------- 4 files changed, 199 insertions(+), 105 deletions(-) create mode 100644 docs/source_docs/user_guide/inputs/images/geometry/mesh_lev0_ex.png diff --git a/docs/source_docs/user_guide/inputs/advanced.rst b/docs/source_docs/user_guide/inputs/advanced.rst index 7ce68e8..ea67991 100644 --- a/docs/source_docs/user_guide/inputs/advanced.rst +++ b/docs/source_docs/user_guide/inputs/advanced.rst @@ -46,3 +46,86 @@ The following inputs must be preceded by "amrex." +------------------------+-----------------------------------------------------------------------+-------------+--------------+ | the_area_init_size | Abort if and overflow is detected. | Int | 0 | +------------------------+-----------------------------------------------------------------------+-------------+--------------+ + + +Load balancing +-------------- + +Note that when running a granular simulation, i.e., no fluid phase, :cpp:`mfix.dual_grid` must be 0. Hence, +the :cpp:`particles.max_grid_size` (in each direction) have no meaning. Therefore the fluid grid and tile +sizes should be set for particle load balancing. It may also be necessary to set the blocking factors to 1. + + +The following inputs must be preceded by "mfix." and determine how we load balance: + ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| | Description | Type | Default | ++======================+=======================================================================+=============+==============+ +| dual_grid | If true then use the "dual_grid" approach for load balancing | Bool | False | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| regrid_int | How often to regrid (in number of steps at level 0) | Int | -1 | +| | if regrid_int = -1 then no regridding will occur | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| load_balance_fluid | Only relevant if (dual_grid); if so do we also regrid mesh data | Int | 0 | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| load_balance_type | What strategy to use for load balancing | String | KnapSack | +| | Options are "KnapSack", "SFC", or "Greedy" | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| knapsack_weight_type | What weighting function to use if using Knapsack load balancing | String | RunTimeCosts | +| | Options are "RunTimeCosts" or "NumParticles"" | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| knapsack_nmax | Maximum number of grids per MPI process if using knapsack algorithm | Int | 128 | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| greedy_dir | The direction in which the greedy algorithm cuts overloaded boxes | Int | 0 | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| greedy_min_grid_size | The minimum particle grid size in the greedy load balance algorithm | Int | 2 | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| greedy_3d | Partition particle grids in 3D with the greedy algorithhm | Bool | False | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| overload_tolerance* | The ratio between the maximum workload and the average workload | Real | 1.2 | +| | in the greedy algorithm | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| underload_tolerance* | The ratio between the minimum workload and the average workload | Real | 0.8 | +| | in the greedy algorithm | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| grid_pruning | Remove all covered grids from the base mesh; this may result in | Bool | False | +| | disjoined grids | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ + +\* The greedy partitioning algorithm uses the tolerances to set the expected +workload range for each rank but doesn't strictly enforce them. The algorithm +creates a partition as balance as possible even if the partition doesn't +meet the tolerances. + +To allow a user to verify the breakdown of fluid grids created before running a full simulation, an input option, +:cpp:`mfix.only_print_grid_report` is supported. By default, it is :cpp:`False`. When set to :cpp:`True`, the run uses +minimal memory to print the grid coverage report and exits immediately after that. + + +The following inputs are defined using the ``particles`` prefix. + ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| | Description | Type | Default | ++======================+=======================================================================+=============+==============+ +| max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | +| | for grids in the ParticleBoxArray if dual_grid is true | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | +| | for grids in the ParticleBoxArray if dual_grid is true | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ +| max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | +| | for grids in the ParticleBoxArray if dual_grid is true. | | | ++----------------------+-----------------------------------------------------------------------+-------------+--------------+ + + +.. todo:: Is this needed anymore? + + The following inputs must be preceded by "particles." + + +----------------------+-----------------------------------------------------------------------+-------------+--------------+ + | | Description | Type | Default | + +======================+=======================================================================+=============+==============+ + | reduceGhostParticles | Remove unused ghost particles from communication, | Bool | True | + | | only supported by GPU build | | | + +----------------------+-----------------------------------------------------------------------+-------------+--------------+ + diff --git a/docs/source_docs/user_guide/inputs/domain.rst b/docs/source_docs/user_guide/inputs/domain.rst index f1a1933..fe18fc1 100644 --- a/docs/source_docs/user_guide/inputs/domain.rst +++ b/docs/source_docs/user_guide/inputs/domain.rst @@ -8,6 +8,8 @@ low and high corners of the cuboid are defined by the ``prob_lo`` and ``prob_hi` The following inputs are defined using the ``geometry`` prefix. +.. _InputsTable_domain: + +-----------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +=================+=======================================================================+=============+===========+ diff --git a/docs/source_docs/user_guide/inputs/images/geometry/mesh_lev0_ex.png b/docs/source_docs/user_guide/inputs/images/geometry/mesh_lev0_ex.png new file mode 100644 index 0000000000000000000000000000000000000000..de1f7c1c1b9721cb93246729404b4271c294bc30 GIT binary patch literal 5097 zcmeAS@N?(olHy`uVBq!ia0y~yVA;;V!05@s%)r1fyX?|u1_lO}bVpxD28NCO+i)SYE^?aOhrJyE!Q66uK=?reYHErCsbIMFx(hhd7f{utY??oYcg1OIRP9&slzP z_NCT_3nFLR{)QOzdxc+WddT^Eqe-TRIMboczkPof-jBXiy2WX>*6QOgr7x8>a7njvt;zE$5WuP<4h;ahm=g1dF?bKX67>b-sUWL*kvNhn`h za(9xRlGyyS-r)+Z|8~E+ci%%?u zI~Mx-MqDy&dBiS!N%Tm{!-XZRMQi5wzNqKz9F~U8dKa_C5AL_VeQJ zt9vG`p8i7mrPrlLYpNeUS-n)h6ag%&sX@PRvZberb8>%c`ae%kRurO$%DLFMs9t zlS;YuyK+3IPZHwy`7XJ5FGyZvW?E{p=6u(kRdPY+g}vX(%6o@T61%*t`d87X+?Q2N z6?aWyKOd}l{%&7$T+qA-DKtKQh%?Xx}{|MD=BiTn2EqGpEnYcQ|Z`QLP$4+)6i9Nmb?@ZmFpub=K#qOGBGQ({78P4k`J93jR ziT?aNtMaT#{OuXCuO#=*@s|A7TEcp^M(B(9)Ni*!k0u=L5T>|;9{mOk1a_4vmlm$=Z3Sy}3K$1eQnK3Dkv>a2@tyBDW9pA0p6 z9CKar!`?Sa$6w!1ySL$K6~j)650X2%+?d|>`Ad1})oS&{Og+2$Uy9-0y0-~ur+M#= zN&dyEuX8K%b@k3Q&$YjvZa=@}{+A1iK}8ooEL|b}FZ22LqL{ z=B510@UFd=71lbnuGWgm(u!Z4+3dISoM{vPokIzyMJ3Q=Y4 zchg?9cV6SZvf6V$@674hrV8Fc7G9O6n`~D6nUm4e#`e}PBsk{l6Q38{yjreu6){I9 zX~!>poa~~0nt96Vi@HxQS8gq9JF)U&T={#aOYfbZJ&oczF=>jM|QSnyV(^aj<7&7;(KF|6Sy`*1r=cz9jZ@yTY z$NMD7p*QfS{f%nrwT`_(Yh7mD`hFw%nyb6s+C}nbw_mK$**V4X_nf^Y>%!LXtvtGC zRZHKbvfeF*(Ryo(bgEWIPp;f4#=J=-tLIkywzG@aLf-{E9d9zb7p;de|vZ4snE}E){9r_&z!zm*wXn|;MIwts?)DY z1QyjM@P~+6)nBvpj?>`}ex)4l%Q;l)hAoD>?@Qgf${4+I_t8(Rr$XO0>dI|=qP3bczWLos z!&6!-4|f|cy(JeKzcxZLaO;G?U)xu&dXQv!@w?*ZB3;%LPpvGy{lUBuulyf6m)@Db zUM%}o|LO0&;aBg!R6aU?`qgz8?HYRAWp;9<=DJ*6+VX40r&k}ozL`+8a>8o4?N=s* z=}znz!Ndgn3`X_ddPF5C^TcU8ZV`2V!qL+7o| zyGk_5J{LZ@bn0+uslmsMw!&xPRjtY#Wl!;m)E9;FX6_8<4|u1%b+_G=4<|d$oW9yQ zXJ5uev&AWM&E^z)c$r$&y6;cA;Mwi=J|M7EPu4ieTt0jIk7KI8&xFO<=iYq2<%Q$a zFW<9TPm8@>bjaoE%87nQ*Ponz=5#7+is!ZU{aP~rmZrareZJ(!{7|D?*>=-ns^`Sd z+8yzEP3=)l@4r*?4EONQoSv~4%{;Ds|YH;PQyEpcM1 zm^WA^bE5=^%X;HQ!p%IdOaFy4r@wAZ*#0o?Rm{$`y62iA zMX3!7%XPmeEkCvN?k&qFk%>%^QQspAKTQ8SPgK9~y2$<5=SF|#e5vO8=eeTqPvWUs zv1q;ba}8{GuTH#tPJich_D6ezJJPrR5c>K2K)l4G{r&pabI%JsnRNEy6RFQSr5j?b zZM`RWo3&3~{V?)X!p=R;I$*g~71OLD^zs)@@HQ)-88f4}H0YhyPCoxQy9@E9TbXyd z6m#tSqAmPh?uB{G?UGe7_C|Z+CpE@wvOII*YuCfKoBO7&mOP<4`|w+_y$*UQCs=o; z7(a1JJ3WDWrmnMI$~K;z6K)z26eya0n$@LKRouMu zFjRnTT>1Zg-_v{QpR@13i->zK_&0y;(?`p8sx5l|2;z)Al}}#9E)?_ViSdsXeail1 z(%IQhZ{6D@ulgivnS_Jh%{R*7r|YInc80nITob&_%uinZ{6#)~UBGFLj(?OUMX zpm+0*%JoCfO}1~^mmU+nl;P5}*X!di%-X#E+H~D6rYDom`WCH?{PMyux7=-AD3?O+ zPO)d1?ZtIZHlNoN0$X6}ZSsAs^7MZxQ<*%X*KKu;iujq{!SZC%*|?omS5y9-oMim` z)k~S3X`b^pKYdm0bFm?2(@pK^Qk(347oB{0^k!?jh{LI(m;Ni$k52P`E4P?&XUgXA zb^BCn-@ZP6W%H6<#mAkw6jFCi`N(}fETdLUg7bvx?7c-_b-voZ-Td^`+}ZCXcK!<6 zrhhx>CiC@p4X~e2Z~a#A`&82TNwXZ@M=WKyB)3yezhlSS+Q->$1! zR`+j{M_@zDCd(gQSI$k;Hk;sW_WSf#`|b(cJKw(14)6bK=zaXm&PU8UQ#P-UImas4 z|4ftNMC+Zl7_r<^JDDQIxw%ix{!6*1@}xqtNO5!O ziPkgI6vd6A+hR`SP(xtn>8DJKj!f#FedI}&uC-D8yH69fXPR?b#!<@x3`Y(l8=0SK z1ta((QElU^-Or+-;$Fcn>aYb*AbN`gD|K;p%nkc!m=%Syq#ajoxi*K~2+gXJ!OGtS#%g!sXF=o@v@O6K-*2bb-JDUJ&p zV>VT8j``)|oU{Pxw^!V|pBc0V~aTiBG#`Bc$UpUr}+*7<_mHSzQu0}U3<)1gL(qt<`e!*D`% zc5Knh;Is?hnMzOZo$_r?_8pV;(jshs`_1omgA7sslGR~QyMETCJ@bv1=}y0;-}cAs z&4;-$H@9-^OxgTAqj3MLquyI)7C)W2D%x;iD&+1)HHMWaGg6-F(zSQ5};A=ALch2mEZ2bK(N?Ys{4O6(&a&_XT z)oD8#&*nSpH?iE|hM8V)#NyWIb021{wkkHcFw^IsckuksEB^ChYRzs*mgX+HWKxv4 z|3;?Iz06leYd7jm|5CT{sk!l%O;tOdYi@ZuZ;S1Pf7hmqGf9_T+=oh2l#k zjQ;W`@3ok2S(TO+`F_rt?eXQQ|APfBzb?Mdd8cEy#?doVGXGu6+Bx;_`Ui1m_o}^~ zJNfIk*V2Xy_}}jQvq5l+V0mEIqaB(5?q%%_-RLG?&9Axno@HeI{(m-^kN?SUp2Bmh zSUp4e-J%owb~@bXug~M%_da&w*K;SUet&(wQshGWueAqnH%b?ZyPgYtm121%Eoy6~ z>osfj*Ke=SUmkSf;J!4L{3Eef!b-P7Z9YG_?1$f%-W}aazmpDfY;XTC31Zk4GocK7 zh~rk~s$8gpm=S#2W5M4w<$l+0uV21xqQh^95B+nM8^1xEIq#Mm)7$jD#hdQc*>3G+ zxpltkdCK0uRaw%UTjpaoc;8>O3w6>VLAO^l-(p7|4tS8@A$~z_S_xzBqN6uYRLh?} zmwNque3`Bvx8~D6hPY3M4b3jOIp678E88dCX;PfO-wn-AUmweI#3 zexq&cdF3pFp4KoPZ%@j(QX+r)h^CCsvboAG)fTVM^j?>UzWzq~iq-aa4;NNE30=f^ zOngE74f)5R=68=96_y?Sv483B1perGze`oS>t283x47T>l{2|{ZROf!a&;#n>Q|PT z{c=0|eWTBenU0U&x=5XhK6lqs;^Dru)*0&sF7{uSi2ko}PT)$7{OKnvRW5b?eZMT= z*&5?x8{`GA{j#sydF)2jI>F`gZ+#>FpS3bK&XNCppZEQV(yq;g?TXzSKi*|#i+}w4 zI6sfc3bDe~AZunNS$2k%Ecff)d|`f}SzfTkshhj%zWveNcC3F-(W9s3{St!PKke*K zJtUIlx#f{hM6!>j@`1`VPYdS#7T?hKW@pwH-;B$(I+OLZCWhx|M`;ybJ6W~HbxZp5 ze=her)_Oc!%eNrK<$jCV-Qz|t!()6GiEjP6CfJPxbc7eX3s}+Mm3t@GbAT zaDn7^Pm1MFw|J(VbA8Uvzj4~m(|d}R>_2v`=3VdY`Fplcem{BJ<-g7E_a<(b>c8;` z$W@D;98CxBsl^GhNF6pZS0Bli@x|^Sgg19*nNhJ7zZfeez6=U)o|X_pFy% z!n9d~0R_D9`X~QmaZBX-xCC>nC}Q!>*kacf{S?RpBb+SAp~Wt~$(699Q<@nrx2 literal 0 HcmV?d00001 diff --git a/docs/source_docs/user_guide/inputs/mesh_and_gridding.rst b/docs/source_docs/user_guide/inputs/mesh_and_gridding.rst index 96f6d3f..62805ad 100644 --- a/docs/source_docs/user_guide/inputs/mesh_and_gridding.rst +++ b/docs/source_docs/user_guide/inputs/mesh_and_gridding.rst @@ -1,84 +1,153 @@ .. sec:InputsMeshAndGridding: -Mesh and gridding -================= - -Mesh, grids and tiles ---------------------- +Mesh, grids, and tiles +====================== Mesh -^^^^ +---- + +.. rubric:: Level-0 mesh + +The following inputs are defined using the ``arm`` prefix. -There are some settings we can specify for the mesh and the automatic mesh -refinement algorithm. These settings must be preceded by "amr." in the input -file. +.. _InputsTable_mesh: +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +| | | | | +======================+=======================================================================+=============+===========+ -| max_level | Maximum level of refinement allowed (0 when single-level) | Int | 0 | +| max_level | Maximum level of refinement. The default value, 0, means that there | int | 0 | +| | is one cell size covering the entire domain (i.e., single-level). | | | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ +| n_cell | Number of cells at level 0 in each coordinate direction | ints | 0 0 0 | ++----------------------+-----------------------------------------------------------------------+-------------+-----------+ + +The level 0 mesh spacing is computed for each direction by dividing the :ref:`domain length` by the +number of cells. The mesh spacing is **required** to be the same in all directions: + +.. math:: + + \frac{\text{prob_hi[0] - prob_lo[0]}}{\text{n_cell[0]}} + = \frac{\text{prob_hi[1] - prob_lo[1]}}{\text{n_cell[1]}} + = \frac{\text{prob_hi[2] - prob_lo[2]}}{\text{n_cell[2]}} + + +The inputs for defining the mesh for a single-level simulation are demonstrated in the +:ref:`following example` and illustrated in :numref:`fig_basic_mesh_ex`. +In this example, the domain is a :math:`4 \times 1 \times 1` cuboid, and there are +:math:`32 \times 8 \times 8` cells in the *X*, *Y*, and *Z* directions, respectively. +The result is a uniform mesh spacing of :math:`0.125` *m* in all three directions. + +.. _inputs_mesh_ex: + +.. code-block:: bash + :caption: Snippet of intpus for mesh example. This is not a complete input file. + + # Define periodicity and domain extents + # ------------------------------------------------------------- + geometry.coord_sys = 0 # Cartesian coordinates + geometry.is_periodic = 0 0 0 # periodicity for each direction + geometry.prob_lo = 0. 0. 0 # lo corner of physical domain. + geometry.prob_hi = 4. 1. 1. # hi corner of physical domain + + # Define the maximum level of refinement and number of cells + # ------------------------------------------------------------- + arm.max_level = 0 + arm.n_cell = 32 8 8 + + +.. _fig_basic_mesh_ex: + +.. figure:: ./images/geometry/mesh_lev0_ex.png + :width: 100% + :align: center + :alt: domain used with box embedded boundary + + Example of a single-level mesh. + +.. warning:: + + MFIX-Exa simulations with a non-uniform mesh will not run. + + +----- + + +.. rubric:: Mesh refinement + + +The following inputs are defined using the ``arm`` prefix. These inputs control the automatic +mesh refinement algorithm and are only applicable when ``amr.max_level > 0``. + ++----------------------+-----------------------------------------------------------------------+-------------+-----------+ +| | Description | Type | Default | ++======================+=======================================================================+=============+===========+ | grid_eff | Threshold value to ensure grids do not contain too large a fraction | Real | 0.7 | | | of un-tagged cells. | | | -| | Applicable only when mesh refinement is enabled ("max_level" > 0). | | | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ -| n_error_buf | Controls the number of tagged cells before grids are defined. Used | Int | 1 | +| n_error_buf | Controls the number of tagged cells before grids are defined. Used | int | 1 | | | to ensure coarse/fine boundaries are not too close to tagged cells. | | | -| | Applicable only when mesh refinement is enabled ("max_level" > 0). | | | -+----------------------+-----------------------------------------------------------------------+-------------+-----------+ -| n_cell | Number of cells at level 0 in each coordinate direction | Ints | 0 0 0 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ +.. caution:: + + **Mesh refinement limitations:** + + * MFIX-Exa simulations with particles currently do not support mesh refinement. + +See the `AMReX documentation `_ +for details on the adaptive mesh refinement algorithms. + +----- + Grids -^^^^^ +----- -The following inputs must be preceded by "amr." and determine how we create the grids. +The following inputs are defined using the ``arm`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | +======================+=======================================================================+=============+===========+ -| max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | +| max_grid_size_x | Maximum number of cells at level 0 in each grid in *X* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ -| max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | +| max_grid_size_y | Maximum number of cells at level 0 in each grid in *Y* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ -| max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | +| max_grid_size_z | Maximum number of cells at level 0 in each grid in *Z* | int | 32 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ -| blocking_factor_x | Each grid must be divisible by blocking_factor_x in x-direction | Int | 8 | +| blocking_factor_x | Each grid in *X* must be divisible by ``blocking_factor_x`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ -| blocking_factor_y | Each grid must be divisible by blocking_factor_y in y-direction | Int | 8 | +| blocking_factor_y | Each grid in *Y* must be divisible by ``blocking_factor_y`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ -| blocking_factor_z | Each grid must be divisible by blocking_factor_z in z-direction | Int | 8 | +| blocking_factor_z | Each grid in *Z* must be divisible by ``blocking_factor_z`` | int | 8 | +----------------------+-----------------------------------------------------------------------+-------------+-----------+ +The domain is decomposed into *grids* by dividing the number of cells by the max grid size +for each direction (e.g., ``n_cells[0]/max_grid_size_x``). The blocking factor ensures that +the grids will be sufficiently coarsenable for good multigrid performance; therefore, the +``max_grid_size`` must be divisible by the corresponding ``blocking_factor``. -The following inputs must be preceded by "particles." +.. note:: + + The `AMReX documentation `_ contains a significant + amount of information on grid creation and load balancing. Users are **strongly** encouraged to + read the relevant sections. + + +----- -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| | Description | Type | Default | -+======================+=======================================================================+=============+==============+ -| max_grid_size_x | Maximum number of cells at level 0 in each grid in x-direction | Int | 32 | -| | for grids in the ParticleBoxArray if dual_grid is true | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| max_grid_size_y | Maximum number of cells at level 0 in each grid in y-direction | Int | 32 | -| | for grids in the ParticleBoxArray if dual_grid is true | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| max_grid_size_z | Maximum number of cells at level 0 in each grid in z-direction | Int | 32 | -| | for grids in the ParticleBoxArray if dual_grid is true. | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ Tiles -^^^^^ +----- -The following inputs must be preceded by "fabarray_mfiter." and determine how we create the logical tiles: +The following inputs are defined using the ``fabarray`` prefix. +----------------------+-----------------------------------------------------------------------+----------+-------------+ | | Description | Type | Default | +======================+=======================================================================+==========+=============+ -| tile_size | Maximum number of cells in each direction for (logical) tiles | IntVect | 1024000 | -| | (3D CPU-only) | | 1024000,8,8 | +| mfiter_tile_size | Maximum number of cells in each direction for (logical) tiles | IntVect | 1024000,8,8 | +----------------------+-----------------------------------------------------------------------+----------+-------------+ -The following inputs must be preceded by "particles." +The following inputs are defined using the ``particles`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+--------------+ | | Description | Type | Default | @@ -87,69 +156,9 @@ The following inputs must be preceded by "particles." | | in the ParticleBoxArray if dual_grid is true. | | | +----------------------+-----------------------------------------------------------------------+-------------+--------------+ +When running on shared memory machines using an OpenMP enabled executable, *grids* are subdivided into +*tiles* and iterated over to improve data locality by cache blocking. -Load balancing --------------- - -Note that when running a granular simulation, i.e., no fluid phase, :cpp:`mfix.dual_grid` must be 0. Hence, -the :cpp:`particles.max_grid_size` (in each direction) have no meaning. Therefore the fluid grid and tile -sizes should be set for particle load balancing. It may also be necessary to set the blocking factors to 1. - - -The following inputs must be preceded by "mfix." and determine how we load balance: - -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| | Description | Type | Default | -+======================+=======================================================================+=============+==============+ -| dual_grid | If true then use the "dual_grid" approach for load balancing | Bool | False | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| regrid_int | How often to regrid (in number of steps at level 0) | Int | -1 | -| | if regrid_int = -1 then no regridding will occur | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| load_balance_fluid | Only relevant if (dual_grid); if so do we also regrid mesh data | Int | 0 | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| load_balance_type | What strategy to use for load balancing | String | KnapSack | -| | Options are "KnapSack", "SFC", or "Greedy" | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| knapsack_weight_type | What weighting function to use if using Knapsack load balancing | String | RunTimeCosts | -| | Options are "RunTimeCosts" or "NumParticles"" | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| knapsack_nmax | Maximum number of grids per MPI process if using knapsack algorithm | Int | 128 | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| greedy_dir | The direction in which the greedy algorithm cuts overloaded boxes | Int | 0 | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| greedy_min_grid_size | The minimum particle grid size in the greedy load balance algorithm | Int | 2 | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| greedy_3d | Partition particle grids in 3D with the greedy algorithhm | Bool | False | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| overload_tolerance* | The ratio between the maximum workload and the average workload | Real | 1.2 | -| | in the greedy algorithm | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| underload_tolerance* | The ratio between the minimum workload and the average workload | Real | 0.8 | -| | in the greedy algorithm | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ -| grid_pruning | Remove all covered grids from the base mesh; this may result in | Bool | False | -| | disjoined grids | | | -+----------------------+-----------------------------------------------------------------------+-------------+--------------+ - -\* The greedy partitioning algorithm uses the tolerances to set the expected -workload range for each rank but doesn't strictly enforce them. The algorithm -creates a partition as balance as possible even if the partition doesn't -meet the tolerances. - -To allow a user to verify the breakdown of fluid grids created before running a full simulation, an input option, -:cpp:`mfix.only_print_grid_report` is supported. By default, it is :cpp:`False`. When set to :cpp:`True`, the run uses -minimal memory to print the grid coverage report and exits immediately after that. - - -.. todo:: Is this needed anymore? - - The following inputs must be preceded by "particles." - - +----------------------+-----------------------------------------------------------------------+-------------+--------------+ - | | Description | Type | Default | - +======================+=======================================================================+=============+==============+ - | reduceGhostParticles | Remove unused ghost particles from communication, | Bool | True | - | | only supported by GPU build | | | - +----------------------+-----------------------------------------------------------------------+-------------+--------------+ +.. note:: + MFIX-Exa disables tiling when using GPU accelerators. -- GitLab From 3a7b4bfc6e2d94a50e927ad65d9513dfff8ddd04 Mon Sep 17 00:00:00 2001 From: Jordan Musser Date: Mon, 29 Jan 2024 10:10:16 -0500 Subject: [PATCH 2/2] Add wdf corrections --- .../source_docs/user_guide/inputs/advanced.rst | 18 +++++++++--------- .../user_guide/inputs/mesh_and_gridding.rst | 6 +++--- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/source_docs/user_guide/inputs/advanced.rst b/docs/source_docs/user_guide/inputs/advanced.rst index ea67991..b650fdc 100644 --- a/docs/source_docs/user_guide/inputs/advanced.rst +++ b/docs/source_docs/user_guide/inputs/advanced.rst @@ -37,15 +37,15 @@ GPU memory The following inputs must be preceded by "amrex." -+------------------------+-----------------------------------------------------------------------+-------------+--------------+ -| | Description | Type | Default | -+========================+=======================================================================+=============+==============+ -| the _arena_is_managed | Abort if an invalid floating point exception is encountered. | Int | 0 | -+------------------------+-----------------------------------------------------------------------+-------------+--------------+ -| abort_on_out_of_gpu_memory | rt if a division by zero is computed. | Int | 0 | -+------------------------+-----------------------------------------------------------------------+-------------+--------------+ -| the_area_init_size | Abort if and overflow is detected. | Int | 0 | -+------------------------+-----------------------------------------------------------------------+-------------+--------------+ ++----------------------------+-----------------------------------------------------------------------+-------------+--------------+ +| | Description | Type | Default | ++============================+=======================================================================+=============+==============+ +| the_arena_is_managed | Abort if an invalid floating point exception is encountered. | Int | 0 | ++----------------------------+-----------------------------------------------------------------------+-------------+--------------+ +| the_arena_init_size | Abort if and overflow is detected. | Int | 0 | ++----------------------------+-----------------------------------------------------------------------+-------------+--------------+ +| abort_on_out_of_gpu_memory | Abort if a division by zero is computed. | Int | 0 | ++----------------------------+-----------------------------------------------------------------------+-------------+--------------+ Load balancing diff --git a/docs/source_docs/user_guide/inputs/mesh_and_gridding.rst b/docs/source_docs/user_guide/inputs/mesh_and_gridding.rst index 62805ad..82fbc73 100644 --- a/docs/source_docs/user_guide/inputs/mesh_and_gridding.rst +++ b/docs/source_docs/user_guide/inputs/mesh_and_gridding.rst @@ -8,7 +8,7 @@ Mesh .. rubric:: Level-0 mesh -The following inputs are defined using the ``arm`` prefix. +The following inputs are defined using the ``amr`` prefix. .. _InputsTable_mesh: @@ -76,7 +76,7 @@ The result is a uniform mesh spacing of :math:`0.125` *m* in all three direction .. rubric:: Mesh refinement -The following inputs are defined using the ``arm`` prefix. These inputs control the automatic +The following inputs are defined using the ``amr`` prefix. These inputs control the automatic mesh refinement algorithm and are only applicable when ``amr.max_level > 0``. +----------------------+-----------------------------------------------------------------------+-------------+-----------+ @@ -103,7 +103,7 @@ for details on the adaptive mesh refinement algorithms. Grids ----- -The following inputs are defined using the ``arm`` prefix. +The following inputs are defined using the ``amr`` prefix. +----------------------+-----------------------------------------------------------------------+-------------+-----------+ | | Description | Type | Default | -- GitLab