Add section on AMReX Submodule

docs/source/BuildingCMake.rst

@@ -13,12 +13,12 @@ The CMake build process is summarized as follows:
     mkdir /path/to/builddir
     cd    /path/to/builddir
     cmake [options] -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel] /path/to/mfix
-    make  
+    make
 
 In the above snippet, ``[options]`` indicates one or more options for the
 customization of the build, as described later in this section.
 If the option ``CMAKE_BUILD_TYPE`` is omitted,
-``CMAKE_BUILD_TYPE=Release`` is assumed. 
+``CMAKE_BUILD_TYPE=Release`` is assumed.
 
 There are two modes to build MFiX-Exa with cmake:
 
@@ -56,7 +56,7 @@ Next, configure, build and install AMReX as follows:
     > cmake -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel] -DENABLE_PARTICLES=yes -DENABLE_AMRDATA=yes -DENABLE_EB=yes [other amrex options] -DCMAKE_INSTALL_PREFIX:PATH=/absolute/path/to/installdir ..
     > make install
 
-The options **ENABLE\_PARTICLES=yes**, **ENABLE\_AMRDATA=yes**, and **ENABLE\_EB=yes** are required by MFiX-Exa. ``[other amrex options]`` in the snippet above refers to any other AMReX configuration option in addition to the required ones. Please refer to the `AMReX user guide <https://amrex-codes.github.io/amrex/docs_html/BuildingAMReX.html#building-with-cmake>`_ for more details on building AMReX with CMake.    
+The options **ENABLE\_PARTICLES=yes**, **ENABLE\_AMRDATA=yes**, and **ENABLE\_EB=yes** are required by MFiX-Exa. ``[other amrex options]`` in the snippet above refers to any other AMReX configuration option in addition to the required ones. Please refer to the `AMReX user guide <https://amrex-codes.github.io/amrex/docs_html/BuildingAMReX.html#building-with-cmake>`_ for more details on building AMReX with CMake.
 
 
 Building MFiX-Exa
@@ -72,7 +72,7 @@ Clone and build MFiX-Exa:
     > cmake -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel] [mfix options] -DAMReX_ROOT=/absolute/path/to/amrex/installdir ..
     > make -j
 
-    
+
 Passing ``-DAMReX_ROOT=/absolute/path/to/amrex/installdir`` instructs CMake to search
 ``/absolute/path/to/amrex/installdir`` before searching system paths
 for an available AMReX installation.
@@ -115,10 +115,10 @@ SUPERBUILD Instructions (recommended)
 -------------------------------------
 
 If no AMReX installation is found on the system, or if one is found but does not meet MFiX-Exa requirements, MFiX-Exa CMake falls back to **SUPERBUILD** mode.
-When building in **SUPERBUILD** mode, the AMReX git repo is checked out via a git submodule before AMReX CMake build system is included into MFiX-Exa CMake infrastructure. Consequently, MFiX-Exa CMake inherents AMReX's CMake targets and configuration options, that is, MFiX-Exa and AMReX are configured and built as a single entity.   
+When building in **SUPERBUILD** mode, the AMReX git repo is checked out via a git submodule before AMReX CMake build system is included into MFiX-Exa CMake infrastructure. Consequently, MFiX-Exa CMake inherents AMReX's CMake targets and configuration options, that is, MFiX-Exa and AMReX are configured and built as a single entity.
 
 
-Assuming no valid AMReX installation is present on the target system, and ``AMReX_ROOT`` is not set in the environment, the following code will build  MFiX-Exa in **SUPERBUILD** mode:  
+Assuming no valid AMReX installation is present on the target system, and ``AMReX_ROOT`` is not set in the environment, the following code will build  MFiX-Exa in **SUPERBUILD** mode:
 
 .. code:: shell
 
@@ -132,6 +132,59 @@ Assuming no valid AMReX installation is present on the target system, and ``AMRe
 ``[amrex options]`` is a list of any of the AMReX configuration options listed in the `AMReX user guide <https://amrex-codes.github.io/amrex/docs_html/BuildingAMReX.html#building-with-cmake>`_
 
 
+Working with the AMReX submodule
+--------------------------------
+
+For MFiX-Exa development or testing, you may need to build with a different
+branch or release of AMReX.
+
+The ``subprojects/amrex`` directory is a Git repo, so use all standard Git
+commands. Either ``cd subprojects/amrex`` to run Git commands in the ``amrex``
+directory, or use ``git -C subprojects/amrex`` in the MFiX-Exa repo. For
+instance, to build with the ``my-amrex-branch`` branch of the AMReX repo:
+
+.. code:: shell
+
+    > git -C subprojects/amrex checkout my-amrex-branch
+    > git status
+    ...
+    modified:   subprojects/amrex (new commits)
+
+The branch ``my-amrex-branch`` will then be used when building MFiX-Exa.
+
+To revert to the default version of the AMReX submodule, run ``git submodule
+update``:
+
+.. code:: shell
+
+    > git submodule update
+    > git status
+    ...
+    nothing to commit, working tree clean
+
+You can edit, commit, pull, and push AMReX changes from ``subprojects/amrex``.
+AMReX development is outside the scope of this document. Run ``git status`` in
+the top-level MFix-Exa repo to see whether the AMReX submodule has new commits,
+modified files, or untracked files.
+
+To update the AMReX submodule reference by MFiX-Exa:
+
+.. code:: shell
+
+    > git add subprojects/amrex
+    > git commit -m 'Updating AMReX version'
+
+This will only update the AMReX SHA-1 referenced by MFiX-Exa. Uncommitted AMReX
+changes and untracked AMReX files under ``subprojects/amrex`` are not added by
+``git add subprojects/amrex``. Change directories to ``subprojects/amrex`` and
+commit any AMReX changes first before ``git add subprojects/amrex``.
+
+.. note::
+
+    Only update the AMReX submodule reference in coordination with the other
+    MFiX-Exa developers!
+
+
 Few more notes on building MFiX-Exa
 -----------------------------------