Commit 6201ff14 authored by Jordan Musser's avatar Jordan Musser
Browse files

Update code testing docs (!158)

parent fbdf45a1
Loading
Loading
Loading
Loading
+1 −3
Changes for docs/source_docs/index.rst: 1 added line, 3 removed lines.
Original line number Diff line number Diff line
@@ -78,10 +78,8 @@ and performance:
   :maxdepth: 0
   :caption: Tests and Benchmarks:

   test_benchmarks/RunningTestSuite
   test_benchmarks/RegressionTesting
   test_benchmarks/CITests
   test_benchmarks/NightlyTests
   test_benchmarks/RegressionTesting
   test_benchmarks/qualitative_bencharks/index
   test_benchmarks/verification/index

+241 −105
Changes for docs/source_docs/test_benchmarks/CITests.rst: 241 added lines, 105 removed lines.
Original line number Diff line number Diff line
.. _Chap:CITesting:

Continuous Integration
======================

The following regression tests are run every time a commit is pushed to the main
MFIX-Exa repository on the NETL gitlab.

For each of the tests in the chart below, there are
three directional variations; these are identified in the repository as, 
for example, FLD01-x, FLD01-y, and FLD01-z.  

For each direction, where appropriate, there are multiple versions, with the following notations:

  * SGS: single grid serial

  * MGS: multiple grid serial

  * TGS: tiled grid serial

  * MGP: multiple grid parallel

Below Ng = number of grids, Npa = number of particles, Np = number of MPI ranks.

All the FLD cases are fluid-only and steady state.
All the DEM cases are particle-only except for DEM06 and DEM07 which are fluid and particles;
these both use the "BVK2" drag type.
In all cases the particle data were read in from "particle_input.dat"

None of these tests have non-rectangular geometry.

"NSW" means "No Slip Wall" and "Per" is "periodic."
"MI/PO" refers to Mass     Inflow at the low end of the domain and Pressure Outflow at the high end.
"PI/PO" refers to Pressure Inflow at the low end of the domain and Pressure Outflow at the high end.

Additional detail about these problems is given in tests/README.md

Single-grid, single-process (SGS) particle-only tests:

+-------+----+----+----+------+------+-------+-----+--------------------+
| Test  | nx | ny | nz | bc_x | bc_y | bc_z  | Npa | Description        |
+=======+====+====+====+======+======+=======+=====+====================+
| DEM01 | 2  | 5  | 5  | NSW  | Per  | Per   | 1   | Freely falling     |
|       |    |    |    |      |      |       |     | particle with      |
|       |    |    |    |      |      |       |     | wall collision     |
+-------+----+----+----+------+------+-------+-----+--------------------+
| DEM02 | 2  | 5  | 5  | NSW  | Per  | Per   | 1   | Multiple bounces   |
|       |    |    |    |      |      |       |     | with bounce height |
|       |    |    |    |      |      |       |     | measured           |
+-------+----+----+----+------+------+-------+-----+--------------------+
| DEM03 | 2  | 5  | 5  | NSW  | Per  | Per   | 2   | Two stacked        |
|       |    |    |    |      |      |       |     | compressed         |
|       |    |    |    |      |      |       |     | particles          |
+-------+----+----+----+------+------+-------+-----+--------------------+
| DEM04 | 4  | 4  | 4  | NSW  | Per  | Per   | 1   | Single particle    |
|       |    |    |    |      |      |       |     | slipping on a      |
|       |    |    |    |      |      |       |     | rough surface      |
+-------+----+----+----+------+------+-------+-----+--------------------+
| DEM05 | 5  |  2 |  5 | Per  | Per  | Per   | 93  | Oblique particle   |
|       |    |    |    |      |      |       |     | collisions         |
|       |    |    |    |      |      |       |     |                    |
+-------+----+----+----+------+------+-------+-----+--------------------+


Steady-state fluid-only tests:

+-------+-----+----+----+----+-------+------+------+-----+----------------------+
| Test  |     | nx | ny | nz | bc_x  | bc_y | bc_z | Ng  | Np                   |
+=======+=====+====+====+====+=======+======+======+=====+======================+
| FLD01 |     | 8  | 8  | 4  | Per   | NSW  | Per  | Poiseuille flow            |
+-------+-----+----+----+----+-------+------+------+-----+----+-----------------+
|       | SGS |    |    |    |       |      |      | 1   | 1  |                 |
|       | MGS |    |    |    |       |      |      | 4   | 1  |                 |
|       | MGP |    |    |    |       |      |      | 4   | 8  |                 |
+-------+-----+----+----+----+-------+------+------+-----+----+-----------------+
| FLD02 |     | 80 |16  | 16 | MI/PO | NSW  | NSW  | Couette flow               |
+-------+-----+----+----+----+-------+------+------+-----+----+-----------------+
|       | SGS |    |    |    |       |      |      | 1   | 1  |                 |
|       | MGS |    |    |    |       |      |      | 40  | 1  |                 | 
|       | MGP |    |    |    |       |      |      | 40  | 8  |                 |
+-------+-----+----+----+----+-------+------+------+-----+----+-----------------+
| FLD03 |     | 8  | 8  | 4  | PI/PO | NSW  | Per  | Poiseuille flow            |
+-------+-----+----+----+----+-------+------+------+-----+----+-----------------+
|       | SGS |    |    |    |       |      |      | 1   | 1  |                 |
|       | MGS |    |    |    |       |      |      | 4   | 1  |                 |
|       | MGP |    |    |    |       |      |      | 4   | 8  |                 |
+-------+-----+----+----+----+-------+------+------+-----+----+-----------------+

Coupled particle/fluid tests:

+-------+-----+----+----+----+------+------+------+------+----+--------------------+
| Test  |     | nx | ny | nz | bc_x | bc_y | bc_z | Npa  | Ng | Np                 |
+=======+=====+====+====+====+======+======+======+======+====+====================+
| DEM06 |     | 50 | 5  | 5  |  NSW | NSW  | NSW  | 1    | Single particle falling | 
|       |     |    |    |    |      |      |      |      | under gravity           |
+-------+-----+----+----+----+------+------+------+------+----+----+---------------+
|       | SGS |    |    |    |      |      |      |      | 1  | 1  |               |
|       | MGS |    |    |    |      |      |      |      | 10 | 1  |               |
|       | MGP |    |    |    |      |      |      |      | 10 | 8  |               |
+-------+-----+----+----+----+------+------+------+------+----+----+---------------+
| DEM07 |     | 20 | 20 | 20 | Per  | Per  | Per  | 1222 | Homogeneous cooling     |
|       |     |    |    |    |      |      |      |      | system                  |
+-------+-----+----+----+----+------+------+------+------+----+----+---------------+
|       | SGS |    |    |    |      |      |      |      | 1  | 1  |               |
|       | MGS |    |    |    |      |      |      |      | 8  | 1  |               |
|       | MGP |    |    |    |      |      |      |      | 8  | 8  |               |
+-------+-----+----+----+----+------+------+------+------+----+----+---------------+
Continuous Integration (CI)
===========================

MFIX-Exa uses a GitLab-based Continuous Integration (CI) pipeline to ensure
code quality, portability, and correctness across multiple compilers, GPU
backends, and build configurations. This page describes what the CI runs,
how to reproduce CI behavior locally, and provides a high-level overview of
the MFIX-Exa test suite.


When the CI Pipeline Runs
-------------------------

The CI pipeline is triggered under the following conditions:

* **Merge requests** – runs linting, warnings builds, and multi-backend builds.
* **Commits to the ``develop`` branch** – runs the full test pipeline.
* **Tag pushes** – used for releases.
* **Manual runs** – triggered from the GitLab web interface.


CI Pipeline Structure
---------------------

The MFIX-Exa CI is organized into two distinct execution paths:

* **Merge Request (MR) pipelines** – run on every push to a merge request.
  These pipelines focus on code quality, portability, and build correctness.
  They include the **Lint**, **Warnings**, and **Build** stages.

* **Post-merge pipelines** – run only after changes are merged into the
  ``develop`` branch. These pipelines run the full **CTest-based test suite**
  and the specialized **Ascent** and **MPMD** tests.

This separation ensures fast feedback during code review while reserving
the more expensive tests for post-merge validation.


Merge Request Pipeline Stages
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Lint**

  Static analysis and formatting checks:

  * Shell scripts (``shellcheck``)
  * Spelling (``codespell``)
  * Tabs and trailing whitespace removal
  * Python linting (``pylint``) for the GUI

**Warnings Builds**

  Compiler-strict builds using GCC and Clang in both Debug and Release modes.
  These builds enable extensive warning flags (``-Wall``, ``-Wextra``,
  ``-Werror``, etc.) to catch portability and correctness issues early.

**Builds**

  Multiple configurations are compiled to ensure portability across compilers
  and GPU backends:

  * CPU-only builds
  * GPU-enabled builds (CUDA, HIP, SYCL)
  * CSG-enabled builds
  * MPMD-enabled builds
  * Unit tests

These MR-stage jobs verify that MFIX-Exa builds cleanly and consistently
before code is merged.


Post-Merge Test Pipeline
~~~~~~~~~~~~~~~~~~~~~~~~

After changes are merged into the ``develop`` branch, the CI runs the full
MFIX-Exa test suite. These jobs are triggered by ``rules:`` blocks that
restrict execution to the ``develop`` branch.

The post-merge pipeline includes:

**CTest-based MFIX-Exa Test Suite**

  Tests are grouped by label:

  * ``SGS`` – single-grid serial tests
  * ``MGS`` – multi-grid serial tests
  * ``TGS`` – tiled-grid serial tests
  * ``MGP`` – multi-grid parallel tests
  * ``TGPO`` – tiled-grid parallel OpenMP tests
  * ``TGSO`` – tiled-grid serial OpenMP tests

**Ascent Visualization Test**

  Runs a small HCS benchmark with Ascent enabled and verifies image output.

**MPMD Tutorial Test**

  Runs the MPMD tutorial example and checks numerical output and generated
  images.


How to Reproduce CI Locally
---------------------------

Building MFIX-Exa
~~~~~~~~~~~~~~~~~

A typical local build mirrors the CI configuration. For example:

.. code-block:: bash

   cmake -S <path-to-mfix> -B build \
         -DCMAKE_BUILD_TYPE=Release \
         -G Ninja

   cmake --build build --target build_tests unit_tests

Additional options (e.g., enabling MPI, OpenMP, GPU backends, or CSG)
can be added as needed.


Running the MFIX-Exa Test Suite
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CTest is used to run the MFIX-Exa test suite. All tests are available
locally, regardless of which subset the CI runs.

List all tests:

.. code-block:: bash

   ctest --test-dir build/ -N

Run all tests:

.. code-block:: bash

   ctest --test-dir build/

Run a specific test by name:

.. code-block:: bash

   ctest --test-dir build/ -R DEM01

Run a specific test by index:

.. code-block:: bash

   ctest --test-dir build/ -I 3,3

Run selected the signle-grid serial DEM tests in the ``y`` direction:

.. code-block:: bash

   ctest --test-dir build/ -R "DEM(01|03|04|05|06)-y_SGS|DEM07_SGS"


Running Unit Tests
~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   cmake --build build --target unit_tests
   ctest --test-dir build/ -R unit_test -V


Summary of MFIX-Exa Test Categories
-----------------------------------

The MFIX-Exa test suite includes fluid-only tests, particle-only tests,
and coupled fluid/particle tests. The CI runs a representative subset of
these, while the full suite is available locally.

Below is a high-level summary of the test categories and representative
cases.

.. list-table::
   :header-rows: 1
   :widths: 15 20 65

   * - Test
     - Type
     - Description
   * - ``FLD01``
     - Fluid
     - 2D channel (Poiseuille) flow in a periodic domain.
   * - ``FLD02``
     - Fluid
     - 2D Couette flow in a channel.
   * - ``FLD03``
     - Fluid
     - Pressure-driven channel flow with specified pressure boundaries.
   * - ``DEM01``
     - DEM
     - Freely falling particle with wall collision; verifies spring-dashpot model.
   * - ``DEM02``
     - DEM
     - Repeated particle bouncing; compares rebound height to theory.
   * - ``DEM03``
     - DEM
     - Two compressed particles between walls; tests multi-particle contact.
   * - ``DEM04``
     - DEM
     - Rolling/slipping particle on a rough surface; tests friction model.
   * - ``DEM05``
     - DEM
     - Oblique particle collisions with a flat wall; tests tangential restitution.
   * - ``DEM06``
     - Coupled
     - Single particle terminal velocity in a fluid; tests drag models.
   * - ``DEM07``
     - Coupled
     - Homogeneous Cooling System (HCS); tests granular temperature decay.


Details for each case are available in ``tests/README.md``.


Relationship to Regression Tests
--------------------------------

The CI pipeline **does not** run the CCSE regression suite.

* The **Regression Tests** page documents the CCSE/AMReX regression framework.
* The CI page documents the **CTest-based MFIX-Exa Test Suite** and infrastructure tests.

These two systems are complementary:

* CI ensures build correctness and basic functional behavior.
* Regression tests ensure numerical consistency across platforms and versions.


Developer Expectations
----------------------

* All CI jobs must pass before merging into ``develop``.
* Lint and warnings jobs help maintain code quality.
* Build jobs ensure portability across compilers and GPU backends.
* Test jobs ensure functional correctness.
* CI artifacts contain detailed logs for debugging failures.
−114

File deleted.

+279 −50

File changed.

Preview size limit exceeded, changes collapsed.

−88

File deleted.