Continuous Integration (CI)¶
MFIX-Exa uses a GitLab-based Continuous Integration (CI) pipeline to ensure code quality, portability, and correctness across multiple compilers, GPU backends, and build configurations. This page describes what the CI runs, how to reproduce CI behavior locally, and provides a high-level overview of the MFIX-Exa test suite.
When the CI Pipeline Runs¶
The CI pipeline is triggered under the following conditions:
Merge requests – runs linting, warnings builds, and multi-backend builds.
Commits to the ``develop`` branch – runs the full test pipeline.
Tag pushes – used for releases.
Manual runs – triggered from the GitLab web interface.
CI Pipeline Structure¶
The MFIX-Exa CI is organized into two distinct execution paths:
Merge Request (MR) pipelines – run on every push to a merge request. These pipelines focus on code quality, portability, and build correctness. They include the Lint, Warnings, and Build stages.
Post-merge pipelines – run only after changes are merged into the
developbranch. These pipelines run the full CTest-based test suite and the specialized Ascent and MPMD tests.
This separation ensures fast feedback during code review while reserving the more expensive tests for post-merge validation.
Merge Request Pipeline Stages¶
Lint
Static analysis and formatting checks:
Shell scripts (
shellcheck)Spelling (
codespell)Tabs and trailing whitespace removal
Python linting (
pylint) for the GUI
Warnings Builds
Compiler-strict builds using GCC and Clang in both Debug and Release modes. These builds enable extensive warning flags (
-Wall,-Wextra,-Werror, etc.) to catch portability and correctness issues early.
Builds
Multiple configurations are compiled to ensure portability across compilers and GPU backends:
CPU-only builds
GPU-enabled builds (CUDA, HIP, SYCL)
CSG-enabled builds
MPMD-enabled builds
Unit tests
These MR-stage jobs verify that MFIX-Exa builds cleanly and consistently before code is merged.
Post-Merge Test Pipeline¶
After changes are merged into the develop branch, the CI runs the full
MFIX-Exa test suite. These jobs are triggered by rules: blocks that
restrict execution to the develop branch.
The post-merge pipeline includes:
CTest-based MFIX-Exa Test Suite
Tests are grouped by label:
SGS– single-grid serial tests
MGS– multi-grid serial tests
TGS– tiled-grid serial tests
MGP– multi-grid parallel tests
TGPO– tiled-grid parallel OpenMP tests
TGSO– tiled-grid serial OpenMP tests
Ascent Visualization Test
Runs a small HCS benchmark with Ascent enabled and verifies image output.
MPMD Tutorial Test
Runs the MPMD tutorial example and checks numerical output and generated images.
How to Reproduce CI Locally¶
Building MFIX-Exa¶
A typical local build mirrors the CI configuration. For example:
cmake -S <path-to-mfix> -B build \
-DCMAKE_BUILD_TYPE=Release \
-G Ninja
cmake --build build --target build_tests unit_tests
Additional options (e.g., enabling MPI, OpenMP, GPU backends, or CSG) can be added as needed.
Running the MFIX-Exa Test Suite¶
CTest is used to run the MFIX-Exa test suite. All tests are available locally, regardless of which subset the CI runs.
List all tests:
ctest --test-dir build/ -N
Run all tests:
ctest --test-dir build/
Run a specific test by name:
ctest --test-dir build/ -R DEM01
Run a specific test by index:
ctest --test-dir build/ -I 3,3
Run selected single-grid serial DEM tests in the y direction:
ctest --test-dir build/ -R "DEM(01|03|04|05|06)-y_SGS|DEM07_SGS"
Running Unit Tests¶
cmake --build build --target unit_tests
ctest --test-dir build/ -R unit_test -V
MFIX-Exa Test Cases¶
The following cases are described in tests/README.md. They cover fluid
flow, DEM and fluid/DEM behavior, fluid-particle coupling, adaptive mesh
refinement, and chemistry.
Fluid (FLD) Cases¶
FLD01: 2D Channel (Poiseuille) flow in a periodic domain [Musser]
FLD02: 2D Couette flow in a channel [Musser]
FLD03: 2D Channel (Poiseuille) flow with specified pressure boundaries [Musser]
FLD04: Tracer transport with and without particles
FLD08: Flow through a rotated pipe
DEM and Fluid/DEM Coupled Cases¶
DEM01: Freely falling particle with wall collision [Musser]
A particle free-falls in a vacuum and collides with a wall. The position and velocity during the three stages (free-fall, collision, rebound) are reported. This case serves to verify the linear spring-dashpot collision model as well as the accuracy of the time-stepping methods.
DEM02: Bouncing particle height [Musser]
A particle free-falls in a vacuum and collides with a wall several times. The resulting height the particle reaches post-collision is reported. This case provides a comparison between the linear spring-dashpot model and the hard-sphere model where collisions are instantaneous.
DEM03: Two stacked, compressed particles [Musser]
Two particles are compressed between two walls, and their center positions are reported. This case serves to verify the linear spring-dashpot collision model through analysis of a multi-particle, enduring collision.
DEM04: Ball slipping on a rough surface [Musser]
A spherical particle is rolled on a rough surface. The angular and translational velocities are reported. This case serves to verify the soft-spring collision model through the analysis of the rolling friction model.
DEM05: Oblique particle collision [Musser]
Ninety-three cases of oblique particle collision are tested simultaneously; the particles collide only with the hard surface, not each other. Rather than having the particles fall vertically and hit an angled surface, the wall is kept level (flat) and the particles are given initial trajectories, resulting in oblique collisions. The particles are initially positioned close to the wall and gravity is suppressed. The post-collision angle and angular velocity are reported for each particle. This case serves to verify the normal and tangential components of the linear spring-dashpot model.
DEM06: Single particle terminal velocity [Musser]
A particle, initially at rest, is released in a uniformly flowing fluid. The particle’s velocity increases until it reaches terminal velocity where the gravitational force balances the fluid-particle drag force. The calculated terminal velocity is compared to the theoretical value.
DEM07: Homogeneous Cooling System [Liu]
Particles with no net flow (zero mean velocities) and specified initial granular temperature are allowed to cool over time. The granular temperature of the system is reported over time.
DEM08: Rigid clumps colliding
Two L-shaped rigid clumps of three overlapping spheres meet off centre in a periodic box
without gravity; after the collision each moves and turns as one body. Each clump is a
template made from its spheres (geometry = spheres, method = file) and placed
where they lie by a clump file (placement = file, orientation = file). The run
script designs the templates with mfix-clump-design first. The position, velocity
and spin of a member of each clump are reported, on one grid and on several.
DEM09: Bonded beam landing on a floor
Four spheres in a line are bonded where they touch at the start and carry shear, bending
and torsion (dem.bonds.solid.model = beam); the beam falls tilted onto the floor of a box.
The positions and spins of its two ends are reported.
DEM10: Clumps built from a template
A superellipsoid grain is filled with member spheres (templates.inp, which the run
script designs into sq.tpl with mfix-clump-design) and an initial condition
region seeds ten rigid clumps of it at random orientations
(ic.<region>.<solid>.template) in the upper part of a box, where they start to fall;
the run ends before they reach the floor. The granular temperature of the members and a
member’s position are reported, on one grid and on several.
DEM11: A clump settling through a fluid
A clump listed in a clump file (placement = file) settles through a viscous fluid,
coupled to it as one body (dem.clump.coupling = body) with the
orientation-dependent drag of the grain’s drag set in its template. A member’s height and velocity
are reported, on one grid and on several.
DEM12: Tangential history when particles are reordered
One particle rests on another with friction and tangential history on, and the tangential spring of their contact swings back and forth. A third particle stored ahead of the pair in the same grid leaves the grid, and AMReX moves the grid’s last particle into its slot, which puts the pair in the opposite order. The run script compares this run with one where the third particle stays put: the pair never touches it, so the pair’s motion must be identical. It changes if the contact’s history follows the particles’ order in the grid instead of the particles.
DEM13: Momentum conservation with tangential history across grids
The particles of DEM07 in a fully periodic box with no gravity, walls or fluid, with friction and tangential history on and the box split into 64 grids. Nothing outside the particles acts on them, so the run script checks that their total momentum keeps its initial value. A contact between particles on different grids is computed on both sides, and the two forces cancel only if both sides use the same tangential history.
DEM14: Torque-free rotation of rigid clumps
Two asymmetric rigid clumps, each a template of four unequal overlapping spheres, meet off center in a periodic box without gravity, walls or fluid, and leave the collision spinning about general axes. From then on each turns freely, an asymmetric top, and the run script checks every millisecond that each clump’s angular momentum about its center keeps its magnitude and its rotational energy (within 1e-9), and that its direction stays within 2.5e-3 of where it started (the positions and velocities written are half a step apart). This is the check of the rigid-body step that comparisons with Yade used to make. On one grid and on several.
DEM15: Hertz collision across a grid boundary
Two spheres meet head-on from either side of the boundary between two grids, with the Hertz contact model and restitution 0.5, and the run script checks that they rebound at 0.25 m/s within 1 %, equal and opposite to 1e-12. Hertz contact damping takes the pair’s effective mass, so the neighbor’s density must reach its ghost; with MPI the two grids are on different ranks. On one rank and on two.
SQP01: Superquadric particle thrown at a wall
A rounded cube (semi-axes 5 mm, exponents m = n = 4) is thrown obliquely at the floor of a closed box with the Hertz contact model and bounces twice. The run is stopped half way, written to a checkpoint and continued from it. Its position, velocity, angular velocity and orientation are reported. This deck is the MFIX-Exa side of a twin with MFIX classic SuperDEM, with which it agrees to 6e-8 of the largest value of each quantity.
Fluid-Particle Coupling (CPL) Cases¶
CPL01: Momentum conservation with lift
Particles at rest sit on the shear layer between two slabs of liquid in a fully periodic box. Drag accelerates them along the flow and lift pushes them across it. With no gravity, walls or inflow, the total momentum of the two phases cannot change, and the run script checks that its change is small next to the momentum the particles exchange with the fluid. This catches a coupling force deposited to the fluid with the wrong sign.
CPL02: Lift vorticity next to a no-slip wall
Nearly immobile particles sit in steady Poiseuille flow between no-slip walls, with
Auton lift on and drag off. The lift on a particle at the first cell centre is checked
against the velocity of that cell and the exact vorticity, which the centred curl
reproduces only if the velocity ghost holds the reflection about the wall value. A
particle a quarter cell from the wall interpolates from the vorticity ghost cells and
must get 0.75 of that force; the run uses amrex.init_snan, so unset ghost cells stop
it.
CPL03: Pressure force next to a no-slip wall
Nearly immobile particles sit in a liquid column at rest between no-slip walls, with gravity on and drag off, so the force on each is its buoyancy. A particle a quarter cell from the wall interpolates the pressure gradient partly from the ghost cells outside the wall, which the projection never fills; it must still get the full buoyancy.
CPL04: Virtual mass force next to a wall
Liquid between free-slip walls is driven from rest by a pressure drop, so the whole column accelerates uniformly. Nearly immobile particles with drag off feel the pressure force plus a constant-coefficient virtual mass force. A particle a quarter cell from the wall interpolates the fluid acceleration partly from the ghost cells outside the wall, which are computed nowhere else; it must get the same force as one at the first cell centre.
CPL05: Interpolation across a periodic boundary next to walls
Two slabs of liquid shear in a domain that is periodic in x and y with free-slip walls in z, so the domain ghost fills run. Nearly immobile particles with drag off and lift on sit at mirror positions of the symmetric flow; one interpolates across the periodic boundary. Their forces must be equal and opposite, which fails if a domain fill overwrites the periodic ghost cells.
Adaptive Mesh Refinement (AMR) Cases¶
AMR01: Restart reproducibility with a refined level
A shear layer in a periodic box with a fixed level-1 patch straddling one interface. A run of 20 steps is restarted from its step-10 checkpoint, and the step-20 plotfiles of both runs must match bit for bit on both levels, with the Godunov and the MOL drivers. This fails if a fine level fills its coarse-fine ghost cells from the coarser level’s previous-step state.
Chemistry (CHEM) Cases¶
CHEM01: Adiabatic reaction in an open channel
A uniform gas at rest undergoes A(g) –> 2B(g) (first order in A) with both channel ends held at the thermodynamic pressure. The reaction drives a symmetric expansion, but every cell follows the same constant-pressure history, so the temperature stays uniform. With equal constant specific heats the end state is closed form (T = 800 K), and the run script checks the minimum and maximum temperature and the residual A against it.
Test-Local User-Defined Functions¶
A test directory may override the default UDFs in src/usr:
*.cppfiles (e.g.,usr_drag.cpp,mfix_pc_usr.cpp) are compiled into the test’s executable in place of the default stubs.mfix_usr_reactions_rates_K.Hsupplies a reaction mechanism. The test then gets its own chemistry library (mfixchem_<test>), built from the same sources asmfixchembut against this header.
Test Case References¶
J. Musser and A. Choudhary, “MFIX Documentation Volume 3: Verification and Validation Manual,” from URL http://mfix.netl.doe.gov
Peiyuan Liu, Timothy Brown, William D. Fullmer, Thomas Hauser, and Christine Hrenya, “A Comprehensive Benchmark Suite for Simulations of Particle Laden Flows Using the Discrete Element Method with Performance Profiles from the Multiphase Flow with Interface eXchanges (MFiX) Code,” Technical Report NREL/TP-2C00-65637, January 2016.
Relationship to Regression Tests¶
The CI pipeline does not run the CCSE regression suite.
The Regression Tests page documents the CCSE/AMReX regression framework.
The CI page documents the CTest-based MFIX-Exa Test Suite and infrastructure tests.
These two systems are complementary:
CI ensures build correctness and basic functional behavior.
Regression tests ensure numerical consistency across platforms and versions.
Developer Expectations¶
All CI jobs must pass before merging into
develop.Lint and warnings jobs help maintain code quality.
Build jobs ensure portability across compilers and GPU backends.
Test jobs ensure functional correctness.
CI artifacts contain detailed logs for debugging failures.