Regression Testing Procedure

MFIX-Exa
1

Hi,

I’m interested in what procedure/tools you use to ensure hard-coded changes to the source files do not impact the existing functionality of the solver.

I looked at the Regression Testing section in the documentation but I think that it is outdated (or the path to the regression test setup file is incorrect).

I decided to search in the other directories for the setup file and found it in two locations:

  • In tools/Python/: I tried to run regtest.py using the instructions in the README file but ran into some errors (I think the setup file assumes you already have benchmark data which actually isn’t there)
  • In tests/: The setup file here seems to be running the same tests as the CCSE Tests → so I assumed this is the right version.

For running the test, I used AMReX’s Regression Testing tool and it worked. I’d like to confirm whether this is the recommended or standard approach, to ensure consistency with best practices. Let me know your thoughts.

NOTE: there was a small change I had to make to AMReX’s Regression Testing tool because of an error I was running into → not sure if this is relevant for this post

Kind Regards,
Tarun

2

Thank you for bringing this up and for your detailed observations. You were correct that the documentation was outdated; it has now been updated to reflect the current regression testing procedure.

The recommended approach for MFIX-Exa regression testing is outlined in the updated documentation. As you noted, this method leverages AMReX’s Regression Testing tool and is the standard way to verify code correctness and ensure that changes do not break existing functionality.

Regarding the Python scripts under tools/Python/: these belong to a different testing infrastructure called RTL2 (Regression Testing Level-2), which is currently not documented. RTL2 runs longer cases designed to assess physical quantities that are typically time- or spatially-averaged. This approach is better suited for evaluating the correctness of the underlying physical models rather than code-level correctness.

1 Like