Hello,
I am running a code with quite a complex USR function and I am experiencing some troubles. It looks like one of the sources of my issue is that USR0 and USR1 are called several times when using DMP. As a test, I did this very simple case in which I am just printing in screen whenever USR0 or USR1 are called. When I use 2 processors, USR0 and USR1 are called twice (once per processor) whereas if y mpirun only using 1 process I have the expected routine calling behavior (USR0 called once at the beginning of the simulation and USR1 being called once in between time-steps).
In DMP mode, you can find out which CPU you are running on by looking at the variable myPE (PE = “Processing Element”, which is DMP jargon for CPU)
As you have seen, the functions are called on each processor. If you only want the function to execute on one processor, do something like this:
SUBROUTINE USR1
USE compar, only: myPE, numPEs
USE USR
IMPLICIT NONE
PRINT *, "Calling USR1 on PE", myPE
if (myPE > 0) RETURN
.... do stuff here
RETURN
END SUBROUTINE USR1
Thank you for your answer. What I do not get is that this only happens if I run from batch. If I run the same code from the GUI, I can select 2 processors in the Y direction and still only call once the UDFs. Is there a command to do this directly from the batch solver?
In the meantime I will implement your code to use only one processor for the UDFs.
The behavior is the same for batch and GUI. It makes sense for the USR functions to be called on each CPU, for cases where each CPU is responsible for some subset of the particles. fluid cells, etc. Don’t think of it as the function being called “multiple times” - USR1 is being called once per time step, but the call is happening in a distributed way.
I find it helps to disable “DIsplay residuals”, this will reduce the amount of messages from the solver so you can spot your own more easily:
I think you have found a bug in the GUI, however. Somehow only the messages from PE 0 are showing up in the GUI console. If you examine the stdout file you will see the messages from the other PEs are there but they are not showing up inside the GUI window. I am not completely sure why this is happening - it’s a bit odd. I’ll try to get this sorted out for the next MFiX release.
It is useful to know this. I am working on a model to solve reactions, convection, diffusion etc. inside some of the particles. I am currently testing and trying to couple the code and MFiX (I may write at some point asking for some support around here, as it is not straightforward). Knowing that UDFs can be called in different processors will in the future accelerate things a bit, as I can manually set which particles are solved by which processor.
I am aware of the warning in the seeding. I just did something silly to seed only one particle of phase 1 into the system, which will be the particle eventually being solved also internally.
Thank you again for your help. I wish you (and your cat) a very nice Christmas!
