Hello everyone,
I have observed the following phenomenon in my CFD-CG-DEM simulations of a fluidized bed: When I repeat the simulations with exactly the same settings, for example, I do not get exactly the same pressure drop profile over time. The simulations are therefore not exactly reproducible. What could be the reason for this? Initially, the simulation results match very well. The deviations then increase over time. Are there any random variables or stochastic processes in the software that could lead to this? It is probably useful to mention that I am computing in parallel on 6x1x5 cores. So maybe using DMP might be an aspect as well?
Many thanks in advance and kind regards
Fritzi
When you do a data reduction (say gather the local sums when computing a dot product), the order of collecting data from all processes is not always going to be the same. For a computer A+B+C isn’t the same as B+C+A. Although the difference is minute, it will lead to different transient results when it accumulates over time. The linear equation solver also has a small random number as a starting point to help convergence. We have looked at this a few years ago and saw a difference when changing the parallel decomposition, compiler and compilation flags (https://mfix.netl.doe.gov/doc/mfix-archive/mfix_current_documentation/MFIX_results_sensitivity_to_Fortran_compilers.pdf). Although this was done with TFM, you will see the same with DEM, and probably even more difference (think how chaotic is a pool break even though it seemingly starts with the same condition).
Instead of comparing instantaneous snap shots, you should look at time-averaged quantities. These should statistically reproducible over a sufficiently long averaging period.
1 Like
Thank you very much for the quick reply. I have a follow-up question: I have carried out a CFD-CG-DEM simulation of a fluidized bed with a very high coarse-grain factor. Due to the high coarse-grain factor, I had to reduce the drag force. Without the reduction, the drag force is overestimated and the particles adhere to the outlet. With the reduction, the bed height is still overestimated, but the particles no longer adhere to the outlet. Now I wanted to write out the drag force and created a new particle settling and wrote out the drag force and the momentum transfer coefficient via user defined variables. But now the particles stick again, although all other settings and the number of parallel cores are the same. If I repeat the simulation without writing them out, the particles do not stick. The files in which the user-defined variables are defined are attached.
So the questions I have are:
- Is it possible that writing out more variables affects the simulation results in some way?
- If so, how?
Many thanks and best regards
Fritzi
drag_gp_des.f (8.4 KB)
drag_gs_des1.f (14.1 KB)
I am assuming you properly allocated the DES_USR_VAR array with des_usr_var_size = 4 . If so, writing data to this array should not affect the results, unless you are using this array somewhere else. You can try to run in debug mode to see if there is an issue with allocation.
Actually, I didn’t allocate it with des_usr_var_size =4. Where should I change it? Should I write
INTEGER :: DES_USR_VAR_SIZE(4)
instead of
INTEGER :: DES_USR_VAR_SIZE
in line 470 of discretelement.f?
What happens, if you don’t do this?
Or do you mean, that I should simply Enable user scalar tracking: 4. That’s what I did.
Yes this is what I meant. setting “Enable user scalar tracking: 4”, will set the keyword des_usr_var_size = 4, and this will be used to allocate the array DES_USR_VAR.