Hi @BimalChhushyabaga
A few comments and suggestions:
-
When looking for help on the forum, it makes our job easier if you provide all the details - attaching case files, specifying DMP/SMP/serial, partitioning scheme, etc. The .mfx file you uploaded was set up for serial, and when I switched to DMP I did not know what partitioning you used and had to guess. I am running with 2x4x2=16 cores, I was not able to run on 32 cores (partitions too small).
-
It also helps to provide a case which does not take all day to run! This makes debugging and testing much easier, with a faster turnaround cycle. This will make life easier for both of us. If possible, try to simplify the case to the simplest/smallest version which shows the same problem.
-
When a case does not work in DMP, always try running in serial mode.
-
WRITE statements are a primitive, but very helpful, debugging tool. Sprinkle them in your code liberally when you’re trying to figure out what’s going on.
With this in mind, I changed your code to the following, dividing the times by 100 and adding WRITE statements:
step_time = 0.004
! Injection cell indices (localized region)
indices = (/ 512, 513, 514, 681, 682, 683 /)
write(*,*) "USR1 TIME", time
IF (time >= 0.005) THEN
p_target = 10000.8 * (time - 0.005)**2
mod_time = MOD(time - 0.005, step_time)
step_index = INT(mod_time / 0.002)
IF (step_index == 0) THEN
DO i = 1, SIZE(indices, 1)
write(*,*) "TIME", time, "PE", myPE,"IDX", indices(i), "P_G", P_g(indices(i)),"P_TARGET", p_target
P_g(indices(i)) = P_g(indices(i)) + p_target
END DO
END IF
END IF
Running in serial mode, after a short time I see (slightly edited for readability):
USR1 TIME 5.00E-003TIME 5.00E-003 PE 0 IDX 512 P_G 102299.97 P_TARGET 1.24E-016
TIME 5.00E-003 PE 0 IDX 513 P_G 102136.65 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 514 P_G 101973.32 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 681 P_G 102136.65 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 682 P_G 101973.32 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 683 P_G 101809.99 P_TARGET 1.249-016
But running in DMP mode I see:
TIME 5.00E-003 PE 0 IDX 512 P_G 9.87654321E+031 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 513 P_G 9.87654321E+031 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 514 P_G 100817.605 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 681 P_G 102628.199 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 682 P_G 9.87654321E+031 P_TARGET 1.249-016
TIME 5.00E-003 PE 0 IDX 683 P_G 9.87654321E+031 P_TARGET 1.249-016
USR1 TIME 5.00E-003
TIME 5.00E-003 PE 3 IDX 512 P_G 100940.715 P_TARGET 1.249-016
TIME 5.00E-003 PE 3 IDX 513 P_G 102366.521 P_TARGET 1.249-016
TIME 5.00E-003 PE 3 IDX 514 P_G 102780.600 P_TARGET 1.249-016
TIME 5.00E-003 PE 3 IDX 681 P_G 102136.606 P_TARGET 1.249-016
TIME 5.00E-003 PE 3 IDX 682 P_G 99975.062 P_TARGET 1.249-016
TIME 5.00E-003 PE 3 IDX 683 P_G 102275.402 P_TARGET 1.249-016
USR1 TIME 5.00E-003
TIME 5.00E-003 PE 2 IDX 512 P_G 100998.270 P_TARGET 1.249-016
TIME 5.00E-003 PE 2 IDX 513 P_G 102399.224 P_TARGET 1.249-016
TIME 5.00E-003 PE 2 IDX 514 P_G 102787.067 P_TARGET 1.249-016
TIME 5.00E-003 PE 2 IDX 681 P_G 102136.613 P_TARGET 1.249-016
TIME 5.00E-003 PE 2 IDX 682 P_G 99885.7389 P_TARGET 1.249-016
TIME 5.00E-003 PE 2 IDX 683 P_G 102213.607 P_TARGET 1.249-016
USR1 TIME 5.00E-003
TIME 5.00E-003 PE 6 IDX 512 P_G 102136.615 P_TARGET 1.249-016
TIME 5.00E-003 PE 6 IDX 513 P_G 101973.290 P_TARGET 1.249-016
TIME 5.00E-003 PE 6 IDX 514 P_G 101809.966 P_TARGET 1.249-016
TIME 5.00E-003 PE 6 IDX 681 P_G 9.87654321E+031 P_TARGET 1.249-016
TIME 5.00E-003 PE 6 IDX 682 P_G 102136.610 P_TARGET 1.249-016
Note that if you run DMP jobs with the “interactive” mode solver in the GUI, you only get output from PE (processing element) 0, the main node, but if you run the batch solver you can see the output from all PEs.
But even looking just at the output from PE 0, there is an giant red flag, which is the number 9.87654321E+031.
This value is used in MFiX to represent invalid/uninitialized data. Whenever you see it, you know something has gone wrong.
What you have to understand here is that the cell indices
indices = (/ 512, 513, 514, 681, 682, 683 /)
are not valid in DEM mode. Each PE covers a different range of (I,J,K) indices and has its own list of IJK cells. Each IJK list of cells start at 1 so it is possible that each PE has a cell 512, but they will all be different cells. So, you are not incrementing the pressure at the cells you intended to. (How did you determine these indices?) It would be better to use I,J,K values and convert to a cell index using the function funijk in the functions module. This will work regardless of which PE we are in, and what DMP decomposition we use. And since your UDF code executes on all PEs you need to check if the cell belongs to the current PE. You can use
IS_ON_myPE_plus2layers(I,J,K)
If this returns .True. it means (I,J,K) is handled by the PE. (It could be an interior cell or a ghost cell.)
So you should do something like:
if (IS_ON_myPE_plus2layers(I,J,K)) then
ijk = funijk(I,J,K)
P_g(ijk) = P_g(ijk) + p_target
endif
See the comments at the top of functions.inc regarding inlining the funijk function. But in your case, since this only executes for a few cells once per time step, it probably does not matter.
Finally,
p_target = 10000.8 * (time - 0.5)**2
looks a bit odd to me - the pressure will continue to rise quadratically, with no limit. Is this really what you want?
