Issues with VDW Forces in DMP Parallel Computations

[Qingsong]

I am currently adding additional particle contact forces based on the existing VDW force. However, when performing DMP parallel computations, I observed significant anomalies in the cohesive force at the boundaries between different partitions.

I have tested the built-in VDW force alone, using the default DMP solver, and the issue still occurs. Notably, this problem does not appear when running single-core computations.

Below are the test source files for the VDW force:

• Figure 1 shows the abnormal behavior after adding my custom contact forces.
• Figure 2 shows the DMP parallel computation of the built-in VDW force using 6 cores (2×3×1).
• Figure 3 shows the result of the built-in VDW force in a single-core computation.

vdw_test.zip (5.4 KB)

[Qingsong]

Hello everyone,

I further tested the VDW force using the fluid_bed_2d tutorial case, setting the inlet gas velocity to 0 so that particles naturally settle and form a packed bed. When solving with the default DMP solver, I still observe abnormal cohesive forces along the DMP partition boundaries.

Under the same conditions, this issue does not occur in single-core computations. I am quite confused and would appreciate any help or suggestions.

For reference, I am currently using version 25.1.2 on Linux for these computations.
fluid_bed_2d_vdw_test.zip (11.4 KB)

Figure 1: Result of DMP parallel computation
Figure 2: Partition map of DMP parallel computation
Figure 3: Result of single-core computation

[jeff.dietiker]

I am looking into it. This may not be an issue with the force calculation itself, but maybe just the way we store it in a different array to save it in the vtp file.

[jeff.dietiker]

calc_force_dem.f (33.1 KB)

Please try with the attached. Copy to you r project directory and rebuild the dmp solver. Let us if this works for you.

[Qingsong]

Thank you very much for your help, Jeff.

At this point, when performing DMP parallel computations, the issue of abnormal values along the partition boundaries has been resolved. However, a new problem has appeared: as the particles settle and form the bed, most of the particles in the central region of the packed bed show cohesive force values that are much lower than those on the sides, which I believe is not normal.
The following two figures show the system at the same time point but with different color bar ranges. As can be seen, the particles in the central region of the bed exhibit almost no cohesive force.

fluid_bed_2d_vdw_test_v2.zip (20.2 KB)

[jeff.dietiker]

The cohesive force stored in the vtp file is the magnitude of the sum of cohesive forces on that particle. Visually, the near zero value seems to occur when you have a fully packed region. I am guessing the sum (resultant) force is near zero because neighbor forces balance out. Regions where packing is more random do not exhibit a near zero resultant. Maybe you can fluidize the bed for some time, turn off the mass inlet, and let particles settle so you have a more random packing.

[Qingsong]

Hi Jeff,

Thank you very much for your suggestion. After adopting it, I obtained a more random packed structure (Figure 1).

Based on the calc_force_dem.f file you provided, I modified the built-in VDW force to implement sintering forces between particles and ran my own case, obtaining the results shown in Figure 2.

When outputting the CohesiveForce values, I found that the forces on particles inside the agglomerates are almost zero, which seems unreasonable for sintered particles. I noticed you had modified the following code;

could it be that the forces between neighboring particles LL and I within the agglomerate are canceling each other out? This would be problematic for simulating the sintering process. Do you have any suggestions on how to handle this?

In addition, based on the fluid_bed_2d example case, I tried to output the X and Y components of the VDW force acting on the particles.

In the current case, gas is injected from the bottom boundary. However, during this process, I observed that at certain times, the particles along the DMP partition boundaries exhibit clear anomalies in the X component of the VDW force. This issue does not occur when running the same case in single-core mode.

Similarly, I observed the same issue with the forces on the particles in the Y direction.

Below is a schematic diagram of the MPI partitions.

Although during later stages of intense motion the differences are not apparent, for simulations of particle sintering involving very small particles, the abnormal reduction of forces at the DMP partition boundaries has a significant impact on the results.

The figure below shows the results of the sintering forces after I modified the cohesive force calculation logic in the latest provided calc_force_dem.f.

I am quite confused about this. Could this be an issue with MPI communication? Or is this potentially a bug?

Below are my test files for the VDW force components. I look forward to your reply!

vdw_test_v4.zip (81.6 KB)

[kerenjie127]

I took a look at your code. You created a new array called vdwForce to store the cohesive force and let MPI handle the exchange for this array as well. I haven’t fully verified your MPI implementation yet.

but from what I found in your code, you have the following block in calc_force_dem.f:

FC_TMP(:) = DIST(:)*FORCE_COH/SQRT(DIST_MAG)
!*******************yq******************************************************
vdwForce(LL,:) = DIST(:)*FORCE_COH/SQRT(DIST_MAG)
!***********************yq************************************************

from my understanding, this vdwForce actually stores the cohesive force between two neighboring particles LL and I. The issue is that this value will be overwritten.

Let’s assume LL = 1. Then I could be 2, 3, 4, or 5. You first calculate vdwForce(1,:) based on LL = 1 and I = 2, but in the same time step, the value is then replaced by the result from LL = 1 and I = 3, and so on. As a result, the final value stored will only be the cohesive force between LL = 1 and I = 5.

I’m not sure what exactly you are trying to obtain, but the current output appears to only reflect the cohesive force between one pair of particles. This could introduce randomness because each processor will have different local particle id for the same particle. Would you be more interested in the accumulated force instead?