Cannot restart DEM

I ran on a cluster a 3D polydisperse DEM simulation with over 2 million particles (with air as a fluid), and when doing a restart_1, I have the following error:

Fatal error in PMPI_Wait: Message truncated, error stack:
PMPI_Wait(219)…: MPI_Wait(request=0x47a60750, status=0x3c32550) failed
MPIR_Wait_impl(100)…: fail failed
MPID_nem_gen2_module_iStartContigMsg(438): Message from rank 180 and tag 5 truncated; 46739176 bytes received but buffer size is 46098720

I have never had an issue like this. Has anyone an idea what is wrong here? (Restart_2 also does not work)

Thanks

Answering my own question … in case it helps someone else
By luck i figured that it was a Dynamic Load Balance issue. When having DBL_DT = … in the *.mfx it had the issue stated. It worked without.

2 Likes

Should be DLB_DT instead of DBL_DT