Simulations with external particle_input file take an excessive amount of time to perform simple GUI tasks because the file is re-read many times. mixer_2025-01-16T103203.570261.zip (23.5 MB)
Description
The attached version of the mixer demo contains a particle_input file with ~258800 lines. The ‘data file particle count’ is currently set to 2858. Various GUI actions, such as changing the particle count, or turning cohesion on / off result in the file being re-read. For 2858 particles, this only takes a few seconds. For 28588 particles, it takes ~80 seconds. For 285880 particles it takes over 10 minutes before the GUI responds to further input. The response time appears to be linear with the number of lines read from the file.
On starting the simulation, the DMP processes start immediately, but the GUI still takes another (file read time) before I can access the VTK display or MFiX status tab
Reading the file seems to start a single thread which takes more than a single core can manage : Linux top shows the mfix process running at >150% until the GUI becomes responsive.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2257521 xxxxxx 20 0 11.6g 3.4g 252180 R 160.4 10.8 2255:21 mfix: mixer.mfx
The ability to view the particles in the particle_input.csv file before running the simulation is a recent addition to MFiX and you have found some shortcomings in the code. Thanks for reporting this, we will have a bug fix as soon as possible.
A few comments:
The particle_input.csv file is being read more times than necessary, as you pointed out. Operations like enabling/disabling cohesion should not trigger reloading the file. I did not notice this while developing this code, because on my system, the time to read the file - even a fairly large file like your 285882 line example - is negligible (a fraction of a second). I suspect you must be using some very slow storage (NFS?) if it’s taking you 10 minutes to read the file. The file is 14MB, if it’s really taking 10 minutes that means you are only getting a read speed of about 24KB/sec (modem speed), which is very poor. Something may be defective or misconfigured in your storage system - you may want to measure I/O performance and see if it can be improved.
2) There’s a control in the model view to enable/disable the particle preview (see above). However this setting seems to be ignored - the file is loaded and particles displayed even when the “particles” button is not pressed in. This is clearly a bug.
There are two places where a particle count can be entered - in the Solids/DEM pane the count limits the number of particles used in the simulation, and in the Model view pane, the count limits the number of particles displayed (see above). However, regardless of these settings, the code is reading the entire file and just displaying the first N rows. This can be improved, only reading the required numbe of lines from the file.
We will be putting together a bug fix release soon, but this may take a few days, so in the meanwhile I suggest you simply disable the particle preview feature to make MFiX usable for you.
Edit the file $CONDA_PREFIX/lib/python3.11/site-packages/mfixgui/vtk_widgets/modeler.py and add a return statement after line 2366:
def update_particles(self):
return
# remove existing actors and mappers - do we need to? Can we just update data?
for actor in self.particles_dict['actors']:
self.vtkrenderer.RemoveActor(actor)
This will disable the particle preview until we are able to get a proper bug fix released. Of course you can remove the return at any time to re-enable the feaure.
Sorry for the trouble, and thanks for the bug report. Feedback from users like you helps us improve MFiX for all.
Thanks Charles. I’ll get our IT people to take a look at why our system is reading the input file so slowly. I agree it must be a local thing - my laptop doesn’t have the same issue. Other files on the linux system can be read quickly - it can animate the result at about a frame per second using .vtp files of 14Mb, which is about twice the speed of my windows laptop reading from the SSD.
Looking at the code, the entire file is read regardless of “Data file particle count” - so it’s not time to read the file, it’s the time required to set up the particle view in VTK. So this doesn’t point to slow file I/O but slow graphics. Are you running this under vglrun? Over a slow network connection, perhaps?
I’m working on speeding up the code - the easiest part is to remove the excessive calls to update_particles which are causing this to happen every time you change something in the GUI - the harder part is understanding why it takes ten minutes to render 285880 particles, since I cannot reproduce this locally.
