The log files .csv not output without GUI

MFiX
#1

Hi,
I’m used to using the MFIX GUI just to describe the system and I use the shell terminal to run the simulations because It’s more comfortable working with a VPN remote connection,

The problem is that only with the GUI the .csv log files of the residuals, dt, nit, solid inventory … located in Output -> Log & residual are generated and written, when running simulations from the terminal these are not generated log files. I have configured with overwrite in these options,

How can I generate these files by running the simulation from the terminal?,

Best Regards.

#2

When the GUI is running, it is collecting the residuals, dt, nit, etc from the http connection and parsing stdout. The solver does not generate the csv files. Some of the information is in the <run_name>.LOG file. You can pipe the stdout to a file as well.

#3

Thanks, I now understand that I must channel the standard output stdout .LOG to a new file, but selectively I must collect the data in an orderly manner for residues, nit, dt, etc. How can I collect this data? Should I write a script to convert that information to .txt or .csv? How should do I do it?

#4

Hi JPGD - I just notice this followup question which did not get a reply - sorry about that!

I’m not sure if you are still pursuing this, but I have a few comments on the topic of remote operation.

It would be easiest if you could just run the full MFiX GUI on the remote host - with some kind of X11 forwarding over your VPN - then all the files would be written for you.

As Justin explained, the MFIX solver job opens up an HTTP port (web service) and the MFIX GUI communicates with the running solver job.

This channel is used to collect status information about the running job (is the job still running? what are the current nit, time step and residuals?). This HTTP channel is also used to control the job (pause/resume). The HTTP channel offers a finer-grained control than the shared filesystem, and it’s possible to monitor at a higher frequency - by default the GUI polls the solver at 10 Hz.

If you enable “developer mode” you can see what these HTTP packets coming back from the solver look like - there’s a tab for MFiX responses where the “live” data is shown. The HTTP protocol is used because the solver and the GUI may not be running on the same node - we support a “remote solver” mode.

However, not all of the communication between the solver and the front end goes over this HTTP channel - we also rely on a shared filesystem (NFS). We have at times discussed implementing a “fully detached” mode where this shared filesystem would not be needed, but there are currently no plans to implement that.

So, if you can’t use X11 forwarding to just run the GUI on the remote node, another option would be to set up an NFS mount across this remote connection, and set up up a job submission script, then you could use the GUI as intended, running locally, with full job control and .csv files written by the GUI.

If that’s not possible, and you want to collect this data, you’ll have to monitor the webserver yourself. When you start the solver job, it will write a file called <run_name>.pid in the project directory.

In that file there’s a line like

url=http://XXX.XXX.XXX.XXX:YYY

Now, you can connect to that URL and get the status data.
If the port is open on your VPN, you can even connect to the remote job with a web browser.

There is a very simple, but undocumented HTTP API for controlling solver jobs. If you look at mfixgui/pymfix_webserver.py you can see the supported commands. In particular, appending /status to the end of the url will get you a status report from the solver, which is formatted as JSON. This contains the high-frequency residuals, etc. If you know a bit of Python it shouldn’t be too hard to write a script that monitors this port (at whatever frequency you prefer) and saves data to a .csv. But, since this functionality is all already present in the MFiX GUI, I suggest you find a way to use that.

Hope this helps,

  • Charles
2 Likes
#6

Hi Charles i’m using MFIX 19.3 and i try the URL in the <run_name>.pid file. When i’m connected displays only this message “hello”. How can i get the status data?

1 Like
#7

Hi Lawrence.

The short answer is - you need to add /status to the URL

The web interface to mfix is not well documented because it is basically there for the GUI to talk to the solver (which may be on a remote node), and for debugging. We didn’t really expect users to access it. But it’s useful, so we should probably document it.

It’s pretty minimal right now.

It’s possible to add requests in a future release if you have special requirements.

Currently, the known requests are:

@F.route('/')
@F.route('/exit')
@F.route('/stop')
@F.route('/pause')
@F.route('/unpause')
@F.route('/status')
@F.route('/reinit', methods=['POST'])    
@F.route('/help')
@F.route('/stdout', methods=['GET'])
@F.route('/stdout/<nbytes>', methods=['GET'])
@F.route('/stderr', methods=['GET'])
@F.route('/stderr/<nbytes>', methods=['GET'])

This can be found in pyfmix_webserver.py

The API is subject to change, but the /status request is central to how the GUI operates and I think it’s safe to say it will not change any time soon!

2 Likes