I want to use OpenMp and MPI together. Due to our cluster limitations, I cannot use GUI. for OpenMP I use:
env OMP_NUM_THREADS=4 ./mfixsolver -f DES_FB1.mfx,
and for MPI , I use:
mpirun -np 4 ./mfixsolver -f DES_FB1.mfx NODESI=2 NODESJ=2.
How can I simlutaneously set the number of nodes for MPI and the number of processors on each node for OpenMP in one command (hybrid parallelization)?
If I want to submit it to queue, then how can I create the sbatch file? would you please help me through these?
Hi, I am not sure my answer may help you or not. Because I am not sure about the command to submit the job in cluster too. For my case, I create the batch file like that
#!/bin/sh
#SBATCH --time=50:00:00
#SBATCH --ntasks=64
#SBATCH --mem-per-cpu=1000MB
module load intel-mpi
mpirun -np 64 ./mfix
I defined the ntasks or thread instead of reserving the node. The HPC will distribute to thread. If only one node don’t have enough thread. It will run to the other node automatically.
Thank you !! domain is decomposed between nodes or threads? I mean I have read that MFiX uses MPI for domain decomposition but now I see that we should set the multiplication of NODESINODESJNODESK equal to number of threads? can you explain me this? I am confused…