Thunderhead Engineering Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Forum moved to https://forum.thunderheadeng.com

Author Topic: OpenMP Threads and Threads MPI  (Read 1865 times)

Dapke36

  • New Member
  • *
  • Posts: 2
    • View Profile
OpenMP Threads and Threads MPI
« on: March 10, 2020, 08:20:13 am »

I'm still trying to wrap my brain around when to use OpenMP and when to use OpenMP Threads MPI (~"MPI) in FDS/Pyrosim.  I read the FDS users guide section 3.3.1, but still struggling on application...and i'm hoping someone can help!

The current machine I am utilizing for smaller runs is 12 GB ram with a Xeon e5 with 4 physical cores reported in that 1 cpu. When I read the user's guide, I am getting confused when they are talking about mapping to cores.

Using that hardware above, lets say I have a model with 4 meshes.  Should I be doing an MPI thread per core, i.e.:

Open MP Threads: 1
Open MP Threads MPI: 4
Open MP Stack: 200M


Or should I be doing 1 mpi thread per physical CPU and 1 OpenMP thread per core of that cpu? i.e.:

Open MP Threads: 4
Open MP Threads MPI: 1
Open MP Stack: 200M


Is MPI supposed to be per core available (parallel or cluster, whatever's linked), or per CPU? 

If there's one MPI per core, I should try to get my # of meshes to match the # of cores available, right? (with exceptions for dense meshes and distribution, understandably) Or is the OpenMP Thread supposed to match the # of cores?


If I double the CPU's on the machine to 2 physical CPU's, each with 4 cores (8 cores total), would it then be 8 MPI processes?

Any insight that can be offered is appreciated!
Logged

chang

  • New Member
  • *
  • Posts: 1
    • View Profile
Re: OpenMP Threads and Threads MPI
« Reply #1 on: September 08, 2020, 04:56:14 pm »

Did you ever figur this out? Very curious myself.
Logged

kazukikaze

  • New Member
  • *
  • Posts: 2
    • View Profile
Re: OpenMP Threads and Threads MPI
« Reply #2 on: December 30, 2020, 10:44:42 pm »

Try and play with those and monitor through task manager or intel/ryzen cpu monitoring software.

I did some experiment myself and manage to run only 8 cores on my 16cores processor using the below parameters.

Open MP threads : 8
Open MP threads MPI : 1
Open MP Stack : 16M

I tried changing the MP threads MPI to 8 but it pushes all 16 cores to work. From what I understand the MPI means the number of Processor that is running. So usually it should be 1 since most computers have only 1 cpu. The Open MP threads is the one that dedicate your cores to your simulation.
Logged

elchanan

  • New Member
  • *
  • Posts: 5
    • View Profile
Re: OpenMP Threads and Threads MPI
« Reply #3 on: January 03, 2021, 05:43:56 am »

MPI uses parallel computing between physical cores (virtual are not advisable). the speed up is in relation to the number of cores used. so theoretically you could have 10 meshes on 10 different cores and the simulation would run much faster than one mesh one one core.
Open MP on the other hand, does parallel computations by dividing the task between a few threads. so the speedup is much more limited.
If you use 2 MPI processes, you will get almost double the speed. If you use 2 OMP, you will get about 20-30% speedup. if you jump to 8, MPI will give about 6-7 times faster simulation and OMP will only double the speed.
So in conclusion, use MPI according to available physical cores. you can try to add OMP if you have available virtual cores. experiment a little with both of them to better understand.
Logged