Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[rahuljindal.66]

Hello all,

I am working on car park fire simulation using pyrosim 2014. The total mesh size is 70,00000. i have tried two cases for multimesh, 12 meshes and 24 meshes. for 100 seconds simulation time , the system is taking almost 11hrs for both the cases. but in case of 12 mesh my CPU usage is 37% while for 24 mesh cpu usage becomes 74%. but total physical memory used is still same. I am not getting why this is not using system full capacity. 

What to do to make my simulation faster.

My system configuration is,

processor- Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.1Ghz 3.1 Ghz (2 processor)

Installed RAM -128Gb

Here is the attached image of FDS.

[Dave McGill]

Hi,

The speed of the simulation will be determined by the mesh that takes the longest to process.  That is usually the mesh with the fire, and it should have the fewest cells, as it will have the most activity. The other meshes should be of approximately equal size.

How many processors are available?

Dave

[rahuljindal.66]

Hey Dave,

In out file there are so many times which time is the CPU time for particular mesh.

there are 2 processors available.

waiting for reply



Time Step  123400   June 15, 2014  13:10:57
       Pressure Iterations:      1
       Maximum Velocity Error:  0.84E-01 on Mesh   1 at (  97  46  22)
       ----------------------------------------------
       Mesh    1, Cycle  123400
       CPU/step:     2.852 s, Total CPU:       97.26 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.90E+00 at ( 95,243, 23)
       Max divergence:  0.79E-01 at ( 95,244, 21)
       Min divergence: -0.11E+00 at ( 95,244, 23)
       Radiation Loss to Boundaries:      -107.224 kW
       Mesh    2, Cycle  123400
       CPU/step:     2.906 s, Total CPU:       98.80 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.91E+00 at ( 89,243, 23)
       Max divergence:  0.13E+00 at ( 89,244, 21)
       Min divergence: -0.29E+00 at ( 89,244, 23)
       Radiation Loss to Boundaries:      -161.255 kW
       Mesh    3, Cycle  123400
       CPU/step:     2.902 s, Total CPU:       98.79 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.90E+00 at ( 69, 15, 23)
       Max divergence:  0.18E+00 at ( 94,227, 22)
       Min divergence: -0.28E+00 at ( 91,217, 17)
       Total Heat Release Rate:              0.011 kW
       Radiation Loss to Boundaries:       -95.964 kW
       Mesh    4, Cycle  123400
       CPU/step:     2.893 s, Total CPU:       98.49 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.90E+00 at ( 55, 95, 23)
       Max divergence:  0.70E+01 at ( 11,223,  3)
       Min divergence: -0.14E+01 at ( 11,224, 16)
       Total Heat Release Rate:           4181.046 kW
       Radiation Loss to Boundaries:     -1256.277 kW
       Mesh    5, Cycle  123400
       CPU/step:     2.877 s, Total CPU:       97.73 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.90E+00 at ( 63, 95, 23)
       Max divergence:  0.20E+00 at ( 93,210, 22)
       Min divergence: -0.14E+00 at ( 92,210, 23)
       Radiation Loss to Boundaries:       -94.470 kW
       Mesh    6, Cycle  123400
       CPU/step:     2.860 s, Total CPU:       96.67 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.90E+00 at ( 58, 94, 23)
       Max divergence:  0.35E-01 at ( 57, 95, 22)
       Min divergence: -0.33E-01 at ( 12,265, 24)
       Radiation Loss to Boundaries:       -33.646 kW
       Mesh    7, Cycle  123400
       CPU/step:     2.862 s, Total CPU:       97.43 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.90E+00 at ( 95, 62, 23)
       Max divergence:  0.13E+00 at ( 95, 63, 24)
       Min divergence: -0.12E+00 at ( 95, 63, 23)
       Radiation Loss to Boundaries:       -59.111 kW
       Mesh    8, Cycle  123400
       CPU/step:     2.883 s, Total CPU:       98.34 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.90E+00 at ( 44, 14, 23)
       Max divergence:  0.11E+00 at ( 45, 15, 21)
       Min divergence: -0.62E-01 at ( 38, 98, 22)
       Radiation Loss to Boundaries:      -100.051 kW
       Mesh    9, Cycle  123400
       CPU/step:     2.907 s, Total CPU:       98.78 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.91E+00 at ( 69, 15, 23)
       Max divergence:  0.44E+00 at ( 70, 14, 22)
       Min divergence: -0.27E+00 at ( 70, 13, 23)
       Radiation Loss to Boundaries:      -137.383 kW
       Mesh   10, Cycle  123400
       CPU/step:     2.902 s, Total CPU:       98.41 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.92E+00 at ( 50, 14, 23)
       Max divergence:  0.35E+00 at ( 51, 15, 22)
       Min divergence: -0.18E+00 at ( 51, 14, 23)
       Radiation Loss to Boundaries:      -119.528 kW
       Mesh   11, Cycle  123400
       CPU/step:     2.874 s, Total CPU:       97.72 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.92E+00 at ( 34, 14, 23)
       Max divergence:  0.57E+00 at ( 35, 14, 22)
       Min divergence: -0.26E+00 at ( 35, 15, 23)
       Radiation Loss to Boundaries:      -103.613 kW
       Mesh   12, Cycle  123400
       CPU/step:     2.821 s, Total CPU:       95.87 hr
       Time step:  0.00627 s, Total time:     881.04 s
       Max CFL number:  0.90E+00 at ( 71, 15, 23)
       Max divergence:  0.34E-01 at ( 18, 40, 11)
       Min divergence: -0.24E-01 at (  3,  2, 24)
       Radiation Loss to Boundaries:       -64.737 kW
 

[Dave McGill]

The load is balanced. I don't know the geometry, the ideal situation would be to have two meshes of equal size. FDS simulations normally take days to run to completion.

Dave

[rahuljindal.66]

The geometry is simple car park. And i wan to add one thing that there are 47 jetfans in domain. which I have defined as HVAC.  These HVAC is making simulation slower. because i tried it without jetfans it was taking almost 1/10 of the time what with jet fans requires. But my doubt is still there , Why system is not using its full capacity.

Does pyrosim has its limit of using system memory???

[Charlie Thornton]

PyroSim does not impose a limit on the amount of system memory available to FDS. Each mesh is associated with an fds6_mpi process and OS-based process resource limits apply. If you are using 64-bit windows and 64-bit PyroSim (with 64-bit FDS) you should have a maximum of 8 TB of memory available per process. It's probably worth double-checking to ensure your are using 64-bit FDS.

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx#memory_limits

[rahuljindal.66]

I am attaching some snapshots of my system configuration and process

Kindly have a look and help me to find out that when the task manager is showing 100% usage why it is not using complete physical memory

[Dave McGill]

Hi,

The amount of memory used is determined by the number of cells and input lines.  If you halved or doubled the amount of memory, this job would still run at the same pace. You have 100% CPU utilization; it can't get any better than that.

Dave

[Lishi]

Hello,
It is normal to take a few days to run a model, if you need to get the results faster, you resizes the mesh, making it bigger, because it reduces the time considerably.

Regards. 


Hola,
Es normal que se demore algunos días en correr un modelo, si necesitas obtener los resultados más rápidamente, modifica el mesh, haciéndolo mas grande, ya que disminuye el tiempo considerablemente.

Saludos  .

[rahuljindal.66]

Thank you all for great response

As Dave said "If you halved or doubled the amount of memory, this job would still run at the same pace. You have 100% CPU utilization; it can't get any better than that."..........Does it mean if I increase the RAM with the same processor it is going to have no effect for the simulation time.

Xeon processor is the latest processor available in market as per my knowledge. so nothing can be done with the system to increase the simulation speed.

Is there any way to increase the simulation speed, what if I use cluster.

[Tobi_Moto]

To much Cached! 8 GB Cache is too high and it makes it slowly!

Presumably you have too many Meshes which actually slowes it because all Meshes have to interchange the data to each other.

[Dave McGill]

Once you have enough memory for a specific job, increasing the amount of memory will have no effect. The only way to reduce the processing time is to use more processors, and even that has a diminishing effect.

[Tobi_Moto]

Cache Memory has nothing to do with regular RAM. I did the same job with the same amount of Grid-Cells with different amount of cores and it gets faster if you have a smaller Cache Memory!

[Charlie Thornton]

As Dave said, once you have enough RAM, adding more will not make the simulation run faster. It looks like you have enough.

Concerning your original post about CPU utilization, the image of your task manager shows that your system has 32 cores (or 16 with hyper threading, but the OS thinks it has 32 so we'll say 32). That means that if you run a 12 mesh simulation, you will only be giving work to 12 of your 32 cores.

12 / 32 = 0.375 --> theoretical max cpu usage = 37.5%

And if you run a 24 mesh simulation, you will only be giving work to 24 of your 32 cores.

24 / 32 = 0.75 --> theoretical max cpu usage = 75%

This matches the cpu utilization you reported in your original post.

I assume that in both cases you have the same number of mesh cells and therefore the memory requirement to run the problem will be roughly the same.

As Dave suggested, as you divide the model into more and more meshes you incur a penalty from the "communication overhead". Any place where the mesh is divided between two processes, FDS has to send information about the cells that touch one another between the processes. The more mesh divisions you have the more this communication overhead will slow down the simulation.

In cluster simulations, communication overhead is very expensive because the data must travel over the network. For this reason, I recommend the cluster feature as a way to overcome memory limitations but not as a way to run a problem faster. Cluster is seldom faster, but if you don't have enough memory to run without cluster, it's the only way to go.

From what I've seen, you get the fastest performance when you use local, mpi-based parallel simulation with the same number of simulation meshes as you have cpu cores. The meshes must be similar in computational load (i.e. balanced). In your case, it looks like this would be 32 meshes.

However, I don't know why your task manager is showing 100% cpu utilization. Are you still using 24 meshes (which you earlier reported at 74% utilization)? Maybe this has something to do with hyper threading and 16 meshes would be better?

[rahuljindal.66]

Now I am using 32 meshes that's why cpu usage it is showing 100%.

after this all discussion can I conclude that the simulation is taking around 240hrs to complete is appropriate and is can not be made faster with this system configuration. only we can reduce mesh size to 16 which make little faster.

If so, what modification required in this configuration so that this kind of simulation can be completed within max 48 hrs or faster than that.