test void fails with when both openmp and mpi are on

[bhermanmit]

Test suite fails when both openmp and mpi are turned on. From an initial assessment, looks like this occurs when macroscopic cross sections are 0 for a material (which is consistent with a void).

[bhermanmit]

I can reproduce the error when I replace the void material with an actual material. When cross sections are calculated for this material, I set material_xs % total to zero such that the neutron a neutron will hit the next surface and act like a void.

[paulromano]

Ok, this was a fun one to figure out. It has nothing to do with void materials really. It has to do with how the fission bank is allocated. In the subroutine create_fission_site, the number of fission sites that are banked is capped by the size of the fission_bank array. What we're seeing here is a combination of two things:

1. When the number of sites is small and the number of threads/processes large, the size of the fission bank for each thread is generally quite small.
2. If the current generation estimate of k-effective is off, too many or too few fission sites get generated. In this case for test_void, the first batch estimate of k-effective is about 0.5, which results in too many sites being generated.

The combination of these two factors means that it is very easy for a single thread on a single processor to reach its maximum number of fission sites. When this happens, the loop at the end of create_fission_sites is not entered, and the random number sequence then gets off, resulting in different answers. So, there's no race condition, and the answers are certainly not "wrong", so I wouldn't really consider this behavior a bug, per se -- it's just that you end up getting a different answer. Possible solutions to get around this:

• Increase the number of sites for this test
• Improve the first batch source guess (a point source seems ok)
• Allocate larger fission banks (probably not wise)

This issue is likely to be the cause of #259 also. I increased the number of particles in test_reflective_cone from 100 to 500 and then answers are consistent even with a large number of threads.

[paulromano]

Fixed in #272