Conversation
|
Hi @ziotom78 , I slightly modified the function |
|
Thanks @sgiardie for having spotted the problem with NumPy types, I have slightly reworked your solution and now everything seems to work as expected. I have added a couple of tests to the suite, but more are needed:
Once this is in place, we just need to add a chapter to the User's Manual. |
Now that PR#136 has been merged, there is no reason to keep using h5py 3.1.
There are some peculiar situations (used mostly in unit tests) where distribute_optimally returns a number of time spans that is smaller than the number of MPI processes. This happens because of the way the painter's algorithm works. Suppose that there are 12 observations of equal length and 9 MPI processes; in this case, the painter's algorithm comes up with 2 observations for the first 6 processes and no observations for the last 3. This is an optimal solution because it minimizes the maximum amount of data per process; any alternative like 2+2+2+1+1+1+1+1+1 would leave 6 processes IDLE for 50% of the time and still require the same time as 2+2+2+2+2+2+0+0+0. This commit fixes a failed assert that assumed that all the processes should be filled with at least one observation.
Given that now distribute_optimally might assign no observations to some MPI processes, this commit makes sure that write_list_of_observations properly works in this case too.
|
Finally the PR has all the features implemented, and all the tests pass. Please have a look at the documentation here and post your comment; I plan to merge this next week. |
|
Warning: the tests have failed because of a failure in fetching PySM maps, but the tests were passing till last commit (which only removed a few debug statements). Until PR#147 is merged, we must cope with these failures. |
2 participants
For now the code uses h5py 3.1, as it is the last version supporting Python 3.6.