Mitigation for https://github.com/casacore/python-casacore/issues/130 #153
Conversation
|
|
There was a problem hiding this comment.
Looks Good To Me!
This PR fixes the version of read_col that is used in the conversion function.
I'd guess that getcolslize() is expected to be affected by the same issue as getcol()?
If that's the case, we should (for another issue...) consider if we want to use the workaround also for the "chunked" siblings read_col_chunk() / load_col_chunk() functions that are used as delayed chunk read when using the delayed read MS functions / in the direct load from MS.
The chances that those functions trigger this issue are much smaller though, as
one would not normally go for chunks of several 10s of GBs. That must be the
reason why the issue had not shown up so far in ngCASA experiments (I think) . From the notebook that Sean attached in #152, it seems that the iterator based workaround could actually bring in some performance improvement, but for these two other functions we would need to compare against getcolslice(). I'm also not sure how the iteration by TIME/ANTENNA1/ANTENNA2 would fit.
|
|
||
| # Workaround for https://github.com/casacore/python-casacore/issues/130 | ||
| # WARNING: Assumes tb_tool is a single measurement set not an MMS. | ||
| # WARNING: Assumes the num_frequencies * num_polarisations > 2**29. If false, |
There was a problem hiding this comment.
Did this mean num_frequencies * num_polarisations < 2**29?
For further information, see #152.