Temporary fix for nan values in flag visibilities #408
Conversation
|
Can one of the admins verify this patch? |
|
ok to test |
|
@ulricharmel Out of interest, are the NaN values in the weight column of the measurement set? |
|
Hi @JSKenyon Sorry for the late replay. Those visibilities are flagged and their weights are zero. I did not check from the measurement set. Maybe those weights have been set to zero by the data handler because they are flagged. |
|
The reason I ask is because this line cannot produce NaN values unless there are already NaN values present in its inputs. I guess my question is in which array are the NaN values living? |
|
I checked it interactively with pdb. The weights were zero. But the values were nan +nanj in the off diagonal. The multiplication line gives a warning about invalid value encounter in multiplication. I tested it and the output of the multiplication is nan+nanj. I also tried multiplying those values with zero and I got nan+nanj with the warning. |
|
Ok, but does |
|
Yes it does |
|
@ulricharmel Ah thanks. That is odd. And are those NaN values in the data column on disk? @o-smirnov What is the expected behaviour here? While I am sure Ulrich's solution will work, I thought that we were cleaning up dodgy input values elsewhere. |
|
I've made fixes that should prevent this using getcolnp. Is your branch in
sync with master. You should not be getting nan values unless there are
actually nan values on the data array on disk.
An upgrade to casacore 3.3 and python-casacore 3.3 should also fix this
problem.
…On Fri, Aug 14, 2020 at 10:55 AM JSKenyon ***@***.***> wrote:
@ulricharmel <https://github.com/ulricharmel> Ah thanks. That is odd. And
are those NaN values in the data column on disk?
@o-smirnov <https://github.com/o-smirnov> What is the expected behaviour
here? While I am sure Ulrich's solution will work, I thought that we were
cleaning up dodgy input values elsewhere.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#408 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB4RE6XJLHYGDFEMID34IJLSAT3ZLANCNFSM4PX3ILRA>
.
--
--
Benjamin Hugo
PhD. student,
Centre for Radio Astronomy Techniques and Technologies
Department of Physics and Electronics
Rhodes University
Junior software developer
Radio Astronomy Research Group
South African Radio Astronomy Observatory
Black River Business Park
Observatory
Cape Town
|
|
@JSKenyon I did not check the MS itself, I checked for a specific time chunk the data that was being passed to the solver. |
|
@bennahugo I will upgrade to python-casacore 3.3 and test. |
|
do make sure you compile your python-casacore 3.3 against casacore 3.3. The
bug is in the C++ code in casacore itself.
…On Fri, Aug 14, 2020 at 11:09 AM ulricharmel ***@***.***> wrote:
@bennahugo <https://github.com/bennahugo> I will upgrade to
python-casacore 3.3 and test.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#408 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB4RE6T5FFQ2RQXU4A3PZP3SAT5LVANCNFSM4PX3ILRA>
.
--
--
Benjamin Hugo
PhD. student,
Centre for Radio Astronomy Techniques and Technologies
Department of Physics and Electronics
Rhodes University
Junior software developer
Radio Astronomy Research Group
South African Radio Astronomy Observatory
Black River Business Park
Observatory
Cape Town
|
|
Sounds like it's a workaround for NaN values in the input that should not have been appearing in the first place (especially from this comment). I have also made further fixes to MS reading in #424. However let's keep this open for the moment, as looking at the code, I can't find where flagged data/model is actually nulled. Except at https://github.com/ratt-ru/CubiCal/blob/master/cubical/solver.py#L108, but this ignore FL.PRIOR. So I suppose it's possible to have a pre-flagged NaN value sneak in and cause some havoc. For the sake of being defensive, we probably need to null it in the |
|
Closing for now - please reopen if this issue persists. |
No description provided.