merge command returned nothing when input msdfs had no confidence column
#350
Conversation
|
I made this decision. Does this seem logical @matentzn ? |
hrshdhgd
changed the title
merge returned nothing when msdfs had no confidence columnmerge command returned nothing when msdfs had no confidence column
hrshdhgd
changed the title
merge command returned nothing when msdfs had no confidence columnmerge command returned nothing when input msdfs had no confidence column
sssom/util.py
Outdated
| @@ -273,6 +273,8 @@ def filter_redundant_rows( | |||
| :param ignore_predicate: If true, the predicate_id column is ignored, defaults to False | |||
| :return: Filtered pandas DataFrame | |||
| """ | |||
| if CONFIDENCE not in df.columns: | |||
There was a problem hiding this comment.
What would happen if we assumed that missing confidence means 100% confidence? I think this assumption is more often true. Also, if you do ad the IF clause here, the dataframe will not be sorted?
There was a problem hiding this comment.
I just realized, line 280 has assign_default_confidence which adds a new column 'confidence' if absent BUT it initiates to np.NaN. Is this what we still intend to do?
There was a problem hiding this comment.
I think so, but it is important that the absence of a confidence value does not lead to a removal of the entire mapping! While it is not the same as confidence 1.0, it should probably be interpreted as such during reconciliation.. Unless you see a strong reason against it?
There was a problem hiding this comment.
The presence of np.NaN is causing the df returned to be empty in the code between lines 297 to 311
There was a problem hiding this comment.
Would it make sense to itiniate it to "0.0" instead of np.Nan
|
Added a flag: |
Co-authored-by: Nico Matentzoglu <nicolas.matentzoglu@gmail.com>
Co-authored-by: Nico Matentzoglu <nicolas.matentzoglu@gmail.com>
|
Good catch! sorry about that! |
|
Done |
|
Nice! Thanks! |
Fixes #348