Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Merge_asof error message unclear when attempting to join on missing values #23189

Closed
svenharris opened this issue Oct 16, 2018 · 1 comment · Fixed by #23190
Closed

BUG: Merge_asof error message unclear when attempting to join on missing values #23189

svenharris opened this issue Oct 16, 2018 · 1 comment · Fixed by #23190
Labels
Error Reporting Incorrect or improved errors from pandas Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@svenharris
Copy link
Contributor

The merge_asof function gives confusing error message when trying to join on columns which contain null values based on it failing the is_monotonic check

See example:

left = pd.DataFrame({'a': [10.0, np.nan, 3.0, 12.0, 15.0],
                     'left_val': ['a', 'b', 'c', 'd', 'e']})
right = pd.DataFrame({'a': [1.0, 5.0, 10.0, 12.0],
                      'right_val': [1, 6, 11, 15]})

# As a responsible pandas user I then sort my left DataFrame
left_sorted = left.sort_values(by="a")


# and try using merge_asof
merge_asof(left_sorted, right, on='a')
ValueError: left keys must be sorted

This can lead to scratching of heads, because hey, the DataFrame was just sorted, and it kind of looks sorted, (also doesn't appear in docs that this isn't allowed).

The proposed change will raise a more meaningful error to lead the user to the cause of the problem.

ValueError: Merge keys contain null values on left side

Working on a pull request with @tolaa06 which should resolve this issue

@svenharris svenharris mentioned this issue Oct 16, 2018
4 tasks
@jreback jreback added Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Reshaping Concat, Merge/Join, Stack/Unstack, Explode Error Reporting Incorrect or improved errors from pandas labels Oct 17, 2018
@jreback jreback added this to the 0.24.0 milestone Oct 17, 2018
@randomgambit
Copy link

ha funny I got this same error this morning and was scratching my head the same. glad to know this is due to nulls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Error Reporting Incorrect or improved errors from pandas Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants