-
Notifications
You must be signed in to change notification settings - Fork 129
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix test failures in duckdb 0.10.0 #1999
Conversation
@@ -257,22 +257,6 @@ def test_duckdb_arrow_array(): | |||
assert len(df) == 2 | |||
|
|||
|
|||
@mark_with_dialects_including("duckdb") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
test no longer needed with later versions of pandas which seems to deal with typing better
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool, makes sense.
Not sure how the lockfile fits with the one in #1998 but presume you know which is more up-to-date
The root cause of the error is that we've been relying on implicit typecasting which is no longer possible
From duckdb 0.10.0 forwards, if dob is a date or timestamp, you cannot do this:
LEVENSHTEIN("dob_l", "dob_r") <= 1
Which means that you have to correctly cast all columns.
Note you can also no longer do
try_strptime(dob_l, dob_r)
either if dob is a datetimeMy proposed 'solution' is to have the function only work in duckdb 0.10.0 if the input is a string (
cast_strings_to_date: bool = True
). But NOT to error out if the user setscast_strings_to_date: bool = False
, because that would break things in duckdb < 0.10.0.I think this makes sense because many of the function arguments only make sense if the input is a string e.g.:
invalid_dates_as_null
cannot be used if the input is a dateseparate_1st_january
When users have problems, we need to tell them to:
example