Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asof join with equality throws Not implemented Error. #9396

Closed
1 task done
michael-hlavacek opened this issue Oct 19, 2023 · 4 comments · Fixed by #9449
Closed
1 task done

Asof join with equality throws Not implemented Error. #9396

michael-hlavacek opened this issue Oct 19, 2023 · 4 comments · Fixed by #9449
Assignees

Comments

@michael-hlavacek
Copy link

What happens?

When I run the example from the documentation for asof join on tables with actual data, I get an Error: Not implemented Error: Unsupported comparison type for ASOF join.

When I remove either the equality condition or data from one of the tables, everything works. So all of

create table prices("when" timestamp with time zone, symbol int, price int);

create table trades("when" timestamp with time zone, symbol int);
insert into trades values ('2020-01-01 00:00:03+00', 1);

SELECT t.*, p.price
FROM trades t ASOF JOIN prices p 
  ON t.symbol = p.symbol AND t.when >= p.when;
create table prices("when" timestamp with time zone, symbol int, price int);
insert into prices values ('2020-01-01 00:00:00+00', 1, 42);

create table trades("when" timestamp with time zone, symbol int);

SELECT t.*, p.price
FROM trades t ASOF JOIN prices p 
  ON t.symbol = p.symbol AND t.when >= p.when;

and

create table prices("when" timestamp with time zone, symbol int, price int);
insert into prices values ('2020-01-01 00:00:00+00', 1, 42);

create table trades("when" timestamp with time zone, symbol int);
insert into trades values ('2020-01-01 00:00:03+00', 1);

SELECT t.*, p.price
FROM trades t ASOF JOIN prices p 
  ON t.when >= p.when;

work fine.

Also, changing the names of all columns and tables preserves the problem.

To Reproduce

create table prices("when" timestamp with time zone, symbol int, price int);
insert into prices values ('2020-01-01 00:00:00+00', 1, 42);

create table trades("when" timestamp with time zone, symbol int);
insert into trades values ('2020-01-01 00:00:03+00', 1);

SELECT t.*, p.price
FROM trades t ASOF JOIN prices p 
  ON t.symbol = p.symbol AND t.when >= p.when;

OS:

Ubuntu x64 in WSL on Windows 11

DuckDB Version:

v0.9.2-dev51 2646836

DuckDB Client:

cli

Full Name:

Míma Hlaváček

Affiliation:

Blindspot.ai

Have you tried this on the latest main branch?

I have tested with a main build

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

  • Yes, I have
@hawkfish
Copy link
Contributor

hawkfish commented Oct 21, 2023

The problem is that the optimiser can tell that your test data causes the inequality to always be true. We should fix this, but it is a somewhat artificial situation.

@hawkfish hawkfish self-assigned this Oct 22, 2023
@michael-hlavacek
Copy link
Author

michael-hlavacek commented Oct 23, 2023

I had this crash on me in a process where we batch-process some data, does not feel that artificial to me. (Not this specific query, of course, but an equivalent one with one equality and one inequality.)

@hawkfish
Copy link
Contributor

Ah sorry, it read like a random finding. But machine generated makes sense. Thanks for isolating it so nicely.

@michael-hlavacek
Copy link
Author

No worries. Thanks for looking into this! :)

hawkfish added a commit to hawkfish/duckdb that referenced this issue Oct 23, 2023
Don't remove the inequality condition for AsOf joins
even if the optimiser can tell it is always true.

fixes: duckdb#9396
fixes: duckdblabs/duckdb-internal#517
Mytherin added a commit that referenced this issue Oct 24, 2023
Issue #9396: AsOf Inequality Optimisation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants