feat(join): leverage band conditions in hash join

[soundOfDestiny]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Leverage band conditions in hash join.
Emit watermarks by band conditions in non-equi conditions in join condition of streaming hash join.
Clean state by band conditions in non-equi conditions in join condition in streaming hash join executor.

Checklist For Contributors

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• I have demonstrated that backward compatibility is not broken by breaking changes and created issues to track deprecated features to be removed in the future. (Please refer to the issue)
• All checks passed in ./risedev check (or alias, ./risedev c)

Checklist For Reviewers

• I have requested macro/micro-benchmarks as this PR can affect performance substantially, and the results are shown.

Documentation

• My PR DOES NOT contain user-facing changes.

Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

• Installation and deployment
• Connector (sources & sinks)
• SQL commands, functions, and operators
• RisingWave cluster configuration changes
• Other (please specify in the release note below)

Release note

[codecov]

Codecov Report

Merging #8749 (22e0d45) into main (4828bb5) will increase coverage by 0.01%.
The diff coverage is 83.05%.

@@            Coverage Diff             @@
##             main    #8749      +/-   ##
==========================================
+ Coverage   70.79%   70.80%   +0.01%     
==========================================
  Files        1171     1171              
  Lines      193943   194337     +394     
==========================================
+ Hits       137298   137608     +310     
- Misses      56645    56729      +84     

Flag	Coverage Δ
rust	70.80% <83.05%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/stream/src/from_proto/hash_join.rs	0.00% <0.00%> (ø)
src/frontend/src/expr/mod.rs	80.16% <56.60%> (-1.90%)	⬇️
...ontend/src/optimizer/plan_node/stream_hash_join.rs	91.86% <85.57%> (-2.73%)	⬇️
src/stream/src/executor/hash_join.rs	96.61% <94.89%> (-0.19%)	⬇️
src/frontend/src/utils/condition.rs	95.19% <95.00%> (-0.01%)	⬇️
...ntend/src/optimizer/plan_node/eq_join_predicate.rs	82.74% <100.00%> (+1.13%)	⬆️
...c/frontend/src/optimizer/plan_node/logical_join.rs	90.33% <100.00%> (+<0.01%)	⬆️
...rc/optimizer/rule/push_calculation_of_join_rule.rs	99.40% <100.00%> (+<0.01%)	⬆️
src/frontend/src/scheduler/distributed/query.rs	69.58% <100.00%> (+0.07%)	⬆️
src/stream/src/executor/test_utils.rs	93.84% <100.00%> (+0.09%)	⬆️

... and 7 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[st1page]

• Could we add a watermark in JoinSide and do the state cleaning logic inside the JoinSide?
• considering if we have a side's entry it is inserted but never matched, so we can not clean. we need to filter the useless data with the watermark when it is inserted.

[yuhao-su]

BandJoin indicates a BETWEEN sematic, while here one side inequality in enough.

[soundOfDestiny]

• Could we add a watermark in JoinSide and do the state cleaning logic inside the JoinSide?
• considering if we have a side's entry it is inserted but never matched, so we can not clean. we need to filter the useless data with the watermark when it is inserted.

• However some watermarks need to be emitted into downstream.
• We cannot filter at present or we cannot pass sanity check, but perhaps we can improve our sanity check in the future.

[yuhao-su]

BandJoin indicates a BETWEEN sematic, while here one side inequality in enough.

[CAJan93]

Some suggestion: Maybe it would also be helpful to add a test that evaluates the query and not just looks at the plan. Inspired by the PSQL tests:

CREATE TABLE t1 (
	c1 int,
	c2 int,
	c3 int,
	c4 timestamptz,
	c5 timestamp,
	c6 varchar,
	c7 varchar,
	c8 int,
	CONSTRAINT t1_pkey PRIMARY KEY (c1)
);
CREATE TABLE t2 (
	c1 int,
	c2 text,
	CONSTRAINT t2_pkey PRIMARY KEY (c1)
);

CREATE TABLE t4 (
	c1 int,
	c2 int,
	c3 text,
	CONSTRAINT t4_pkey PRIMARY KEY (c1)
);

INSERT INTO t4
	SELECT id,
	       id + 1,
	       'AAA'
	FROM generate_series(1, 100, 1) as t(id);
DELETE FROM t4 WHERE c1 % 3 != 0;

INSERT INTO t1
	SELECT id,
	       id % 10,
	       id,
	       '1970-01-01'::timestamptz + ((id % 100) || ' days')::interval,
	       '1970-01-01'::timestamp + ((id % 100) || ' days')::interval,
	       id % 10,
	       id % 10,
	       1
	FROM generate_series(1, 1000, 1) as t(id);

INSERT INTO t2
	SELECT id,
	       'AAA'
	FROM generate_series(1, 100, 1) t(id);

SELECT t1.c1, t2.c1 FROM (SELECT c1 FROM t4 WHERE c1 between 50 and 60) t1 FULL JOIN t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1;

[soundOfDestiny]

Some suggestion: Maybe it would also be helpful to add a test that evaluates the query and not just looks at the plan. Inspired by the PSQL tests:

CREATE TABLE t1 (
	c1 int,
	c2 int,
	c3 int,
	c4 timestamptz,
	c5 timestamp,
	c6 varchar,
	c7 varchar,
	c8 int,
	CONSTRAINT t1_pkey PRIMARY KEY (c1)
);
CREATE TABLE t2 (
	c1 int,
	c2 text,
	CONSTRAINT t2_pkey PRIMARY KEY (c1)
);

CREATE TABLE t4 (
	c1 int,
	c2 int,
	c3 text,
	CONSTRAINT t4_pkey PRIMARY KEY (c1)
);

INSERT INTO t4
	SELECT id,
	       id + 1,
	       'AAA'
	FROM generate_series(1, 100, 1) as t(id);
DELETE FROM t4 WHERE c1 % 3 != 0;

INSERT INTO t1
	SELECT id,
	       id % 10,
	       id,
	       '1970-01-01'::timestamptz + ((id % 100) || ' days')::interval,
	       '1970-01-01'::timestamp + ((id % 100) || ' days')::interval,
	       id % 10,
	       id % 10,
	       1
	FROM generate_series(1, 1000, 1) as t(id);

INSERT INTO t2
	SELECT id,
	       'AAA'
	FROM generate_series(1, 100, 1) t(id);

SELECT t1.c1, t2.c1 FROM (SELECT c1 FROM t4 WHERE c1 between 50 and 60) t1 FULL JOIN t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1;

The example is extremely helpful. Thank you very much.

[chenzl25]

LGTM! Thank you for your effort.