test: Add nexmark test, kafka test for backwards compatibility testing

[kwannoel]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Closes #9973

Changes to test framework

1. Major refactoring of backwards compatibility test. It now uses sqllogictest for the testing part, rather than handrolled test framework.
2. Nexmark test (without source, only RW tables)
3. Kafka test (with kafka source)
4. We provide a script to make it easy to run backwards compat tests locally, see the developer guide.
5. Tpch test.

Other improvements

1. We fix ./risedev k on full cluster with kafka. Previously kafka did not exit cleanly. The draw back is if kafka is enabled, we need to wait 5s. We could also use jps but I don't want to include an external dependency on risedev k.
2. Always fetch pre-built binaries for the backwards compat test. For old version, we just download the risingwave binary from github. For current version, just from the buildkite build step.
3. When we restart the cluster, we restart ALL components, not just RW.
   This is because we use risedev to manage components, and it is easier to just kill everything, rather than restart a subset of components.

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• All checks passed in ./risedev check (or alias, ./risedev c)
• My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

Documentation

• My PR contains user-facing changes.

Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

• Installation and deployment
• Connector (sources & sinks)
• SQL commands, functions, and operators
• RisingWave cluster configuration changes
• Other (please specify in the release note below)

Release note

[codecov]

Codecov Report

Merging #10413 (8f821fc) into main (52192e6) will increase coverage by 0.00%.
Report is 1 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main   #10413   +/-   ##
=======================================
  Coverage   69.76%   69.76%           
=======================================
  Files        1404     1404           
  Lines      235002   235002           
=======================================
+ Hits       163953   163955    +2     
+ Misses      71049    71047    -2     

Flag	Coverage Δ
rust	69.76% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 4 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[github-actions]

This PR has been open for 60 days with no activity. Could you please update the status? Feel free to ping a reviewer if you are waiting for review.

[kwannoel]

Just testing it on CI.

[lmatz]

LGTM, thanks!

[fuyufjh]

The rest LGTM

[fuyufjh]

Is there any reason to place is as a separated folder (backwards-compat-tests/)?

The previous convention seems to be placing

• slt tests in e2e_test/
• run_local.sh in scripts/

[kwannoel]

I prefer all backwards-compat-tests components:

• utils.sh
• run_local.sh
• slt files

To be all placed together, rather than split it into different directories for maintainability reasons.

For run_local.sh I will add a risedev backwards-compat-test for it to make it discoverable.

[kwannoel]

For run_local.sh I will add a risedev backwards-compat-test for it to make it discoverable.

Done.