tests: small change in tests folder will cause a full rebuild of binary

[skyzh]

Cause by #376

Is there any way that build.rs only detects only file name instead of full file content?

[arkbriar]

According to cargo: change detection of build script, re-run of build scripts only relies on the modification of filesystem "last-modified" timestamp.

To avoid re-run each time when only file content changes, we can manually maintain a list of the test cases (filenames) in a single file instead and declare it as the trigger.

[skyzh]

Good idea!

[skyzh]

... so we can create a "test_cases.rs", which includes a function, and include!("tests/test_cases.rs") in build.rs?

Meanwhile, in the rerun-if-changed config, we can only specify test_cases.rs as dependency.

[arkbriar]

... so we can create a "test_cases.rs", which includes a function, and include!("tests/test_cases.rs") in build.rs?

Meanwhile, in the rerun-if-changed config, we can only specify test_cases.rs as dependency.

What is the function in "test_cases.rs" for? If it's for scanning the dir, the build script can not be triggered when there're modifications in tests/sql.

IMO only a list of file names is needed. Developers should maintain the list whenever a test file is added or removed.

Besides, we can add a check on the list to make sure all files are included (and no non-existing files).

[skyzh]

You're right. Ignore my comments above 🤪

[skyzh]

Then I think a better approach is to:

• maintain a list of files in a vector in test_cases.rs, and generate the corresponding test case functions.
• check if all files matching the filename pattern are included.

cc @wangrunji0408

[xxchan]

I'm a little bit confused: if we need to maintain a list of files now, what's the difference from the situaion before the auto discovery?

[skyzh]

That's true... Maybe we should find another way to do auto-discovery of tests...

[Fedomn]

How about moving the integration test into a separate crate? Because now test file generation depends on cargo build script but currently cargo does not support test build scripts.