Skip to content

feat(test): treat model file paths as test selectors - #6061

Open
tripleaceme wants to merge 1 commit into
SQLMesh:mainfrom
tripleaceme:test-path-selectors
Open

feat(test): treat model file paths as test selectors#6061
tripleaceme wants to merge 1 commit into
SQLMesh:mainfrom
tripleaceme:test-path-selectors

Conversation

@tripleaceme

Copy link
Copy Markdown
Contributor

Description

Closes #6023.

sqlmesh test already took multiple positional arguments, but they were only ever looked up in the test-file index. A model file matched nothing and the command exited 0, so a pre-commit hook passing staged model files silently ran no tests.

Each positional argument now resolves to one of:

  • a test file → every test in it
  • file.yaml::test_name → that one test
  • a model file → the tests targeting that model

The results are unioned. Per your note on the issue, they are also deduplicated on fully_qualified_test_name, so overlapping selectors don't cause duplicate runs — sqlmesh test models/a.sql tests/test_a.yaml, where the YAML holds a's tests, runs each of them once, and tests/a.yaml::test_x tests/a.yaml collapses to one run too.

An argument resolving to neither a known test nor a known model file is now an error rather than a silent skip. That part is opt-in through raise_on_unknown_paths and only the CLI turns it on, because LSPContext.get_document_tests calls select_tests with whatever document happens to be open and depends on getting an empty list back — raising unconditionally would make the extension throw on every non-test file. A known model that simply has no tests still selects nothing and is not an error, since failing a hook because a model lacks a unit test isn't the intent here.

--select-model is unchanged: it still narrows the selection rather than adding to it.

Two notes on cost, since path matching is on the hot path for the pre-commit use case:

  • Selectors are matched exactly as given before any normalization, so the existing absolute-path callers pay nothing new, and normalization is os.path.abspath rather than resolve() — no stat or symlink syscalls per model.
  • The model-path index is built at most once per call, and only if some selector turns out not to be a test file. Passing 50 staged model files doesn't rebuild it 50 times.

Test Plan

Nine tests in tests/core/test_test.py:

  • test_model_path_selects_its_tests / test_model_path_without_tests_selects_nothing
  • test_model_and_test_paths_are_unioned_without_duplicates — the dedup case from your comment
  • test_overlapping_yaml_and_named_test_are_deduplicated
  • test_relative_paths_select_tests — pre-commit passes repo-relative paths
  • test_unknown_path_is_ignored_by_default — guards the LSP's behavior
  • test_unknown_path_errors_when_requested / test_unknown_test_name_errors_when_requested
  • test_select_model_still_filters_path_selection

Two in tests/cli/test_cli.py covering the CLI wiring and the non-zero exit on an unknown path.

pytest tests/core/test_test.py tests/cli/test_cli.py tests/lsp
222 passed, 1 skipped

The 4 remaining failures in those files (test_pyspark_python_model, three test_dlt_*) are pre-existing on main in my environment — missing pyspark and dlt.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

`sqlmesh test` already accepted multiple positional arguments, but they
were only ever looked up in the test-file index. Passing a model file
matched nothing and exited 0, so a commit hook handing over staged model
files silently ran no tests at all.

Each positional argument now resolves to a test file, a
`file.yaml::test_name`, or a model file, in which case the tests
targeting that model are selected. The results are unioned and
deduplicated on the fully qualified test name, so a model file and the
test file holding its tests select the same test once rather than
running it twice.

An argument that resolves to neither a known test nor a known model file
is now an error instead of a silent skip. That is opt-in via
raise_on_unknown_paths and only the CLI turns it on, because
LSPContext.get_document_tests selects against arbitrary open documents
and relies on an empty result. A known model that simply has no tests
still selects nothing, which is not an error.

Selectors are matched as given before being normalized, so relative
paths work from the project root without any resolve() calls per model.
The model path index is built at most once per call, and only when a
selector is not a test file.

Signed-off-by: Adegbite Ayoade <tripleaceme@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sqlmesh test should treat model file paths as test selectors

1 participant