Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Selectors in selectors.yml do not default to --indirect-selection or DBT_INDIRECT_SELECTION #7673

Closed
2 tasks done
dbeatty10 opened this issue May 21, 2023 · 3 comments · Fixed by #10009
Closed
2 tasks done
Assignees
Labels
bug Something isn't working node selection Functionality and syntax for selecting DAG nodes

Comments

@dbeatty10
Copy link
Contributor

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When a YAML selector definition does not include indirect_selection, then neither the --indirect-selection CLI argument nor the DBT_INDIRECT_SELECTION environment variable have any effect.

Instead, these YAML selectors always default to eager, no matter what.

Expected Behavior

This says:

If provided, a YAML selector's indirect_selection value will take precedence over the CLI flag --indirect-selection.

And this says:

If defined field in selector, override CLI flag

Both of those imply that the intent was that selectors would use get_flags().INDIRECT_SELECTION or flags.INDIRECT_SELECTION when indirect_selection is not defined within a selector in selectors.yml.

Steps To Reproduce

Reprex

Project contents

models/_models.yml

version: 2

models:
  - name: model_b
    columns:
      - name: id
        tests:
          - relationships:
              to: ref('model_a')
              field: id
  - name: model_c
    columns:
      - name: id
        tests:
          - relationships:
              to: ref('model_b')
              field: id

models/model_a.sql

select 1 as id

models/model_b.sql

select * from {{ ref("model_a") }}

models/model_c.sql

select * from {{ ref("model_b") }}

selectors.yml

selectors:
  - name: my_selector
    definition:
      method: fqn
      value: model_b

Try it

Compare the output of these commands:

dbt ls -s model_b --indirect-selection eager
dbt ls -s model_b --indirect-selection cautious
dbt ls -s model_b --indirect-selection buildable

with the output of these:

dbt ls --selector my_selector --indirect-selection eager
dbt ls --selector my_selector --indirect-selection cautious
dbt ls --selector my_selector --indirect-selection buildable

Expectations

Eager:

my_project.model_b
my_project.relationships_model_b_id__id__ref_model_a_
my_project.relationships_model_c_id__id__ref_model_b_

Buildable:

my_project.model_b
my_project.relationships_model_b_id__id__ref_model_a_

Cautious:

my_project.model_b

Diagram

image

Relevant log output

No response

Environment

- OS:
- Python:
- dbt: 1.5.0

Which database adapter are you using with dbt?

No response

Additional Context

I tried a sampling of versions between dbt v1.0 and v1.5 and none of them took the --indirect-selection CLI argument into account for selectors.

I didn't include the empty mode for indirect selection in the reprex because I already had a diagram without it, and it doesn't affect explaining the bug report one way or the other. buildable isn't a relevant option prior to v1.4 and could have been left out as well, but it was already in the diagram so it gets to come along for the ride 😃

Copy link
Contributor

github-actions bot commented Jan 8, 2024

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

@github-actions github-actions bot added the stale Issues that have gone stale label Jan 8, 2024
Copy link
Contributor

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jan 15, 2024
@dbeatty10
Copy link
Contributor Author

There's some better diagrams of each of the indirect selection diagrams here: #9746 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working node selection Functionality and syntax for selecting DAG nodes
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants