[Epic] state:modified should Actually (only) select the modified resources

[graciegoheen]

We currently call out a lot of caveats to state comparison in our docs.

There are many scenarios where executing dbt list --select state:modified over or under selects the appropriate resources. This leads to:

• confusion "why are these models selected as state:modified, I haven't changed them"
• performance concerns "why am I running 60 models when I only changed 2"
• potentially bad code sneaking it's way into production "i checked my changes using a Slim CI job, but these models weren't run, now production is broken"

While it's unlikely we can cover ever single edge case here, we should improve state:modified so that it Ideally actually only selects the modified resources.

This epic will track the relevant bugs/enhancements related to state:modified.

Issues

Beta Give feedback

1. [state:modified] Introduce tests to capture current state of env-aware configuration and state:modified #10002
   state +1
   
2. [Bug] semantic models always included as part of state:modified #9548
   backport 1.7.latest bug +2
   
3. [jinja if statement] [false positives] environment-aware logic in config should not cause resources to Always selected by state:modified #9563
   enhancement state state: modified +3
4. [macro] [false positives] environment-aware logic in config should not cause resources to Always selected by state:modified #9564
   enhancement state state: modified +3
5. [env_vars] [false positives] environment-aware logic for sources should not cause sources to always be marked as modified #9573
   enhancement state state: modified +3
6. [vars] [false negatives] Changes in used variables do not cause models to be selected by state:modified #4304
   paper_cut state state: modified +3
7. [CT-3166] [Bug] State modified --select state:modified ignores new sources #8727
   Refinement bug state: modified +3
8. [CT-333] Macro for warehouse selection dbt-snowflake#103
   enhancement help_wanted +2

[matt-winkler]

What would be lost by disallowing env-aware logic outside the dbt_project.yml?

[patkearns10]

Lots of examples here: https://discourse.getdbt.com/t/why-are-ci-jobs-state-method-building-more-less-models-nodes-than-expected/7504
An example of env_var logic here , which seems impossible to keep out of state:modified