[CT-729] [Bug] unrendered_config in manifest.json doesn't update correctly from model properties yaml

[AnnaMChoi-CD]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Config changes from model properties yaml are not correctly propagated to the manifest.json, thus not showing up as a state change when using the state:modified selector.

Expected Behavior

I expect the config changes from the model properties yaml to be correctly propagated to the manifest.json - "unrendered_config", just like it does from the config changes made in dbt_project.yml.

This will fix the state:modified selector to correctly detect my config changes made in the model properties yaml files.

Steps To Reproduce

1. Clone the jaffle shop example, and set up the profiles.yml to get dbt working.
2. Run dbt compile. Look for customers model in target/manifest.json - has "unrendered_config": {"materialized": "table"}
3. Copy target/manifest.json to top directory.
4. Go to models/schemas.yaml and add the config block for customers model.

- name: customers
   config:
     materialized: view

5. Run dbt compile. Look for customers model in target/manifest.json - still has "unrendered_config": {"materialized": "table"}. The materialization config change hasn't been propagated
   5-1. Side-effect (which is the real problem for me): Run dbt ls -s state:modified --state .

The selection criterion 'state:modified' does not match any nodes
No nodes selected!

6. Go to dbt_project.yml, and update the model config as below:

models:
  jaffle_shop:
      materialized: table
      customers:
        materialized: view
      staging:
        materialized: view

7. Run dbt compile. Look for customers model in target/manifest.json - now the updated config is shown here: "unrendered_config": {"materialized": "table"}.
   7-1. Run dbt ls -s state:modified --state .

jaffle_shop.customers
jaffle_shop.not_null_customers_customer_id
jaffle_shop.relationships_orders_customer_id__customer_id__ref_customers_
jaffle_shop.unique_customers_customer_id

I expect the same result from step 5-1.

Relevant log output

No response

Environment

- OS: MacOS 12.1
- Python: 3.8.9
- dbt: 1.1.0 (dbt-bigquery 1.1.0)

What database are you using dbt with?

bigquery

Additional Context

Similar issue is reported for the jinja config macro:
#3680
But the fix for the jinja config macro won't necessarily fix this issue.

[jtcohen6]

Agree that #3576 and #3680 are related to the larger topic here, but actually separate from the specific bug reported here.

The bug here seems much easier to reproduce, and (hopefully) to get to the bottom of why configurations defined in standard .yml files aren't being reflected in unrendered_config.

Ideally, unrendered_config would include (as the name says) the unrendered versions of configs in .yml files, such that if your config looks like:

- name: customers
   config:
     materialized: "{{ 'view' if target.name == 'dev' else 'table' }}"

We'd store that whole Jinja expression, rather than 'view' or 'table' (depending on the environment), leading to cross-env "false positives" for state:modified.

As a practical matter, even just storing 'view' or 'table' sounds better than the status quo, where the materialization change fails to propagate at all, leading to a "false negative" worse.