Fix: prevent AttributeError when a DBT project is run with dbt-core >= 1.9.0

[erindru]

Prior to this change, if SQLMesh tries to load a DBT project and dbt-core==1.9.0 or newer is in use, the following error occurs:

AttributeError: 'Flags' object has no attribute 'state_modified_compare_more_unrendered_values'

The Flags being referred to are the project_flags which we currently hardcode to None.

This wasn't a problem until DBT 1.9.0 added a bunch of unguarded checks for get_flags().state_modified_compare_more_unrendered_values.

I checked where this was meant to be set in DBT and it turns out that flags.set_from_args() will automatically include these default flags if both project_dir and profiles_dir are specified.

Today, we only specify profiles_dir. It looks like project_dir is expected to be the directory containing dbt_project.yml and it will merge values from the flags member of that config file with the hardcoded defaults.

So this PR sets project_dir to be passed to flags.set_from_args() which triggers the default ProjectFlags logic and SQLMesh runs successfully.

Note: The reason that our CI hasnt picked this up already is because pip resolves dbt-core to version 1.8.8 when make install-cicd-test is run. This is due to conflicts in transitive dependencies. As such, im not entirely sure how to write an automated test for this without adding a bunch of complexity / test harnesses for multiple DBT versions

[tobymao]

can we test this?

[erindru]

Not easily because dbt-core==1.8.8 is resolved in our cicd environment (due to both dbt-sqlserver and dbt-clickhouse not being compatible with 1.9.0 at the moment) and this issue only occurs on dbt-core>=1.9.0.

In my local environment, without this patch, if I force install dbt-core==1.9.1:

$ pytest tests/dbt/
...SNIP a heap of test failures with
E       AttributeError: 'Flags' object has no attribute 'require_batched_execution_for_custom_microbatch_strategy'

13 failed, 64 passed, 2 skipped, 638 warnings, 45 errors in 92.84s (0:01:32)

After this patch:

$ pytest tests/dbt/
...SNIP

122 passed, 2 skipped, 4197 warnings in 219.86s (0:03:39)

When our cicd environment starts resolving dbt-core>=1.9.0 (once all the dependencies are compatible) then this will be covered by existing tests