[CT-2169] [Bug] All global configs should also be settable in ProjectFlags

[jtcohen6]

Opening from #6903 (comment)

There are some "global configs" that are supported as params (flag / env var), but missing in ProjectFlags.

Acceptance criteria:

• All "global configs" should be settable in three places (CLI flag, env var, dbt_project.yml). These decorators, documented here:
  https://docs.getdbt.com/reference/global-configs
• EXCEPT configs whose type is click.Path (--log-path, --target-path), which do not make sense to define in a separate file

Relevant, but very much out of scope for this issue:

• [CT-1470] Whither UserConfig? #6207

[jtcohen6]

But maybe not --log-path or --target-path. Not sure any "path"-specific configs make a whole lot of sense! :)

[jtcohen6]

Switching this from tech_debt to bug, because it's actually unexpected behavior, given our current documentation

[emmyoop]

Add a test to pull out everything global and ensure there are configs.

[dbeatty10]

Caution

This issue is in the same area of code as #9183 / #9289.

Any solutions to #9183 and #7036 should be compared to each other to make sure one doesn't accidentally stomp on / overwrite the solution of the other.

[dbeatty10]

I took a swing at auditing our global configs versus UserConfig and recorded them in this internal spreadsheet.

Global configs missing from UserConfig

After filtering out flags that are hidden=True or click.Path or excluded for other reasons*, these are the global configs missing from UserConfig:

• print
• quiet
• log_cache_events

Exclusions

Previously decided here to exclude profile and target from this effort.

Excluded version since it is a unique flag that merely displays the relevant versions before exiting any further execution.

Hidden flags that are missing from UserConfig

• deprecated_print
• enable_legacy_logger
• log_file_max_bytes
• macro_debugging
• partial_parse_file_diff
• single_threaded

UserConfig missing from global configs

There are were a handful that are missing the opposite direction (but are now fixed by #9040):

• indirect_selection
• log_format
• warn_error
• warn_error_options

For refinement: determine if these are okay and/or desired.

All of these except for indirect_selection are covered by #8715 / #9040.

[jtcohen6]

The remaining scope of this issue is to decide whether these "global" flags should be settable in dbt_project.yml:

• print
• quiet
• log_cache_events

The question being: Do these feel different enough from all other global configs to justify the inconsistency? Right now, we only disallow setting file-path flags (--log-path, --target-path, --state) in the project flags.

[dbeatty10]

print, quiet, and log_cache_events don't seem different enough to me to justify the inconsistency.

At various times, I've found myself tying to set each of these within profiles.yml at different times. Granted, print and log_cache_events would be most likely to be used while troubleshooting. And my uses of quiet were specifically to get parseable output from dbt list.

But it seems the least amount of effort and confusion within the code base and the documentation to make them global in dbt_project.yml. Do you buy it @jtcohen6?