SQLFluff fails when dbt model is disabled

[dwallace0723]

if a dbt model is disabled (i.e. has configuration of enabled = false) sqlfluff will fail with the following internal error:

dbt@e5d0121500ec:/workspaces/dbt-dutchie$ sqlfluff lint models/shared/ga_daily_sessions.sql
WARNING    Unable to lint models/shared/ga_daily_sessions.sql due to an internal error. Please report this as an issue with your query's contents and stacktrace below!
To hide this warning, add the failing file to .sqlfluffignore
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/sqlfluff/core/linter.py", line 1404, in lint_path
    self.lint_string(
  File "/usr/local/lib/python3.8/site-packages/sqlfluff/core/linter.py", line 1186, in lint_string
    parsed = self.parse_string(in_str=in_str, fname=fname, config=config)
  File "/usr/local/lib/python3.8/site-packages/sqlfluff/core/linter.py", line 874, in parse_string
    templated_file, templater_violations = self.templater.process(
  File "/usr/local/lib/python3.8/site-packages/sqlfluff/core/templaters/dbt.py", line 188, in process
    return self._unsafe_process(fname, in_str, config)
  File "/usr/local/lib/python3.8/site-packages/sqlfluff/core/templaters/dbt.py", line 230, in _unsafe_process
    raise RuntimeError("File %s was not found in dbt project" % fname)
RuntimeError: File models/shared/ga_daily_sessions.sql was not found in dbt project

This error seems a little misleading. Additionally, I'm curious about the design decision to not lint dbt models that are disabled.

Expected Behaviour

For sqlfluff to be able to lint disabled models. Or at least have a more helpful error message.

Steps to Reproduce

Disable a dbt model and try to lint it.

Version

$ sqlfluff --version
sqlfluff, version 0.4.1

Configuration

[sqlfluff]
dialect = snowflake
templater = dbt
# dbt templating does not keep trailing new lines (L009)
exclude_rules = L007,L009,L031,L033,L034

[sqlfluff:rules]
max_line_length = 120
comma_style = trailing

[sqlfluff:rules:L010]
capitalisation_policy = upper

[alanmcruickshank]

@dwallace0723 - I'm pretty sure in this case, a disabled model isn't compiled. The dbt templater uses the dbt compiler internally to process the file before linting. If it's never compiled then the templater won't be able to find it to lint it.

I'm tempted to say that the solution here is a clearer error message which says the user will either need to enable the model, or use a different templater. Using the basic jinja templater would also work, but would fail for any files which use any more advanced dbt syntax.

More advanced solutions would involve:

• Falling back to the jinja templater if the dbt templater fails?
• Hacking the dbt compiler to force enable a model in some way if it's specified directly?

@dmateusp , as the expert on the dbt templater, what do you think?

[dwallace0723]

I'm tempted to say that the solution here is a clearer error message which says the user will either need to enable the model, or use a different templater.

@alanmcruickshank what are your thoughts on just skipping disabled dbt models and displaying a message saying the file/model was skipped or couldnt be found? dbt does something similar when trying to run disabled models:

dbt@1432a0e63563:/workspaces/dbt-dutchie$ dbt run --models ga_daily_sessions
Running with dbt=0.19.1
Found 330 models, 370 tests, 20 snapshots, 0 analyses, 440 macros, 1 operation, 0 seed files, 141 sources, 0 exposures
The selector 'ga_daily_sessions' does not match any nodes and will be ignored
WARNING: Nothing to do. Try checking your model configs and model specification args

[dmateusp]

With the PR I opened, here's the new behavior:

🚀 sqlfluff lint models/my_new_project/disabled_model.sql
=== [dbt templater] Compiling dbt project...
WARNING    Skipped file models/my_new_project/disabled_model.sql because the model was disabled

[polmonso]

Can we silence the warning as well?