[Feature] dbt snapshot --defer

[iljau]

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

Implement "dbt snapshot --defer", which would behave similar to "dbt run --defer".

Describe alternatives you've considered

No response

Who will this benefit?

When trying to set up "slim ci", one step in my case would be to run snapshots based on other dbt models.

dbt snapshot --select 'dbt.*,state:modified' --state manifests/staging

When running this command, generated references point to current schema, which doesn't have these tables, as they weren't generated during preceding run command:
dbt run --models state:modified --defer --state manifests/airflow/staging

Are you interested in contributing this feature?

No response

Anything else?

https://docs.getdbt.com/reference/node-selection/defer
https://docs.getdbt.com/docs/guides/best-practices#run-only-modified-models-to-test-changes-slim-ci

dbt version: 0.20.1

[jtcohen6]

@iljau Thanks for opening!

The steps here are quite straightforward. I'm going to mark this a good first issue, and I'd welcome a PR from a community member.

1. Include snapshot in the subcommands that get the --defer flag:
   

   dbt-core/core/dbt/main.py

   Lines 1091 to 1092 in f79a968

   	# --defer
   	_add_defer_argument(run_sub, test_sub, build_sub)

2. Remove defer_to_manifest override in SnapshotTask, so as to inherit the implementation from RunTask:
   

   dbt-core/core/dbt/task/snapshot.py

   Lines 25 to 27 in f79a968

   	def defer_to_manifest(self, adapter, selected_uids):
   	# snapshots don't defer
   	return

3. Then, we'd want to add/adjust a test case combining snapshots + deferral:
   

   dbt-core/test/integration/062_defer_state_test/test_defer_state.py

   Lines 154 to 161 in f79a968

   	# these should work without --defer!
   	self.run_dbt(['snapshot'])
   	# make sure these commands don't work with --defer
   	with pytest.raises(SystemExit):
   	self.run_dbt(['seed', '--defer'])
   	with pytest.raises(SystemExit):
   	self.run_dbt(['snapshot', '--defer'])

Is this a change you'd be interested in contributing?

one step further

At this point, there's no good reason not to support --defer for compile, too, and thereby docs generate as well (#3741). (The only reason I can come up with: we might eventually want to rename/rework the UX of deferral (#2968)).

For that support:

• Add the --defer flag to the compile and docs subparsers, too
• Move defer_to_manifest up from RunTask into CompileTask (its parent)

alternative

The build task, new in v0.21, supports both snapshots and deferral:

$ dbt build --resource-type snapshot --defer --state manifests/airflow/staging`

So, that's something you can use in the meantime.

[kadero]

Hello @jtcohen6 @iljau 👋,

Thank you for opening this issue and providing a step by step solution 🙂.
Happy to work on this feature and to raise a PR for it. Ok for you?

🙏

[iljau]

I switched to using dbt build and don't have immediate plans to work on a fix.

[ciejer]

Hi @jtcohen6 - just checking in.
Is there another issue you're aware of that covers your "one step further" note from above? We're missing deferral in catalog generation (dbt docs generate); followed the link here from #3741 but the PR for this does not appear to close that issue.