Skip to content

Cloud defer: ref() resolves to dev schema instead of deferred production schema #14230

Description

@dataders

Bug Description

When using Cloud-based defer (via defer_env_id), ref() resolves to the current dev target schema instead of the deferred production schema. The deferred manifest is downloaded successfully, but the deferral step appears to be a no-op — Finished deferring state [0.00s] — and compiled SQL references the dev schema where upstream models don't exist.

Local --defer --state works correctly (tested on preview.105 with Snowflake). The bug appears specific to the Cloud defer path using defer_env_id.

Customer Evidence (BigQuery, preview.108)

Deferred manifest shows production schema

The downloaded manifest at target/dbt_cloud_defer/manifest.json has the correct production coordinates:

{
  "schema": "analytics",
  "database": "inq-warehouse",
  "alias": "stg_blue_whale_places"
}

Compiled SQL uses dev schema (wrong)

target/compiled/models/dim_places.sql:

select * from `inq-warehouse`.`dbt_bwaligorski`.`stg_blue_whale_places`

Expected: inq-warehouse.analytics.stg_blue_whale_places
Actual: inq-warehouse.dbt_bwaligorski.stg_blue_whale_places (dev schema)

Logs show defer is a no-op

19:29:46.347508 [info ]: INFO Using defer_env_id '171533' for manifest download
19:29:46.347569 [info ]: DOWNLOADING deferral manifest
19:29:47.001461 [info ]: DOWNLOADED deferral manifest to target/dbt_cloud_defer/manifest.json
19:29:47.711930 [info ]: Started scheduling
19:29:47.712093 [info ]: Finished scheduling [  0.00s]
19:29:47.712820 [debug]: Initializing schema store with entry: Frontier(CanonicalFqn { catalog: "inq-warehouse", schema: "dbt_bwaligorski", table: "stg_blue_whale_places" })
19:29:47.713513 [info ]: Started deferring state
19:29:47.713545 [info ]: Finished deferring state [  0.00s]   ← no-op!

The schema store is initialized with the dev schema (dbt_bwaligorski) and deferring state takes 0.00s, meaning no refs are updated.

Downstream failure

[warn ]: dbt1014: Failed to download model schema for 'model.inquirer_dbt.stg_blue_whale_places'.
  [BigQuery] googleapi: Error 404: Not found: Dataset inq-warehouse:dbt_bwaligorski, notFound

Local Repro Attempt (Snowflake, preview.105)

Using jaffle_shop with Snowflake, two targets:

  • prod: schema dataders (all models built)
  • dev: schema dataders_defer_test (empty)
# Build prod state
dbt seed --target prod && dbt run --target prod

# Save manifest
cp target/manifest.json state/manifest.json

# Reproduce
dbt clean && dbt build --select customers --defer --state state/

Result: Local --state defer works correctly. Logs show:

Started deferring state
Finished deferring state [  0.96s]    ← actual work done

Compiled SQL correctly references prod schema:

select * from DBT_TEST.dataders.stg_customers   -- correct!

Summary

Scenario Defer Duration Compiled Schema Result
Cloud defer (defer_env_id, preview.108) 0.00s dev (dbt_bwaligorski) BROKEN
Local --state (preview.105) 0.96s prod (dataders) Works

The Cloud defer path downloads the manifest but does not apply it to update ref resolution. The local --state path correctly reads the manifest and remaps refs to the production schema.

Metadata

Metadata

Assignees

No one assigned

    Labels

    deferralengine:v2Concerns the dbt Fusion (v2) engine.type:bugA defect: Fusion behaves incorrectly versus expected/reference behavior.

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions