Skip to content

VDE_MODE only derived from project config, not from state #5887

Description

@davem-bis

Steps to Reproduce

  1. The prod environment is initially planned with vde_mode = All
  2. vde_mode is set to dev_only
  3. A plan of a dev environment will assume that prod is set as physical models, and perform table copy operations as if this is the case, despite prod still being in vde_mode = All.

For Databricks, this can cause errors as SQLMesh attempts to clone the prod environment assuming it is a table, and fails with [DELTA_CLONE_UNSUPPORTED_SOURCE] as the prod model is still a view at this point.

Location in Code

The VDE mode is set in the self.config object of the loader, which is set as self.context.configs[self.config_path] in sqlmesh/core/loader.py::Loader::__init__.

Proposed Solution

When in a plan or run, VDE mode should be derived from both the config, and the current state of the prod environment in the target gateway. Potentially a warning should be raised when they do not agree.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions