Currently, the plan's behavior varies depending on whether the target dev environment already exists:
- If the target dev environment doesn't exist, the plan compares local models against the current state of the production environment, effectively creating a copy of production with the planned changes
- If the target dev environment already exists, the local models are compared against the current state of this target dev environment
This difference in behavior is pretty nuanced and often confuses users. Additionally, the plan's behavior can be considered inconsistent, as changes may be categorized differently depending on whether the target environment already exists.
At the same time, it appears that the user expects all changes to be made in relation to production regardless of the current state of the dev environment. Additionally, this ensures a certain level of consistency and predictability in change categorization.
The downside of always comparing to production is that incremental changes may lead to additional backfills, since intermediate changes are lost. For example, when a breaking change to model A is followed by a non-breaking change to the same model, downstream models will be backfilled only once when comparing against the target dev environment - but twice when comparing to production. This is because, in the latter case, the change will be categorized as breaking both times.
Therefore, we can introduce this behavior as an opt-in feature, making it configurable as part of the plan configuration. The tradeoff described above should be thoroughly documented.
Note that if the production environment doesn't exist yet, SQLMesh should still attempt to compare against the target environment.
Currently, the plan's behavior varies depending on whether the target dev environment already exists:
This difference in behavior is pretty nuanced and often confuses users. Additionally, the plan's behavior can be considered inconsistent, as changes may be categorized differently depending on whether the target environment already exists.
At the same time, it appears that the user expects all changes to be made in relation to production regardless of the current state of the dev environment. Additionally, this ensures a certain level of consistency and predictability in change categorization.
The downside of always comparing to production is that incremental changes may lead to additional backfills, since intermediate changes are lost. For example, when a breaking change to model A is followed by a non-breaking change to the same model, downstream models will be backfilled only once when comparing against the target dev environment - but twice when comparing to production. This is because, in the latter case, the change will be categorized as breaking both times.
Therefore, we can introduce this behavior as an opt-in feature, making it configurable as part of the plan configuration. The tradeoff described above should be thoroughly documented.
Note that if the production environment doesn't exist yet, SQLMesh should still attempt to compare against the target environment.