You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The snowplow_web_base_sessions_lifecycle_manifest model's schema is defined by the snowplow__manifest_custom_schema variable. Since this model is a 'proper' dbt model, dbt will create this schema before doing anything else in the run if it doesn't exist already.
The snowplow_web_incremental_manifest table's schema is also defined by the snowplow__manifest_custom_schema variable. This however is a manually created table via DDL.
Since the manifest schema is created by dbt, when we go to create the snowplow_web_incremental_manifest table, the manifest schema exists.
Potential implementation
A user overrides the snowplow_web_base_sessions_lifecycle_manifest model's schema but not by changing the snowplow__manifest_custom_schema variable accordingly (this can be achieved by amending the dbt_project.yml)
The snowplow_web_incremental_manifest tables schema is still defined by the snowplow__manifest_custom_schema variable. This schema will no longer be generated automatically by dbt since the snowplow_web_base_sessions_lifecycle_manifest model is not using the same schema. This would cause the CREATE TABLE DDL to fail as the schema doesn't exist.
This is why the create_snowplow_manifest_schema step is included as a fail safe.
Issue
The adapter.create_schema method I use here to manually create the schema, under the hood calls CREATE SCHEMA IF NOT EXISTS.
If the user does not have schema creation permissions this will fail.
Despite dbt recommending having such permissions for all dbt users, one of our customers using the package does not allow their dbt users to have such permissions. Rather a super user manually creates all the schemas they need for dbt.
This meant every time they ran the package it failed due to this step, despite the required schema already existing.
Fix
We should remove the schema creation step to be sure the package will run irrespective of the users permissions.
The text was updated successfully, but these errors were encountered:
Currently
snowplow_web_base_sessions_lifecycle_manifest
model's schema is defined by thesnowplow__manifest_custom_schema
variable. Since this model is a 'proper' dbt model, dbt will create this schema before doing anything else in the run if it doesn't exist already.snowplow_web_incremental_manifest
table's schema is also defined by thesnowplow__manifest_custom_schema
variable. This however is a manually created table via DDL.snowplow_web_incremental_manifest
table, the manifest schema exists.Potential implementation
snowplow_web_base_sessions_lifecycle_manifest
model's schema but not by changing thesnowplow__manifest_custom_schema
variable accordingly (this can be achieved by amending thedbt_project.yml
)snowplow_web_incremental_manifest
tables schema is still defined by thesnowplow__manifest_custom_schema
variable. This schema will no longer be generated automatically by dbt since thesnowplow_web_base_sessions_lifecycle_manifest
model is not using the same schema. This would cause theCREATE TABLE
DDL to fail as the schema doesn't exist.create_snowplow_manifest_schema
step is included as a fail safe.Issue
adapter.create_schema
method I use here to manually create the schema, under the hood callsCREATE SCHEMA IF NOT EXISTS
.Fix
The text was updated successfully, but these errors were encountered: