Skip to content

Conversation

migmartri
Copy link
Member

@migmartri migmartri commented Jul 7, 2023

NOTE: This is needed because we've moved to a better database migration mechanism #221 and now the migration process is not automatic in the app but as part of the deployment process.

Updates our Helm Chart to perform database migrations. It's been implemented using helm hooks and kubernetes jobs as suggested here.

I've also seen some Bitnami Charts following the same pattern, so I'll be stick to hooks + jobs instead of init containers for now.

Example of job output for the initial import

$ k logs chainloop-controlplane-migrate-5vjw7                                                                                                                      
Migrating to version 20230706165452 (1 migrations in total):                                                                                                                                               
                                                                                                                                                                                                           
  -- migrating version 20230706165452                                                                                                                                                                      
    -> CREATE TABLE "organizations" ("id" uuid NOT NULL, "name" character varying NOT NULL DEFAULT 'default', "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY ("id"));            
    -> CREATE TABLE "workflow_contracts" ("id" uuid NOT NULL, "name" character varying NOT NULL, "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "deleted_at" timestamptz NULL, "organization_
workflow_contracts" uuid NULL, PRIMARY KEY ("id"), CONSTRAINT "workflow_contracts_organizations_workflow_contracts" FOREIGN KEY ("organization_workflow_contracts") REFERENCES "organizations" ("id") ON UP
DATE NO ACTION ON DELETE CASCADE);                                                                                                                                                                         
    -> CREATE TABLE "workflows" ("id" uuid NOT NULL, "name" character varying NOT NULL, "project" character varying NULL, "team" character varying NULL, "runs_count" bigint NOT NULL DEFAULT 0, "created_a
t" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "deleted_at" timestamptz NULL, "organization_id" uuid NOT NULL, "workflow_contract" uuid NOT NULL, PRIMARY KEY ("id"), CONSTRAINT "workflows_organizatio
ns_workflows" FOREIGN KEY ("organization_id") REFERENCES "organizations" ("id") ON UPDATE NO ACTION ON DELETE CASCADE, CONSTRAINT "workflows_workflow_contracts_contract" FOREIGN KEY ("workflow_contract")
 REFERENCES "workflow_contracts" ("id") ON UPDATE NO ACTION ON DELETE NO ACTION);                                                                                                                          
    -> CREATE TABLE "robot_accounts" ("id" uuid NOT NULL, "name" character varying NOT NULL, "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "revoked_at" timestamptz NULL, "workflow_robotacc
ounts" uuid NULL, PRIMARY KEY ("id"), CONSTRAINT "robot_accounts_workflows_robotaccounts" FOREIGN KEY ("workflow_robotaccounts") REFERENCES "workflows" ("id") ON UPDATE NO ACTION ON DELETE CASCADE);     
    -> CREATE TABLE "workflow_contract_versions" ("id" uuid NOT NULL, "body" bytea NOT NULL, "revision" bigint NOT NULL DEFAULT 1, "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "workflow_c
ontract_versions" uuid NULL, PRIMARY KEY ("id"), CONSTRAINT "workflow_contract_versions_workflow_contracts_versions" FOREIGN KEY ("workflow_contract_versions") REFERENCES "workflow_contracts" ("id") ON U
PDATE NO ACTION ON DELETE SET NULL);                                                                                                                                                                       
    -> CREATE TABLE "workflow_runs" ("id" uuid NOT NULL, "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "finished_at" timestamptz NULL, "state" character varying NOT NULL DEFAULT 'initializ
ed', "reason" text NULL, "run_url" character varying NULL, "runner_type" character varying NULL, "attestation" jsonb NULL, "robot_account_workflowruns" uuid NULL, "workflow_workflowruns" uuid NULL, "work
flow_run_contract_version" uuid NULL, PRIMARY KEY ("id"), CONSTRAINT "workflow_runs_robot_accounts_workflowruns" FOREIGN KEY ("robot_account_workflowruns") REFERENCES "robot_accounts" ("id") ON UPDATE NO
 ACTION ON DELETE SET NULL, CONSTRAINT "workflow_runs_workflow_contract_versions_contract_version" FOREIGN KEY ("workflow_run_contract_version") REFERENCES "workflow_contract_versions" ("id") ON UPDATE N
O ACTION ON DELETE CASCADE, CONSTRAINT "workflow_runs_workflows_workflowruns" FOREIGN KEY ("workflow_workflowruns") REFERENCES "workflows" ("id") ON UPDATE NO ACTION ON DELETE CASCADE);                  
    -> CREATE INDEX "workflowrun_created_at_id" ON "workflow_runs" ("created_at", "id");                                                                                                                   
    -> CREATE INDEX "workflowrun_created_at_state" ON "workflow_runs" ("created_at", "state");                                                                                                             
    -> CREATE TABLE "integrations" ("id" uuid NOT NULL, "kind" character varying NOT NULL, "description" character varying NULL, "secret_name" character varying NOT NULL, "created_at" timestamptz NOT NUL
L DEFAULT CURRENT_TIMESTAMP, "configuration" bytea NULL, "deleted_at" timestamptz NULL, "organization_integrations" uuid NOT NULL, PRIMARY KEY ("id"), CONSTRAINT "integrations_organizations_integrations"
 FOREIGN KEY ("organization_integrations") REFERENCES "organizations" ("id") ON UPDATE NO ACTION ON DELETE CASCADE);                                                                                       
    -> CREATE TABLE "integration_attachments" ("id" uuid NOT NULL, "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "configuration" bytea NULL, "deleted_at" timestamptz NULL, "integration_att
achment_integration" uuid NOT NULL, "integration_attachment_workflow" uuid NOT NULL, PRIMARY KEY ("id"), CONSTRAINT "integration_attachments_integrations_integration" FOREIGN KEY ("integration_attachment
_integration") REFERENCES "integrations" ("id") ON UPDATE NO ACTION ON DELETE CASCADE, CONSTRAINT "integration_attachments_workflows_workflow" FOREIGN KEY ("integration_attachment_workflow") REFERENCES "
workflows" ("id") ON UPDATE NO ACTION ON DELETE CASCADE);                                                                                                                                                  
    -> CREATE TABLE "users" ("id" uuid NOT NULL, "email" character varying NOT NULL, "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY ("id"));                                     
    -> CREATE UNIQUE INDEX "users_email_key" ON "users" ("email");                                                                                                                                         
    -> CREATE TABLE "memberships" ("id" uuid NOT NULL, "current" boolean NOT NULL DEFAULT false, "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "updated_at" timestamptz NOT NULL DEFAULT CUR
RENT_TIMESTAMP, "organization_memberships" uuid NOT NULL, "user_memberships" uuid NOT NULL, PRIMARY KEY ("id"), CONSTRAINT "memberships_organizations_memberships" FOREIGN KEY ("organization_memberships")
 REFERENCES "organizations" ("id") ON UPDATE NO ACTION ON DELETE CASCADE, CONSTRAINT "memberships_users_memberships" FOREIGN KEY ("user_memberships") REFERENCES "users" ("id") ON UPDATE NO ACTION ON DELE
TE CASCADE);                                                                                                                                                                                               
    -> CREATE UNIQUE INDEX "membership_organization_memberships_user_memberships" ON "memberships" ("organization_memberships", "user_memberships");                                                       
    -> CREATE TABLE "oci_repositories" ("id" uuid NOT NULL, "repo" character varying NOT NULL, "secret_name" character varying NOT NULL, "created_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "vali
dation_status" character varying NOT NULL DEFAULT 'OK', "validated_at" timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, "organization_oci_repositories" uuid NOT NULL, PRIMARY KEY ("id"), CONSTRAINT "oci_r
epositories_organizations_oci_repositories" FOREIGN KEY ("organization_oci_repositories") REFERENCES "organizations" ("id") ON UPDATE NO ACTION ON DELETE CASCADE);                            
  -- ok (48.602248ms)                                                                                                                                                                                      
                                                                                                                                                                                                           
  -------------------------                                                                                                                                                                                
  -- 76.798634ms                                                                                                                                                                                           
  -- 1 migrations                                                                                                                                                                                          
  -- 15 sql statements                           

Closes #221

Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
@migmartri migmartri requested a review from danlishka July 7, 2023 11:34
type: application
# Bump the patch (not minor, not major) version on each change in the Chart Source code
version: 1.7.0
version: 1.7.1
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll bundle this change with the automatic PR that will be created when we release next week.

Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
@migmartri migmartri merged commit ed21026 into chainloop-dev:main Jul 10, 2023
@migmartri migmartri deleted the atlas-migration-deployment branch July 10, 2023 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

enhancement: atlas versioned migrations support
2 participants