Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix migration v292 #30153

Merged
merged 16 commits into from
Mar 28, 2024
Merged

Fix migration v292 #30153

merged 16 commits into from
Mar 28, 2024

Conversation

lunny
Copy link
Member

@lunny lunny commented Mar 28, 2024

Fix #29874 (comment)

  • The migration of v292 will miss many projects. These projects will have no default board. This PR introduced a new migration number and removed v292 migration.

  • This PR also added the missed transactions on project-related operations.

  • Only SetDefaultBoard will remove duplicated defaults but not in GetDefaultBoard

@lunny lunny requested a review from denyskon March 28, 2024 03:22
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Mar 28, 2024
@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 28, 2024
@github-actions github-actions bot added modifies/go Pull requests that update Go code modifies/migrations labels Mar 28, 2024
@lunny lunny requested a review from wxiaoguang March 28, 2024 03:24
@wxiaoguang wxiaoguang removed their request for review March 28, 2024 03:56
@pull-request-size pull-request-size bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 28, 2024
@lunny lunny added the type/bug label Mar 28, 2024
@lunny lunny changed the title Improve migration v292 Fix migration v292 Mar 28, 2024
@lunny lunny added the skip-changelog This PR is irrelevant for the (next) changelog, for example bug fixes for unreleased features. label Mar 28, 2024
models/migrations/v1_22/v293_test.go Show resolved Hide resolved
models/migrations/v1_22/v293.go Show resolved Hide resolved
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Mar 28, 2024
@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Mar 28, 2024
@denyskon denyskon self-requested a review March 28, 2024 14:47
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. labels Mar 28, 2024
@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Mar 28, 2024
@6543 6543 enabled auto-merge (squash) March 28, 2024 15:47
@6543 6543 added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Mar 28, 2024
@6543 6543 merged commit 40cdc84 into go-gitea:main Mar 28, 2024
26 checks passed
@GiteaBot GiteaBot added this to the 1.23.0 milestone Mar 28, 2024
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Mar 28, 2024
@techknowlogick techknowlogick modified the milestones: 1.23.0, 1.22.0 Mar 28, 2024
@lunny lunny deleted the lunny/improve_migration_v292 branch March 29, 2024 02:09
zjjhot added a commit to zjjhot/gitea that referenced this pull request Mar 29, 2024
* upstream/main:
  Remove fomantic checkbox module (go-gitea#30162)
  Refactor topic Find functions and add more tests for pagination (go-gitea#30127)
  replace jquery-minicolors with coloris (go-gitea#30055)
  Add API for `Variables` (go-gitea#29520)
  Fix `DEFAULT_SHOW_FULL_NAME=false` has no effect in commit list and commit graph page (go-gitea#30096)
  Fix migration v292 (go-gitea#30153)
  Adjust VS Code debug filename match in .gitignore (go-gitea#30158)
  Prevent re-review and dismiss review actions on closed and merged PRs (go-gitea#30065)
  Render code tags in commit messages (go-gitea#30146)
  Bump `@github/relative-time-element` to v4.4.0 (go-gitea#30154)
  Migrate font-family to tailwind (go-gitea#30118)
  Move from `max( id )` to `max( index )` for latest commit statuses (go-gitea#30076)
  Remember login for a month by default (go-gitea#30150)
@melroy89
Copy link
Contributor

melroy89 commented Jun 12, 2024

I believe I still have an issue with the new migration. It says: "Ensure every project has exactly one default column". It tries to do some SQL queriy on the project and project_board tables. Migration seems to fail with error: pq: invalid input syntax for type bigint: "true". Help?? I'm stuck on this 293 migration. My docker container keeps restating in a loop now. This happens after I updated to v1.22. On version 1.21 everything worked fine. Migration 293 is failing for me!

I'm using Docker gitea/gitea:latest-rootless image with PostgreSQL v14.

2024/06/12 13:41:30 ...ations/migrations.go:642:Migrate() [I] [SQL] SELECT tablename FROM pg_tables WHERE schemaname = $1 [public] - 371.738µs
2024/06/12 13:41:30 .../xorm@v1.3.8/sync.go:30:Sync() [I] [SQL] SELECT column_name, column_default, is_nullable, data_type, character_maximum_length, description,
    CASE WHEN p.contype = 'p' THEN true ELSE false END AS primarykey,
    CASE WHEN p.contype = 'u' THEN true ELSE false END AS uniquekey
FROM pg_attribute f
    JOIN pg_class c ON c.oid = f.attrelid JOIN pg_type t ON t.oid = f.atttypid
    LEFT JOIN pg_attrdef d ON d.adrelid = c.oid AND d.adnum = f.attnum
    LEFT JOIN pg_description de ON f.attrelid=de.objoid AND f.attnum=de.objsubid
    LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
    LEFT JOIN pg_constraint p ON p.conrelid = c.oid AND f.attnum = ANY (p.conkey)
    LEFT JOIN pg_class AS g ON p.confrelid = g.oid
    LEFT JOIN INFORMATION_SCHEMA.COLUMNS s ON s.column_name=f.attname AND c.relname=s.table_name
WHERE n.nspname= s.table_schema AND c.relkind = 'r' AND c.relname = $1 AND s.table_schema = $2 AND f.attnum > 0 ORDER BY f.attnum; [version public] - 2.886482ms
2024/06/12 13:41:30 .../xorm@v1.3.8/sync.go:30:Sync() [I] [SQL] SELECT indexname, indexdef FROM pg_indexes WHERE tablename=$1 AND schemaname=$2 [version public] - 431.557µs
2024/06/12 13:41:30 routers/common/db.go:31:InitDBEngine() [W] Table version column id db type is BIGINT, struct type is BIGSERIAL
2024/06/12 13:41:30 ...orm@v1.3.8/engine.go:1252:Get() [I] [SQL] SELECT "id", "version" FROM "version" WHERE "id"=$1 LIMIT 1 [1] - 223.978µs
2024/06/12 13:41:30 ...ations/migrations.go:688:Migrate() [I] Migration[293]: Ensure every project has exactly one default column
2024/06/12 13:41:30 routers/common/db.go:46:migrateWithSetting() [I] [SQL] BEGIN TRANSACTION [] - 53.6µs
2024/06/12 13:41:30 ...ations/v1_22/v293.go:55:CheckProjectColumnsConsistency() [I] [SQL] SELECT project.id as id, project.creator_id, project_board.id as board_id FROM "project" LEFT JOIN "project_board" ON project_board.project_id = project.id AND project_board."default"=$1 WHERE (project_board.id is NULL OR project_board.id = 0) LIMIT 50 [true] - 210.268µs
2024/06/12 13:41:30 ...ations/migrations.go:691:Migrate() [I] [SQL] ROLLBACK [] - 54.09µs
2024/06/12 13:41:30 routers/common/db.go:36:InitDBEngine() [E] ORM engine initialization attempt #8/10 failed. Error: migrate: migration[293]: Ensure every project has exactly one default column failed: pq: invalid input syntax for type bigint: "true"
2024/06/12 13:41:30 routers/common/db.go:37:InitDBEngine() [I] Backing off for 3 seconds
2024/06/12 13:41:31 ...eful/manager_unix.go:144:handleSignals() [W] PID 7. Received SIGTERM. Shutting down...

When I currently dump the project & project_board schema, it looks like this (hope this helps):

Project table:

CREATE TABLE "public"."project" (
    "id" bigint DEFAULT GENERATED BY DEFAULT AS IDENTITY NOT NULL,
    "title" text NOT NULL,
    "description" text,
    "repo_id" bigint,
    "creator_id" bigint NOT NULL,
    "is_closed" boolean,
    "board_type" bigint,
    "type" bigint,
    "created_unix" bigint,
    "updated_unix" bigint,
    "closed_date_unix" bigint,
    "card_type" integer DEFAULT '0' NOT NULL,
    "owner_id" bigint,
    CONSTRAINT "project_pkey" PRIMARY KEY ("id")
) WITH (oids = false);

Project board table:

CREATE TABLE "public"."project_board" (
    "id" bigint DEFAULT GENERATED BY DEFAULT AS IDENTITY NOT NULL,
    "title" text,
    "default" bigint DEFAULT '0' NOT NULL,
    "project_id" bigint NOT NULL,
    "creator_id" bigint NOT NULL,
    "created_unix" bigint,
    "updated_unix" bigint,
    "sorting" integer DEFAULT '0' NOT NULL,
    "color" character varying(7),
    CONSTRAINT "project_board_pkey" PRIMARY KEY ("id")
) WITH (oids = false);

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/go Pull requests that update Go code modifies/migrations size/L Denotes a PR that changes 100-499 lines, ignoring generated files. skip-changelog This PR is irrelevant for the (next) changelog, for example bug fixes for unreleased features. type/bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants