Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres: Modifying enum fails migration if the enum is used in an array column #7217

Closed
2 of 21 tasks
dapetcu21 opened this issue Dec 22, 2020 · 4 comments · Fixed by #7419
Closed
2 of 21 tasks

Postgres: Modifying enum fails migration if the enum is used in an array column #7217

dapetcu21 opened this issue Dec 22, 2020 · 4 comments · Fixed by #7419

Comments

@dapetcu21
Copy link

dapetcu21 commented Dec 22, 2020

Issue Description

If an enum is used as the array item type in a column with array: true, then adding new enum values to said enum causes a crash when migrating. See code sample below.

Expected Behavior

The migration should complete without a hitch.

Actual Behavior

This crash:

query: SELECT * FROM current_schema()
query: SELECT "udt_schema", "udt_name" FROM "information_schema"."columns" WHERE "table_schema" = 'public' AND "table_name" = 'user' AND "column_name"='roles'
query: ALTER TYPE "public"."_user_roles_enum" RENAME TO "user_roles_enum_old"
query failed: ALTER TYPE "public"."_user_roles_enum" RENAME TO "user_roles_enum_old"
error: error: cannot alter array type user_roles_enum[]
    at Parser.parseErrorMessage (/Users/da_petcu21/Work/rethink-backend/node_modules/pg-protocol/src/parser.ts:357:11)
    at Parser.handlePacket (/Users/da_petcu21/Work/rethink-backend/node_modules/pg-protocol/src/parser.ts:186:21)
    at Parser.parse (/Users/da_petcu21/Work/rethink-backend/node_modules/pg-protocol/src/parser.ts:101:30)
    at Socket.<anonymous> (/Users/da_petcu21/Work/rethink-backend/node_modules/pg-protocol/src/index.ts:7:48)
    at Socket.emit (events.js:315:20)
    at Socket.EventEmitter.emit (domain.js:486:12)
    at addChunk (_stream_readable.js:309:12)
    at readableAddChunk (_stream_readable.js:284:9)
    at Socket.Readable.push (_stream_readable.js:223:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
  length: 177,
  severity: 'ERROR',
  code: '42809',
  detail: undefined,
  hint: 'You can alter type user_roles_enum, which will alter the array type as well.',
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'typecmds.c',
  line: '3305',
  routine: 'RenameType'
}
query: ROLLBACK
[1608669311589] ERROR    (QueryFailedError/39871 on salem.local): cannot alter array type user_roles_enum[]
    QueryFailedError: cannot alter array type user_roles_enum[]
        at new QueryFailedError (/Users/da_petcu21/Work/rethink-backend/src/error/QueryFailedError.ts:9:9)
        at Query.callback (/Users/da_petcu21/Work/rethink-backend/src/driver/postgres/PostgresQueryRunner.ts:220:30)
        at Query.handleError (/Users/da_petcu21/Work/rethink-backend/node_modules/pg/lib/query.js:128:19)
        at Client._handleErrorMessage (/Users/da_petcu21/Work/rethink-backend/node_modules/pg/lib/client.js:335:17)
        at Connection.emit (events.js:315:20)
        at Connection.EventEmitter.emit (domain.js:486:12)
        at /Users/da_petcu21/Work/rethink-backend/node_modules/pg/lib/connection.js:115:12
        at Parser.parse (/Users/da_petcu21/Work/rethink-backend/node_modules/pg-protocol/src/parser.ts:102:9)
        at Socket.<anonymous> (/Users/da_petcu21/Work/rethink-backend/node_modules/pg-protocol/src/index.ts:7:48)
        at Socket.emit (events.js:315:20)

Steps to Reproduce

  1. Create the following entity.
  2. Run the server once.
  3. Add a new enum value to UserRole.
  4. Run the server again and watch the crash happen.
export enum UserRole {
  PLAYER = 'PLAYER',
  FULL_GAME = 'FULL_GAME',
  SUPERVISOR = 'SUPERVISOR',
  REPORTS = 'REPORTS',
  ADMIN = 'ADMIN',
}

@Entity()
export class User {
  @PrimaryGeneratedColumn('uuid')
  id: string;

  @Column({
    type: 'enum',
    enum: UserRole,
    array: true,
    default: [UserRole.PLAYER],
  })
  roles: UserRole[];
}

My Environment

Dependency Version
Operating System macOS 11.0.1
Node.js version v14.15.1
Typescript version v4.1.3
TypeORM version v0.2.29

Additional Context

Relevant Database Driver(s)

  • aurora-data-api
  • aurora-data-api-pg
  • better-sqlite3
  • cockroachdb
  • cordova
  • expo
  • mongodb
  • mysql
  • nativescript
  • oracle
  • postgres
  • react-native
  • sap
  • sqlite
  • sqlite-abstract
  • sqljs
  • sqlserver

Are you willing to resolve this issue by submitting a Pull Request?

  • Yes, I have the time, and I know how to start.
  • Yes, I have the time, but I don't know how to start. I would need guidance.
  • No, I don't have the time, although I believe I could do it if I had the time...
  • No, I don't have the time and I wouldn't even know how to start.
@karousn
Copy link

karousn commented Jan 12, 2021

any workaround for this issue ?

@pmsfc
Copy link

pmsfc commented Jan 21, 2021

This is a bug from Typeorm, it's adding a _ on the rename query (happens on up and down migration).
If you look closely "public"."_user_roles_enum" should be "public"."user_roles_enum"

query failed: ALTER TYPE "public"."_user_roles_enum" RENAME TO "user_roles_enum_old"

Just remove it and you should be fine.

The problem occurs on

const result = await this.query(`SELECT "udt_schema", "udt_name" ` +

    protected async getEnumTypeName(table: Table, column: TableColumn) {
        const currentSchemaQuery = await this.query(`SELECT * FROM current_schema()`);
        const currentSchema = currentSchemaQuery[0]["current_schema"];
        let [schema, name] = table.name.split(".");
        if (!name) {
            name = schema;
            schema = this.driver.options.schema || currentSchema;
        }
        const result = await this.query(`SELECT "udt_schema", "udt_name" ` +
            `FROM "information_schema"."columns" WHERE "table_schema" = '${schema}' AND "table_name" = '${name}' AND "column_name"='${column.name}'`);
        return {
            enumTypeSchema: result[0]["udt_schema"],
            enumTypeName: result[0]["udt_name"]
        };
    }

Because udt_name starts with an underscore when the enum is an array on that table.
I can submit a PR since most of the investigation is done.

@geric
Copy link

geric commented Feb 14, 2021

I also encountered it on typeorm 0.2.29. I followed @pmsfc advice and it works perfectly. The cause was indeed the appended "_".

@niederschlag
Copy link

niederschlag commented Feb 20, 2021

There's a PR for this issue: #6126

It's ready for merge but stuck for a couple of months though

AlexMesser added a commit that referenced this issue Feb 25, 2021
AlexMesser added a commit that referenced this issue Mar 5, 2021
* fix #5371

* fix #6471;
fix: `enumName` changes not handled;
fix: `enumName` does not handle table schema;

* fixed falling test;

* added test for #7217

* fix #6047, #7283;

* fix #5871

* added support for `enumName` in `joinColumns` (#5729)

* fix #5478

* fixed falling test;
updated `postgres-enum` test;

* added column `array` property change detection (#5882);
updated `postgres-enum` test;

* fix #5275

* added validation for `enum` property (#2233)

* fix #5648

* improved missing "enum" or "enumName" properties validation;

* fix #4897, #6376

* lint fix;

* fixed falling tests;

* fixed falling tests;

* removed .only

* fix #6115
gogotaro added a commit to flowaccount/typeorm that referenced this issue Mar 17, 2021
* docs: fix small typo on package.json script example (typeorm#7408)

Add missing colon in JSON property at `package.json` `"script"` example

* feat: output Javascript Migrations instead of TypeScript (typeorm#7294)

* docs / test: Added tests and documentation for Feature 7253 - Migrations Javascript output

* Change in the test

* test: Re-arranged the tests to move them to the core tests directory

* tests: Adjusted Tests a bit

* tests - renamed tests to follow the other functional tests naming

* tests - renamed tests to follow the other functional tests naming

* tests - Fixed issues with the test connections setup

* tests - Removed unnecesary restore

* fix: improve EntityManager.save() return type (typeorm#7391)

This brings it in line with the equivalent method in Repository.

* fix: resolve issue building tree entities with embeded primary column (typeorm#7416)

Closes: typeorm#7415

* Adjust mongodb driver options & connect driver to support replica set (typeorm#7402)

- Dupplicate buildDriverOptions for mongodb especially
- Add hostReplicaSet to MongoConnectionOptions properties for collect host replica list
- Adjust buildConnectionUrl to build replica set connection url

* fix: performance issues of `RelationId`. (typeorm#7318)

* test: relationId is too slow

* perf: RelationId is too slow

When we join a lot of relations we can get 1k+ records from db even it is only 10 entities. Then when relationId are loaded the query contains too many duplciates for the same ids. The solution is to check that the relationId query fetches only unique values.

Closes: typeorm#5691

* fix: Array type default value should not generate SQL commands without change (typeorm#7409)

* fix(1532) Array type default value should not generate SQL commands without change

* Update PostgresDriver.ts

* removed `arrayCast` from `normalizeDefault` since casting for default value is already removed in `PostgresQueryRunner.loadTables()` method;
* removed support for function definition in `default` because function syntax suppose to support raw sql, we don't have to confuse things by applying custom modifications.

* Update User.ts

removed incorrect `default` definition with functions

Co-authored-by: AlexMesser <dmzt08@gmail.com>

* feat: add check and dryrun to migration generate (typeorm#7275)

Adds support for “check” and “drynrun” modes to the migration generate command.

Fixes typeorm#3037
Refs typeorm#6978

* chore: typescript version upgrade (typeorm#7422)

* chore: dependencies update (typeorm#7424)

* typescript version upgrade

* fixing linting

* fixing mongo query runner issues

* fixing linting

* updated all dependencies

* fixes typeorm#7418

* fixes typeorm#7418

* adding missing ILike operator docs (took from next branch)

* fix: mongodb connectionurl parse options (#1)

* fix: mongodb connectionurl parse options

- Loop every options in mongodb connection url and turn it as object to merge with connection url object before return of method "parseMongoDBConnectionUrl"
- unit test of mongodb replicaset parse connectionurl of typeorm#7401
- unit test of mongodb options parse connectionurl of typeorm#7437

* fix: add semicolon by lint suggestion

/home/circleci/typeorm/src/driver/DriverUtils.ts
  192:39  error  Missing semicolon  @typescript-eslint/semi

* chore: @beamdev package scope (#2)

* chore: update master (typeorm#3)

* fix: fixed all known enum issues (typeorm#7419)

* fix typeorm#5371

* fix typeorm#6471;
fix: `enumName` changes not handled;
fix: `enumName` does not handle table schema;

* fixed falling test;

* added test for typeorm#7217

* fix typeorm#6047, typeorm#7283;

* fix typeorm#5871

* added support for `enumName` in `joinColumns` (typeorm#5729)

* fix typeorm#5478

* fixed falling test;
updated `postgres-enum` test;

* added column `array` property change detection (typeorm#5882);
updated `postgres-enum` test;

* fix typeorm#5275

* added validation for `enum` property (typeorm#2233)

* fix typeorm#5648

* improved missing "enum" or "enumName" properties validation;

* fix typeorm#4897, typeorm#6376

* lint fix;

* fixed falling tests;

* fixed falling tests;

* removed .only

* fix typeorm#6115

* refactor: improve README.md and DEVLOPER.md code examples formatting (typeorm#7436)

* fix: correctly get referenceColumn value in `getEntityValueMap` (typeorm#7005)

* test: add test case (typeorm#7002)

* fix: correctly get referenceColumn value in `getEntityValueMap`

* test: reproduction for issue typeorm#3246 (typeorm#3247)

* Add reproduction for issue 3246

* Update test/github-issues/3246/issue-3246.ts

Co-authored-by: Json Choi <1890mah@gmail.com>

Co-authored-by: Dan Imbrogno <dan.imbrogno@gmail.com>
Co-authored-by: AlexMesser <dmzt08@gmail.com>
Co-authored-by: Json Choi <1890mah@gmail.com>

* code refactoring in test;

* added test for typeorm#2758

* feat: allow to pass the given table name as string in RelationDecorators (typeorm#7448)

* feat(RelationDecorators): allow to pass the given table name as string

* Update EntityMetadataBuilder.ts

added parentheses;

Co-authored-by: Emily Marigold Klassen <forivall@users.noreply.github.com>

* feat: add option for installing package using CLI (typeorm#6889)

* init cli: add options for installing package

* yarg choice, add await, revert formatter changes

* init flag - set default to npm

Co-authored-by: AlexMesser <dmzt08@gmail.com>
Co-authored-by: Henry Boisdequin <boisdequinhenry19@gmail.com>
Co-authored-by: Json Choi <1890mah@gmail.com>
Co-authored-by: Dan Imbrogno <41128441+danimbrogno-pml@users.noreply.github.com>
Co-authored-by: Dan Imbrogno <dan.imbrogno@gmail.com>
Co-authored-by: Emily Marigold Klassen <forivall@gmail.com>
Co-authored-by: Emily Marigold Klassen <forivall@users.noreply.github.com>
Co-authored-by: Gaurav Sharma <gtpan77@gmail.com>

* chore: update master typeorm/typeorm (typeorm#5)

* fix: fixed all known enum issues (typeorm#7419)

* fix typeorm#5371

* fix typeorm#6471;
fix: `enumName` changes not handled;
fix: `enumName` does not handle table schema;

* fixed falling test;

* added test for typeorm#7217

* fix typeorm#6047, typeorm#7283;

* fix typeorm#5871

* added support for `enumName` in `joinColumns` (typeorm#5729)

* fix typeorm#5478

* fixed falling test;
updated `postgres-enum` test;

* added column `array` property change detection (typeorm#5882);
updated `postgres-enum` test;

* fix typeorm#5275

* added validation for `enum` property (typeorm#2233)

* fix typeorm#5648

* improved missing "enum" or "enumName" properties validation;

* fix typeorm#4897, typeorm#6376

* lint fix;

* fixed falling tests;

* fixed falling tests;

* removed .only

* fix typeorm#6115

* refactor: improve README.md and DEVLOPER.md code examples formatting (typeorm#7436)

* fix: correctly get referenceColumn value in `getEntityValueMap` (typeorm#7005)

* test: add test case (typeorm#7002)

* fix: correctly get referenceColumn value in `getEntityValueMap`

* test: reproduction for issue typeorm#3246 (typeorm#3247)

* Add reproduction for issue 3246

* Update test/github-issues/3246/issue-3246.ts

Co-authored-by: Json Choi <1890mah@gmail.com>

Co-authored-by: Dan Imbrogno <dan.imbrogno@gmail.com>
Co-authored-by: AlexMesser <dmzt08@gmail.com>
Co-authored-by: Json Choi <1890mah@gmail.com>

* code refactoring in test;

* added test for typeorm#2758

* feat: allow to pass the given table name as string in RelationDecorators (typeorm#7448)

* feat(RelationDecorators): allow to pass the given table name as string

* Update EntityMetadataBuilder.ts

added parentheses;

Co-authored-by: Emily Marigold Klassen <forivall@users.noreply.github.com>

* feat: add option for installing package using CLI (typeorm#6889)

* init cli: add options for installing package

* yarg choice, add await, revert formatter changes

* init flag - set default to npm

Co-authored-by: AlexMesser <dmzt08@gmail.com>
Co-authored-by: Henry Boisdequin <boisdequinhenry19@gmail.com>
Co-authored-by: Json Choi <1890mah@gmail.com>
Co-authored-by: Dan Imbrogno <41128441+danimbrogno-pml@users.noreply.github.com>
Co-authored-by: Dan Imbrogno <dan.imbrogno@gmail.com>
Co-authored-by: Emily Marigold Klassen <forivall@gmail.com>
Co-authored-by: Emily Marigold Klassen <forivall@users.noreply.github.com>
Co-authored-by: Gaurav Sharma <gtpan77@gmail.com>

Co-authored-by: rccursach <rccursach@gmail.com>
Co-authored-by: Jorge Luis Vargas <Jorge.Vargas@albelli.com>
Co-authored-by: Anthony Rosequist <arosequist@users.noreply.github.com>
Co-authored-by: Tomas Zaluckij <mrtomaszal@gmail.com>
Co-authored-by: MG <m@sudo.eu>
Co-authored-by: Ed Mitchell <edeesis@gmail.com>
Co-authored-by: AlexMesser <dmzt08@gmail.com>
Co-authored-by: Christian Holm <cho@cubitech.dk>
Co-authored-by: Umed Khudoiberdiev <pleerock.me@gmail.com>
Co-authored-by: Henry Boisdequin <boisdequinhenry19@gmail.com>
Co-authored-by: Json Choi <1890mah@gmail.com>
Co-authored-by: Dan Imbrogno <41128441+danimbrogno-pml@users.noreply.github.com>
Co-authored-by: Dan Imbrogno <dan.imbrogno@gmail.com>
Co-authored-by: Emily Marigold Klassen <forivall@gmail.com>
Co-authored-by: Emily Marigold Klassen <forivall@users.noreply.github.com>
Co-authored-by: Gaurav Sharma <gtpan77@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants