Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PostgreSQL Error: Incompatible columnType for auto-incrementing column ("id") #4599

Open
shobhitsinghal624 opened this issue Feb 26, 2019 · 14 comments
Labels
helpful info or workaround orm Related to models, datastores, orm config, Waterline, sails-hook-orm, etc.

Comments

@shobhitsinghal624
Copy link

Sails version: 1.1.0
Node version: v10.12.0
NPM version: 6.8.0
DB adapter name: N/A
DB adapter version: N/A
Operating system: macOS


I'm getting this error when I try a many-to-many association with id as string type is both the models. No errors when at least one model has an id of type number.
Error:

Error: Incompatible `columnType` for auto-incrementing column ("id").  Expecting `columnType` to be left undefined, or to be set explicitly to SERIAL, BIGSERIAL, or SMALLSERIAL.  But instead got a different numeric PostgreSQL column type, "integer", which unfortunately does not support auto-increment.  To resolve this, please remove this explicit `columnType`, or set it to an auto-increment-compatible PostgreSQL column type.

Steps to reproduce the issue provided below.

Create models with many-to-many association:

// api/models/User.js
module.exports = {
  attributes: {
    id: {
      type: 'string',
      required: true,
      unique: true,
    },
    name: {
      type: 'string',
      required: true,
    },
    teams: {
      collection: 'Team',
      via: 'members',
    },
  },
};
// api/models/Team.js
module.exports = {
  attributes: {
    id: {
      type: 'string',
      required: true,
      unique: true,
    },
    name: {
      type: 'string',
      required: true,
    },
    members: {
      collection: 'User',
      via: 'teams',
    },
  },
};

Use PostgreSQL in config/datastores.js:

// config/datastores.js
module.exports.datastores = {
  default: {
    adapter: 'sails-postgresql',
    url: 'postgresql://tempuser:temppassword@127.0.0.1:5432/tempdb',
  },
};

Make sure default attributes are unchanged in config/models.js:

// config/models.js
module.exports.models = {
  schema: true,
  migrate: 'safe',
  attributes: {
    createdAt: { type: 'number', autoCreatedAt: true, },
    updatedAt: { type: 'number', autoUpdatedAt: true, },
    id: { type: 'number', autoIncrement: true, },
  },
  dataEncryptionKeys: {
    default: 'YpzV6pxGlNVrwYjTu8B106/WV08HmxbY/WpahQh95sM='
  },
  cascadeOnDestroy: true
};

Run npm i sails-postgresql --save to install sails-postgresql
Run sails lift --drop to get the error.

Full error log:

$ sails lift --drop

 info: Starting app...

 info: ·• Auto-migrating...  (drop)
error: A hook (`orm`) failed to load!
error:
error: Error: Incompatible `columnType` for auto-incrementing column ("id").  Expecting `columnType` to be left undefined, or to be set explicitly to SERIAL, BIGSERIAL, or SMALLSERIAL.  But instead got a different numeric PostgreSQL column type, "integer", which unfortunately does not support auto-increment.  To resolve this, please remove this explicit `columnType`, or set it to an auto-increment-compatible PostgreSQL column type.
    at map (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/sails-postgresql/helpers/private/schema/build-schema.js:76:13)
    at /Users/shobhit.singhal/workspace/sailsbugreport/node_modules/@sailshq/lodash/lib/index.js:2453:27
    at /Users/shobhit.singhal/workspace/sailsbugreport/node_modules/@sailshq/lodash/lib/index.js:3253:15
    at baseForOwn (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/@sailshq/lodash/lib/index.js:2223:14)
    at /Users/shobhit.singhal/workspace/sailsbugreport/node_modules/@sailshq/lodash/lib/index.js:3223:18
    at baseMap (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/@sailshq/lodash/lib/index.js:2452:7)
    at Function.map (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/@sailshq/lodash/lib/index.js:6911:14)
    at Object.buildSchema (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/sails-postgresql/helpers/private/schema/build-schema.js:23:19)
    at afterNamespaceCreation (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/sails-postgresql/helpers/define.js:156:35)
    at createSchemaNamespace (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/sails-postgresql/helpers/define.js:113:18)
    at spawnConnectionCb (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/sails-postgresql/helpers/define.js:127:9)
    at Object.success (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/sails-postgresql/helpers/private/connection/spawn-connection.js:116:14)
    at /Users/shobhit.singhal/workspace/sailsbugreport/node_modules/machine/lib/private/help-build-machine.js:1517:30
    at proceedToFinalAfterExecLC (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/parley/lib/private/Deferred.js:1181:16)
    at proceedToInterceptsAndChecks (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/parley/lib/private/Deferred.js:909:12)
    at proceedToAfterExecSpinlocks (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/parley/lib/private/Deferred.js:841:10)
    at /Users/shobhit.singhal/workspace/sailsbugreport/node_modules/parley/lib/private/Deferred.js:303:7
    at /Users/shobhit.singhal/workspace/sailsbugreport/node_modules/machine/lib/private/help-build-machine.js:954:24
    at Function.handlerCbs.success (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/machine/lib/private/help-build-machine.js:814:26)
    at cb (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/machinepack-postgresql/machines/get-connection.js:87:20)
    at BoundPool.<anonymous> (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/pg-pool/index.js:137:9)
    at Pool.dispense [as _dispense] (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/generic-pool/lib/generic-pool.js:310:14)
    at Pool.acquire (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/generic-pool/lib/generic-pool.js:391:8)
    at BoundPool.<anonymous> (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/pg-pool/index.js:111:15)
    at BoundPool.Pool._promiseNoCallback (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/pg-pool/index.js:75:7)
    at BoundPool.Pool.connect (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/pg-pool/index.js:109:15)
    at Object.getConnection (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/machinepack-postgresql/machines/get-connection.js:74:25)
    at wrapper (/Users/shobhit.singhal/workspace/sailsbugreport/node_modules/@sailshq/lodash/lib/index.js:3275:19)

error: Could not load Sails app.
error:
error: Tips:
error:  • First, take a look at the error message above.
error:  • Make sure you've installed dependencies with `npm install`.
error:  • Check that this app was built for a compatible version of Sails.
error:  • Have a question or need help?  (http://sailsjs.com/support)
@sailsbot
Copy link

@shobhitsinghal624 Thanks for posting, we'll take a look as soon as possible.


For help with questions about Sails, click here. If you’re interested in hiring @sailsbot and her minions in Austin, click here.

@da-snap
Copy link

da-snap commented Mar 4, 2019

I had the same issue. As a workaround it helped to downgrade sails-postgresql to version 1.0.0

@johnabrams7 johnabrams7 added orm Related to models, datastores, orm config, Waterline, sails-hook-orm, etc. bug labels Mar 4, 2019
@nahanil
Copy link

nahanil commented Mar 15, 2019

@shobhitsinghal624 Not sure if explicitly adding the join table will help, but it looks like that where it might be getting stuck generating the schema stuff. https://sailsjs.com/documentation/concepts/models-and-orm/associations/through-associations

Basically, create a new model (I tried with api/models/Through.js)

module.exports = {
  attributes: {
    members:{
      model:'user'
    },
    teams: {
      model: 'team'
    }
  }
}

Then make sure the existing models know to use this join table, ie in User.js

    teams: {
      collection: 'Team',
      via: 'members',
      through: 'through' // or whatever you call it
    },

Then also in Team.js

Then I create()-ed a couple of users and a team and can query it as expected..

Team.find({}).populate('members').then(r => console.log(JSON.stringify(r, null, 2)))
 [
  {
    "members": [
      {
        "createdAt": 1552678830790,
        "updatedAt": 1552678830790,
        "id": "a",
        "name": "bob"
      },
      {
        "createdAt": 1552678839204,
        "updatedAt": 1552678839204,
        "id": "b",
        "name": "tom"
      }
    ],
    "createdAt": 1552678892003,
    "updatedAt": 1552678892003,
    "id": "NYC",
    "name": "New York Nuggins"
  }
]

🐛 FWIW adding the "through model", but not linking it to the other two models with the through attribute squelches the errors. Waterline can somehow create both the through model's table, and the default team_members__user_teams (which is where the join rows end up and through stays empty). 🐛

Edit: I tried this with Sails 1.1.0 / Node v8.15.0 but I experienced your described errors with sails lift --drop. Not sure if Sails may still have other issues running with v10.

@raqem
Copy link
Contributor

raqem commented Mar 18, 2019

@texh,
You experienced this issue using Node V8.15.0?
@shobhitsinghal624, @da-snap,
As a side note I did want to confirm that Sails V1 and Node V10 dont seem to play nice all the time. I would recommend downgrading your Node version to V8.

@nahanil
Copy link

nahanil commented Mar 19, 2019

@raqem Starting from sails new and with node v8.15.0 I added the changes mentioned in the original post here and found the same issue mentioned.

I suppose I was hinting that the changes I made worked in my environment, but other weird things may happen if OP is running v10

@shobhitsinghal624
Copy link
Author

@raqem Tried again with node v8, still getting the error.

I think its a bug in sails-postgresql and specifically in this diff - Link. Though I am not able to pinpoint what the issue is. Maybe you can figure it out.

@da-snap
Copy link

da-snap commented Mar 19, 2019

@raqem I also got the error with Node v8.15.0.
The error was introduced in sails-postgresql v1.0.1.
I think it is depending on this feature.

@nahanil
Copy link

nahanil commented Mar 19, 2019

Error: Incompatible columnType for auto-incrementing column ("id"). Expecting columnType to be left undefined, or to be set explicitly to SERIAL, BIGSERIAL, or SMALLSERIAL. But instead got a different numeric PostgreSQL column type, "integer", which unfortunately does not support auto-increment. To resolve this, please remove this explicit columnType, or set it to an auto-increment-compatible PostgreSQL column type.

I think this could be an issue with whatever builds the "join table" for the many-to-many relationship & happens before helpers/private/schema/build-schema.js.

If you take a look at the data for the user-defined models that's being fed into buildSchema, the attribute's 'columnType's are things like _string and _number which get picked up an turned into actual postgres types around here, but the columnType of the join tables ID is 'integer', which gets left as integer and can't be auto-incremented (without adding DEFAULT nextval('tablename_colname_seq') NOT NULL) so buildSchema croaks.

Adding something like the following around here gets the app lifted & tables created, but idk if it's The Right Way™. I think integer should probably have been translated to '_numberkey' before it gets this far 🤷‍♂️

case 'integer':          computedColumnType = (attribute.autoIncrement ? 'SERIAL' : 'INTEGER'); break;

@rachaelshaw
Copy link
Member

@shobhitsinghal624 would have to dig into the code a bit more to confirm this, but I suspect that adding a custom definition for one of the attributes in config/models.js doesn't completely replace the default definition: it probably just overrides the values you explicitly include. Have you tried adding autoIncrement: false to your id definitions to see if that makes a difference?

(btw, any particular reason why you're overriding the id attribute just for those models, rather than using a consistent id type across the whole app?)

@shobhitsinghal624
Copy link
Author

@rachaelshaw just tried with autoIncrement: false and still getting the same error. Also tried commenting out id: { type: 'number', autoIncrement: true, } from config/models.js, but got the error.

I'm unable to use consistent id across the whole app because of issue #4591

@rachaelshaw
Copy link
Member

@shobhitsinghal624 just realized the error is coming from the join model, which is using the default settings for id. The workaround here would be to manually create the join model (e.g. TeamMembership.

Also, talked to @mikermcneil about this one, and he suggested that, instead of overriding the ID settings, it would be a good idea to use a different (custom) attribute for the via (instead of id).

Here's a similar issue that you might find helpful: #4542

@shobhitsinghal624
Copy link
Author

@rachaelshaw: Found another workaround for this issue. First, the underlying problems that lead to this issue:

  1. sails-postresql adapter needs any autoIncrement attribute to have a columnType of SERIAL or BIGSERIAL. This is checked here.
  2. sails-postresql assigns columnType based on model definitions here. But, there is no definition for attribute.columnType = 'integer', which gets passed through as is.
  3. waterline-schema assigns a default id attribute to join tables, in which the columnType is set as integer, and autoIncrement set as true. This happens here
  4. Finally, (3) and (2) above cause the check in (1) to fail.

Workaround is to set default id attributes in config.models as per below:

// config/models.js
module.exports.models = {
  schema: true,
  migrate: 'safe',
  attributes: {
    createdAt: { type: 'number', autoCreatedAt: true, },
    updatedAt: { type: 'number', autoUpdatedAt: true, },
    id: {
      type: 'number',
      autoIncrement: true,
      autoMigrations: {
        autoIncrement: true,
        columnType: '_number',
        unique: true,
      },
    },
  },
  dataEncryptionKeys: {
    default: 'YpzV6pxGlNVrwYjTu8B106/WV08HmxbY/WpahQh95sM='
  },
  cascadeOnDestroy: true
};

This leads to waterline-schema setting columnType as _number here after which sails-postgresql correctly sets computedColumnType as SERIAL here

@raqem
Copy link
Contributor

raqem commented May 21, 2019

@shobhitsinghal624 thank alot for revisiting this issue and providing your work around. Hopefully this can help others in the future.

@Enteee
Copy link

Enteee commented Jun 11, 2019

thank you @shobhitsinghal624. I am seeing the exact same issue and your workaround helped. In my app I heavily rely on uuids for id's. Since I am often mixing instances of different model types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
helpful info or workaround orm Related to models, datastores, orm config, Waterline, sails-hook-orm, etc.
Development

No branches or pull requests

9 participants