Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep getting unique validation error: How to allow duplicate entries in N:M (belongsToMany)? #3220

Open
maplesap opened this issue Feb 25, 2015 · 32 comments

Comments

@maplesap
Copy link

@maplesap maplesap commented Feb 25, 2015

The unique validation check is preventing me from creating duplicate entries in my join table:

// models/user.js
User.belongsToMany(models.item, { through: models.history, as: "Owners', 
  foreignKey: { 
    name: "itemId",
    allowNull: true,
    unique: false
  }
});

// models/item.js
Item.belongsToMany(models.user, { through: models.history, as: "Owned", 
  foreignKey: {
    unique: false
  }
});

// models/history.js
id: {
  type: DataTypes.INTEGER,
  primaryKey: true,
  autoIncrement: true
},
other: DataTypes.STRING

When a user buys the same item again, resulting in duplicate combinations of itemId and userId in history table, sequelize will throw validation errors:

{ name: 'SequelizeUniqueConstraintError',
  message: 'Validation error',
....
message: "userId" must be unique
type: 'unique violation'
....
message: "itemId" must be unique
type: 'unique violation'

But I have set unique: false on the foreignkeys, and have created a primaryKey for history table, so this validation error confused me. What's the proper way to allow duplicate entries in N:M?

@mickhansen
Copy link
Contributor

@mickhansen mickhansen commented Feb 25, 2015

unique entries in N:M is currently not supported. You can use unique: false for the options on the N:M relation but it only gets you halfway there (the setters will not work ideally so you manually have to create entries).

@maplesap
Copy link
Author

@maplesap maplesap commented Feb 25, 2015

Can you explain further with what you mean "so you manually have to create entries". Right now to create the history, I just do:

models.History.create({userId: user_id, itemId: item_id});

Which triggers the unique validation errors if a duplicate of userId and itemId already exists.

Do you mean I should remove my belongsToMany relations on user and item models, and I should just manually create the join table like:

// models/history.js
id: {
  type: DataTypes.INTEGER,
  primaryKey: true,
  autoIncrement: true
},
userId: DataTypes.INTEGER,
itemId: DataTypes.INTEGER,
other: DataTypes.STRING
@mickhansen
Copy link
Contributor

@mickhansen mickhansen commented Feb 25, 2015

Ah well that's great, that's what you have to do.
You might need to tweak the belongsToMany call to accept uniques:

Item.belongsToMany(models.user, { through: models.history, as: "Owned", unique: false});

(the unique: false prop is on the relation, not on the foreign key option)

@maplesap
Copy link
Author

@maplesap maplesap commented Feb 25, 2015

Hm, I'm still getting the unique validation errors for both itemId and userId:

User.belongsToMany(models.item, { through: models.history, as: "Owners', unique: false,
  foreignKey: { 
    allowNull: true  }
});

Item.belongsToMany(models.user, { through: models.history, as: "Owned", unique: false  });
@mickhansen
Copy link
Contributor

@mickhansen mickhansen commented Feb 25, 2015

@maplesap what does the create table sql look like?

@maplesap
Copy link
Author

@maplesap maplesap commented Feb 25, 2015

This is the output:

Executing (default): INSERT INTO "History" ("id","other","userId","itemId","updatedAt","createdAt") VALUES (DEFAULT,"blah",1,2,'2015-02-25 17:12:20.221 +00:00','2015-02-25 17:12:20.221 +00:00') RETURNING *;
{ name: 'SequelizeUniqueConstraintError',
  message: 'Validation error',
  errors: 
   [ { message: '"itemId" must be unique',
       type: 'unique violation',
       path: '"itemId"',
       value: '2' },
     { message: '"userId" must be unique',
       type: 'unique violation',
       path: '"userId"',
       value: '1' } ],
  fields: { '"itemId"': '2', '"userId"': '1' } }
@mickhansen
Copy link
Contributor

@mickhansen mickhansen commented Feb 25, 2015

@maplesap that's not the create table statement though :)
I assume you are using sync({force: true}), otherwise the constraints in your database won't change.

@maplesap
Copy link
Author

@maplesap maplesap commented Feb 25, 2015

Okay! I created a small version with the problem. Here is the entire code:

app.js

var express = require('express');
var app = express();
var http = require('http').Server(app);

var bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(express.static(__dirname + '/public'));

app.post("/test", function(req, res) {

    models.history.create({other: "blah", itemId: 1, userId: 1}).catch(function(error) {
        console.log(error);
    });

});

var models = require('./models')
models.sequelize.sync({logging: console.log}).then(function() {
    http.listen(3000, function() {});
});

models/user.js

"use strict";

module.exports = function(sequelize, DataTypes) {
  var User = sequelize.define("user", {
    name: DataTypes.STRING
  }, {
    timestamps: false,
    classMethods: {
        associate: function(models) {
            User.belongsToMany(models.item, { through: models.history, as: 'Owners', unique: false, foreignKey: { name: 'userId', allowNull: true }});
        }
    }

  });

  return User;
};

models/item.js

"use strict";

module.exports = function(sequelize, DataTypes) {
    var Item = sequelize.define('item', {
        name: DataTypes.STRING
    }, {
        timestamps: false,
        classMethods: {
            associate: function(models) {
                Item.belongsToMany(models.user, { through: models.history, as: 'Owned', unique: false });
            }
        }
    });

    return Item;
};

models/history.js

"use strict";

module.exports = function(sequelize, DataTypes) {
    var History = sequelize.define("history", {
        id: {
            type: DataTypes.INTEGER,
            primaryKey: true,
            autoIncrement: true,
        },
        other: DataTypes.STRING
    });

    return History;
};

index.html

<html>
    <body>
        <script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
    </body>
</html>

<script>
$.post("/test");
</script>

Here is the output when I first run the app:

Executing (default): CREATE TABLE IF NOT EXISTS "items" ("id"   SERIAL , "name" VARCHAR(255), PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND a.attnum = ANY(ix.indkey) AND t.relkind = 'r' and t.relname = 'items' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Executing (default): CREATE TABLE IF NOT EXISTS "users" ("id"   SERIAL , "name" VARCHAR(255), PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND a.attnum = ANY(ix.indkey) AND t.relkind = 'r' and t.relname = 'users' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Executing (default): CREATE TABLE IF NOT EXISTS "histories" ("id"  SERIAL , "other" VARCHAR(255), "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "updatedAt" TIMESTAMP WITH TIME ZONE NOT NULL, "itemId" INTEGER REFERENCES "items" ("id") ON DELETE CASCADE ON UPDATE CASCADE, "userId" INTEGER REFERENCES "users" ("id") ON DELETE CASCADE ON UPDATE CASCADE, UNIQUE ("itemId", "userId"), PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND a.attnum = ANY(ix.indkey) AND t.relkind = 'r' and t.relname = 'histories' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;

When I insert a duplicate combination of userId and itemId, the output is:

Executing (default): INSERT INTO "histories" ("id","other","itemId","userId","updatedAt","createdAt") VALUES (DEFAULT,'blah',1,1,'2015-02-25 21:28:09.957 +00:00','2015-02-25 21:28:09.957 +00:00') RETURNING *;
{ name: 'SequelizeUniqueConstraintError',
  message: 'Validation error',
  errors: 
   [ { message: '"itemId" must be unique',
       type: 'unique violation',
       path: '"itemId"',
       value: '1' },
     { message: '"userId" must be unique',
       type: 'unique violation',
       path: '"userId"',
       value: '1' } ],
  fields: { '"itemId"': '1', '"userId"': '1' } }
@maplesap
Copy link
Author

@maplesap maplesap commented Feb 28, 2015

For the moment I kind of cheated and created belongsTo for history:

history
belongsTo(models.user)
belongsTo(models.item)

Which won't cause the unique validation error to occur and I can set my ids.

It seems like setting unique: false is getting ignored for belongsToMany? This line:
https://github.com/sequelize/sequelize/blob/master/lib/dialects/postgres/query-generator.js#L69

Is returning true and uniqueKeys always gives fields: [ 'itemId', 'userId' ]. So somewhere before this must be skipping the unique check.

@andycbriggs
Copy link

@andycbriggs andycbriggs commented Mar 16, 2015

I'm seeing this same issue and have created this short gist where I was able to replicate the problem using some examples from the docs.

https://gist.github.com/kaidoe/a8a46fc56e09f67ea956

@chrisjhoughton
Copy link

@chrisjhoughton chrisjhoughton commented Jun 18, 2015

Forgive me if this a noob thing to say, I'm new to Sequelize and am a little rusty on SQL, but it looks like when you create belongsToMany tables, for example:

| productId | userId |       createdAt        |
|-----------|--------|------------------------|
|       123 |    456 | 2015-05-10 23:00:00+00 |

It's using the productId, userId together as the primary key. While this could work fine for pure unique relationships, it doesn't handle the circumstances where the same relation can be created multiple times - because the primary key has to be unique.

For example, purchasing/viewing the same product multiple times.

To get around this I've created a separate id primary key on the relational table, using an auto increment.

Don't know if this is the best solution but it certainly works.

@mickhansen
Copy link
Contributor

@mickhansen mickhansen commented Jun 18, 2015

@chrisjhoughton I'm afraid we have very poor support for non-unique N:M relations at the time.

@chrisjhoughton
Copy link

@chrisjhoughton chrisjhoughton commented Jun 18, 2015

@mickhansen no problem, the workaround above is working great!

@j7caiman
Copy link

@j7caiman j7caiman commented Jan 14, 2016

@chrisjhoughton
Hi - I'm having the same problem as you I think, but even when I try your workaround, it doesn't seem to work. Is this what you were doing?

var Doctor = sequelize.define('doctor', {
    name: Sequelize.STRING
});

var Patient = sequelize.define('patient', {
    name: Sequelize.STRING
});

var Appointment = sequelize.define('appointment', {
    id: {
        type: Sequelize.INTEGER,
        primaryKey: true,
        autoIncrement: true
    },
    date: Sequelize.DATE,
    paymentType: Sequelize.STRING
});

Doctor.belongsToMany(Patient, {through: Appointment, unique: false, foreignKey: {name: "patientId", unique: false}});
Patient.belongsToMany(Doctor, {through: Appointment, unique: false, foreignKey: {name: "doctorId", unique: false}});

Even still, it creates:

    "appointments_doctorId_patientId_key" UNIQUE CONSTRAINT, btree ("doctorId", "patientId")

I realize some of those unique: true may be redundant / nonsensical but I cannot figure out how to make it work.

@zachguo
Copy link

@zachguo zachguo commented Jan 23, 2016

@j7caiman In your last two lines, try following instead

Appointment.belongsTo(Patient, ...
Appointment.belongsTo(Doctor, ...
@zachguo
Copy link

@zachguo zachguo commented Jan 23, 2016

This should be mentioned in docs very explicitly.

@j7caiman
Copy link

@j7caiman j7caiman commented Jan 23, 2016

@zachguo: Unless there's something in the ... that you suggested that I'm missing, adding those two associations has no effect on my table structure.

Secondly, I wasn't able to find the information you suggested in the docs anywhere. Here's where I looked: http://sequelize.readthedocs.org/en/latest/api/associations/belongs-to-many/
and: http://sequelize.readthedocs.org/en/latest/api/associations/

@statyan
Copy link

@statyan statyan commented Feb 21, 2016

I met the same thing: unique: false does not prevents creation of unique index. First I came to google and found this post. And I've tried as @mickhansen said:

Ah well that's great, that's what you have to do.
You might need to tweak the belongsToMany call to accept uniques:
Item.belongsToMany(models.user, { through: models.history, as: "Owned", unique: false});

But this is not correct. After deep debugging I've found that belongsToMany expects the complex object 'through' if we want to set additional options to the relation. So the correct code is:

Item.belongsToMany(models.user, { through: { model: models.history, as: "Owned", unique: false}});

As I've found this, I search the sequel docs for word 'unique'. And found this feature in n:m chapter:
http://docs.sequelizejs.com/en/latest/docs/associations/?highlight=unique:%20false#nm

See the example for Post:

    Post.belongsToMany(Tag, {
        through: {
            model: ItemTag,
            unique: false,
            scope: {
                taggable: 'post'
            }
        },
        foreignKey: 'taggable_id',
        constraints: false
});

P.S. Maybe this options may be covered in manual more clearly. I've read man several times, but 'unique: false' feature was discovered only with Google.

@mickhansen
Copy link
Contributor

@mickhansen mickhansen commented Feb 22, 2016

Hmm, my mistake then, ideally it should work simply as an option on the relation though.

@marcusfeeney
Copy link

@marcusfeeney marcusfeeney commented May 10, 2016

Any update on this issue?

@armandopadilla
Copy link

@armandopadilla armandopadilla commented Aug 31, 2016

Im also encountering this issue. seems like unique: false, both in through and outside it do nothing. Digging into the library, i THINK, the code doesnt take into account when this.through.unique is false. It checks for everything BUT false. I was looking at the associations/belongs-to-many.js file.

@Iliyass
Copy link

@Iliyass Iliyass commented Sep 9, 2016

after tried @statyan solution of unique:false in Postgres it doesn't create a unique index on the foreign keys, but it still issue an update when a the same foreign keys exists.

amadk added a commit to amadk/escape-reality that referenced this issue Nov 30, 2016
- There's a bug in Sequelize "BelongsToMany" where it refuses to accept
multiple foreign keys ("user_id and photo_id must be unique"), but this is
not correct error for a join table because the fields all of their own unique
IDs. We found a Github thread where people discovered this same bug, but
it hasn't been fixed by the Sequelize team:

sequelize/sequelize#3220

- Amad and Tony
amadk added a commit to lowtalkers/escape-reality that referenced this issue Nov 30, 2016
- There's a bug in Sequelize "BelongsToMany" where it refuses to accept
multiple foreign keys ("user_id and photo_id must be unique"), but this is
not correct error for a join table because the fields all of their own unique
IDs. We found a Github thread where people discovered this same bug, but
it hasn't been fixed by the Sequelize team:

sequelize/sequelize#3220

- Amad and Tony
@stale stale bot added the stale label Jun 29, 2017
@stale

This comment was marked as outdated.

@ernani
Copy link

@ernani ernani commented Aug 18, 2017

Hi, is this still ongoing?

I need the relation table to avoid having a composite primary key with the 2 relation fields, is that possible?

Perhaps we can give it the field name for the primary key?

Shall I need to create my own model for this and then set unique: false on the BelongsToMany ?

@bharathvaj-ganesan
Copy link

@bharathvaj-ganesan bharathvaj-ganesan commented Jan 4, 2018

The unique constraint error deletes the previous records too. This isn't expected. Please resolve the issue

@napindc
Copy link

@napindc napindc commented Feb 27, 2018

I'm also running into this issue

@felixSabatie
Copy link

@felixSabatie felixSabatie commented Jul 14, 2018

I also have this issue

@Andiedie
Copy link

@Andiedie Andiedie commented Feb 17, 2019

+1

1 similar comment
@cguertin14
Copy link

@cguertin14 cguertin14 commented Dec 1, 2019

+1

@papb papb reopened this Jan 16, 2020
@stale stale bot removed the stale label Jan 16, 2020
@papb
Copy link
Member

@papb papb commented Jan 16, 2020

For anyone with this issue, here is a suggested workaround:

// Instead of using a Many-to-Many relationship:
A.belongsToMany(B, { through: C });
B.belongsToMany(C, { through: C });

// Throw away the Many-to-Many relationship altogether and treat C as a full-fledged standard model
A.hasMany(C);
C.belongsTo(A);
B.hasMany(C);
C.belongsTo(B);

However the solution above of course will change how you fundamentally perform includes and the like.

// This way, instead of
A.findAll({
  include: B
});

// You will have to do
A.findAll({
  include: {
    model: C
    include: B
  }
})
// and the structure of the query result will be a bit different but all the content you need will be there
@papb
Copy link
Member

@papb papb commented Jan 16, 2020

Although the workaround above is a drastic conceptual change and would take a lot of effort to convert from what should be working, since it's not working I doubt anyone will really have lots of code to change.

Hopefully the above is a workaround that helps everyone.

Let me know if it doesn't work for any of you

@Xlider
Copy link

@Xlider Xlider commented Jul 23, 2020

This code will help you. Look at this:
I have found that primaryKey must be false and I did it with adding to my associations primaryKey: false

// Tag model
  let Tag = sequelize.define(
    "Tag",
    {
      type: {type: DataTypes.ENUM, values: ['A', 'B'], unique: 'item_tag_taggable', defaultValue: 'A', allowNull: false},
      userId: {type: DataTypes.INTEGER, unique: 'item_tag_taggable'},
      relId: {type: DataTypes.INTEGER, unique: 'item_tag_taggable', references: null},
    },
    {
        charset: "utf8",
        collate: "utf8_general_ci",
        underscored: true,
        timestamps: false,
    }
  );

// Associations

ItemA.belongsToMany(models.User, { through: { model:"Tag", unique: false, scope: { type: 'A' }}, as: "ItemATags", foreignKey: { name: "relId", primaryKey: false, references: null }, constraints: false});
ItemB.belongsToMany(models.User, { through: { model:"Tag", unique: false, scope: { type: 'B' }}, as: "ItemBTags", foreignKey: { name: "relId", primaryKey: false, references: null }, constraints: false});

User.belongsToMany(models.ItemA, { through: { model: "Tag", unique: false }, as: "ItemATags", foreignKey: { name: 'userId', primaryKey: false}});
User.belongsToMany(models.ItemB, { through: { model: "Tag", unique: false }, as: "ItemBTags", foreignKey: { name: 'userId', primaryKey: false}});

Main thing there primaryKey must be false
And I didn't optimize my code, you can do it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.