Ignore duplicates on save

[robertmylne]

Currently if you run Model.save(models) and the Model class has a unique constraint it won't save anything, is there anyway to just ignore duplicates and still mass save the rest. I know I could loop over them and save 1 by 1 but this is sooooo much slower.

@Entity({name: 'posts'})
export class Post extends BaseEntity {

    @PrimaryGeneratedColumn()
    id: number

    @Index({ unique: true })
    @Column()
    name: string
}

Posts.save(posts)

UnhandledPromiseRejectionWarning: QueryFailedError: ER_DUP_ENTRY: Duplicate entry 'Post 7' for key 'IDX_e55fd28485892c76253624dee3'

This means that if I have Post 1, Post 2, Post 3... etc. None will get saved if Post 7 errors as a duplicate.

[dsbert]

This is the documentation for save

save - Saves a given entity or array of entities. If the entity already exist in the database, it is updated. If the entity does not exist in the database, it is inserted. It saves all given entities in a single transaction

In your example, is the error resulting because you are trying to save a new entity with the same name as an existing entity?

[robertmylne]

The error is happening because the column name has a unique constraint. So therefore when using the save method nothing saves. There should be a way to just ignore the entities that are duplicates and still save the rest of the entities, instead, nothing gets saved and it displays the duplicate entry error.

Example:

Database

|id|name|
|1|Post 12|
|2|Post 7|

Entities

let posts = [
{name: "Post 1"},
{name: "Post 2"},
{name: "Post 7"},
]
Posts.save(posts)

This throws the error ER_DUP_ENTRY: Duplicate entry 'Post 7' because Post 7 already exists. But it does not save Post 1 and Post 2 where the duplicate error did not trigger.

[dsbert]

TypeORM has no idea what your intent is when you try to save the new Post 7 object and it does not know that it already exists in the database (the identity field is missing).

If you are trying to save a post that already exists, then you should pass the id value as well. What happens if you do this?

let posts = [
{name: "Post 1"},
{name: "Post 2"},
{id: 2, name: "Post 7"},
]
Posts.save(posts)

[robertmylne]

I don't think you understand my issue. I do not want to update the post, I want it to just not throw the error when it runs into a name field that exists in an entity and not save anything. How do I ignore the save operation on entities that trigger a duplicate error.

Also I can not pass an id as these are new entities not existing entities. The post model is used as an example.

Basically, if I want to save an array of posts and on inserting into the database it encounters a post where the name field already exists (due to the unique constraint on the database column) I want it to ignore that post and just keep saving the rest of the entities. What it currently does is throw an exception and save none of them.

[nebkat]

Firstly, if you only intend on inserting entities you can use Posts.insert() instead of Posts.save(). This will perform an efficient insert query without checking if the entities already exist.

Secondly, if you want it to ignore errors like this, that is a custom behavior so you will have to use a query builder like so:

Posts.createQueryBuilder('posts')
    .insert()
    .values(posts)
    .orIgnore()
    .execute();

This is unlikely to ever be supported using Repository.insert(), but the query builder solution should be sufficient for your application.

[WygorFonseca]

Can use

this.yourEntityRepository
      .createQueryBuilder()
      .insert()
      .orIgnore(true)
      .into(YourEntity)
      .values(arrayData)
      .updateEntity(false)
      .execute();