Determine conflict strategy for inserts with a function

[thelinuxlich]

I'm inserting in batches with conflict: replace, but I stumbled upon a case where I'd need to change the strategy to error if the field status of my data is false. Without the possibility to do this dynamically I will have to avoid batch inserting for this specific table.

[danielmewes]

Good idea. We should support it in the same way as #873 .

[marshall007]

Just ran into this as well. I'm basically looking to do a replace while preserving a couple fields if they don't exist in the document being inserted. For example:

table.insert(values, {
  conflict: function (old_val, new_val) {
    return old_val.pluck('foo', 'bar').merge(new_val);
  }
})

[mlucy]

I like the idea of passing the conflict resolution function as a conflict optarg.

[mlucy]

We should also consider making the format be function (id, old_val, new_val) for consistency with #873.

[danielmewes]

@mlucy Would the id be the primary key of the document? This seems somewhat different from #873 where the first argument would be the field name. I don't think it makes sense to add the id here.

[mlucy]

Bumping to subsequent; #873 got bumped and we should settle this at the same time (hopefully for 2.2).

[thelinuxlich]

Is this happening on 2.1 or 2.2? Now I need to insert in batches with conflict = update and there is a field that could be missing when updating that I would like to add(but only if it doesn't exist). Are there any alternatives?

[danielmewes]

@thelinuxlich This is not happening for 2.1, hopefully for 2.2. Sorry.

Assuming that you need this to be atomic, I'm not sure if there's a work-around at the moment.
@mlucy do you have any idea for @thelinuxlich 's use case?

[mlucy]

I don't think there's any way to do this while preserving the performance benefits of batched inserts in 2.0 or 2.1. Hopefully we can get this in for 2.2.

[mlucy]

Bumping to 2.2 polish.

[marshall007]

@danielmewes oops, sorry. At first glance I was thinking (as you just said) that the callback would be fired for every document with old_val = null in the case of new documents. It's definitely the cleanest solution to #4410 we've considered, but I think I agree. It's not worth making the more typical cases less convenient.

[danielmewes]

Looks like we agree on

.insert(documents, {conflict: function(id, old_val, new_val) { ... } })

where the conflict function is only called when there actually is a conflict.
Marking settled.

[thelinuxlich]

Another very good use case for this feature would be when you save timestamps for document updating and when replacing in batches, you want to drop documents which have a older timestamp than the document to be replaced, thus guaranteeing some kind of ordering.

[danielmewes]

@thelinuxlich Very interesting use case!

[thelinuxlich]

I found another use case, when you want to limit the key quantity of a embedded object, very useful.

[marshall007]

@thelinuxlich what do you mean? Can you give an example where that's useful?

[thelinuxlich]

@marshall007 when you let the user save N fields inside a embedded object but you want to limit the max quantity of fields. An example from my code(limiiting to 30 fields):

r.table("attributes").get(item.id).replace(row => {
    return r.branch(row.eq(null),
        item, {
            id: item.id,
            client_id: item.client_id,
            visitor_id: item.visitor_id,
            fields: r.branch(row("fields").keys().setUnion(r(item.fields).keys()).count().gt(30),
                row("fields").merge(r(item.fields).pluck(row("fields").keys().setIntersection(r(item.fields).keys()))),
                row("fields").merge(item.fields)
            )
        }
    );
}, {
    durability: "soft"
})

[nighelles]

This was in review #3523.

[nighelles]

This is in next.