Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@Unique: replace on conflict flag #509

Closed
greenrobot opened this issue Jul 25, 2018 · 64 comments
Closed

@Unique: replace on conflict flag #509

greenrobot opened this issue Jul 25, 2018 · 64 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@greenrobot
Copy link
Member

Right now, @Unique throws an exception if there's a violation.

We could also offer to replace old entities with newer ones, e.g. by doing a @Unique(replaceOnConflict = true).

Keep in mind: If there are multiple unique properties, on entity might replace several others.

@greenrobot greenrobot added the enhancement New feature or request label Jul 25, 2018
@avcial
Copy link

avcial commented Jul 25, 2018

Actually requirement is not completely replacing, just update fields which are not annotated as @id or @unique

@greenrobot
Copy link
Member Author

@avcial Not sure if I follow. Example?

@kordianbruck
Copy link

@greenrobot you can update the fields instead of delete+create ("replace") objects that have a unique property (like primary keys and explicit @unique), but that's just an implementation detail. Difference is, that PKs will stay the same an FK will not break while updating a unique key with replaceOnConflict.

@avcial
Copy link

avcial commented Jul 26, 2018

@kordianbruck you are soo right
As i mentioned before here, mine and in my opinion the most common case is that caching server datas to device for giving faster responses, but the necessity of UpdateOnConflict is that what if my users last name has changed but i don't want to lose my ObjectBox db ObjectId and relations on it.
Scenario :
Get datas from WebAPi you have empty table, you inserted all of them with logic [long id,string ServerGuid, string FirstName, string LastName].
Then you want to update your ObjectBox user table because you know you have some new users or some of your users might change their name or maybe they got married and their last name changed and you don't want to show old datas on your screen. So you get the new list from your api and then you give it to a function whic adds new users with new objectIds but it also checks the older records based on your second column(string ServerGuid) that is because your Server doesn't know ObjectBox ObjectId's of your devices and when you got data which are inserted before you just check the fileds(FirstName and LastName) which are not annotated by @id or @unique or maybe they are annotated on consciously like @UpdateOnConflict and update them with new datas which comes from the current entity.

@greenrobot
Copy link
Member Author

greenrobot commented Jul 26, 2018

Yeah, it makes sense to keep the ID, because it is used in relations and you do not want to cut those off.
So it should rather be @Unique(onConflict = UPDATE) with some enum {FAIL, REPLACE, UPDATE}.

We could extend further extend the enum with ABORT_TX (e.g. check SQLite ON CONFLICT).

HOWEVER, I'm getting doubts if it's a good idea to tie it to an annotation. Because annotations on two different properties could be contradicting. And also, behavior is less flexible.

The more flexible approach would be to overload the put method and pass either a ConflictResolution enum or flags (more hacky but more future proof) additionally.

@kordianbruck
Copy link

There is a bit more specific documentation in the SQLite docs: https://sqlite.org/lang_conflict.html

So yes, its a good idea and general industry standard when talking about SQLite based things: https://developer.android.com/reference/android/arch/persistence/room/OnConflictStrategy

Maybe SQLite is not the reference here, but I think when devs come from using an ORM like room, they at least expect similar ways how those things work 👍

@avcial
Copy link

avcial commented Jul 26, 2018

I know the logic will be a mess but when you add the exception throwing on conflict at ver2.0.0 i expected that maybe you would return the original entity inside the error so we can do whatever we want to do with conflicted row, maybe try to put it again after some changes??

@greenrobot
Copy link
Member Author

@kordianbruck Not sure what you are arguing for, could you explain? Do it as annotations? SQLite supports conflict resolution within column definition (inflexible), but also inside INSERT/UPSERT (flexible). I tend to prefer the flexible option, e.g. box.put(person, REPLACE) because at another code section you might want to do box.put(person, IGNORE) because you might have another use case.

@avcial Maybe be a mess for lists, but OK for single entities; e.g. Person conflict = box.putOrGet(person). Same with lists would be possible but you would loose the connection of offending and existing entity.

@avcial
Copy link

avcial commented Jul 26, 2018

if performance is good enough it does't matter if i loose connection.
maybe the method will return me the list which is handled for the choosen strategy(ies)??

@kordianbruck
Copy link

@greenrobot sorry, I could've been more clear on that: I would vouch to use the same naming conventions as in SQLite for ROLLBACK, ABORT and FAIL so that developers coming freshly to objectbox already can guess what that flag is doing from their previous SQLite experience.

For our use cases we would be fine with annotated support. If the more flexible option is doable without modifying the API too much that also might be a good idea - no clue tho, how useful it is.

@avcial
Copy link

avcial commented Aug 3, 2018

Throwing exception break's box.put(Collection ) method loop, there is a need for quick solution for that

@Mistic92
Copy link

Mistic92 commented Aug 5, 2018

Yep, I can't update via put method. I'm getting crash when entity with unique annotation exists but I just want to update

@avcial
Copy link

avcial commented Aug 7, 2018

Any upgrade or time estimation?

@wheelergames
Copy link

Jumping on the bandwagon for this. I want to do put(Collections) with Entities that have a Unique field. If I have a clash the try catch just throws an exception and stops. I'd love to have that overloaded method to say on unique clash, do update or something like that.

@greenrobot
Copy link
Member Author

greenrobot commented Aug 7, 2018

Couple of questions to better understand requirements:

  1. Do you use @Unique to emulate e.g. a string ID?
  2. Or, do you have multiple @Unique properties in a single entity or do you plan to?
  3. Any preference to put the conflict strategy on @Unique or, more flexible, with put()? And why?

@wheelergames
Copy link

To answer your questions

  1. Yes
  2. No
  3. no pref, maybe in put() as I'm not an expert on annotations

So my use case might have that extra level of annoyance/complication, as my Box has a ToMany with it...

I want to have a box/entity of words, to form a dictionary, then I want a separate box for a difficulty level, which links on many to many to the words.

i.e. a word like cat will exist on easy and hard words, but a word like supercilious would only be in hard

I will upload all the words from one dictionary e.g. easy, which will populate the Word Box and the WordDifficulty Box, then upload all the hard words, which will

  1. add a new word if needed
  2. if it exists add a 'row' in the WordDifficulty for hard

(I'd like to do the put for this as a List so use the collections version of put)

@avcial
Copy link

avcial commented Aug 7, 2018

Good to be aware about conflicts on Uniqly annotated fields but throwing exception breaks put function and not giving any referance of conflicted entity, to update or do something with that entity, i have to querry it to find which entity was registered to db before, i need to know that, if its possible i need to make changes on that verry quickly like change its "LastSyncDate" field.

@wheelergames
Copy link

The error message actually says which IDs have clashed, but you'd need to somehow regex the string in order to find that out, which is not exactly ideal!

@mesterj
Copy link

mesterj commented Aug 10, 2018

@unique(replaceOnConflict = true) this behaviour will default? I think true would be better then throwing exception. When new version will available? I am waiting :-)

@avcial
Copy link

avcial commented Aug 17, 2018

any news?

herbeth1u added a commit to herbeth1u/VNDBA that referenced this issue Sep 22, 2018
@Queatz
Copy link

Queatz commented Sep 23, 2018

Should be in the .put, because one may want to do different things in different flows. Looking for this feature, too. 👍

@greenrobot
Copy link
Member Author

@Queatz Do you have a real (seen in practice) use case?

Both options have up- and down-sides. With put, there can be multiple unique properties that work differently. And a put may span over relations... Thus, I'd like to continue with property-based on-conflicts.

@greenrobot
Copy link
Member Author

Current proposal:
@Unique has the default on-conflict strategy FAIL; any unique violation throws an exception.

@Unique(onConflict=IGNORE): ignore the offending object (no changes to the existing conflicting object). If there are multiple unique properties in an entity, this strategy is evaluated first:
if the property conflicts, no other properties will be checked for conflicts.

@Unique(onConflict=REPLACE): the offending object replaces the existing conflicting object (deletes it). If there are multiple properties using this strategy, a single put can potentially replace (delete) multiple
existing objects.

@Unique(onConflict=UPDATE): the offending object overwrites the existing conflicting object while keeping its ID. Thus, all relations pointing to the existing entity stay intact.
This is useful for a "secondary" ID, e.g. a string "ID". Within an entity, this strategy may be used once only (update target would be ambiguous otherwise).

Multiple strategies in an entity are processed in this order: IGNORE, FAIL, REPLACE, UPDATE.

@Queatz
Copy link

Queatz commented Sep 27, 2018

In my app, I basically have a simple implementation of Offline First:

@Entity
class Message {
    @Id long localId;
    String id;
    ...
}

When a user sends a message, it creates a new Message with a null id. When a sync happens, the Message is sent to the server and an id is returned, which then gets set on the Message. Any subsequent load from the server will update the local model.

However, there is a slight chance that the Message can be loaded from some other server endpoint before the id is successfully returned and set on the local model. So I end up with duplicates.

Ideally I could do:

@Unique(onConflict=UPDATE) for any object coming from the server
@Unique(onConflict=IGNORE) for any object created locally (i.e. when the server ID is added to the local object, and that ID already exists on an object due to a race condition, just ignore adding the ID, because the model is outdated anyways.

For this app, I could get by with only being able to pick one option per field, however I can foresee running into a wall there by having different requirements in different scenarios.

@greenrobot
Copy link
Member Author

greenrobot commented Sep 28, 2018

@Queatz Don't see duplicates in this scenario. Check this example and let me know if and where we diverged:

  1. Your entity with localId 42 is being uploaded
  2. Server assigns id "foo"
  3. "foo" object gets to the client on a different route, it's inserted with localId 43 in another thread (race condition to 1. step)
  4. First step (upload) gets "foo" back; with onConflict=UPDATE it will update into localID 43 which got there first (object with ID 42 is removed)

No dups. However, any relations to localId 42 will be deleted. If that is a problem, server-side queuing should solve it. Nobody said sync is easy... ;-)

@Queatz
Copy link

Queatz commented Sep 28, 2018

Sync is hard xD Can't wait to play around with objectbox sync when it's ready. :)

onConflict=UPDATE would solve the dupe, but then any extra details the server added will be removed (if i understand correctly)

@Reginer
Copy link

Reginer commented Jun 15, 2020

The flower has faded three times .

It's going to bloom again this year .

@valposv
Copy link

valposv commented Oct 14, 2020

Any updates?

@tushar09
Copy link

tushar09 commented Oct 24, 2020

I have migrated from greendao to objectBox, and stuck in this @unique. Any update please? Should I revert back my whole project again? Or I have to insert data iteratively?

@zhieci
Copy link

zhieci commented May 10, 2021

When my only value is a string, I can only insert data iteratively at present. I don't think it's a good way. Is there a better way?

@greenrobot-team
Copy link
Member

You may want to try the new @Unique(onConflict = ConflictStrategy.REPLACE) of the 2.9.2-RC preview release.

Currently there can only be a single property annotated with this.

@Reginer
Copy link

Reginer commented Jun 1, 2021

You may want to try the new @Unique(onConflict = ConflictStrategy.REPLACE) of the 2.9.2-RC preview release.

Currently there can only be a single property annotated with this.

Yes , Yes !!!!!

@Queatz
Copy link

Queatz commented Jun 30, 2021

You may want to try the new @Unique(onConflict = ConflictStrategy.REPLACE) of the 2.9.2-RC preview release.

Currently there can only be a single property annotated with this.

Thank you so much for this! :) So far with my testing it seems to be working great and my app is running noticeably smoother than the old hack I had!

@greenrobot-team
Copy link
Member

This is now available with the 3.0.1 release.

@RobbWatershed
Copy link

@greenrobot don't wanna be the party pooper here, but what happened to onConflict=IGNORE along the way ?

@greenrobot-team
Copy link
Member

@RobbWatershed Got mislead by the title and closed this. But yes, the later proposal has more options like IGNORE: #509 (comment)

Re-opening this for now, I guess. But maybe this should get its own issue, this one has been getting rather long.

@greenrobot
Copy link
Member Author

We really should move the remaining option(s) to a new issue to start from a clean slate... Note though that other options are not "sync friendly"...

@RobbWatershed
Copy link

Thanks for the quick reply guys. Shall I create a new issue then ? I don't mind at all

@greenrobot-team
Copy link
Member

Closing this then. For more conflict strategy options vote and comment on #1023.

@AdelKanso
Copy link

The same behavior is happening on Dart, does the new ConflictStrategy.replace available for dart or just for java, cause I'm stuck with the same problem.
I'm trying to accomplish on unique key conflict, I don't want to remove the old object and insert a new one, I want the same object to be updated. @greenrobot-team

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests