zdiffstore

[hellolucky]

Hello,

I use sorted set to store my photo list.When A follow B, A's follow photo list and B's photo list will be zunionstore to A's follow list. But when A unfollow B, there is no zdiffstore command to handle this.Will Redis add zdiffstore command?

Felix

[kgcrom]

How about using zinterstore?

zunionstore - zinterstore = list that you want
it's implement client-side.

I think that implement server side in a similar way.

[hellolucky]

Sure, it's similar!

But it is slow than zdiffstore!

zdiffstore: 1 command
zunionstore - zinterstore: 3 command (zunionstore、zinterstore、store back to some key)

I use zrange & zrem to solve this problem, but it is very slow

[parrish]

Just noticed this. I implemented it in #448
We've also been discussing it on the Google Group

[atomAltera]

I have the same problems. When user A unfollows user B, or user B deletes, all there post have to be deleted from user A feed

[timherby]

Would love to see this functionality merged in. I have the same situation (updating elements in a feed, and need a quick way to sync to an in-memory copy. I only use the sortedSet to keep the entries in-order. The scores are immutable.

[let4be]

too bad we still have this inconsistency between normal and ordered sets...

[jdejoode]

I agree that zdiffstore would be a valuable tool. In my case it would reduce memory usage by 50%.

[rburkat]

We have been using zdiffstore for years, but it's becoming a pain to patch and maintain every release our selves so perhaps I'm missing something why zdiffstore is not getting added.

Our use case is a photo popular suggestion subsection of our site. A per user set of daily viewed photos is maintained. We want to show to any users a set of popular/good photos that they have not seen yet. The idea is the most popular photos seen by a users friends, but NOT seen by the users would make a good suggestions.

1. QUERY:0.002242: zunionstore argsize(2309) zunionstore all the friends photo sets. In this case it's passing arg size of 2309 friends that I have and doing a union of all their photo view sets.
2. QUERY:0.006051: zdiffstore argsize(2) zdiffstore the created set in 1 of all my friends' viewed photos with my photo view set.
3. QUERY:0.000919: zrevrange argsize(2) zrevrange the set in 2 to get a list of most popular photos viewed by friends but not viewed by me yet.

This very quickly gives as a popular sorted set of photos that were seen by my friends but have not been seen by my.
Keep in mind that we have 10s of millions of photos views per day so this has to be fast and efficient.

This seems like a really nice and efficient way to solve this problem. It's fast enough to be realtime and has been running production for years. Am I missing something how this can be done without zdiffstore in an efficient way?

[tsiege]

Is there any news on this? It would be very helpful for a project of mine to run zdiffstore on sorted sets. To me @parrish's PR would make sorted sets feature complete. I'm surprised this issue has languished for so long. Is there a reason not to that we're unaware of?

[rdehuyss]

Also in need of this...

[rdehuyss]

Tagging @yossigo @natoscott so they get notified.

[felipou]

I'm currently working on getting a pull request ready for this. I took @parrish 's PR as reference to reimplement this in the current codebase, should be ready soon. But I have a couple questions for now (later we can move the discussion to the PR):

• The ZUNION and ZINTER commands have an AGGREGATE option, does it make sense for ZDIFF? If so, what exactly should it do? Would it be like ZUNION, but with inverted weights except for the first set (in case of AGGREGATE SUM)? What about MIN/MAX?
• If there is no AGGREGATE, does it make any sense to have the WEIGHTS argument option?

[oranagra]

@felipou thank you. from what i can tell the AGGREGATE and WEIGHTS options have no meaning for a DIFF operation, but maybe i'm missing some interesting use case.
I tried to skim though the various discussions, but didn't find anything.
Did you see some request for that? did the inverted weights idea you mentioned come up somewhere?
please provide a reference if there is one.

@itamarhaber are you aware of some use case or request for such?

[felipou]

No, I didn't see anything, it just came up while I analysed the code.

Since the ZINTER and ZUNION are implemented together, and the pull request I used as reference also added ZDIFF to the same function, I'll have to actively ignore these options in the case of ZDIFF (as the reference PR did). But no worries, that's quite simple.

I'll finish up the pull request without these options then.

[itamarhaber]

I'm not aware of any previous discussions on the matter, excluding one that I had several years ago with a certain Dr. Defining the meaning of these options in the context of ZDIFF, afaicr, is tricky and the value is debatable. I also motion for not supporting these, at least in the initial release of the command.

[felipou]

Just created the PR: #7961