Argument serialization into redis #357

Closed
wants to merge 8 commits into from

5 participants

@elado

ActiveRecord, classes, modules and other parameters are serialized into custom strings such as "SIDEKIQ@User@2" and "SIDEKIQ@SomeClass" (similar to DelayedJob) in order to have minimum concern about deserialization on the worker methods (used w/ #delay)/worker classes and minimum storage (not storing full serialized AR object but only its ID).

Every class can implement serialization by an instance method sidekiq_serialize that returns a string prefixed with SIDEKIQ followed by @ and the class name, followed by an optional @ with arguments for the deserialization.
Deserialization is implement by by a class method sidekiq_deserialize(string) which gets the argument after the second @ and should return the instance.

ActiveRecord instances will serialize to SIDEKIQ@User@2, and when deserialized, the User will be extracted and constantized, then 2 will be passed to User.sidekiq_deserialize(2) which will return User.where(id: 2)

elado added some commits Aug 18, 2012
@elado elado argument serialization into redis
ActiveRecord, classes, modules and other parameters are serialized to strings such as "AR:User:2" and "CLASS:SomeClass" in order to have minimum concern about deserialization on the worker methods (used w/ #delay) and classes.
3a09556
@elado elado custom serialization/deserialization methods for any class instead ju…
…st hardcoded AR
8585312
@elado elado removed unnecessary prefixes (AR: and CLASS:) 0559df1
@travisbot

This pull request passes (merged 0559df1 into 254c204).

@mperham
Owner

I wish you had opened an issue first as I'm not really interested in complicating the serialization any further. Have you seen sidekiq/yaml_patch?

@elado
  1. The yaml_patch still seems to return the entire attributes hash, and if changed it'll affect the entire yaml serialization. Not necessarily expected.
  2. You can't pass classes to a worker as an argument (not top priority be see below what I think is important)
  3. Custom serialization can be used for various situations

I think that it's important to

a) be able to switch between delayed method on non-delayed method
b) write your delayed methods as if you wrote a regular method, meaning using real AR instances and not IDs
c) write less obvious code, DRY

For example:

class TwitterService
  def self.followers_for_user(user)
     ...
  end
end

user = User.first

# urgent to fetch followers
TwitterService.followers_for_user(user)

# not urgent
TwitterService.delay.followers_for_user(user)

And you can switch between the two options just by delay. No need to worry about passing only ids, creating worker classes etc.

It's OK if you don't pull; I just had to have it for my app.

I might convert this into a plugin instead of rewriting your extensions (as I didn't touch the rest of the code).

And thanks for the wonderful gem :)

@travisbot

This pull request fails (merged 3bb3148 into 254c204).

@travisbot

This pull request fails (merged 4708c1b into 254c204).

@travisbot

This pull request fails (merged d9a9754 into 254c204).

@travisbot

This pull request passes (merged 170fc54 into 254c204).

@KensoDev

I am 👍 on this

@felipecsl

👍

@mperham
Owner

Your arguments will be whatever you pass in within the limitations of the serialization scheme in use. If you pass in a full Ruby object, that full Ruby object will be serialized. As part of the Best Practices I strongly urge people to pass IDs, not objects, in their messages but if you want to pass objects, you should get the full object including any transient state and the state of the object at the time delay was called.

@mperham mperham closed this Aug 20, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment