Consider providing serialization-aware consumer API #6

michaelklishin opened this Issue Apr 19, 2012 · 11 comments


None yet

4 participants


Many scenarios in message consumer applications are very common. For example, JSON or MsgPack used for serialization. We should investigate a convenient way to plug common serialization formats into consumer API. Then payload attribute can be a data structure instead of a bag of bytes.


yes, and clojure data structure (de)serialization:

(defn serialize [data]
  (binding [*print-dup* true] (pr-str data)))

(defn deserialize [str]
  (with-in-str str (read))

suggested content-type: application/clojure

@carlhoerberg has generic serialization support, could be worth checking out..


We probably can take a page from Welle which in turn steals this idea from Sumo. I don't want this to replace langohr.consumers/subscribe, though, just be an useful option to choose from.


maybe even include content-encoding, eg. automatic gzip (de)compression of payloads..


I've written my own abstraction on top that uses a ring style to attach additional behaviors like this. So I have a wrap-decode-payload wrapper that support JSON or EDN based on content-type. Similarly, I have a wrap-encode-payload for going the other way and several other wrappers for other purposes. Gzip could be supported the same way. This pattern seems to work pretty well.


@mlimotte yes, I was contemplating doing just that.


Has the release of transit any incidence over this discussion?


Transit doesn't change anything for us. We can't support a single transport only.


OK. Thanks.

Just to be clear: is it up to everyone to build their abstractions on top of langohr for serialization/deserialization, or are there features/best practices/examples that can help when starting out?


Well, if you can deserialize a byte array in your consumer, you are all set today.

We can provide a protocol and a deserializing consumer function that will do it for you and call your handler with a Clojure data structure, that's pretty much it.


Thanks, Michael, I'll take the first suggestion for now.

Coming from Ruby, I'm always surprised when there's no native data structures straight off the box.
Not a big deal, but it's a philosophy thing that puzzles me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment