Skip to content

Serialization and partitioning strategies for using Thrift with clj-pail.

License

Notifications You must be signed in to change notification settings

EricGebhart/pail-thrift

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

pail-thrift

Build Status

Serialization and partitioning strategies for using Thrift with clj-pail. Makes defining PailStructure classes for Thrift schemas easier.

Usage

Add pail-thrift to your project's dependencies. If you're using Leiningen, your project.clj should look something like this:

(defproject ...
  :dependencies [[pail-thrift VERSION]])

Where VERSION is the latest version on Clojars.

Defining a PailStructure

PailStructure classes are defined with the gen-structure macro from clj-pail. pail-thrift provides serializers and partitioners that can be used with the gen-structure macro.

(ns example.pail
  (:require [clj-pail.structure :refer [gen-structure]]
            [pail-thrift.serializer :as s]
            [pail-thrift.partitioner :as p])
  (:import (example.thrift MyUnion))
  (:gen-class))

(gen-structure example.pail.PailStructure
               :type MyUnion
               :serializer (s/thrift-serializer MyUnion)
               :partitioner (p/union-partitioner MyUnion))

In the above example, we define a PailStructure that serializes the example.thrift.MyUnion type using the default Thrift serialization protocol. The PailStructure will also be vertically partitioned by the active field of each union.

Controlling the Serialization Protocol

The previous example uses the default Thrift serialization protocol. The protocol can be specified as an additional argument to the thrift-serializer function. The protocols are defined in clj-thrift:

(require '[clj-thrift.protocol.factory :as protocol])

(s/thrift-serializer MyUnion (protocol/compact))

Vertical Partitioning

A PailStructure is vertically partitioned according to the partitioner supplied as the :partitioner keyword argument of gen-structure. pail-thrift provides generic partitioners, but you may want the partitioner to be specific to your application. For this reason, the generalized partitioners are designed to be composed by application-specific ones.

Generalized partitioners are defined in pail-thrift.partitioner. Currently, there are 2 partitioners. Both partition data based on the fields in a union. One partitioner uses field ids the other uses field names.

License

Copyright © 2013 David Cuddeback

Distributed under the MIT License.

About

Serialization and partitioning strategies for using Thrift with clj-pail.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Clojure 100.0%