Avro Loader

[theJohnnyBrown]

TODO

• Add tests. I've run this code against fairly complicated avro data, but it's private so I'll need to write some example data.
• Add license to files and tidy up the code.
• parameterize the jar locations in load-avro.
• (Stretch goal) add store-avro and local/store

[mbossenbroek]

If we need these on the classpath, would it be easier to just add them as dependencies that get pulled into the uberjar? The purpose of this was more for the annoying jars that aren't in maven.

[mbossenbroek]

Looks good! I'm looking forward to seeing the sample data & tests. I think that'll make it a lot more clear how the schema is being used.

[theJohnnyBrown]

It would be great to hear about the gradle/clojure workflow people are using on this project - does anyone know how to get a repl with the dependencies loaded? I've resorted to creating another leningen project, with a dependency on pigpen-avro, opening core.clj in a cider buffer and eval-ing it in that repl.

EDIT: there's ../gradlew clojureRepl but that seems to have issues for me.

[theJohnnyBrown]

@mbossenbroek, I think the issues you mentioned have been addressed. I'll save the storage side for next time (Or for someone else!). Anything else you need to see?

[mbossenbroek]

Everything except load-avro should be private

[theJohnnyBrown]

dotted-args->nested-map needs to be public so we can require it from pig

[mbossenbroek]

Fair enough. In other places where that's the case, I'll put the logic in a different namespace and just expose the public API by copying the symbols there. Take a look at pigpen.core for examples. It also makes testing easier because you can access all of the internals easily.

[mbossenbroek]

Looks great! Just a couple of polish items. Thanks for the work!

[mbossenbroek]

Almost forgot - what issues are you seeing with ./gradlew clojureRepl? IIRC, I'm getting that from clojuresque and it should work. Personally, I use eclipse and CCW - one keystroke and I have a repl in my IDE. IntelliJ+Cursive should work similarly.

[theJohnnyBrown]

Here's the output from running clojureRepl: https://gist.github.com/theJohnnyBrown/faa5782176ebe327dade

[theJohnnyBrown]

updated

[mbossenbroek]

Released in 0.2.13