Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
heavy lifting in clojure
branch: master

This branch is 4 commits ahead, 1 commit behind bguthrie:master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
src/crane
test/crane
.gitignore
README
project.clj

README

To build:

1. Download and install leiningen http://github.com/technomancy/leiningen
2. $ lein deps
3. $ lein install

core libs for s3, ec2, ssh.

build upon the core is remote repl and cluster capabilities.

in a couple cases, we use an erlang convention of the ! syntax representing "reomote" (on another process) execution.
the first argument to ! in erlang is the pid, wheras in our api, it is an open ssh session or socket.
there is no ! operator or fn, we just use the ! suffix convention to communicate remote execution.

remote shell execution.
(sh! session "tar -xzf repl.tar.gz")

remote repl evaluation.
eval! 
(eval! socket (execute (workflow some-cascading-workflow)))

TODO: add examples for usages.


TODO:
the persistent shell session using jsch ChannelShell is shaky at best, although exec works fine.
eval! may be simpler if we could use LineNumberingPushbackReader see comment in remote_repl.clj
stuff to poll and pull info from hadoop tracker url into repl
Something went wrong with that request. Please try again.