Switch branches/tags
Nothing to show
Commits on Mar 5, 2011
Commits on Mar 3, 2011
Commits on Feb 27, 2011
  1. handle for a filesytem file object should be an attr_accessor so, whe…

    thedatachef committed Feb 27, 2011
    …n all else fails, one can have access to the file handle
  2. updated readme

    thedatachef committed Feb 27, 2011
Commits on Feb 26, 2011
Commits on Feb 25, 2011
  1. s3 filesystem in process of being implemented, uses right_aws gem, so…

    thedatachef committed Feb 25, 2011
    …mewhat cludgy but so far works
Commits on Feb 24, 2011
Commits on Feb 23, 2011
Commits on Feb 21, 2011
  1. removed hadoop merge file

    thedatachef committed Feb 21, 2011
  2. got rid of idempotency at the script level, not all scripts have file…

    thedatachef committed Feb 21, 2011
    …system output and idempotency should be left to the workflow designer themselves, updated example to reflect this, stripped out pig_classpath and pig_options but added the ability to set env for any script via the "env" method (way more flexible)
Commits on Feb 18, 2011
  1. local mode should actually work now (at least for pig scripts and r s…

    thedatachef committed Feb 18, 2011
    …cripts), still broken for wukong scripts since --run=local is broken
Commits on Feb 17, 2011
Commits on Feb 16, 2011
  1. simple script for fetching hadoop logs in the "correct" way, not inte…

    thedatachef committed Feb 16, 2011
    …grated into workflow yet, still need to work out a sane way to get the job id for a currently running job
  2. minor (but really annoying) bug in hadoop filesystem, needs to add a …

    thedatachef committed Feb 16, 2011
    …"/" after the path to hadoop conf if it isnt already there
  3. notes on logging, thinking it might be helpful to have a http filesys…

    thedatachef committed Feb 16, 2011
    …tem for doing a directory listing...? workflows need a method for getting/setting logdir
Commits on Feb 15, 2011
  1. filesystem should be autoloaded when called, when open is called on t…

    thedatachef committed Feb 15, 2011
    …he hadoop filesystem with a block, it MUST close itself after
Commits on Feb 12, 2011
  1. updated readme

    thedatachef committed Feb 12, 2011
  2. made example work with new abstraction, changed script to use new abs…

    thedatachef committed Feb 12, 2011
    …traction, added methods for merging hdfs files and copying to local without concatenating them
Commits on Feb 11, 2011
  1. hadoop file system abstraction should be done, now to re-integrate wi…

    thedatachef committed Feb 11, 2011
    …th rest of swineherd for all those things that broke
  2. basefilesystem should have the correct interface, going to use it now…

    thedatachef committed Feb 11, 2011
    … to define hadoopfilesystem
Commits on Feb 10, 2011
  1. Added file system abstraction sample implementation

    Kurt Bollacker
    Kurt Bollacker committed Feb 10, 2011
Commits on Feb 9, 2011
  1. HDFS.mkdir_p for idempotently creating a directory and not throwing a…

    thedatachef committed Feb 9, 2011
    …n error if it already exists
Commits on Feb 8, 2011
Commits on Feb 1, 2011
Commits on Jan 6, 2011
  1. example makes use of all the current types of scripts and does so ide…

    thedatachef committed Jan 6, 2011
    …mpotently, treat this as the canonical swineherd example