Permalink
Commits on Dec 15, 2011
  1. - shifted Thrift from 0.5.0 to the 0.7.0

    - ability to test just one specified class 'ant test-class -Dtest.class=com.twitter.elephantbird.pig.load.TestJsonLoader'
    committed Dec 15, 2011
Commits on Nov 30, 2011
  1. 2.1.2

    Raghu Angadi committed Nov 30, 2011
  2. Two ThriftToPig schema fixes and tests (Jonathan Coveney)

      - tuple wrapping for Pig9
      - bug with nested containers
    Squashed commit of the following:
    
    commit 961be195d3365e3a3ff737be3a65f6d9041124ec
    Merge: 342f336 3973933
    Author: Raghu Angadi <raghu@twitter.com>
    Date:   Tue Nov 29 16:20:11 2011 -0800
    
        Merge branch 'jco_pig9_fix' of https://github.com/jcoveney/elephant-bird into jcoveney-jco_pig9_fix
    
        Conflicts:
        	src/test/com/twitter/elephantbird/pig/piggybank/TestThriftToPig.java
    
    commit 3973933
    Author: Jonathan Coveney <jcoveney@twitter.com>
    Date:   Mon Nov 28 20:43:15 2011 -0800
    
        Refine ThriftToPig and tests
    
        There were some issues with elephantbird's handling of lists of lists.
        This refines elephantbird to deal with these issues in both Pig8 and
        pig9, and adds tests.
    
        I also added some commented out tests for map functionality, which
        currently is broken.
    
    commit 61464b9
    Author: Jonathan Coveney <jcoveney@twitter.com>
    Date:   Mon Nov 28 20:42:51 2011 -0800
    
        A newer pig9 jar with an important patch
    
    commit 2d4e425
    Author: Jonathan Coveney <jcoveney@twitter.com>
    Date:   Thu Nov 17 16:05:22 2011 -0800
    
        Update code
    
    commit 4488a22
    Author: Jonathan Coveney <jcoveney@twitter.com>
    Date:   Wed Nov 16 20:47:48 2011 -0800
    
        Move logic around
    
    commit 7445a0d
    Author: Jonathan Coveney <jcoveney@twitter.com>
    Date:   Wed Nov 16 20:00:46 2011 -0800
    
        Refine ThriftToPig changes
    
        It used to be the convention in elephantbird that if you made a bag from
        a list, the element would be named "_tuple." but this no longer makes
        any sense in pig9, when the tuples are explicit. As such, I've removed
        it.
    
    commit ab98602
    Author: Jonathan Coveney <jcoveney@twitter.com>
    Date:   Wed Nov 16 17:34:04 2011 -0800
    
        remove old pig jar
    
    commit c3fa36b
    Author: Jonathan Coveney <jcoveney@twitter.com>
    Date:   Wed Nov 16 17:32:15 2011 -0800
    
        Fix compatibility issue with pig9, upgrade to pig9
    
        Pig9 changed the construction of bags, which was creating an issue in
        the case of lists and sets with structs as elements. I added a test for
        this case, and made it such that the change is consistent with both pig8
        and pig9.
    
        Note: the test will fail on pig8.
    Raghu Angadi committed Nov 30, 2011
Commits on Nov 24, 2011
  1. Merge pull request #106 from rangadi/restructure_multi_loader

    Restructure multi loader
    rangadi committed Nov 24, 2011
  2. forgot to add this file in prev commit (5d48835).

    Raghu Angadi committed Nov 24, 2011
  3. Add FilterLoadFunc. simplifies MultiFormatLoader.

    Raghu Angadi committed Nov 24, 2011
Commits on Nov 23, 2011
  1. Merge pull request #97 from rangadi/multi_lzo_reader

    Multi lzo reader
    rangadi committed Nov 23, 2011
  2. Merge pull request #92 from rangadi/projection_support

    Projection support for protobuf and thrift loaders
    rangadi committed Nov 23, 2011
  3. Merge remote branch 'upstream/eb-dev' into projection_support

    Conflicts:
    	src/test/com/twitter/elephantbird/pig/piggybank/TestThriftToPig.java
    Raghu Angadi committed Nov 23, 2011
  4. project support review comments:

       - rename Projected*Tuple to Project*TupleFactory
       - remove double use of contextSignature
       - remove trailing '_' and add couple of @SuppressWarnings
    Raghu Angadi committed Nov 23, 2011
  5. add comment about RunWith/SuiteClasses for lzo tests.

    Raghu Angadi committed Nov 23, 2011
  6. MultiFormatLoader: cut some more fat

    Raghu Angadi committed Nov 23, 2011
  7. MultiInputFormat : review comments

    Raghu Angadi committed Nov 23, 2011
Commits on Nov 22, 2011
  1. Merge pull request #102 from miguno/issue65

    Issue 65: LzoJsonRecordReader and JsonLoader may fail with NullPointerException
    rangadi committed Nov 22, 2011
Commits on Nov 19, 2011
  1. prep for 2.1.2

    Raghu Angadi committed Nov 19, 2011
  2. 2.1.1

    Raghu Angadi committed Nov 19, 2011
  3. Merge branch 'master' into eb-dev

    Raghu Angadi committed Nov 19, 2011
Commits on Nov 18, 2011
  1. Merge pull request #104 from billonahill/bg-analytics-1471

    Adding Pig schema - to - ProtoDescriptor functionality with tests
    rangadi committed Nov 18, 2011
Commits on Nov 17, 2011
  1. removing log check

    billonahill committed Nov 17, 2011
  2. Adding schemaToProtoDescriptor functionality with tests

    Add new version of getMessageFieldNames that takes FieldDescriptors instead of protobufs class.
    billonahill committed Nov 17, 2011
Commits on Nov 15, 2011
  1. Merge pull request #100 from dvryaboy/eb-dev

    Minor Doc update
    dvryaboy committed Nov 15, 2011
  2. Issue 65: LzoJsonRecordReader and JsonLoader may fail with NullPointe…

    …rException
    
    This patch prevents LzoJsonRecordReader and JsonLoader from failing with
    a NPE when JSONParser#parse() returns a null value.  In this case a warning
    is logged and the values 'false' (LzoJsonRecordReader) and 'null'
    (JsonLoader) are returned, respectively.
    
    JsonLoader#parseStringToTuple() will also increment the counter
    JsonLoaderCounters.LinesParseError if this happens.
    
    More information is available at:
    twitter#65
    miguno committed Nov 15, 2011
Commits on Nov 11, 2011
  1. clarified docs a bit more

    dvryaboy committed Nov 11, 2011
Commits on Nov 10, 2011
  1. Some Readme improvements

    dvryaboy committed Nov 10, 2011
Commits on Nov 9, 2011
  1. add TestMultiFormatLoader.

    Raghu Angadi committed Nov 9, 2011
  2. add test.libary.path option to set java.library.path (needed to run l…

    …zo tests)
    Raghu Angadi committed Nov 9, 2011
  3. add MultiFormatLoader and MultiInputFormat.

    currently can handle mix of B64Line and BlockStorage files
    The correct record reader used at runtime based on the input
    file header.
    more formats will be supported overtime.
    Raghu Angadi committed Nov 9, 2011
Commits on Nov 4, 2011
  1. prep for 2.1.1

    Raghu Angadi committed Nov 4, 2011
  2. 2.1.0

    Raghu Angadi committed Nov 4, 2011
  3. Merge pull request #95 from rangadi/allow_enum_keys

    Allow enum keys in thrift maps
    rangadi committed Nov 4, 2011
  4. add unit test for Thrift map with enum keys

    Raghu Angadi committed Nov 4, 2011
  5. ENUMs are implicitly converted to Strings for PIG

    allow ENUM along with a STRING for Thrift map key type
    Raghu Angadi committed Nov 4, 2011