Permalink
Commits on Feb 29, 2012
Commits on Feb 9, 2012
  1. MapReduce input reader providing access to the Logs API.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4190
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@293 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Feb 9, 2012
Commits on Feb 8, 2012
  1. Add google storage-specific existence errors.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4189
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@292 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Feb 8, 2012
  2. You have to specify parent entity if you want your jobs to start

    transactionally. This is the only way you can save anything to datastore
    from inside transaction. Concerns only huge mapreduce specifications
    that do not fit into task payload.
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4188
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@291 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Feb 8, 2012
  3. Introducing new Finalize rpc call which is not used yet.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4187
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@290 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Feb 8, 2012
  4. Add an option to force_writes in mapper db operation.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4186
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@289 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Feb 8, 2012
Commits on Jan 17, 2012
  1. Do not mark output as partial if key is going to change anyway.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4093
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@288 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 17, 2012
Commits on Jan 13, 2012
  1. Giving more details on assertion.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4081
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@287 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 13, 2012
  2. More tests for corruption in Python levelDB log reader.

    Now reproduces the corruption warnings and behaviors from C++ example cases that I could find.
    
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4080
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@286 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 13, 2012
  3. Properly handle ordering corruption in levelDB log reader.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4079
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@285 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 13, 2012
Commits on Jan 5, 2012
  1. Using thread local to store current mapper context.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4041
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@284 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 5, 2012
  2. Correctly initializing thread locals in all threads.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4040
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@283 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 5, 2012
Commits on Jan 4, 2012
  1. Fixing jquery css selector usages.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4036
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@282 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 4, 2012
  2. Internal change

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4035
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@281 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 4, 2012
  3. appengine_mapreduce: Switch to jquery-1.6.1.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4034
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@280 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 4, 2012
Commits on Jan 3, 2012
  1. Full combiner support for both poor-man and rich-man shufflers.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4028
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@279 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 3, 2012
  2. Revision created by MOE tool push_codebase.

    Synching external repo to internal repo.
    
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4027
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@278 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 3, 2012
  3. Cleaned up a few small instances where Exceptions were incorrect. e.g…

    …. raise NotImplemented instead of NotImplementedError
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4026
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@277 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 3, 2012
  4. Instrumenting various places in shuffler code.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4025
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@276 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 3, 2012
  5. Clone mapreduce and mapper parameters

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4024
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@275 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 3, 2012
  6. Fix access to blobstore entities when namespace is set.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4023
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@274 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 3, 2012
  7. Resetting namespace prior BlobInfo/BlobFileIndex lookup.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4022
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@273 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 3, 2012
  8. Better handling of empty datasets:

    - Datastore reader always generates a shard with namespace range
    - Kickoff job calls callback when job is finished.
    
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=4021
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@272 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Jan 3, 2012
Commits on Dec 15, 2011
  1. Never override active flag in shard state.

    Fixes issue 116.
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3972
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@271 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Dec 15, 2011
  2. Fixing open source test by not using C-extension methods.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3971
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@270 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Dec 15, 2011
  3. Breaking down large KeyValues outputs into multiple protos.

    This improves reading memory overhead and makes it possible to implement
    combiners.
    
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3970
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@269 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Dec 15, 2011
  4. Exporting counters from mapper pipeline.

    This will open way to exporting counters from mapreduce pipeline.
    
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3969
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@268 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Dec 15, 2011
  5. UI improvements:

    - JSON-ify properties
    - large value displayed in twisties
    - better graph display
    
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3968
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@267 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Dec 15, 2011
  6. [Cleanup] Removing unneeded shuffling format.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3967
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@266 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Dec 15, 2011
Commits on Nov 29, 2011
  1. Rich man shuffler support in mapreduce library.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3879
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@265 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Nov 29, 2011
  2. Make the default acl None / null so that the default bucket object acl

    can be utilized.
    Basically if no ACL is specified, the default acl specified for a bucket
    is used. This defaults to project-private as discussed in another thread.
    
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3878
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@264 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Nov 29, 2011
Commits on Nov 15, 2011
  1. Do not try to queue tasks in system queues.

    Fixes issue 120 in public tracker.
    
    
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3722
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@263 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Nov 15, 2011
Commits on Nov 10, 2011
  1. First cut of combiner support in poor man shuffler.

    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3668
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@262 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Nov 10, 2011
  2. Populate the filename when creating a file to match the new backend c…

    …hanges.
    
    Revision created by MOE tool push_codebase.
    MOE_MIGRATION=3667
    
    
    git-svn-id: https://appengine-mapreduce.googlecode.com/svn/trunk@261 441dd3cb-e453-d85f-364a-b28e0c74875c
    appengine.mirrorbot@gmail.com committed Nov 10, 2011