dumbo cat silently fails when JobHistory logging is enabled #19

Closed
emf opened this Issue Sep 17, 2010 · 1 comment

Projects

None yet

2 participants

@emf

Causes frustration with new users that don't know why it's mysteriously failing and haven't figured out either of the workarounds yet.

proposed patch included.. perhaps not the best solution, but better than failing silently.

diff --git a/dumbo/backends/streaming.py b/dumbo/backends/streaming.py
index f71e611..7755fbe 100644
--- a/dumbo/backends/streaming.py
+++ b/dumbo/backends/streaming.py
@@ -230,6 +230,8 @@ class StreamingFileSystem(FileSystem):
                 subpaths = [path]
             ls.close()
             for subpath in subpaths:
+                if subpath.endswith("/_logs"):
+                    continue
                 dumptb = os.popen('%s %s/bin/hadoop jar %s dumptb %s 2> /dev/null'
                                   % (hadenv, self.hadoop, streamingjar, subpath))
                 ascodeopt = getopt(opts, 'ascode')
@klbostee
Owner

bugfix contributed by Erik Fichtner (closed by 550528b)

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment