LocalDataSegmentPusher: Fix for Hadoop + relative paths.

[gianm]

This makes the path-for-hadoop stuff work with local-mode hadoop when you have a relative local storageDirectory (like simply "segments" if you want to store stuff in segments/ in your current working directory).

[drcrallen]

is it at all possible to use standard URI building things here? Either hadoop FS path building or the java File and Path constructs?

[gianm]

I was avoiding the Hadoop classes since this isn't a hadoop related module. Could probably use the Java stuff a bit more though, let me see

[gianm]

@drcrallen .toURI.toString works too, updated the patch. It generates different uris ("file:/x/y/z" rather than "file:///x/y/z") but they still work with hadoop.

[himanshug]

"It generates different uris ("file:/x/y/z" rather than "file:///x/y/z") but they still work with hadoop."
is that a hadoop bug :) or is that supposed to work really?

[gianm]

Do you mean, is it a bug that "file:/blah/blah/blah" works?

I don't think it's a bug. That's what you get when you do a toURI on a File in Java, so I think it makes sense that it should work in Hadoop. It also appears to also be OK by RFC 3986 in that "file:/blah" is a uri with scheme "file", no authority, and path "/blah". "file:///blah" is the same thing with an empty string authority instead of no authority, which should get interpreted the same way. At least in my reading of the thing.

      URI         = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

      hier-part   = "//" authority path-abempty
                  / path-absolute
                  / path-rootless
                  / path-empty

      path          = path-abempty    ; begins with "/" or is empty
                    / path-absolute   ; begins with "/" but not "//"
                    / path-noscheme   ; begins with a non-colon segment
                    / path-rootless   ; begins with a segment
                    / path-empty      ; zero characters

and

   If the URI scheme defines a default for host, then that default
   applies when the host subcomponent is undefined or when the
   registered name is empty (zero length).  For example, the "file" URI
   scheme is defined so that no authority, an empty host, and
   "localhost" all mean the end-user's machine, whereas the "http"
   scheme considers a missing authority or empty host invalid.

[himanshug]

LGTM
ok sounds like its not a hadoop bug but should work.