[FLINK-8475][config][docs] Integrate FS options #5459
Conversation
| .key("fs.default-scheme") | ||
| .noDefaultValue(); | ||
| .noDefaultValue() | ||
| .withDescription("The default filesystem scheme, used for paths that do not declare a scheme explicitly."); |
There was a problem hiding this comment.
Why does this not have the whole long text that the option has in the current documentation?
There was a problem hiding this comment.
IMO the missing text isn't really needed, but I'll add it back to keep this PR as a straight port of the existing docs and merge it afterwards.
There was a problem hiding this comment.
ah I remember now, looking at the description:
[Part1]
The default filesystem scheme to be used
[Part2]
, with the necessary authority to contact, e.g. the host:port of the NameNode in the case of HDFS (if needed).
[Part3]
By default, this is set to file:/// which points to the local filesystem. This means that the local filesystem is going to be used to search for user-specified files without an explicit scheme definition.
[Part4]
This scheme is used ONLY if no other scheme is specified (explicitly) in the user-provided URI.
Part 1 and 4 are contained in the description, part 3 was left out since file:/// isn't the documented default. Only part 2 is really missing.
There was a problem hiding this comment.
Ah, because we have no default in here but isn't the FileSystem code using that as the default fallback?
There was a problem hiding this comment.
Yes, in a practical sense "file:///" is the default due to how FileSystem works. We can't define it as the official default at the moment because of OS compatibility, see LocalFileSystem:
LOCAL_URI = OperatingSystem.isWindows() ? URI.create("file:/") : URI.create("file:///");
Not sure whether that is really necessary though, but the scheme parsing by Paths is a bit wonky on Windows.
There was a problem hiding this comment.
Ok, then I'd say merge it like this.
aljoscha
left a comment
aljoscha
left a comment
There was a problem hiding this comment.
Looks good except that one question I had.
|
merging |
3 participants
What is the purpose of the change
This PR ports the fileystem ConfigConstants to
ConfigOptionsand integrates them into the configuration docs generator.Brief change log
config.md