HDDS-9804. ozone freon dfsg does not work with relative --path#5704
HDDS-9804. ozone freon dfsg does not work with relative --path#5704adoroszlai merged 2 commits intoapache:masterfrom
Conversation
Change-Id: I5829de1d89dbc203e41d7c07d310435e1d683eb5
| String scheme = uri.getScheme(); | ||
| if (scheme == null) { | ||
| throw new IllegalArgumentException("--path requires FQDN"); | ||
| } |
There was a problem hiding this comment.
Shouldn't we use the default FS in that case? If no FS is specified explicitly, the defaultFs should be used, right? Something like this:
if (scheme == null) {
URI defaultUri = FileSystem.getDefaultUri(configuration)
uri = new URI(defaultUri.getScheme(), defaultUri.getAuthority(), uri.getPath());
}
There was a problem hiding this comment.
We don't even need to completely replace uri, can use scheme of default URI only for disabling the cache.
| String scheme = uri.getScheme(); | ||
| if (scheme == null) { |
There was a problem hiding this comment.
does this work?
if (!uri.isAbsolute()) {
adoroszlai
left a comment
There was a problem hiding this comment.
Thanks @jojochuang for finding this bug, and working on the fix.
file system cache is not disabled properly and all threads share the same FS object
Instead of forcing full URLs, we should fix the way cache is disabled.
| String scheme = uri.getScheme(); | ||
| if (scheme == null) { | ||
| throw new IllegalArgumentException("--path requires FQDN"); | ||
| } |
There was a problem hiding this comment.
We don't even need to completely replace uri, can use scheme of default URI only for disabling the cache.
| String disableCacheName = String.format("fs.%s.impl.disable.cache", | ||
| uri.getScheme()); |
There was a problem hiding this comment.
| String disableCacheName = String.format("fs.%s.impl.disable.cache", | |
| uri.getScheme()); | |
| String scheme = Optional.ofNullable(uri.getScheme()) | |
| .orElseGet(() -> FileSystem.getDefaultUri(configuration).getScheme()); | |
| String disableCacheName = String.format("fs.%s.impl.disable.cache", scheme); |
(also add import java.util.Optional)
Or can use if block, if you prefer.
There was a problem hiding this comment.
Maybe should also fail if scheme is file.
adoroszlai
left a comment
There was a problem hiding this comment.
Thanks @jojochuang for updating the patch.
|
Thanks @jojochuang for the patch, @ayushtkn, @nandakumar131 for the review. |
What changes were proposed in this pull request?
Add check to prevent invalid --path parameter in ozone freon dfsg command.
Please describe your PR in detail:
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-9804
How was this patch tested?
Manually tested on a cluster.