Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

URL encoding of directories #19

Closed
hilljb opened this issue Jun 17, 2015 · 4 comments
Closed

URL encoding of directories #19

hilljb opened this issue Jun 17, 2015 · 4 comments

Comments

@hilljb
Copy link

hilljb commented Jun 17, 2015

Many times, you need characters such as '=' in your HDFS path names. For instance, a partitioned Hive table can have '../year=2015/month=6/day=17' as the path name of a directory. Impala, for instance, uses this naming convention by default for partitioned tables.

Using hdfs.InsecureClient(url, user).list(hdfs_path) on such a path results in

HdfsError: File ../year%3D2015/month%3D6/day%3D30 does not exist.

@mtth
Copy link
Owner

mtth commented Jun 17, 2015

Thanks for pointing this out. Let me know if you know of any special characters other than = that we shouldn't be encoding. I'll push a fix to stop encoding them (reserved characters usually need to, see #11).

@mtth mtth closed this as completed Jun 17, 2015
@mtth mtth reopened this Jun 17, 2015
@mtth
Copy link
Owner

mtth commented Jun 18, 2015

= won't be encoded after 6581150 (version >=1.0.1). Feel free to reopen if you need other characters.

@mtth mtth closed this as completed Jun 18, 2015
@hilljb
Copy link
Author

hilljb commented Jun 18, 2015

Awesome! Thanks!

Off the top of my head, I can't think of any other characters that would be affected by this.

@sbakiu
Copy link

sbakiu commented May 7, 2017

I am having the same issue for directory names containing german special characters: ä,ö,ü,ß,Ä,Ö,Ü. Maybe you could include these characters also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants