Windows file manager and OSX Finder fails on file names with comma

[jonasbardino]

One of our users wanted to upload a bunch of files with several exotic characters in file names. We managed to change our own code base to support most except ones with commas.
Apparently the native Microsoft Win 7 WebDAV client does not URL-encode paths passed in the
HTTP_AUTHORIZATION header and that causes troubles with the headerparser regex in http_authenticator.py.

An example HTTP_AUTHORIZATION value would be

'Digest
username="bardino@...",realm="/",nonce="MTQ...",uri="/test-enc/a,b.txt",cnonce="647...",nc=00000001,algorithm=MD5,response="af...",qop="auth"'

The regexp headerparser thus splits after the comma in /test-enc/a,b.txt causing auth errors like:

<140650507945728> [10:53:59.701] wsgidav.http_authenticator:
computeDigestResponse('/', 'bardino@...', ...):
4177dc6ee07fcaa2455855132cdeb7f8 != af0259a96cfd680c478b875e4b44e4f1

Testing the same with the cadaver client works and debug show that the paths get urlencoded there.

I'll see if I can provide a server side hotfix to address it.

[jonasbardino]

I found out that the same issue applies for OSX Finder and it can be fixed with the patch in pull request #24.

[mar10]

Cool you tracked that down.
Are you using a custom storage provider or run on Unix?
AFAIK on Windows (and OSX?) a ',' is not allowed for filenames, so I am wondering what happens if you download or WebDAV-edit such a file inside your browser or Fileexplorer on a windows box.

Did you try filenames with or quotes ("), equal sign (=), or double comma (,,)?

Anyway we should add a 'HOTFIX' comment near, so we can clean-up all those client fixes, document them or even make them configurable sometime later.
(I am also wondering, if we should emit a warning, if your fix kicks in?)

[jonasbardino]

We run our own fork of the wsgidav service on a few Linux servers in our grid setups. Thus in principle we could allow almost any characters on the server side, but we enforce some restrictions to prevent side effects when using the files in other contexts.

Comma is allowed in file names on Windows as long as you don't use the old FAT16/32 file systems:
https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations
However, to answer your question, a similar issue arises if you (by other means) create files with e.g. colon in the name. Then the Windows file manager refuses all access to the file with an error message. Fair enough really.
I tried various files with all kind of exotic characters on Win 7 and OSX. now most just work, like those with one or more commas, single quotes and equals signs. Double quote is illegal on both platforms, and we don't support it either, so that isn't a worry in terms of parsing.
Apart from this comma issue I noticed now that semicolon wreaks havoc from ALL clients in some operations. E.g. move 'abc' to 'abc;new' will result in an attempt to rename 'abc' to 'abc'. Again probably because of interference with query string separators. Maybe it's better to investigate and create a separate issue for that? Other query string separators like '&' may also cause problems in those operations.

I'm not 100% sure if the lack of url-encoding paths is actually a violation of the HTTP protocol but AFAICT the header values should be possible to just split on commas and that's not the case then.
Anyway, I can add a HOTFIX comment and a log warning when it is used - I'll make another pull request then.

[mar10]

excellent, I will merge then :)
(And from my point, those problems with operations esoteric characters like & or ';' can wait, until someone opens a concrete issue. Seems that nobody missed it so far).

btw. I am currently working (low priority) on a python3 port (#4). Is this something you could use?

[jonasbardino]

I tried to add the latest changes to the pull request - hope it works :-)

I'll probably dig into that other issue with semicolon etc. when I find the time but let's just leave it for now.

We are bound to python2 for a number of reasons so a python3 version is not urgent for us in any way, but it's always nice to be prepared for the future.

[mar10]

merged - thanks again!