Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Unexpected responses by the WebDAV-interface #1442
we're stumbling over an issue with responses of the WebDAV-interface that always returns the same contents, leading to an infinite recursion.
we are accessing the exist-instance via an nginx-proxy with this configuration:
here's an example from the xml-response to a web browser, but the behaviour is identical with a dav-client like the Oxygen editor is providing:
and so on..
looking at nginx' logs, we noticed that all subsequent requests after the initial one simply appended an additional
so, our first attempt for a workaround was this:
now, everything looks as expected in the web browser, but the dav-client always shows an additional (ghost) directory contained in any directory that has the same name as the containing one, but yields a proper
i can provide additional information as needed.
this is observed w/ eXist-db 3.2.0 / 691bcd6 and nginx 1.10.0 on a Ubuntu Xenial, but the issue is also known for previous versions in conjunction with the Apache httpd.
So from what we have seen (the webdav code is there for quite some time), the interface is pretty stable. When direct accessing webdav, there are very little issues.
So from my point, it looks like nginx is breaking stuff, and the solution should be found in nginx.
So the next step to do is to compare the actual HTTP calls between NGINX and eXist-db (headers, data) with the calls done when connected with eXist-db directly.
Another thing...... the output you report
is NOT part of the webdav specification! This XML document has been added as a convenience for an enduser, but it is not webdav. WebDAV is far more complex. This is not the interface Oxygen uses, But i agree it is the same data that Oxygen receives.
thanks for your responses.
as Apache httpd does the same, i'd consider the to be at least a documentation bug. are such deployments behind proxies intended to be supported anyway? another issue we're confronted with is that the web root response of an exist-instance contains a link to assets from the
yep, and that conveniance was easier to post here; the behaviour is identical with the xml output and a WebDAV client.
can you elaborate more specifically what you mean with 'mounted' here? on the server or on the client having mounted a webdav resource from or to the said paths?
shall we start with the endless recursion or with the ghost folders?
Well it is not part of a test plan, nor we (I?) intended to have these interfaces tested behind one of the many potential third party reverse proxies. The 'core' interface works, that should count :-)
when you mount the webDav interface from e.g. macOS Finder, the mount point should be on 'http://host:8080/exist/webdav/db/' as documented on http://exist-db.org/exist/apps/doc/webdav.xml ; From a long time ago [the extension has been developed 5 years ago!] I remember that when mounting on
The extension works for 5+ years already more or less without too many issues.....
okay, we looked at the issue with this condensed setup as nginx configuration (not including more general stuff):
any request that matches the first declaration works fine, the aliased access shows a ghost folder named
comparing both dimensions (working/non-working, proxied/unproxied data) reveals no differences in requests or responses; except for cookie related headers that shouldn't matter at all (dav-related rfcs only mention 'cooking recipes' once).
i also investigated other client's behaviour and so far Oxygen is the most forgiving with its ghost folders, others do not work at all on aliased locations.
i'll further look into posssible solutions and probably post a proposal to extend the docs next week.
okay, i tested the whole setup with another WebDAV-implementation as upstream server with exactly the same results. now i wonder,
referenced this issue
Jun 27, 2017
Same problem with Konqueror and curl. Reports an error, but whole file is actually transferred. To reproduce, open a webdav folder in Konqueror and try to copy an xml file to another folder with ctrl-c, ..., ctrl-v. You'll get an error popup, and a file called orig.xml.part, which actually has the whole file in it.
@htInEdin please specify exist-db version, curl versions and conqueror version. and the form of the webdav http end-point definition in your app.
control-c control-v sounds to me like a KDE specific operation, so maybe it is a bug in there? For sure I have no idea what webdav calls are done under the hood, please check the logging.
Dannes Wessels writes:
@htInEdin please specify exist-db version, curl versions and conqueror version
Versions: Exist-db: 4.4 curl: 7.29 Linux: Scientific Linux 7.5 Konqueror: 4.14.8 Qt: 4.8.7 KDE Pform: 4.14.8
and the form of the webdav http end-point definition in your app.
No idea about what curl or Konqueror are using in that regard -- how would I find out?
control-c control-v sounds to me like a KDE specific operation, so maybe it is a bug in there?
Not if curl exhibits the same problem. Also it's at least odd that this problem occurs with .xml files but not .js files, which again suggests the problem is not at the client end...
For sure I have no idea what webdav calls are done under the hood, please check the logging.
Will see what I can find tomorrow -- relevant machine is at work and I'm now at home. ht -- Henry S. Thompson, School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440 Fax: (44) 131 650-4587, e-mail: email@example.com URL: http://www.ltg.ed.ac.uk/~ht/ [mail from me _always_ has a .sig like this -- mail without it is forged spam] The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
@htInEdin what is the URL you are connecting to? (leave out the hostname for obvious privacy reasons) ; it is the webdav URL you typed in to connect with the database.
with the URL I should be able to re-play the scenario in curl. Please could you share the exact curl command with me?
@funkyfuture yes true, but I have no idea what is happening under the hood.
The URL which gives the bad behaviour in Konqueror is
The complete relevant curl session is
The file is indeed 401 bytes long, and it's all there.
ah ok. thnx. it is all clear for me now.
In exist-db there is not something like actual document size: the actual document size that is serialized out of the database model highly depends on the serialization parameters, indenting, inclusion of PIs etc etc.
So as a result, we make a guesstimation on the size by looking a the amount of datapages the document uses. nr of pages * size of page (4k) = total size. That explains the 4k value.
It is implemented this way for performance reasons. The only alternative is to pre-serialize the whole document and measure the size before the document is actually transferred. This is a very expensive operation.
Most webDAV clients work OK with this approach. Some clients don't. Or did and don't. Or didnt and do. You get the idea.
Leaving out the content-length header cannot be done by default, although the specs allows. Some clients really expect this header to be set.
So with the curl command nothing is wrong; the document is correctly transferred, you see the report that a content length was reported, but less bytes were effectively transferred.
So workaround is set the environment variable "org.exist.webdav.PROPFIND_METHOD_XML_SIZE" and/or "org.exist.webdav.GET_METHOD_XML_SIZE" to the value NULL, EXACT or APPROXIMATE
As you can read in the code, this part is a can of worms in the WEBDAV clients, it is never OK for every one....
OK, so, following your suggestion solves the original problem, but took three of us all morning to actually implement.
When you document this, you should explain how to do it. I believe the right way to do it is by editting
The change to