Unexpected responses by the WebDAV-interface #1442
Comments
So from what we have seen (the webdav code is there for quite some time), the interface is pretty stable. When direct accessing webdav, there are very little issues. So from my point, it looks like nginx is breaking stuff, and the solution should be found in nginx. So the next step to do is to compare the actual HTTP calls between NGINX and eXist-db (headers, data) with the calls done when connected with eXist-db directly. |
Another thing...... the output you report
is NOT part of the webdav specification! This XML document has been added as a convenience for an enduser, but it is not webdav. WebDAV is far more complex. This is not the interface Oxygen uses, But i agree it is the same data that Oxygen receives. |
I think I remember that I have seen ghosts directories a long time ago when things are mounted on |
thanks for your responses.
as Apache httpd does the same, i'd consider the to be at least a documentation bug. are such deployments behind proxies intended to be supported anyway? another issue we're confronted with is that the web root response of an exist-instance contains a link to assets from the
yep, and that conveniance was easier to post here; the behaviour is identical with the xml output and a WebDAV client.
can you elaborate more specifically what you mean with 'mounted' here? on the server or on the client having mounted a webdav resource from or to the said paths?
shall we start with the endless recursion or with the ghost folders? |
Well it is not part of a test plan, nor we (I?) intended to have these interfaces tested behind one of the many potential third party reverse proxies. The 'core' interface works, that should count :-)
when you mount the webDav interface from e.g. macOS Finder, the mount point should be on 'http://host:8080/exist/webdav/db/' as documented on http://exist-db.org/exist/apps/doc/webdav.xml ; From a long time ago [the extension has been developed 5 years ago!] I remember that when mounting on The extension works for 5+ years already more or less without too many issues..... |
okay, we looked at the issue with this condensed setup as nginx configuration (not including more general stuff):
any request that matches the first declaration works fine, the aliased access shows a ghost folder named comparing both dimensions (working/non-working, proxied/unproxied data) reveals no differences in requests or responses; except for cookie related headers that shouldn't matter at all (dav-related rfcs only mention 'cooking recipes' once). i also investigated other client's behaviour and so far Oxygen is the most forgiving with its ghost folders, others do not work at all on aliased locations. i'll further look into posssible solutions and probably post a proposal to extend the docs next week. |
okay, i tested the whole setup with another WebDAV-implementation as upstream server with exactly the same results. now i wonder,
|
hi, if something is really not working, as you investigated, some explanation in the documentation makes sense. For the second point, I am not sure what you exactly want or need? |
the second thing would be a 'native' support by eXist-db to facilitate aliased access to WebDAV resources. but it's propably too much effort as a 3rd-party library is used for the WebDAV service, right? |
Same problem with Konqueror and curl. Reports an error, but whole file is actually transferred. To reproduce, open a webdav folder in Konqueror and try to copy an xml file to another folder with ctrl-c, ..., ctrl-v. You'll get an error popup, and a file called orig.xml.part, which actually has the whole file in it. |
@htInEdin please specify exist-db version, curl versions and conqueror version. and the form of the webdav http end-point definition in your app. control-c control-v sounds to me like a KDE specific operation, so maybe it is a bug in there? For sure I have no idea what webdav calls are done under the hood, please check the logging. |
Dannes Wessels writes:
@htInEdin please specify exist-db version, curl versions and conqueror
version
Versions:
Exist-db: 4.4
curl: 7.29
Linux: Scientific Linux 7.5
Konqueror: 4.14.8
Qt: 4.8.7
KDE Pform: 4.14.8
and the form of the webdav http end-point definition in your app.
No idea about what curl or Konqueror are using in that regard -- how
would I find out?
control-c control-v sounds to me like a KDE specific operation, so
maybe it is a bug in there?
Not if curl exhibits the same problem. Also it's at least odd that this
problem occurs with .xml files but not .js files, which again suggests
the problem is not at the client end...
For sure I have no idea what webdav calls are done under the hood,
please check the logging.
Will see what I can find tomorrow -- relevant machine is at work and I'm
now at home.
ht
--
Henry S. Thompson, School of Informatics, University of Edinburgh
10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
Fax: (44) 131 650-4587, e-mail: ht@inf.ed.ac.uk
URL: http://www.ltg.ed.ac.uk/~ht/
[mail from me _always_ has a .sig like this -- mail without it is forged spam]
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
|
@htInEdin could you also post your proxy configuration and the Jetty config?
i think this refers to keyboard commands for clipboard interactions. |
@htInEdin what is the URL you are connecting to? (leave out the hostname for obvious privacy reasons) ; it is the webdav URL you typed in to connect with the database. with the URL I should be able to re-play the scenario in curl. Please could you share the exact curl command with me? @funkyfuture yes true, but I have no idea what is happening under the hood. |
The URL which gives the bad behaviour in Konqueror is The complete relevant curl session is
The file is indeed 401 bytes long, and it's all there. |
No proxy, not sure which file you mean for jetty config, but all xml files under |
ah ok. thnx. it is all clear for me now. In exist-db there is not something like actual document size: the actual document size that is serialized out of the database model highly depends on the serialization parameters, indenting, inclusion of PIs etc etc. So as a result, we make a guesstimation on the size by looking a the amount of datapages the document uses. nr of pages * size of page (4k) = total size. That explains the 4k value. It is implemented this way for performance reasons. The only alternative is to pre-serialize the whole document and measure the size before the document is actually transferred. This is a very expensive operation. Most webDAV clients work OK with this approach. Some clients don't. Or did and don't. Or didnt and do. You get the idea. Leaving out the content-length header cannot be done by default, although the specs allows. Some clients really expect this header to be set. So with the curl command nothing is wrong; the document is correctly transferred, you see the report that a content length was reported, but less bytes were effectively transferred. so I wrote in the sourcecode some instructions how to work around this. But you are right, this is missing in the documentation. So workaround is set the environment variable "org.exist.webdav.PROPFIND_METHOD_XML_SIZE" and/or "org.exist.webdav.GET_METHOD_XML_SIZE" to the value NULL, EXACT or APPROXIMATE As you can read in the code, this part is a can of worms in the WEBDAV clients, it is never OK for every one.... |
OK, so, following your suggestion solves the original problem, but took three of us all morning to actually implement. When you document this, you should explain how to do it. I believe the right way to do it is by editting
with
The change to |
@htInEdin Would you be able to send a Pull Request to improve our documentation? https://github.com/exist-db/documentation |
Will try, but given the way the code is written it's not easy. Am I allowed to suggest reverting a change to the code (in |
Dear @htInEdin We have a similar problem here. Our Oxygen Author cannot open files from the eXist-db when they are larger than 1 MB. In this case, Oxygen creates a Truncated Chunk error. We described our problem already in the Oxygen forum here: https://www.oxygenxml.com/forum/topic20090.html#p54080 Oxygen support gave me a reference to your solution here above, working with $EXIST_HOME/vm.properties but it didn't help. Have you ever noticed such a Truncated Chunk problem? |
we're stumbling over an issue with responses of the WebDAV-interface that always returns the same contents, leading to an infinite recursion.
we are accessing the exist-instance via an nginx-proxy with this configuration:
here's an example from the xml-response to a web browser, but the behaviour is identical with a dav-client like the Oxygen editor is providing:
http://ourhost/collections
yet
http://ourhost/collections/OurProject
yields the same:and so on..
looking at nginx' logs, we noticed that all subsequent requests after the initial one simply appended an additional
/db
to the uri.so, our first attempt for a workaround was this:
now, everything looks as expected in the web browser, but the dav-client always shows an additional (ghost) directory contained in any directory that has the same name as the containing one, but yields a proper
404
when requesting its contents.i can provide additional information as needed.
this is observed w/ eXist-db 3.2.0 / 691bcd6 and nginx 1.10.0 on a Ubuntu Xenial, but the issue is also known for previous versions in conjunction with the Apache httpd.
The text was updated successfully, but these errors were encountered: