Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add extra layer in DAV to accomodate for other services like versions, trashbin, etc #12543

Closed
PVince81 opened this issue Dec 2, 2014 · 51 comments

Comments

@PVince81
Copy link
Contributor

PVince81 commented Dec 2, 2014

See #12504 (comment)

The idea is to add an extra subdir in the WebDAV URL.
Since there is a risk of breakage if we do it in the existing https://domain.tld/owncloud/remote.php/webdav then I suggest to add another entry point called "dav":

  • files: https://domain.tld/owncloud/remote.php/dav/files
  • versions: https://domain.tld/owncloud/remote.php/dav/files_versions
  • trashbin: https://domain.tld/owncloud/remote.php/dav/files_trashbin
  • favorites: https://domain.tld/owncloud/remote.php/dav/files_favorites
  • get files by tag: https://domain.tld/owncloud/remote.php/dav/tags/tagname/
  • thumbnails: https://domain.tld/owncloud/remote.php/dav/thumbnails
  • calendar: https://domain.tld/owncloud/remote.php/dav/calendars
  • contacts: https://domain.tld/owncloud/remote.php/dav/addressbooks
  • chunks: https://domain.tld/owncloud/remote.php/dav/files_chunks/folder/file.txt/{transferid}/{chunkIndex}
  • users: https://domain.tld/owncloud/remote.php/dav/users
  • groups: https://domain.tld/owncloud/remote.php/dav/groups
  • ...

and we keep the old WebDAV one https://domain.tld/owncloud/remote.php/webdav for backwards compatibility.

Note: I guess some of the "tags/favorites" can probably be achieved using a "SEARCH" method instead.

@PVince81
Copy link
Contributor Author

PVince81 commented Dec 2, 2014

That's really cool, it means that tag management could simply be done by getting/posting/putting/deleting to: https://domain.tld/owncloud/remote.php/dav/tags/

@DeepDiver1975
Copy link
Member

That's really cool, it means that tag management could simply be done by getting/posting/putting/deleting to: https://domain.tld/owncloud/remote.php/dav/tags/

which will then finally be our REST-API 🙊

@DeepDiver1975
Copy link
Member

and we keep the old WebDAV one https://domain.tld/owncloud/remote.php/webdav for backwards compatibility.

I like that 👍

@icewind1991
Copy link
Contributor

How would those api's map to webdav?

(Also we can drop the /dav and just use remote.php/tags etc as dav enpoints)

@DeepDiver1975
Copy link
Member

how to bring in pure share information?

https://domain.tld/owncloud/remote.php/dav/shares

@DeepDiver1975
Copy link
Member

(Also we can drop the /dav and just use remote.php/tags etc as dav enpoints)

but then you need to duplicte a hell of code to fure up the sabre server - furthermore the used objects have dependencies - e.g. the principal route I'm referring to - see #12504 (comment)

furthermore - there is one webdav entrypoint then - this is desired

@schiessle
Copy link
Contributor

We are talking about files only, right? Or could https://domain.tld/owncloud/remote.php/dav/ also be the entry point for additional caldav/cardav properties?

If we talk about files only I would remove files_ and just call them https://domain.tld/owncloud/remote.php/dav/versions and so on.

@schiessle
Copy link
Contributor

how to bring in pure share information?

Yes, I think it would make a lot of sense to use the same API for shares.

@DeepDiver1975
Copy link
Member

We are talking about files only, right? Or could https://domain.tld/owncloud/remote.php/dav/ also be the entry point for additional caldav/cardav properties?

I already updated the description above - everything should be under one url - refs #5939

@icewind1991
Copy link
Contributor

I'm not sure if putting everything under the same dav root makes it easier than making things a seperate dav endpoint, things like card and caldav need to setup sabre/dav in a different way then regular webdav.

Trying to move all the different backend logics in a single dav enpoint would only make things complicated imo

@PVince81
Copy link
Contributor Author

PVince81 commented Dec 2, 2014

This is something that we should probably experiment at some point (hence the "research" label).
This will probably need some sub-routing to allow apps to plugin under "remote.php/dav".
Every such sub-route would need to register the appropriate Sabre plugins which might include app-specific/item-specific plugins (for ex for tags).

@icewind1991 if carddav/caldav wouldn't work well, would having tags/sharing and others still work ? Or did you mean none of them would work ?

@PVince81
Copy link
Contributor Author

PVince81 commented Dec 2, 2014

@evert pointed out here #12504 (comment) that we could also include the user name in the WebDAV URL.

This also means that we could have truly unique resource URLs.

@icewind1991
Copy link
Contributor

Other might work and even card/caldav can probably be made to work with it, but I think that instead of trying to push separate backends in a single dav endpoint we should try and reduce the boiler place needed to setup new dav endpoints and make it as easy as possible for apps to do that.

sabre/dav provides a great system for creating multiple dav endpoints with seperate logic behind it, implementing the same thing on another layer seems like a waste

@schiessle
Copy link
Contributor

FWIW, I tend to agree with Robin. Having separate endpoints for separate backends makes it a lot easier to define a nice API which only contains stuff which is really needed for the given backend. For example calendar and contact doesn't have a trashbin or versions. If everything uses the same endpoint it would depend on the resource you access which service exists and which not. Or you will specify the backend indirectly with URLs like .../dav/files_versions. Further we would have to make sure that we name the resources in a way that it is unique across all back-ends. Probably it would be nicer to have separate caldav, carddav and webdav endpoints and every endpoints defines the services which are really needed in the best possible way for the backend.

Maybe I'm influences by the share API, but I think it makes it much easier for both the developer who write the API and the developer who will use it later if you keep it separated and create the best possible experience for each endpoint.

@evert
Copy link

evert commented Dec 2, 2014

I'm curious what the perceived drawback is of splitting these up. There's definitely some overhead to combining all endpoints into a single sabredav server.php, but it's not that much.

You would still only have some functionality at segments of your API instead of everywhere, but you also get some benefits... both caldav and carddav need principals, and if one day you also want to support RFC3744 (ACL) on files, you're already prepared for it.

You can totally just push down specific functionality (versions, trash) further down the tree.

Is it possible that this issue exist because all functionality is provided by one or a few node-classes in owncloud?

@DeepDiver1975
Copy link
Member

Is it possible that this issue exist because all functionality is provided by one or a few node-classes in owncloud?

as of today we have three node implementation:

  • files
  • calender
  • contacts

we will for sure add more implementations for all these various resources as described above - in case we walk this path - to-be-decided

@PVince81
Copy link
Contributor Author

PVince81 commented Dec 9, 2014

Things to think about:

  • "remote.php" => "dav.php" ? "api.php" ?
  • integration with app framework (middleware?)
  • how external apps can hook into existing endpoints (add methods, properties, etc), possibly using SabreDAV plugins
  • wrap real objects (ex: User) into Sabre Nodes
  • content-type to return JSON: (see Replace WebDAV with REST #12504 (comment))
  • whether to use a single instance of SabreDAV or multiple instances

@DeepDiver1975
Copy link
Member

regarding the discussion about using a single sabredav object to handle all resource - please have a look at this code piece from the group-office project. This is an implementation of the root collection:

http://sourceforge.net/p/group-office/code/HEAD/tree/trunk/www/modules/dav/fs/RootDirectory.php

with some dynamic function we can provide the same mechanism for us - I see no need to duplicate the work in multiple endpoints

@icewind1991
Copy link
Contributor

But they dont seem to use a custom tree implementation

@DeepDiver1975
Copy link
Member

But they dont seem to use a custom tree implementation

well - they do - but their object tree holds no special logic for files ... I see your point now ... let's see how this evolves 🙈

@PVince81
Copy link
Contributor Author

I think multiple DAV instances is the way to go, as each instance can have different plugins.

I did an experiment here with a tags endpoint, for fun: #12867

@DeepDiver1975
Copy link
Member

I think multiple DAV instances is the way to go, as each instance can have different plugins.

maybe add a very simple instance to the root of the entrypoint which lists all available resources.

regarding the routing of that new dav entrypoint we have multiple options:

  • reuse remote.php and the remote tag in info.xml
  • introduce a new tag in info.xml
  • extend our current routing capabilities in routes.php and App::registerRoutes - which implies the entry piont will be index.php
  • introduce a new entrypoint dav.php or dav/dav.php - later might help to not clutter the root namespace

@schiessle
Copy link
Contributor

I still find it a bit confusing that we have so much files related end points on the same level, wouldn't it make sense to group them all below https://domain.tld/owncloud/remote.php/dav/files ?

So that we would have

  • https://domain.tld/owncloud/remote.php/dav/files
  • https://domain.tld/owncloud/remote.php/dav/files/versions
  • https://domain.tld/owncloud/remote.php/dav/files/deleted
  • https://domain.tld/owncloud/remote.php/dav/files/favorites
  • https://domain.tld/owncloud/remote.php/dav/files/tags
  • https://domain.tld/owncloud/remote.php/dav/files/thumbnails
  • https://domain.tld/owncloud/remote.php/dav/files/chunks
  • https://domain.tld/owncloud/remote.php/dav/calendars
  • https://domain.tld/owncloud/remote.php/dav/addressbooks
  • https://domain.tld/owncloud/remote.php/dav/users
  • https://domain.tld/owncloud/remote.php/dav/groups

@DeepDiver1975
Copy link
Member

https://domain.tld/owncloud/remote.php/dav/files
https://domain.tld/owncloud/remote.php/dav/files/versions

this will not work because under files the files will be listed and this will conflict with files/versions

@PVince81
Copy link
Contributor Author

I think we also need to give a good though about how to represent relations / how to link entities of different type. For example file <-> tags, file <-> versions.

One of the trickiest parts could be the sharing API.

Also:

  • how to sort ?
  • how to do pagination ?

@MorrisJobke
Copy link
Contributor

@PVince81 the pagination is implemented by @butonic via the search spec of WebDAV

@butonic
Copy link
Member

butonic commented Dec 19, 2014

@PVince81 see #12884 for a research PR that implements SEARCH. DAV:basicsearch has several components:

o DAV:select provides the result record definition.
o DAV:from defines the scope.
o DAV:where defines the criteria.
o DAV:orderby defines the sort order of the result set.
o DAV:limit provides constraints on the query as a whole.

@PVince81
Copy link
Contributor Author

Does it mean "SEARCH" needs to be used in all cases, as replacement for PROPFIND ? (some kind of paginated PROPFIND ?)

@DeepDiver1975 DeepDiver1975 modified the milestone: backlog Mar 21, 2015
@PVince81
Copy link
Contributor Author

Some thoughts regardfing the versions endpoint: #10571

@PVince81
Copy link
Contributor Author

Raised #16524 for @butonic's suggestion of using http-patch from Sabre for chunking

@PVince81
Copy link
Contributor Author

@icewind1991 I see your point about CalDAV/CardDAV needing to setup Sabre differently.
Maybe we can provide a utility method that creates a Sabre server with the basic plugins (MaintenanceCheckPlugin, Auth, etc) and let the implementer add their additional plugins ?

If you go the single endpoint route proposed by @DeepDiver1975, it would mean that every plugin needs to be able to filter itself out by checking if the path is actually for them. For example a CardDAV plugin would need to check whether the requested route is indeed under "dav/carddav". If yes, then go on, else return. I don't think plugins should be aware of that, so for now I'd rather go with @icewind1991's suggestion and a utility method.

@evert
Copy link

evert commented May 25, 2015

Note that all sabre/dav plugins are already built to only add their own behavior for relevant paths. Generally they don't really do that based on the path though, but usually:

  1. the type of request (MKCALENDAR is only relevant for CalDAV).
  2. the content-type of the request
  3. the type of node (calendaring-related behavior would kick in on nodes implementing Sabre\CalDAV\ICalendar).

To a certain extent the system is designed to do this, but I do agree that there's a bit of overhead. There's also great advantages though. If you follow both the 'single root' and the 'embed the user id in the url' rules, it means that you can actually use the 'path' of any resource as a primary key for many things, which is really the correct way to do a lot of this stuff.

@PVince81
Copy link
Contributor Author

In the case of a mixed tree, I guess it might work as we'd have an implementation of ObjectTree that sits at the base and delivers different node implementations for different paths. So if the plugins are all working with specific node types (ex: trash plugin working with TrashedFile nodes), then it could work.

Note that in the case of trashed files or versions, we wouldn't invent a new content-type but reuse the original file's content-type, so content-type based detection would not work here.

@evert
Copy link

evert commented May 26, 2015

Hm.. can you get rid of the custom objecttree implementation? I think your implementation might be a lot simpler if you do. All you need is a root node that correctly instantiates these sub-nodes, instead of doing it on the objecttree-level.

@PVince81
Copy link
Contributor Author

Right, the root node. I think I need to do more exercises with Sabre 😉

@PVince81
Copy link
Contributor Author

@DeepDiver1975 you forgot to advertise your POC here 😉

https://github.com/DeepDiver1975/dav

@oparoz
Copy link
Contributor

oparoz commented Aug 15, 2015

So, will part of this be in 8.2?

I will be able to get the list of files contained in a folder from my mobile app, posting the path to
https://domain.tld/owncloud/remote.php/dav/files/list
and get a JSON response?

Will uploading be supported?
https://domain.tld/owncloud/remote.php/dav/files/upload

If the endpoint is going to change, thumbnails should be replaced by preview
https://domain.tld/owncloud/remote.php/dav/preview/1024/1024/jpg

@PVince81
Copy link
Contributor Author

Unlikely.

The linked repo is just a proof of concept and more design/research work needs to be done to decide what the best approach is for the endpoints.

@PVince81
Copy link
Contributor Author

@DeepDiver1975 the new layer has been added in 9.0.
Closing.

The other services/endpoints can be added as part of their respective apps/features.

@PVince81
Copy link
Contributor Author

Well, unless we want to keep this ticket to discuss possible extensions like search and returning JSON objects ? I'll reopen 😄

@PVince81 PVince81 reopened this Dec 18, 2015
@Yetangitu
Copy link

Is there any documentation - or code which could serve as such - on how to add a new endpoint from within an application?

@PVince81
Copy link
Contributor Author

PVince81 commented Mar 9, 2016

There is currently no public API for external apps to add their own endpoints in "remote.php/dav".
Currently all endpoints there are implemented by core features.

@DeepDiver1975 we need to discuss this.

For now, apps should still register endpoints like they did before under "remote.php/myappendpoint" pre 9.0.

@oparoz
Copy link
Contributor

oparoz commented Mar 9, 2016

I think it's a good thing because if an app starts registering preview, thumbnails or even upload, it will get messy unless there is a blacklist in place

@PVince81
Copy link
Contributor Author

We already have the layer, closing in favor of #20047

@lock
Copy link

lock bot commented Aug 3, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Aug 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

10 participants