Move resources with redirect

[wouteraj]

As an app developer, I want to move a resource without invalidating the original URL.

[pietercolpaert]

Related issues in the specs

• Proposal: Support COPY use case solid/specification#19 ==> COPY + DELETE use case might work, but the problem is the original URL is not kept, which could be solved however by a redirect and an owl:sameAs or equivalent, but then you’d need to be able to infer this URI changed at some point
• API methods for copy/move/rename solid/solid-spec#156 ==> related issue without helpful answers for this issue (yet)

Proposal

In order to be able to move resources from one ldp:Container to another, and in order to support the idea of (i) Cool URIs don’t change and (ii) not potentially leaking the original container this resource has been part of, we need to have opaque URIs for resources by default.

Consequence: when POST-ing to a specific container, we should not take the container URI as the base URI to generate an identifier for this resource. We should just have an opaque URI based on the base URI of the Solid pod. This way, multiple containers could also contain the same ldp:Resource, or moving resources from one container to another is just a matter of updating container metadata (issue in the CSS about this: #1027).

This may have an impact on solid/specification#35 as this way the parent container cannot be automatically inferred based on URI semantics.

[pietercolpaert]

In this comment I’m reiterating what I said above, as my proposal didn’t explain all possible scenarios:

Moving a resource can entail two things

1. Renaming an ldp:Resource

The default strategy in CSS must be to create opaque IRIs without any semantics behind them (thus POST /container/ will result in a new resource that has a IRI /{serverdecided}). When the resource’s name changes, this won’t affect the IRI, as it would simply be PATCH-ing the rdfs:label’s value.

An optional strategy in CSS for more human readable IRIs is to add a slug when POST-ing that will be used by the CSS to create a IRI. This enables clients to have more control of the IRI-structure, and thus a client can decide the server-side identifier.

Since the client chose the slug, it is now in control of the identifier, and not the server. If the client wants to rename the resource to something else, it is up to the client to re-POST this entity with a different slug, remove the old one, and add a redirect resource in the old location with #162. The client may also add an owl:sameAs from the new IRI to the old IRI in the new resource.

For resolving the problem of inefficient use of bandwidth when POSTing the full resource all over again, I refer to issue solid/specification#19 - this is out of scope for a PR that fixes this one.

While I’m in favor of removing this from the Solid spec, a client may still implement URI slash semantics (https://solidproject.org/TR/protocol#uri-slash-semantics) if it wishes to based on the slugs, but the server should not be aware of this. Important limitation: using a hierarchical slug and cache semantics there would not be a way to make this container not contain this resource anymore.

2. Changing the container(s) an ldp:Resource is contained in

In 1, we made slash semantics a concern of the client. The server can thus without any problem update the metadata (if this becomes possible) of a container to indicate containment: it can remove the ldp:contains in the original container, and add it in the new one (#1027)

When POST is done on a container, it should create an opaque IRI that is not prefixed by the container’s IRI in order to not confuse developers in the future when this resource would have changed containers solid/specification#35 (comment)

Next steps before this issue can be closed

@joachimvh Is removing the container’s name from the POST location debatable? What changes would be needed to the code-base? Should I close this issue and open another one instead with this as its focus?

[joachimvh]

Going to go with a high-level answer first before going deeper into some specific parts and what implementation would require. Not sure if you just wanted me to reply to your last line, but it felt relevant I also mentioned everything else.

How this server relates to the Solid Specification

First of all I don't think that this is the relevant location to raise some of the issues that you're raising. This server is an implementation of the Solid specification. With many components that provide features and options that go beyond the spec yes, but at its core it does fully follow the spec (or at least we try to). So we can never add a feature/component here that breaks with that.

On the other hand, this server was designed to be easily adaptable, so it could be used to implement certain features before they are in the spec and then perhaps even be used as relevant examples in discussions there. But in case you want to add a feature that would make it so the server does not fully conform to the spec anymore, I feel that this should not be part of this repository but would be better suited in either a fork or a repository containing a specific module that can be injected using the Components.js configuration.

I'm not saying that (some of) the features here can't be implemented without breaking the spec, although I do think for some of them it would increase the internal complexity trying to balance between both requirements. I do also think we have to at least be a bit careful about user expectations, specifically on the file system. Currently there is a 1-to-1 mapping between how resources are stored on the file system, and many users expect that to be the case. AFAIK there are tools out there built on that expectation, just like for some it is expected that you can start the server on a specific folder on your system the files in there are immediately accessible through the expected URLs. Now this is not part of the spec, so I would actually be ok with changing that if there is a good and relevant reason to do so, but I do think we have to keep this in mind.

Another consequence of following the spec is that requirements specified there were not always designed to be modular, since changing that would cause the server to not implement Solid correctly anymore. This might also make it more difficult to support some of the changes suggested here.

Moving a resource

Your description of moving a resource contains several different feature changes that I'll try to separate here to make it clearer how each of them might impact the server. I might have missed some specifics or misinterpreted some, but I think these are the 3 most relevant points:

1. Create an opaque URI for resources created through POST.
2. Manage container containment triples.
3. (Optionally) Use the slug to let clients determine the URI structure so they can use slash semantics.

I'll cover what is needed to add them below. The problems covered are just those that I can immediately think of, chances are there are a few more I'm not immediately thinking of since these changes would touch many of the components of the server.

1. Opaque URLs when POSTing

I actually made the assumption here that you meant only having opaque URLs when doing a POST. In the case of a PUT or PATCH you can target a non-existing URL to create a new resource at that location. In case you do want to change PUT/PATCH there would be several other issues which I won't go into now.

Just creating the opaque URL would require a small change in the DataAccessorBasedStore in the function that creates the new URL. The main repercussion is that we lose the hierarchical nature of the URLs, meaning that when given a resource URL it would no longer be possible to determine the parent container. Some cases where this is relevant (there are probably more):

• No longer possible to find the relevant ACL resource in case a parent ACL is needed.
• No longer possible to determine if a resource is part of a specific pod. This is needed because pod owners always have control access to their resources.
  One solution to make this work would be let a resource store metadata of what its parent container is, but then you have duplication of (meta)data which brings its own risks.

The other issue is of course that this breaks with the current spec, so this would cause this server to not be a valid Solid server anymore.

2. Manage containment triples

This would mostly require changes in the DataAccessors since there are responsible for returning container metadata. For the memory and file data accessor it would require storing them explicitly instead of implicitly as is done right now. The SPARQL data accessor actually already stores them explicitly.

To actually add some sort of MOVE API to the server we probably would have to extend the ResourceStore interface to have a corresponding function. The DataAccessorBasedStore could then request metadata from the relevant containers and update their containment metadata.

3. Let clients determine the URL when they provide a slug

If the other points get implemented I'm actually not sure why you would need this. If clients want to choose the exact URL they can use PUT. This would actually make the spec stricter since there the slug header is defined as a suggestion the client can give to the server, but the server is not required to follow that suggestion. Hierarchical slugs are also not a thing right now (since the slug is just a suggestion to the server).

[pietercolpaert]

Thanks for the elaborate response!

Main take-aways for me:

1. Given persistent URIs and slash semantics, membership of a container is de facto permanent. We could however add another organizational layer such as SKOS, Hydra, LDN or TREE, as mentioned by @kjetilk in Add the semantics of slashes, which is shared by client and server solid/specification#35 (comment), that then can support moving virtually.
2. If we’d want to take URI Slash Semantics out of the Solid spec, Web ACL would break, which is also required in the spec. However ACP is upcoming, so maybe we can make an experimental branch where we test ACP, disable Web ACL and URI Slash semantics to then support moving resources on the LDP layer itself.

[joachimvh]

2. If we’d want to take URI Slash Semantics out of the Solid spec, Web ACL would break, which is also required in the spec. However ACP is upcoming, so maybe we can make an experimental branch where we test ACP, disable Web ACL and URI Slash semantics to then support moving resources on the LDP layer itself.

It's not only ACL that would break. Another example is Storage:

Clients can determine the storage of a resource by moving up the URI path hierarchy until the response includes a Link header with rel="type" targeting http://www.w3.org/ns/pim/space#Storage.

Which is also related to the issue of ownership I mentioned above besides the ACL issue.

[RubenVerborgh]

^ I think it would be a good exercise to list all things that depend on slash semantics, so we understand the consequences. Plus, everything in the specs that depends on slash semantics might want to link explicitly to the requirement (@csarven).

[acoburn]

However ACP is upcoming, so maybe we can make an experimental branch where we test ACP

ACP propagation (i.e., acp:memberAccessControl) relies on slash semantics

[pietercolpaert]

ACP propagation (i.e., acp:memberAccessControl) relies on slash semantics

@acoburn I should look more closely to ACP, but does this mean if slash semantics would disappear, that we’d need to start redesigning ACP from scratch? Or would this mean only some minor algorithm changes?

[acoburn]

Major vs minor changes depends entirely on context. Given the importance of access control in Solid, I would consider that to be a substantial re-envisoning of ACP. That said, changing slash semantics would be a major re-envisioning of all of Solid; TBH, I thought slash semantics was a settled point of discussion.

[kjetilk]

Yes, I think slash semantics is pretty settled. I also think that many reasons for moving things around should not require changing identifier, and that should rely on a knowledge organisation system on top of the containment hierarchy.

However, I think there will also be a residue, cases where changing the identifier is the right thing to do. I have thought that one (good) way to address that is by leaving behind an auxiliary resource (think of an .htaccess, but in RDF :-) ) that instructs the server to actually redirect (e.g. a 301) erect a tombstone (so, 410), etc.

I think we need to put effort into generalize auxiliary resources after we've published 0.9, and this could be one of the things that are addressed. We're not quite there, but soon! :-)

[pietercolpaert]

Ok, I will close this issue without action and resolve it by summarizing @kjetilk’s point: LDP containers as used within the Solid spec is a lower-level construct. In order to support functioanlity like moving resources, we should use higher level organizational specifications such as SKOS, Hydra, TREE or AS2.0.

While I still believe slash semantics are a mistake and using an extra level of knowledge management on top of LDP makes it more complex for app developers, I’m convinced by the above examples that fighting against it at this stage would just be lost time.