Is Solid containment strictly hierarchical?

[kjetilk]

Another topic that @csarven and I discussed is if the Solid view of LDP is a strictly hierarchical structure.

LDP itself does not require this, and Linked Data itself does not tend towards trees, but we are talking mostly in terms of a folder hierarchy where the presence of / in the request URI carries significant information about the parent-child relationship.

Therefore, we should clarify whether we intend the folder hierarchy to be strictly so. This has great implications for the design, and influences issues like #39 and #68 .

[kjetilk]

I'll just share some of my initial thoughts around this, hope we can quickly have a discussion on this.

I would normally not advocate strict hierarchies, but I have come to view this as a possibility, and something that could be a constraint on LDP that can be relaxed later that could have several simplifying effects.

The cost of constraining Solid this way doesn't need to be very dramatic. Even though strict hierarchical systems have major shortcomings for knowledge organization, in my experience, we can very well build a non-hierarchical knowledge organization system on the top of a hierarchical containment hierarchy. The containment hierarchy simply does not need to play any important role in knowledge organization. It seems to me the containment hierarchy is mostly important to apps that group their writes and for the ACL system to assign common authorizations to those resources.

In a hierarchical system where identifiers for parent containers can be determined simply by looking at the request URI, containment triples can very easily be computed on the fly for an incoming request, avoiding atomicity problems that might arise from more advanced structures that might need to persist information about the containment in some other form.

[jaxoncreed]

I think it would be a good idea to make the container structure optional in Solid so any resource can go at any path. For example if I put a resource at /a/b/c/resource.ttl, Solid will not required that /a, /a/b and /a/b/c be containers. It could very well set it up that way, but it would be optional.

Let's imagine that we are creating a web-site that we want to be Solid compatible. Essentially, our site would look the same for a user using a web browser, but when requests are sent to urls with a accept: text/turtle header, turtle will be returned. It would be fantastic if eventually all web-sites were Solid compatible.

However, many sites have url schemes that don't follow a container structure. For example, on reddit, a url might look like https://www.reddit.com/r/SOLID/, but reddit.com/r doesn't exists. It redirects to reddit.com/subreddits, so technically the container that holds reddit.com/r/solid is actually at reddit.com/subreddits.

Also a site might want to use the scheme /resource/edit to show an edit ui. Technically, asking for turtle on /resource would should return the resource itself and /resource/edit should return a form ontology. But, you couldn't put a form ontology in /resource/edit because /resource is not a container.

Finally, allowing a resource to be at any url independent of containers would encourage containers that contain resources that are on completely different servers, which is much more in line with the decentralized nature of Solid.

[dmitrizagidulin]

My personal vote would be in the direction of required hierarchical containers. Solid's core model so far has been - it's a web-ization of the file system, directories and files. Which are necessarily hierarchical.

[justinwb]

The cost of constraining Solid this way doesn't need to be very dramatic. Even though strict hierarchical systems have major shortcomings for knowledge organization, in my experience, we can very well build a non-hierarchical knowledge organization system on the top of a hierarchical containment hierarchy. The containment hierarchy simply does not need to play any important role in knowledge organization. It seems to me the containment hierarchy is mostly important to apps that group their writes and for the ACL system to assign common authorizations to those resources.

Agree completely with @kjetilk here. I think we can provide something along these lines that is simple for development, intuitive for authorization, but also leaves us with room to expand long term. We are experimenting with exactly this as part of the work in the data interoperability panel related to the intersection of discovery, validation, shapes, footprints, and authorization.

[csarven]

Bi-directional linking can accommodate hierarchical linking.

We have the notion of parent-to-child but not child-to-parent - that's one thing that can be formalised. Servers wishing to expose either or both of those kinds of connections between its resources should be able to do so.

[csarven]

What are the reasons to prohibit non-hierarchical containment in a given system?

As @jaxoncreed highlights the scenario where servers manage resources in containers which are not necessarily in the same storage / system should be recognised as within the scope of Solid.

[kjetilk]

Just to add some context from @timbl here, he wrote in this comment, that:

If you save a file as /foo/bar/baz then /foo/ will exist can contain /foo/bar/ and /foo/bar/will exist and contain /foo/bar/baz

A solid store is hierarchical. The slashes in the URIs represent that hierarchy.

So, we have a hierarchical design from the outset, the question is really whether the containment is strictly hierarchical. I.e., can anything exist outside of the hierarchy, e.g. #97 , do we have an analog to UNIX ln (or ln -s)?

The advantage of strict hierarchy is that management of containment triples can be left entirely to the server, and the server may therefore reject any request that contains containment triples in the body of the message. It also means that we avoid atomicity issues since the containment triples can be generated from the hierarchy only, and no knowledge is needed beyond the structure of the URIs. Since the hardest problems in a decentralized architecture revolves around ACID properties, that is likely to be quite a simplifying trait. That'd be my main reason to consider a strict hierarchy.

It also has a bunch of consequences for how the HTTP verbs are defined too, e.g. there is no verb that doesn't have consequences for containment, but I think this is mainly a design question that should be resolved before going too detailed. We know that both hierarchical and non-hierarchical containment can be done with LDP, and we also know that hierarchical constraint can be relaxed later.

[JordanShurmer]

There's a difference between something existing at /foo/bar/baz and /foo/bar actually being an ldp:container and actually containing baz. The NSS has hierarchical urls of course, but that doesn't imply containment hierarchy. So, I don't think the questions is actually "can anything exist outside the (containment) hierarchy". The question here seems to be (as it was originally framed) "should there even be an enforced containment hierarchy". #97 deals with whether things should exist outside of containers or not.

It seems like this suggestion (to enforce containment hierarchy) is simultaneously enforcing ldp:containers everywhere and ignoring them entirely. It would force every path that has "children" to be a container, and yet it would make the ldp:container vocab meaningless since the child/parent relationship is based on path not on the containment triples.

The advantage of strict hierarchy is that management of containment triples can be left entirely to the server

Unless I'm mistaken this is true by definition, regardless of strict hierarchy or not.

Personally, I agree with @csarven here - unless there's some reason to explicitly prohibit non-hierarchical containment than it should not be done. I don't think advantages of hierarchical containment are reasons to prohibit non-hierarchical containment. It should be up to the implementer to decide if the advantage is worth it or not. In other words, the implementer should be able to decide where to put a resource when it's being added to a container. And containment should be determined based on the containment triples, not assumptions about the urls.

[JordanShurmer]

I'll bring my comment over from #97 and apply it to this discussion as well.

The more I think about it, I think hierarchical containment is the way to go. One very important thing that Solid must provide is the ability for the Pod owner to know and control all the data in her pod. So, if containment is not derived from hierarchy, that would imply that Solid must not support POSTing a resource directly, one would have to go through a container url to create new resources.

I personally like the ability to POST a new resource directly (note that NSS currently doesn't support this though?) and think it is something Solid should support... and so I think deriving containment from hierarchy is a better solution than prohibiting POSTing a resource.

[csarven]

Proposal following F2F meeting of 2019-10-29T17:17:00-01:00

There is a 1-1 correspondence between containment triples and relative reference within the path name hierarchy eg:

GET http://example.org/container/

<> ldp:contains <resource> .

maps to:

http://example.org/container/resource

Non-strictly hierarchical containment can potentially introduce non-uniform URI patterns.

Hence it is generally preferable to design so that containment is strictly hierarchical.

[JordanShurmer]

I think a good way to solve both this and #97 in the spec would be to say something along the lines of

• A Solid Server MUST derive the containment triples of an LDPC from all the resources which are located under that container in the url hierarchy.

Expressing it like this would still allow POSTing a resource directly, and it would implicitly be added to the container it lives under.

(although, there is surely a more clear way to say the same thing as "under that container in the url hierarchy")

[akuckartz]

How was this resolved?

[csarven]

@akuckartz It was based on #98 (comment) and text committed in 717ed8a and related following 2f528f1 . It was part of PR #157 but I didn't backlink at the time. Have been backlinking on following PR.