How to refer to “the” Solid spec?

[michielbdejong]

E.g. here I think @RubenVerborgh 's wording:

This should likely not be in the core spec (especially since it is unclear what subset should be supported). Can be in an extension though.

Is going down the wrong route. A clearer wording would IMHO be:

This should likely not be in the Solid spec (especially since it is unclear what subset should be supported). Web servers could still support it, independently of whether they (also) support the Solid spec though.

We should strive to have one single feature set, which all pod servers support, end of story. We could introduce discoverable performance-improving extensions, but I would say let's just look at each feature intelligently and make a decision to either include it or not. Life is difficult enough without such differences between pod servers! :)

[RubenVerborgh]

We should strive to have one single feature set, which all pod servers support, end of story.

Not entirely the end.

There is one (set of) spec(s) that should be implemented by a server to be considered Solid compatible. Any Solid app will work with them then. So far I agree.

But there are a set of optional specs that a server could support to optimize things.
Cfr. HTTP and its many extensions. These have popped up in discussions surrounding search, and will be necessary. (Theory at https://ruben.verborgh.org/articles/web-api-ecosystem/.)

So when I say "core" spec, I mean:

• the spec that all servers should conform to
• the currently unwritten spec of what we actually want that to be (for lack of a written version, the Solid spec v0.7 is a proxy, but not a very good one yet)

We already see this in the current v0.7, where some features are not considered core/crucial, and are only suggested/loosely defined.

Happy to not use the term "core", but the above distinction is important. There will be a core/base/mandatory spec, and extension points (optional = apps MUST work without them, just less optimally).

[RubenVerborgh]

Specifically on the topic of #162, this is a good example of an optional part that can speed up things, but is not necessary for a functioning app.

[michielbdejong]

No, I understand that you would like to have a core + extensions spec (like e.g. the XMPP spec - I understand what you are requesting), but the possible gain from that simply does not weigh against the cost of it.

We should just have one spec. If there are ambiguities about what is in or out, then let's create issues for those resolve each of them on the next CG call. It's clear that SPARQL on GET is 'in' because both server implementations support it and also the current spec text says so.

So you can still propose that we drop it in the next spec version, of course, and we can discuss the pros and cons of that in #162. But we should use the term 'Solid spec' instead of 'core spec', and the term 'other web protocols' instead of 'extensions'.

[RubenVerborgh]

This:

No, I understand that you would like to have a core + extensions spec

seemingly conflicts with this:

We should just have one spec

Because we will find ourselves in the situation where we/Solid publish multiple specs, and referring to the main/core/base spec as "the spec" will be less clear then.

and the term 'other web protocols' instead of 'extensions'.

They might also be "Solid" specs, not just Web protocols.

I could imagine (top of my head):

• Solid shapes
• Solid search
• Solid indexing
• Solid querying
• Solid full-text search

Honestly, no strong feeling. I'm just looking for something more clear than "the Solid spec" to refer to "the thing that will form when we take the bits of pieces of v0.7 and create an actual unambigious spec". So happy to stop using "core spec", when given an unambiguous alternative 🙂

[RubenVerborgh]

the next spec version

Made #170

It's clear that SPARQL on GET is 'in' because both server implementations support it and also the current spec text says so.

This is not clear to everyone at the moment. The current spec document hasn't been proofread, discussed, etc. It is more documentation of implementations than anything else. So we should not consider this document as authoritative in any way. The implementations even diverge as to the point that clients do not even know what queries they can send to the server, so it is not a meaningful part of the spec.

In absence of a definition of "a subset of SPARQL 1.1" and "basic SELECT, INSERT and DELETE statements", it simply cannot be implemented. (See #170)

So I strongly urge anyone to not rely on what is written in the spec. The only safe assumption at the moment is LDP+WAC (and some form of WebID-TLS or WebID-OIDC, but the latter spec is incomplete).

[michielbdejong]

Now you are talking about sub-specs which is different from extensions, so there indeed it's not such a problem. So how about we call https://github.com/solid/solid-spec#solid-specification either 'the solid spec' (if we mean the whole thing including sub-specs) or 'the main solid spec' if we mean just the parent, and we call https://github.com/solid/solid-spec/blob/master/api-rest.md etc. the 'sub-specs'?

[RubenVerborgh]

“main” is fine, but I wasn’t specifically aiming to talk about subspecs (they should likely just be sections in the main spec).

My concern is still about a naming distinction between the/all mandatory spec(s) and optional Solid extension specs. They will all be Solid specs; so “the” spec is not meaningful (but “main” can be).

In any case, this thread confirms the necessity and urgency of #170 (which I will follow up on with priority).

[michielbdejong]

Can you give a list of those optional Solid extension specs?

[RubenVerborgh]

I could imagine (top of my head):

* Solid shapes

* Solid search

* Solid indexing

* Solid querying

* Solid full-text search

[michielbdejong]

Solid shapes is a good example, but it's not an extension, it's an inter-app interoperability layer on top of the Solid spec. We could consider it part of the spec, then maybe we should say that the Solid spec prescribes behaviour for both pod servers and apps, I'm not opposed to that! So then we have 'the pod server spec' prescribing server behaviour and 'the shapes spec' prescribing app behaviour.

For search, indexing, querying, and full-text search, I don't know what you're referring to, but I think what you mean is that servers may implement other web protocols apart from the Solid spec (or 'pod server spec' is maybe a better term indeed).

[RubenVerborgh]

'the pod server spec'

I like the direction where this is going; but we might want to emphasize that the spec is a contract between app and servers. I.e., it's not just useful when you are building a server, also when you are building an app.

[justinwb]

I like the direction where this is going; but we might want to emphasize that the spec is a contract between app and servers. I.e., it's not just useful when you are building a server, also when you are building an app.

Can't stress the importance of this enough. For example, application and data discovery is central to application interoperability. I argue that this belongs in the specification - because what good is a solid server if applications that interface with it don't function in a consistent way? This consistency is core to the value of solid, otherwise you have a decentralized ecosystem where nothing works with each other.

The caveat then, is that application and data discovery relies on things like shapes, indexing, and querying, so their inclusion in some form becomes necessary as well.

[kjetilk]

We should also keep in mind the general architectural principle of orthogonal specifications. That is to say, there is not one Solid spec, but a group of orthogonal specifications making up what we momentarily define to be Solid, and what the Solid test suite can verify. I think it is very important to adhere to this principle of orthogonal specifications.

That being said, I felt 20 years ago, and I think I have observed it in others, that orthogonality makes it harder to gain a fairly good overview and understanding of the technology, and so what we understand and communicate around Solid should reflect that. It is more an issue of presentation than of actual spec structure though.

[michielbdejong]

there is not one Solid spec, but a group of orthogonal specifications making up what we momentarily define to be Solid, and what the Solid test suite can verify

It is more an issue of presentation

Yes, agree! But we need better names, because in the current situation we have misunderstandings about what people mean with "... is in the spec". To make it clear that the main spec refers to sub-specs, and that the current main spec only describes behaviour of pod servers, whereas there will probably be other spec parts around app behaviour, how about the following wording and terminology:

• all together, we have "the Solid spec collection"
• it contains two main parts, one normative for pod providers, namely "the pod server spec collection" and one non-normative advice for app developers, namely "the data conventions spec collection".
• the pod server spec collection is further split up in a "main pod server spec" and "pod server sub specs", which is any document referred to as normative by the main pod server spec.

Currently, the only requirement on a solid app is that it works with solid pods that adhere to version 0.7 of the pod server spec collection. In the future, the plan is to make the data conventions spec collection normative as well.

Currently, all pod providers should adhere to version 0.7 of the pod server spec collection. Apart from that, they are allowed to offer other (discoverable) web protocols such as pod-wide SPARQL search etc. In the future, after we have gained some experience with how to build apps for v0.7 pods, we will use the lessons learned to make some simplifications and improvements, and introduce version 1.0 of the pod server spec collection.

[justinwb]

We should also keep in mind the general architectural principle of orthogonal specifications. That is to say, there is not one Solid spec, but a group of orthogonal specifications making up what we momentarily define to be Solid, and what the Solid test suite can verify. I think it is very important to adhere to this principle of orthogonal specifications.

We have to be careful to keep the mission in perspective. While it is tempting as engineers to think of a spec as a bunch of interchangeable and modular pieces, this can lead to problems with interoperability in the absence of a governing specification that mandates how to tie these things together in a consistent way. Seamless interoperability across solid servers and pods everywhere is paramount to the Solid mission. If there is any fundamental inconsistency in server implementations we will end up with a partitioned ecosystem, and will have fallen short of our goal.

I'm not lobbying against having a set of companion specifications that can interchanged on the server side to satisfy functionalities defined in the Solid specification, but the resulting interface for clients must be consistent for a stable specification that can support a stable ecosystem.

[kjetilk]

While it is tempting as engineers to think of a spec as a bunch of interchangeable and modular pieces, this can lead to problems with interoperability in the absence of a governing specification that mandates how to tie these things together in a consistent way.

Indeed, but not maintaining orthogonality can lead to that the platform cannot evolve to meet new challenges. It is a delicate balance, but I think that a clear presentation of what constitutes a Solid server should take care of it.

[RubenVerborgh]

I actually think there is a resolution now:

• There is the v0.8 draft spec (this repo)
• There is the Solid specification repository with
  • an entry document at https://solid.github.io/specification/
  • more documents at https://solid.github.io/*

[kjetilk]

I actually think there is a resolution now:

Right.

* There is the v0.8 draft spec (this repo)

* There is the Solid specification repository with
  
  * an entry document at https://solid.github.io/specification/
  * more documents at https://solid.github.io/*

We shouldn't use those domains though, we need to have it under solidproject.org.

[RubenVerborgh]

Indeed, tracking in https://github.com/solid/specification/issues/23.