App. X: Define custom license namespaces

[swinslow]

This commit is an initial attempt to define the custom license namespace concept that was agreed upon in Spring 2019, as reflected in #113 and tech team minutes, but which was not subsequently specified.

This defines the DNS-style version of the LicenseRef- format. It also describes the process for a namespace maintainer to create and publish an SPDX document at a URL within that domain, where anyone can find the license text corresponding to each identifier defined within that namespace.

This is a first cut to capture discussions from nearly a year ago, and I am sure that things could be worded better. Please, please throw rocks at it. :) cc @goneall @kestewart @MarkAtwood @pombredanne @zvr @tsteenbe @iamwillbar as well as anyone else who wants to weigh in.

Fixes #113

Signed-off-by: Steve Winslow steve@swinslow.net

[goneall]

@swinslow Thanks for pulling this together! A nicely detailed proposal.

It would be good to have the legal team review - especially Mark Atwood who made the original proposal.

I recall 2 separate discussions which would impact the proposal. I can't find the notes on the exact discussions, so this is a bit from memory. I'm hoping some folks on the tech or legal team can provide a bit more detail.

• Optional rather than required registration of the namespace - For the DNS namespace, the registration and public documentation of the licenses would be encouraged, but not required. I recall some on the legal team wanted to maintain an internal list and still allow references in SPDX documents.
• Non-DNS formatted namespaces - I believe this came from a tech call. There was a concern that some maintainers of license lists may not have a DNS registered domain. I recall we agreed that non-DNS formatted namespaces were required to register their namespaces to avoid namespace conflicts.

[zvr]

As I recall, we had discussed two (independent) license namespaces:

1. the "DNS" namespace, which is described in this PR, where the owner of example.com could use identifiers like LicenseRef-.example.com.-something (and obliviously publish the licenses)

2. the "reserved name" namespace, where identifiers would be of the form LicenseRef--example-something where the SPDX group would serve as a simply "registry" for the names (example, in this case) and the "owner" would publish the licenses via SPDX. Examples of such namespaces could be large organizations, scanning tools, etc. We had not advanced in the discussion about how to operate the registry, etc.

The former was called dash-dot and the latter dash-dash, from the characters following LicenseRef.

[swinslow]

Hi @goneall and @zvr, thanks for your comments. I agree, in #113 we had at least originally discussed having a second format option which was a "flat" or organization-name-based namespace as well.

I had omitted it from the draft here, because to @zvr's point I don't think we had worked our a process where that would be authenticated or managed. For DNS names, it's pretty much an objective check to see -- a yes/no of whether or not the URL they're seeking to register is within the domain name from the DNS entry. For an organization name, it's more of a subjective question of, do we know that this person represents XYZ organization or company? Maybe that's acceptable, I'd just feel better if we had a process in place for overseeing this (or at least agreement on who would handle it).

@goneall, thanks for the details on your understanding of registration being optional for DNS identifiers. I guess my question then is, how people would be able to derive the actual license text -- if all they have is a DNS-based identifier, but it's unregistered so there is no link back to an SPDX document with the license text. Maybe that's something we can discuss during the joint call tomorrow.

[swinslow]

cc @jlovejoy -- apologies, I meant to include you in the list of mass-cc's at the top of this pull request!

This is my draft proposal for the "license list namespace" writeup, to implement what we'd discussed and agreed on last year. Grateful if you are able to weigh in as well, if you have input here.

[swinslow]

Suggested on tech team call - also recommend that "--" and "-." should not be used for LicenseRef's that aren't part of namespaces.

Important to define hierarchy of resolution

[swinslow]

Syntax should be both accessible, obvious, intuitive, compatible and automatable

[swinslow]

This is still open, and subject to outcome of conversation in the main thread.

[goneall]

An alternative proposal - we could use the external document referenced licenseRef's as defined in the License Expresions Appendix - Simple License Expressions.

We could use a custom syntax for the External Document Ref to denote a "registered external document" containing licenses.

[swinslow]

Proposals from 2020-03-03 tech team call (thanks @zvr!):

NamespaceRef-amazon:LicenseRef-MyLicense


NamespaceRef-([^:]+):LicenseRef-(.+)

look up \2 in namespace \1

[goneall]

Updated proposal from the call:

(NamespaceRef-|^(DocumentRef-))([^:]+):LicenseRef-(.+)

Examples:

NamespaceRef-amazon:LicenseRef-myLicense
amazon:LicenseRef-myLicense

The two examples above would both refer to the same external list of licenses.

[zvr]

We can even simplify it further, deleting the NamespaceRef- part, going towards:
intel:LicenseRef-MyLicense-1.0
If the name includes a colon (':'), the part before the colon is the namespace (or something starting with DocumentRef-).

[comment can be disregarded; had it opened for too long while on phonecalls and @goneall already provided the info]

[swinslow]

Could we swap it around? with LicenseRef- still as the prefix? So for instance:

LicenseRef-intel:MyLicense-1.0

That way LicenseRef- is still consistently the prefix for non-official license identifiers. And after parsing that, you can look for a colon (or not) to determine whether it's namespaced (or not).

[goneall]

Could we swap it around? with LicenseRef- still as the prefix?

@swinslow you raise a good point about making it easier to parse (both for computers and humans).

Since the DocumentRef-XXX: is already a prefix, we already have situations where we have non-listed licenses that do not start with LicenseRef-.

I think the cleanest approach would be to require the NamespaceRef-XXX, that way we could easily recognize we are dealing with namespaces. The rules for parsing would be:

• If the license ID starts with LicenseRef- then it will be found in the SPDX document
• If the license ID starts with 'DocumementRef-` then it will be found in an external SPDX document defined by the external document reference section of the SPDX document
• If the license ID starts with NamespaceRef- then it will be found in an SPDX document defined by the external license namespace git repository
• If none of the above apply, then it is an SPDX listed license

If we don't require the NamespaceRef- prefix, then the rules would be:

• If the license ID starts with LicenseRef- then it will be found in the SPDX document
• If the license ID contains a colon ':' and starts with 'DocumementRef-` then it will be found in an external SPDX document defined by the external document reference section of the SPDX document
• If the license ID contains a colon ':' and does NOT starts with 'DocumementRef-` then it will be found in an SPDX document defined by the external license namespace git repository
• If none of the above apply, then it is an SPDX listed license

I'm OK with not requiring the NamespaceRef- as it does make the ID's more verbose.

[swinslow]

Thanks @goneall. Appreciate you spelling these out. I'm in favor of the first of the two formats you proposed, using the appropriate prefix to indicate which type of reference it is.

I do feel that the "namespace+colon without other prefix" makes the custom identifiers look too similar to official identifiers that are on the license list. I think it's a plus, not a minus, that the *Ref- prefix makes that clear upfront.

[zvr]

Actually, we could have it both ways:

• If the license ID starts with LicenseRef- then it will be found in the SPDX document
• If the license ID starts with DocumementRef- then it will be found in an external SPDX document defined by the external document reference section of the SPDX document
• If the license ID starts with NamespaceRef- then it will be found in an SPDX document defined in the registration of the external license namespace
• If the license ID contains a colon : (and does not start with DocumementRef-) then it will be found in an SPDX document defined in the registration of the external license namespace
• If none of the above apply, then it is a short identifier of a license in the SPDX license list

This way the "NamespaceRef-" prefix is optional.
I envision that it will be used in SPDX documents (and the prefix may be used for other uses in a 3.x spec), but in standalone uses people might choose the shorter format.

Example in a source file:

SPDX-License-Id: intel:LicenseRef-ISSL-202002

[zvr]

Heh, of course if we go by my comment above, there is nothing stopping us from saying that the fourth alternative is actually:

SPDX-License-Id: intel:ISSL-202002

since the presence of the colon signifies a non-standard namespace, which makes the need for the "LicenseRef-" prefix redundant.

To put it another way, if : must always be followed by LicenseRef-, one can substitute :LicenseRef- by : without introducing any misunderstandings.

[swinslow]

I really think it needs to be crystal-clear to people about whether they're using a curated license identifier that's on the SPDX License List, or a custom license namespace.

If it's on the License List then they know that it's been vetted, it has a stable URL and a settled reference text / markup.

If it's a custom namespace then anyone using it is buying into the controller of that namespace being able to change its meaning at their whim.

I'm really concerned about an assumption that the broader community will be able to distinguish between the implications of these two very different cases, just based on the presence of a colon. I'd strongly encourage that the custom namespace route have something that makes it really explicit, to a casual observer, that it's not an official and vetted license / identifier. That's why I'm a fan of something that puts NamespaceRef- or LicenseRef- front and center.

(cc @jlovejoy for visibility)

[goneall]

To put it another way, if : must always be followed by LicenseRef-, one can substitute :LicenseRef- by : without introducing any misunderstandings.

This is true for the syntax within the SPDX document making the external reference.

The proposed solution for finding the actual text of the referenced licenses is to use SPDX documents.

If we use an SPDX document to document the licenses referenced by the namespace, they would need to be preceeded by LicenseRef- to match the current spec. If we don't use the existing SPDX document spec, we would be free to not require the LicenseRef- prefix.

[swinslow]

All, I've updated the PR to add in the second, "organization name" format, and made some other conforming changes.

I believe the only remaining open issue is whether to use the identifier format that was agreed upon last year, or whether to change it to something else (such as by incorporating NamespaceRef- as a prefix, or using colons, etc).

My guess is that if we are taking one of those different approaches, there will need to be more extensive changes to e.g. the license expression syntax and grammar, as well as getting buy-in from the legal team on another joint legal/tech call. So I'd suggest that if we want to go that route, then we defer all of this to 3.0. I'm not sure what the timing looks like for 2.2, but if we want to add this in the near-term then I'd encourage we use the format that is described here, since (I think) it won't require those sort of further changes and discussions.

[kestewart]

Based on discussions on the call today, there is strong support for Namespace- proposal, and possibly using ":" symbol to separate. Alexios has volunteered to write up a pull request for this in 3.0, and Steve has volunteered to help, and coordinate with legal team. Since this seems to be the direction, we'll close this for 2.2 and not apply.