Report detected license expressions in ScanCode

[pombredanne]

Multiple licenses can be and are detected together when appropriate rules exist. However, even though we report these with the same detected start and end line and we know internally that they are detected together and we know if they are license choices or conjunctive licenses, we only report discrete and distinct licenses, and do not provide all these detection details.

We should return all the information that we detect, especially the interesting license choices, possibly as SPDX-style license expressions.

This is especially relevant for multiple choices that exists in Qt and similar (see #73)

[pombredanne]

The reporting as a license expression should be a rather straightforward addition to the license detection, since we have already all the data when we match a license RULE.
The logic will be the following: Once we have matched a text region (start/end lines) to a detection rule:

• if there is more than one license in the rule, create a license expression:
  • if the rule is a license_choice: yes, create this expression as joining with an OR all the rules licenses
  • else, create this expression as joining with an AND

At a later stage, we can replace the list of licenses in all rules (with a one time migration script) by a plain license expression and then support more complex expressions such as (XXXX with YYY-exception) or ZZZZ

At a high level, the key thing to understand is that we already match to "expression-like" rules
we just do not expose that info in the scan reports.

[pombredanne]

The latest https://github.com/nexB/license-expression/releases/tag/v0.6 now fully supports expressions "with exceptions" and expressions using license names containing "or later". This is now good enough to be used and support the implementation of expression in ScanCode

[pombredanne]

The new approach is this:

• exception licenses will be flagged with an is_exception boolean tag. Note that there there is no consistent way to tell which license an exception may apply too all the times. It may be more than one possibility or not defined, hence why we only will use a flag.

• licenses that are for this "or later" versions will be flagged with an is_or_later boolean tag. e.g. mpl-1.1 or gpl-2.0-plus will have this flag.

• for an is_or_later license, the base_license field will point to the base license for this license. For instance gpl-2.0-plus will have base_license: gpl-2.0. But mpl-1.1 has no base version.

• licenses that are versioned will have a next_version field that points to the license key of the next version when available. For instance gpl-2.0-plus will have next_version: gpl-3.0-plus and gpl-2.0 will have next_version: gpl-3.0 but gpl-3.0 will not have a next_version because there is no such thing yet. mpl-1.1 will have next_version: mpl-2.0

• a new standard_notice field will be added to store the standard notice that may exist for some licenses such as Apache or A/L/GPL or MPL licenses and in particular to have the standard "or later" notice for the license keys that are for an "or later license"

• Support for a preferred license version when there is an "or later" case would be something that would be added to a user config file at a later time with Add File-based user preferences and scan configuration #520

[pombredanne]

The final implemented approach is this:

• "exception" licenses are flagged with an is_exception boolean attribute.
• a new standard_notice field has been added to store the standard notice that may exist for some licenses such as Apache or A/L/GPL or MPL licenses and in particular to have the standard "or later" notice for the license keys that are for an "or later license"

This simple and more than good enough.

Closing now.