Merge pull request #23 from hesa/hesa-normalize-output

Normalize output, add doc

FLAME.md

@@ -53,18 +53,20 @@ BSD-3-Clause AND LicenseRef-flame-x11-keith-packard
 
 ## Get a license expression with compatible licenses
 
-You want an expression with licenses that are supported by (osadl_matrix)[https://github.com/priv-kweihmann/osadl-matrix]:
+You want an expression with licenses that are supported by [osadl_matrix](https://github.com/priv-kweihmann/osadl-matrix):
 ```
 $ flame compat "BSD3 & x11-keith-packard"
 BSD-3-Clause AND HPND
 ```
 
-You want an expression with licenses that are supported by (osadl_matrix)[https://github.com/priv-kweihmann/osadl-matrix] from `BBSD3 & x11-keith-packard` with info on how the data was found:
+You want an expression with licenses that are supported by [osadl_matrix](https://github.com/priv-kweihmann/osadl-matrix) from `BBSD3 & x11-keith-packard` with info on how the data was found:
+```
 $ flame --verbose compat "BSD3 & x11-keith-packard"
 BSD-3-Clause AND HPND
  * "BSD3" -> "BSD-3-Clause" via "alias" -> "BSD-3-Clause" via "direct"
  * "&" -> "AND" via "operator"
  * "x11-keith-packard" -> "LicenseRef-flame-x11-keith-packard" via "scancode_key" -> "HPND" via "compatibility_as"
+```
 
 ## List aliases
 
@@ -78,9 +80,19 @@ GPL2 -> GPL-2.0-only
 ....
 ``` 
 
+## List operators
+
+To list all the supported operators (incomplete listing below):
+``` 
+$ flame operators
+OR -> OR
+or -> OR
+....
+``` 
+
 ## Compatibilities
 
-To list all licenses that has a license with same compatibility as an license known to (osadl_matrix)[https://github.com/priv-kweihmann/osadl-matrix] (incomplete listing below):
+To list all licenses that has a license with same compatibility as an license known to [osadl_matrix](https://github.com/priv-kweihmann/osadl-matrix) (incomplete listing below):
 ``` 
 $ flame compats
 LicenseRef-flame-x11-keith-packard -> HPND

Makefile

@@ -52,7 +52,7 @@ check-py-cli:
 	@echo "OK"
 
 	@echo -n "Check cli (alias): "
-	@PYTHONPATH=./python python3 ./python/flame/__main__.py expression "BSD3 and BSD3" > /dev/null
+	@PYTHONPATH=./python python3 ./python/flame/__main__.py license "BSD3 and BSD3" > /dev/null
 	@echo "OK"
 
 	@echo -n "Check cli (compat): "

README.md

@@ -14,26 +14,61 @@ A database with meta data for FOSS licenses adding useful information to existin
 
 * license text
 
+# Background
+
+There are lots of software licenses out there (e.g. see [ScanCode LicenseDB](https://scancode-licensedb.aboutcode.org/)), some of them are FOSS and some not. In this project we focus on FOSS licenses.
+
+## License name proliferation
+
+When you're working with compliance you are used to liceses called differently in source code or by tools (e.g. `GPLv2`, `GPL (v2)` and `GNU General Public License Version 2`) when all you really want too see is the [SPDX identifier](https://spdx.org/licenses/) `GPL-2.0-only`. A seasoned compliance engineer or lawyer knows this already, but we need this information machine readable.
+
+## License proliferation
+
+Another problem you face when working with compliance is the need to check whether the licenses in a combined work are compatible. One example is the [`X11-Style (Keith Packard)`](https://scancode-licensedb.aboutcode.org/x11-keith-packard.html) license, which really is the same license as the [Historical Permission Notice and Disclaimer - sell variant](https://spdx.org/licenses/HPND-sell-variant.html). `X11-Style (Keith Packard)` is not supported in for example the OSADL matrix, but `HPND-sell-variant` is. Again, a seasoned license engineer or lawyer knows which licenses are compatible and not, but we need to make it possible for a machine to assist us. 
+
+# About
+
+This projet aims at providing a database with:
+
+* "all" different names for a license in a database
+
+* mappings from one license to another license which is supported by the OSADL matrix
+
+and, to make the database easier to use:
+
+* a Python API
+
+* command line tool
+
 # Database
 
 The data can be found in the [var directory](https://github.com/hesa/foss-licenses/tree/main/var). Each license has a JSON file with meta information and a LICENSE file with the license text.
 
 # Tools and APIs
 
-* [FLAME](https://github.com/hesa/foss-licenses/blob/main/FLAME.md) - command line program
+* [flame](https://github.com/hesa/foss-licenses/blob/main/FLAME.md) - command line program
+
+* [Python API](https://github.com/hesa/foss-licenses/blob/main/PYTHON_API.md)
 
-* [PYTHON API](https://github.com/hesa/foss-licenses/blob/main/PYTHON_API.md) - Python API
+# Contributions
+
+More info soon.
 
 # Related tools and projects
 
 * [flict](https://github.com/vinland-technology/flict) - FOSS License Compatibility Tool 
 
+* [License Compatibility Matrix](https://www.osadl.org/Access-to-raw-data.oss-compliance-raw-data-access.0.html) - a matrix with license compatibilities
+
 * [scancode](https://github.com/nexB/scancode-toolkit) - ScanCode toolkit
 
 * [ScanCode LicenseDB](https://scancode-licensedb.aboutcode.org/) - a database with licenses
 
 # Acknowledgements
 
-* [Nexb](https://www.nexb.com/) for their general and generous work in FOSS compliance
+* [Nexb](https://www.nexb.com/) for their general and generous work in FOSS compliance, especially [scancode](https://github.com/nexB/scancode-toolkit) and [ScanCode LicenseDB](https://scancode-licensedb.aboutcode.org/).
+
+
+* [Max Huber](https://github.com/maxhbr) for [LDBcollector](https://github.com/maxhbr/LDBcollector)
 
-* [Max Huber](https://github.com/maxhbr) for [LDBcollector](https://github.com/maxhbr/LDBcollector) 
+* [OSADL](https://www.osadl.org) for their [License Compatibility Matrix](https://www.osadl.org/Access-to-raw-data.oss-compliance-raw-data-access.0.html)

python/flame/__main__.py

@@ -44,31 +44,31 @@ def parse():
 
     subparsers = parser.add_subparsers(help='Sub commands')
 
+    # license
+    parser_e = subparsers.add_parser(
+        'license', help='Convert license to SPDX identifiers and syntax')
+    parser_e.set_defaults(which='license', func=license)
+    parser_e.add_argument('license', type=str, help='license expression to fix')
+
     # compatibility
     parser_c = subparsers.add_parser(
-        'compat', help='display license with same compatibility as supplied license')
+        'compat', help='Convert license to using licenses compatible with OSADL\'s matrix')
     parser_c.set_defaults(which='compat', func=compatibility)
     parser_c.add_argument('license', type=str, help='license name to display')
 
-    # expressions
-    parser_e = subparsers.add_parser(
-        'expression', help='not yet')
-    parser_e.set_defaults(which='expression', func=expressions)
-    parser_e.add_argument('license', type=str, help='license expression to fix')
-
     # aliases
     parser_a = subparsers.add_parser(
-        'aliases', help='show all aliases')
+        'aliases', help='Display all aliases')
     parser_a.set_defaults(which='aliases', func=aliases)
 
     # compatbilities
     parser_cs = subparsers.add_parser(
-        'compats', help='show all compatibilities')
+        'compats', help='Display all compatibilities')
     parser_cs.set_defaults(which='compats', func=compats)
 
     # operators
     parser_os = subparsers.add_parser(
-        'operators', help='show all operators')
+        'operators', help='Display all operators')
     parser_os.set_defaults(which='operators', func=operators)
 
     # licenses
@@ -100,8 +100,8 @@ def compatibility(ldb, formatter, args):
     compatibilities = ldb.expression_compatibility_as(args.license)
     return formatter.format_compatibilities(compatibilities, args.verbose)
 
-def expressions(ldb, formatter, args):
-    expression = ldb.expression(args.license)
+def license(ldb, formatter, args):
+    expression = ldb.expression_license(args.license)
     return formatter.format_expression(expression, args.verbose)
 
 def main():

python/flame/format.py

@@ -5,9 +5,9 @@
 import json
 import yaml
 
-OUTPUT_FORMAT_JSON = "Json"
-OUTPUT_FORMAT_YAML = "Yaml"
-OUTPUT_FORMAT_TEXT = "text"
+OUTPUT_FORMAT_JSON = 'Json'
+OUTPUT_FORMAT_YAML = 'Yaml'
+OUTPUT_FORMAT_TEXT = 'text'
 OUTPUT_FORMATS = [OUTPUT_FORMAT_JSON, OUTPUT_FORMAT_YAML, OUTPUT_FORMAT_TEXT]
 
 class OutputFormatterFactory():
@@ -68,7 +68,7 @@ def format_identified_list(self, all_aliases, verbose):
         return json.dumps(all_aliases, indent=4)
 
     def format_error(self, error, verbose):
-        return json.dumps({"error": f'{error}'}, indent=4)
+        return json.dumps({'error': f'{error}'}, indent=4)
 
     def format_licenses(self, licenses, verbose):
         return json.dumps(licenses, indent=4)
@@ -97,7 +97,7 @@ def format_identified_list(self, all_aliases, verbose):
         return yaml.dump(all_aliases)
 
     def format_error(self, error, verbose):
-        return yaml.dump({"error": f'{error}'})
+        return yaml.dump({'error': f'{error}'})
 
     def format_licenses(self, licenses, verbose):
         return yaml.dump(licenses)
@@ -112,7 +112,7 @@ class TextOutputFormatter(OutputFormatter):
 
     def format_compat(self, compat, verbose):
         ret = []
-        id_lic = compat["identified_license"]
+        id_lic = compat['identified_license']
         if verbose:
             ret.append(f'queried_name: {id_lic["queried_name"]}')
             ret.append(f'name: {id_lic["name"]}')
@@ -135,18 +135,14 @@ def format_compatibilities(self, compats, verbose):
                 else:
                     ret.append(f' * "{lic_elem["queried_name"]}" -> "{lic_elem["name"]}" via "{lic_elem["identified_via"]}" -> "{compat["name"]}" via "{compat["compat_identification"]["compatibility"]["identified_via"]}"')
 
-        return "\n".join(ret)
+        return '\n'.join(ret)
 
     def format_compat_list(self, all_compats, verbose):
-        ret = []
-        for comp in all_compats:
-            ret.append(f'{comp["spdxid"]} -> {comp["compatibility_as"]}')
-
-        return "\n".join(ret)
+        return '\n'.join([f'{comp["spdxid"]} -> {comp["compatibility_as"]}' for comp in all_compats])
 
     def format_identified(self, identified, verbose):
         ret = []
-        id_lic = identified["identified_element"]
+        id_lic = identified['identified_element']
         ret.append(f'{id_lic["name"]}')
         if verbose:
             ret.append(f' * "{id_lic["queried_name"]}" -> "{id_lic["name"]}" via "{id_lic["identified_via"]}"')
@@ -158,23 +154,19 @@ def format_expression(self, expression, verbose):
         ret.append(f'{id_lic}')
         if verbose:
             for identification in expression['identifications']:
-                id_elem = identification["identified_element"]
+                id_elem = identification['identified_element']
                 ret.append(f' * "{id_elem["queried_name"]}" -> "{id_elem["name"]} via "{id_elem["identified_via"]}"')
-        return "\n".join(ret)
+        return '\n'.join(ret)
 
     def format_identified_list(self, all_aliases, verbose):
-        ret = []
-        for alias, value in all_aliases.items():
-            ret.append(f'{alias} -> {value}')
-
-        return "\n".join(ret)
+        return '\n'.join([f'{k} -> {v}' for k, v in all_aliases.items()])
 
     def format_licenses(self, licenses, verbose):
         licenses.sort()
-        return "\n".join(licenses)
+        return '\n'.join(licenses)
 
     def format_operators(self, operators, verbose):
-        return "\n".join(operators)
+        return '\n'.join([f'{k} -> {v}' for k, v in operators.items()])
 
     def format_error(self, error, verbose):
         return f'Error, {error}'

python/flame/license_db.py

@@ -111,7 +111,7 @@ def __identify_license(self, name):
             'identified_via': ret_id,
         }
 
-    def expression(self, license_expression):
+    def expression_license(self, license_expression):
         new_expression = license_expression
         license_parts = []
         license_list = []
@@ -205,7 +205,7 @@ def compatibility_as(self, license_name):
         }
 
     def expression_compatibility_as(self, license_expression):
-        expression_full = self.expression(license_expression)
+        expression_full = self.expression_license(license_expression)
         compats = []
         for identification in expression_full['identifications']:
             lic_elem = identification['identified_element']['name']

tests/python/test_ldb.py

@@ -43,7 +43,7 @@ def test_identify_with_blank():
     assert lic['license']['spdxid'] == "GPL-2.0-or-later"
 
 def test_identify_expression():
-    lic = ldb.expression("BSD-3-Clause and BSD3")
+    lic = ldb.expression_license("BSD-3-Clause and BSD3")
     assert lic['identified_license'] == "BSD-3-Clause AND BSD-3-Clause"
 
 def test_fail_identify():