Extend hermes context to depict all metadata of a pyproject.toml

[notactuallyfinn]

A lot of metadata can be collected when harvesting a pyproject.toml.
However, not all relevant data can be displayed properly with schema and codemeta fields.
The following data causes difficulties:

• The permitted Python versions, as there may be several and they may also be restricted by a wide variety of conditions.
• The (optional) dependencies on other libraries, as the versions can also be given by logical expressions, which can make the versions dependent on each other, on optional features, the Python version, etc. (see https://packaging.python.org/en/latest/specifications/dependency-specifiers/#dependency-specifiers).
• The licenses for the distribution and use of the software, as these may be connected by logical links (see https://packaging.python.org/en/latest/glossary/#term-License-Expression)
• The ReadMe(s), as no URL is given, but one or more files or even just text, but a URL would be necessary for the codemeta field.
• The classifieres (see https://pypi.org/classifiers/)
• The URLs that do not reference the repository

Therefore I suggest the following:

• Adding a version data type that extends or replaces the allowed data types of version in schema.org (text or number). It should be able to represent the grammar (https://packaging.python.org/en/latest/specifications/dependency-specifiers/#grammar) completely and as unambiguously as possible.
• Add a license data type that supplements the permitted data types of license in schema.org (URL or CreativeWork). It would have two fields for text or itself and another for text, which only allows the logical operators.
• Add a readme field, which has one field each for text and file type.
• Add a classifier field that only allows the values mentioned here (https://pypi.org/classifiers/).
• Add a field for URLs, which has a text field and the field for a URL to combine purpose and URL.

Related

• Develop and publish hermes and related vocabularies #393

[led02]

Great overview, thanks.

Before extending the vocabulary, I feel we should do some decisions:

• Which pyproject.toml schema do we want to support? In my opinion we should support people in their decision to migrate to the standardized format, i.e., only go for PEP-style pyproject files.
• I do not see any discussion on what subset we want to support, we should be greedy here!
• I think to a certain extend this information is covered by https://schema.org/SoftwareSourceCode (in doubt using generic relations like hasPart or isPartOf).
• We should decide how to use this "additional vocabulary" (e.g., explicitly exposing it in a hermes-context that implies we can process it, or encouraging people to freely use other well-defined contexts (whereof I prefer the latter solution)).

Edit: I also think your tabular overview is a great documentation and we should think about having some common template for different harvesters for quality control...
Edit 2: Not that it is of any great worth but ... well this is not what I meant with "great documentation" or "common template" https://codemeta.github.io/crosswalk/python/ ;)

[sdruskat]

As an addition, the crosswalk seems incomplete to me as is. I'll remove it here, and re-add it to the internal issue in the GitLab for future discussion.

Edit: I also think your tabular overview is a great documentation and we should think about having some common template for different harvesters for quality control...

While I generally agree, I don't think tabular overviews are ideal for this. The CodeMeta project had tried this and it showed that the relations between fields can be much more complex than can be described in a tabular format. They're currently using SSSOM to re-do some crosswalks, so we could look into using something similar to require from new harvesters for documentation, but this needs to be discussed. Perhaps we can find something more usable and less esoteric ;).

[led02]

While I generally agree, I don't think tabular overviews are ideal for this. The CodeMeta project had tried this and it showed that the relations between fields can be much more complex than can be described in a tabular format. They're currently using SSSOM to re-do some crosswalks, so we could look into using something similar to require from new harvesters for documentation, but this needs to be discussed. Perhaps we can find something more usable and less esoteric ;).

Okay, let's discuss this further (maybe in an own issue?)

What is the exact purpose of this table? While I'm a fan of fancy, machine-readable stuff, here I think it is more important for (user side) documentation and support while developing harvesters. This is also user-oriented - since we want to make it easy to use and extend. For the implementation, we do not want something fully automatic - we implement the mapping logic in Python! For documentation of the mappings applied, looking at SSSOM might be of interest.

Hence, I'll stick with "providing some template table to help the user design their harvester mapping".

Looking at the "mapping" in the deposition phase, I might have a totally different view there... ;)

[led02]

Two further remarks / ideas:

• For the version support, I think using a string representation is fine. However, one might decide to add a more specific type, similar to https://schema.org/DateTime that describes the expected format. In effect, one could do this by expecting a certain type in the schema, using additional vocabulary or just keep it to the use to add:

{
  "@context": [
    "codemeta...",
    {
      "SemanticVersion": {"@id": "https://...hermes-context.../SemanticVersion"},
      "version": {"@type": "SemanticVersion", "@id": "schema:version"}
    }
  ],
  "version": "1.2.3"
},

{
  "@context": [
    "codemeta...",
    {
      "SemanticVersion": {"@id": "https://...hermes-context.../SemanticVersion"},
      "hermes:version": {"@type": "SemanticVersion", "@id": "schema:version"}
    }
  ],
  "hermes:version": "1.2.3"
},

{
  "@context": [
    "codemeta...",
    {
      "SemanticVersion": {"@id": "https://...hermes-context.../SemanticVersion"}    }
  ],
  "version": {"@type": "SemanticVersion", "@value": "1.2.3"}
},

• More complex license information is complicated right now, standards like SPDX are still developing.
  For now, I think we should not enter this discussion too much and be happy what is available - we can already assign a number of different licenses (as noone verbids us to store arrays behind the license key).

[sdruskat]

* More complex license information is complicated right now, standards like SPDX are still developing.
  For now, I think we should not enter this discussion too much and be happy what is available - we can already assign a number of different licenses (as noone verbids us to store arrays behind the `license` key).

Hm, I don't know. This is another case where CodeMeta falls behind the state of the art, as it doesn't support composite license expressions. Perhaps we should define license in our own vocabulary as taking either a composite license expression string, or look at the RDF representations 🤷?