appstreamcli validate fails to identify WITH operator

[balasankarc]

I am trying to validate the following metadata xml file

<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2019 Swathanthra Malayalam Computing <contact@smc.org.in> -->
<component type="font">
    <id>in.org.smc.dyuthi</id>
    <name>Dyuthi</name>
    <metadata_license>CC0-1.0</metadata_license>
    <project_license>OFL-1.1 OR (GPL-3.0-or-later WITH Font-exception-2.0)</project_license>
    <description>
        <p>
            Dyuthi Malayalam Typeface. Ornamental font suitable for titles.
        </p>
    </description>
    <summary>Dyuthi Malayalam Font</summary>
    <url type="homepage">https://smc.org.in/fonts</url>
    <url type="bugtracker">https://gitlab.com/smc/fonts/dyuthi/issues</url>
    <update_contact>contact_at_smc_org_in</update_contact>
    <provides>
        <font>Dyuthi Regular</font>
    </provides>
    <languages>
        <lang>ml</lang>
    </languages>
</component>

and appstreamcli validate returns an error

$ appstreamcli validate in.org.smc.dyuthi.metainfo.xml
W: in.org.smc.dyuthi:7: spdx-license-unknown WITH

Validation failed: warnings: 1

The font is released under OFL-1.1 or GPL-3.0+ font exception.

Version 0.12.9 of appstreamcli

$ appstreamcli --version                                
AppStream version: 0.12.9

[ximion]

Oh, that's a new change - previously SPDX had this attached as suffix to the license name, like GPL-2.0-with-classpath-exception.
We need to validate this correctly though, as it looks like to do that properly, AppStream will need a list of accepted exceptions. I'll add this to the todo list for the next release.

[hughsie]

FWIW, appstream-glib only supports the old syntax, e.g. https://github.com/hughsie/appstream-glib/blob/master/libappstream-glib/as-utils.c#L388 -- I'm getting a bit tired of the SPDX "API" breaks.

[balasankarc]

@ximion @hughsie So, I assume tools like GNOME Software (which AFAIK uses appstream-glib under the hood) will fail to parse such licenses (that is GPL-3.0-or-later won't work, but GPL-3.0+ will) ?

[hughsie]

will fail to parse such licenses

We convert many of the new-style licenses into old-style. :/

[ximion]

AppStream does the exact same as appstream-glib, only the old syntax is supported.
Just when we adapted to the "then latest" syntax, SPDX updated again with yet another incompatible model, so at the point we stopped implementing it. It broke about three times so far, and I don't know if we have any guarantee that this was truly the last time.

that is GPL-3.0-or-later won't work, but GPL-3.0+ will

I think both should work in AppStream, they'll just get translated. We will also support a "GPL-3.0+" syntax probably forever, even if the SPDX and apparently FSF discourage that, otherwise we'd break backwards compatibility in a major way.

I do agree though that we should deal with this problem, either by properly implementing the new SPDX, or instead just not failing if statements like WITH are encountered.

[ximion]

I've implemented this now, so the next version of AppStream should recognize the operator.
License exceptions are also separated from licenses now, but share the same token prefix internally when tokenized. While feeling impure, this has a few advantages, especially since SPDX seems to pretty much treat license exceptions like licenses. It also avoids introducing more breakage.
Please test this if you can :-)

[balasankarc]

@ximion I built and tested it and it still seems broken to me (with the XML file in this issue's description)

$ ~/git/appstream/build/tools/appstreamcli --version                              
AppStream version: 0.12.10
$ ~/git/appstream/build/tools/appstreamcli validate in.org.smc.dyuthi.metainfo.xml
W: in.org.smc.dyuthi:7: spdx-license-unknown ^

Validation failed: warnings: 1

[ximion]

Jup, the validator didn't know about the change yet. Thanks for testing! This should work now, also for validation :-)

[balasankarc]

Tested and confirmed. Validator no longer complains. Thanks @ximion