Release url

[Jehan]

This is a proposition for allowing <url> inside the <release> tag as discussed in the pull request #158.
We do very nice news post on gimp.org now, with a lot of screenshots, explanations and much more, and a link to these would be very good in a release listing for people who want to know more.
And for release notes too. :-)

[Jehan]

Ok something is wrong with this PR. Why does it say it has 3 commits?!

[Jehan]

Ok I fixed the branch. Something was wrong with my rebasing obviously. All good now. :-)

[hughsie]

I'd rather see <screenshot> tags inside the <release> tags. It means we can show something sexier in gnome-software...

[Jehan]

I'd rather see <screenshot> tags inside the <release> tags. It means we can show something sexier in gnome-software...

That is not contradictory. We can improve the release information within the appstream metadata (I think adding per-release screenshots is indeed a very good idea) while still acknowledging the fact that release information also exists through other medium. There are several reasons why this is important:

(1) When a project (such as GIMP) publishes news and release notes on the web, we are not going to stop doing so in favor there are appstream metadata (which is not even used by all OSes; only the web news is usable everywhere across platforms, hence is a preferred medium). So we are duplicating the work.

(2) Our web news are edited multiple times for days and even weeks before a release and we usually do the last edits just hours, if not minutes, before the actual release. This means that the web news will be usually English only at first. This is not a big problem since we can still translate the news in other languages later on, if we want. The link will stay the same anyway so the same link will become multi-language information over time.
On the other hand, the appstream metadata needs to be finished and freeze long enough before release so that it can be translated in as many languages as possible, because the translations are all included together with the release. If it has no translation by the time we release, this is too late: people in all language apart from English won't have any release description. We cannot add the translation later (well we can: at next release! This means that release information in non-English will always be 1 release later, making it a lot less useful).

Web news and appstream release description is definitely not a similar process, writing and localization-wise, therefore the release info within the metadata just cannot be treated the same as the web news.

(3) In any case, the <description> formatting is currently just too basic (just paragraphs <p> and lists, no kind of emphasis — bold nor italic, no code formatting — inline or in blocks, no links, no videos which we also have a lot in GIMP news to demonstrate some more complex features, no titles, even less several levels of titles, no quoting, basically nothing in advanced formatting).

So yes, we can still add <screenshot> tags inside <release>, I think this is a very good idea. And as I said, this is not a contradiction to my proposition. That is a first step towards improved appstream's release description and indeed will make it a bit more sexy and interesting. But this is still not enough and with the current lack of formatting, we cannot have long news because they will just be huge blocks of unreadable text with no titles to separate things nor formatting to make things more digestible.
This cannot be compared to a well formatted release note with full-featured HTML.

(4) Even if <description> suddenly got all the features of HTML (hence making my point (3) void, yet neither (1) nor (2)), we don't really want to just make a copy of our web news inside appstream.
First because I really don't think that appstream should be that detailed (that's made to be seen in a software installer or similar interfaces; would you really display huge news?). But also because if we were to do this, we don't want the versions to diverge or fall behind if we were to update one. Basically we'd want to keep them in sync automatically.
That would be hard since one is non-updatable after the release by definition. Also anyway the web not being semantic enough, we all know that this is an annoying task to do.
I mean, that's just a start for huge headaches!

(5) Finally we have web links, they are made to be used, aren't they? Why would we go all the way to absolutely embed all the same data as we already have inside appstream? It's not like appstream is going to replace the web (or is even competing at it since that is not meant to!) anytime soon. ;-)
Web links are made to be shared. Is there a reason why you would not like to?
I don't see a problem to have some good enough appstream description (yet not as detailed as the web one) of a release, maybe with some screenshot of the coolest features indeed, and then a "know more" link which would send to much more detailed and fancy web page with videos, images, parts, code examples (for GIMP plug-in API, we have some, sometimes), and more.

Here some examples of news for the 2 latest development releases of GIMP:
https://www.gimp.org/news/2017/12/12/gimp-2-9-8-released/
https://www.gimp.org/news/2017/08/24/gimp-2-9-6-released/
…

And the news of GIMP 2.10 will be even huger since it could encompass all the info of all 2.9.x releases!

[ximion]

Quite honestly, I think adding the release information to AppStream metainfo files is a flawed concept from the start, and I am not sure if we should continue this path and add even more stuff to the release tag.

Instead, I think it might make sense to create a new AppStream-derived file format (based on XML or something else) that is linked from the metainfo file. So the metainfo file would just contain information on which releases exist or even just which release is "current", while all other information (are there new releases? what is the description of the new release? potential screenshots?) is fetched from a web URL referenced in the metainfo file.
Linux distributions could mirror that information, similarly to how we already deal with screenshots.

Doing that would kill a lot of issues, like the GIMP case of release information being updated post-release and the problem of having to touch the metainfo file so often to make a new release. If a different format from XML is allowed or chosen for writing release details, it might even resolve the "writing release descriptions in XML is awful" issue (although I don't know yet if that really would be a good idea).

I am - at the moment - a bit opposed to adding a release-URL tag, because I think projects will use this to not write release descriptions for AppStream anymore, and instead just use the new url to redirect to their web pages. This will overall result in an awful experience when people click on web-URLs in software centers to get update information, especially if no release description is written at all.

[hughsie]

is fetched from a web URL referenced in the metainfo file

We're making gnome-software work more and more "offline" for internet-expensive deployments.

the problem of having to touch the metainfo file so often to make a new release

Well, either you have translated release notes, or you don't. I don't see how splitting this to another file actually helps anything.

and instead just use the new url to redirect

This is a valid concern. We could certainly make this a validation failure.

[Jehan]

and instead just use the new url to redirect

This is a valid concern.

I don't understand this fear of URLs. Web news won't disappear, even if the formatting capabilities of appstream were the best in the world.
In any case, for GIMP, we will fill the <description> of <release> (future ones, we won't go back in time on the dozens of release we had; we just added dates for these). We won't just put a URL because (1) the field exists so we may as well fill it well (2) I assume that a release with a description properly filled will be better rendered than one with none or just a URL and (3) as you said, not everyone has network!

Yet that is clear that we will always put more efforts on our web news, however fancy is the appstream format. The web news will obviously remain our most read text regarding a new release for a foreseeable future, not appstream. Also I don't think that the appstream format is really meant to have overlong and detailed release descriptions, am I right?

We could certainly make this a validation failure.

As I said, we will add release descriptions for now on, but we won't do archaeological research for past releases. I already had all past twelve 2.8.x releases with only the release dates. If you add a validation test on presence of a description, we will likely have to remove our past releases (unless someone else wants to play archaeologist, that won't be me).
Or make it more subtle, like only the last version must have a <description> to validate, or whatnot.

Well, either you have translated release notes, or you don't. I don't see how splitting this to another file actually helps anything.

The difference is that appstream data is distributed within the software package, which means that if release notes are not translated by the day of the release, it is too late.
From what I understood, the proposition of @ximion implies a file on the web, hence which can still receive updates after releases for GNOME Software (or others) to be able to add more translations after release, for instance, or fixes to the release data.
This being said, your concern of wanting to be as offline as possible is very valid. Appstream data being delivered with the software means once installed, it doesn't need network.

As a conclusion, I don't have really an opinion on whether it is better to have a new linked format or just improve the current format.

But what I can say (to focus back to the topic at hand) is that web URLs exists, whether appstream exists or not, and this will likely stay the preferred way of communicating fancy information to people for a long time. Trying to deny this is not the best idea IMO.

[Jehan]

For information, our current <release> tag for the last dev release (2.9.8), as proposed by a contributor. It ends with a link to the release website post: https://git.gnome.org/browse/gimp/tree/desktop/org.gimp.GIMP.appdata.xml.in.in#n119

In appstream-util, strict validation fails with:

• style-invalid         : <p> cannot contain a hyperlink [For more information, see https://www.gimp.org/news/2017/12/12/gimp-2-9-8-released/]
• style-invalid         : <p> does not end in '.|:|!' [For more information, see https://www.gimp.org/news/2017/12/12/gimp-2-9-8-released/]

And anyway it is not as a nice as whether we could just use <url> for installer software to do something with it.

[ximion]

@hughsie

is fetched from a web URL referenced in the metainfo file

We're making gnome-software work more and more "offline" for internet-expensive deployments.

And I am all for that, but release information does not need to be present offline. You may want to show it when browsing the catalog, in which case we already download screenshots and potentially other stuff, or when upgrading software. And in the "upgrade software" case, and internet connection needs to be there anyway.
Also, the AppStream metadata generator on the distro side could be instructed to include e.g. information on the last release into the collection data, even if the release info has been split out.

and instead just use the new url to redirect

This is a valid concern. We could certainly make this a validation failure.

What I am concerned about is not just making the release data a solution that works. That is an easy task. It absolutely has to be a solution that upstream projects want to use, and ideally want to use as their primary means of distributing release information. If using the release tag is annoying or if it is not featureful enough, we will get sub par metadata, or upstreams will just insert a weblink instead of actually useful information (at the same time, of course, there must be a limit on what is supported in the release tags).

[Jehan]

It absolutely has to be a solution that upstream projects want to use

Indeed.

and ideally want to use as their primary means of distributing release information

I'm sorry but it won't (or at least not in a foreseeable future). There are a few obvious reasons to this:

1. AFAIK this is a Linux-only feature. Well I'm not sure if appstream is available on other platforms, but if so, it is not widely used yet, right? GIMP for instance is also released on Windows, macOS, BSDs. So release information will need to be published through platform-agnostic (and widely spread) technology.
2. As already discussed on issue How detailed should be <release> tags? #171, AppStream <release> tag is meant for different purpose. I quote @hughsie:

I would expect the release info to be readable (so less than say 4 paragraphs) and it should be designed for a non-technical user. The primary purpose is so the user can decide "do I want to do this update" and the secondary purpose IMHO is for marketing.

Even our website news are beyond this description. They are longer, with a lot of links, images, informative texts, and with advanced/detailed description on the most noteworthy features.

3. Anyway the web is currently the first medium for distributing news. This is not even a question of technology. That's just how things are currently. Maybe things will change in 10 years, I don't know. But for now, this is our project "primary means of distributing release information" and this won't change in a day. And I am guessing that this is the case for most projects.

So hoping for appstream to become the new primary means, why not as a very long term goal. But that is just not realistic for short term and such a description is the best way for projects to actually not want to use <release> tags.

Once again, I am not saying this against appstream. I think it's cool technology/spec. Don't misunderstand. I just hope you can understand projects point of view. :-)

[Jehan]

Could we revive the discussion?
I am really annoyed by our current usage of appdata <release> even though I am the one who pushed for this on GIMP (actually I am basically the only one who seem to care about updating these).

Basically we are told by appstream-util validation (and @hughsie) that our <description> for <release> are too long. I wouldn't mind so much if we had <url> so that anyone who wants to know more about a release can just click to get to our blog post we post on gimp.org at every release.

Right now, I feel like we are stuck in this half-assed situation where we are forced to write down very uninformative release notes (in appdata) while forbidden to link to the more complete notes with nice descriptions, screenshots and images.

[hughsie]

I'm now a proponent for a <url xml:lang="fr_FR" type="more-info">https://gimp.org/notes.fr.html</url> that's available for the release tags. It's certainly something I can see being supported in gnome-software.

[ximion]

I am not against adding a (localized) url tag to release for more detailed release information on upstream's website, quite the contrary. I don't want to add it though before rethinking and updating the current workflow of how release notes are written in metainfo files, because otherwise we will end up with a situation where the existence of the url tag will become a convenient excuse to not improve the release tag, and a convenient way for upstream projects to not write proper release notes in metainfo files at all (I fear you'd end up with one paragraph saying "New release" followed by a read more link, which renders the whole release notes in AppStream pretty useless).

In any case, this is on my long todo list, but unfortunately so far I haven't had the time to work on it.

[ximion]

I still want to improve the releases workflow to make people use it better, however, adding an URL to releases to point to detailed release notes makes sense, so that is now in the spec.
I will merge this PR and then adjust the specification wording to match the current implementation, as discussed with @hughsie on IRC.
See the linked commit above for the implementation.
Thank you for your patch and patience, @Jehan !

[Jehan]

Thanks all! I just updated our appdata file to add URL "details" links. :-)