Add <internet/> element to <requires>/<recommends>

[pwithnall]

It would be interesting to expose metadata about whether an application requires or recommends internet access to work, or whether it can work fully offline. For users who have limited or sporadic internet access this would allow them to choose applications which work well in those situations. Such users could, for example, be on a mobile data plan for their internet (which runs out), be travelling with patchy signal, or be on a satellite internet connection which has high enough latency to make it annoying to use.

This is something we’d be interested in presenting in the hardware context tile in gnome-software, which currently summarises the existing <requires>/<recommends> elements.

Suggested XML would be an <internet> element whose value is the minimum internet speed the app requires, in megabits/second. A value of 0 would mean that the internet is required, but a specific speed isn’t needed. An offline="yes|no" element would indicate whether the app can work when offline (by pre-caching content).

Using <internet> inside <requires> would indicate that the app is useless without some internet access at some point. Using <internet> inside <recommends> would indicate that the app can largely work without the internet, but its experience would be enhanced by internet access.

Examples:

<requires><internet>0</internet></requires> <!-- App can’t function without some form of internet connection -->
<recommends><internet>2</internet></requires> <!-- App works better if it has a >=2MBit/s internet connection -->
<requires><internet offline="yes">0</internet></requires> <!-- App can work if offline, but needs an internet connection at some point, perhaps when first run to download and cache some data -->

How does that sound?

/cc @wjt because you had some ideas about this

[wjt]

The speed case is interesting, I hadn't thought about that. The cases I was interested in are:

1. App does not need an internet connection. (Example: Encyclopedia on Endless OS, Contrast, Teleport)
2. App is enhanced by an internet connection but doesn't need one. (Example: web browsers, which can open local content; games like OpenTTD which are usable offline but really you want to go online at least once to download some campaigns; Vorta, which can be used to back up to local disks or remote disks.)
3. App is useless without an internet connection. (Example: Dropbox, Spotify, etc.)

(I mention Teleport as an example because it has the --share=network Flatpak permission. Absence of this permission implies the app is usable without an internet connection, but its presence does not mean the app needs an internet connection. man flatpak-build-finish also documents that --share=network is needed to fully use Bluetooth.)

[pwithnall]

• App does not need an internet connection. (Example: Encyclopedia on Endless OS, Contrast, Teleport)

This can’t be represented in my proposal above. Perhaps we need an additional high-level element to go alongside <requires> and <recommends> which is <unneeded> (better naming suggestion welcome). Then these apps would use <unneeded><internet/></unneeded>.

I think currently appstream uses the semantics that if <requires> is present, and a child element is missing from it (say, <input>keyboard</input>), that’s how you express that a keyboard is not needed. However, I don’t think that can work in a forwards-compatible way as new elements (like <internet>) are added to the spec. Metainfo files written against an older version of the spec, when interpreted by a newer client, would suddenly acquire meanings like “explicitly doesn’t need the internet” which the authors didn’t intend when they originally wrote the metainfo file.

That’s why I suggest a new high-level element like <unneeded> might be needed. Thoughts welcome.

• App is enhanced by an internet connection but doesn't need one. (Example: web browsers, which can open local content; games like OpenTTD which are usable offline but really you want to go online at least once to download some campaigns; Vorta, which can be used to back up to local disks or remote disks.)

In my proposal above, this would be represented as <recommends><internet>0</internet></recommends>.

• App is useless without an internet connection. (Example: Dropbox, Spotify, etc.)

This would be <requires><internet>0</internet></requires>. Or perhaps just <requires><internet/></requires> and my proposal can be modified to say that a missing value in the <internet> element is equivalent to a 0 value, i.e. no particular speed requirement.

[ximion]

• App does not need an internet connection. (Example: Encyclopedia on Endless OS, Contrast, Teleport)

This can’t be represented in my proposal above. Perhaps we need an additional high-level element to go alongside <requires> and <recommends> which is <unneeded> (better naming suggestion welcome). Then these apps would use <unneeded><internet/></unneeded>.

I'm still quite a bit against "negative descriptions" in AppStream metadata. The data should describe what the application can do, not what it can't. Same for the ongoing discussion of maybe adding a supported tag: An app would show what it can support, and everything else can be assumed unsupported.
So, in this case, unlike the application specifies that it needs an internet connection to work, it should be assumed that the app can mostly work without internet connection if that property isn't present.

I think currently appstream uses the semantics that if <requires> is present, and a child element is missing from it (say, <input>keyboard</input>), that’s how you express that a keyboard is not needed. However, I don’t think that can work in a forwards-compatible way as new elements (like <internet>) are added to the spec. Metainfo files written against an older version of the spec, when interpreted by a newer client, would suddenly acquire meanings like “explicitly doesn’t need the internet” which the authors didn’t intend when they originally wrote the metainfo file.

That’s why I suggest a new high-level element like <unneeded> might be needed. Thoughts welcome.

That is true, which means without a negative-X tag, software centers would either need to wait for app authors to adjust their metadata, or not wait for that to happen and put pressure on the app authors to fix their data. I think both would be fair.
What I am trying to avoid is a massive combinatorial explosion in AppStream, where you have requires, not-requires, suggests, not-suggests, recommends, not-recommends, supported, not-supported, etc. AppStream already defines a ton of relation types (extends/requires/recommends/suggests/provides), adding negations would make this really unwieldy at some point and much harder to give guidance to metadata authors on what to do. Also, software centers making decisions would become quite a bit more complex than just testing for "app provides X".

• App is enhanced by an internet connection but doesn't need one. (Example: web browsers, which can open local content; games like OpenTTD which are usable offline but really you want to go online at least once to download some campaigns; Vorta, which can be used to back up to local disks or remote disks.)

In my proposal above, this would be represented as <recommends><internet>0</internet></recommends>.

Jup, that makes sense to me too. If we would have a supports tag (which is still open for debate...) it would be a toss between recommends and supports (the latter being even weaker than a recommendation).

• App is useless without an internet connection. (Example: Dropbox, Spotify, etc.)

This would be <requires><internet>0</internet></requires>. Or perhaps just <requires><internet/></requires> and my proposal can be modified to say that a missing value in the <internet> element is equivalent to a 0 value, i.e. no particular speed requirement.

Also makes sense to me :-)

[pwithnall]

That is true, which means without a negative-X tag, software centers would either need to wait for app authors to adjust their metadata, or not wait for that to happen and put pressure on the app authors to fix their data. I think both would be fair.

I don’t think either would work. I don’t have evidence to back this, but from my experience with gnome-software it seems like there’s a long tail of apps who don’t update their metadata at all, and a small number of apps who are proactive adopters of new metainfo stuff. So waiting won’t work.

Putting pressure on app authors to fix their data also won’t work particularly well for most app authors — there’s not enough of a direct feedback channel from us to them. We could should incorrect data in the UI (in this case, that would mean saying an app doesn’t require internet access even though it potentially does) alongside a “Something wrong with this data?” link. But I think the number of people who will try to update the data will be vastly outnumbered by the people who will get silently confused by it.

I can’t think of another way to handle the transitional period without making it explicit which metainfo files have been updated with the correct <internet/> data.

What I am trying to avoid is a massive combinatorial explosion in AppStream, where you have requires, not-requires, suggests, not-suggests, recommends, not-recommends, supported, not-supported, etc.

I agree, but in this case I can’t see any other way.

Note that I’m not suggesting adding negations in general, just this specific one. I think unneeded exists at one end of the requires/recommends/suggests/provides/unneeded spectrum, rather than being one case of negating an element in that spectrum.

For example, I can’t see a not-recommends or not-suggests ever being useful.

🤷

[ximion]

I think it's actually okay for GNOME Software to simply say it doesn't know about certain data, rather than guessing. Instead of adding any "not" directives, we could also version metainfo files and ambed the AppStream spec version in them, but I can't see people ever updating that properly...

I agree, but in this case I can’t see any other way.
Note that I’m not suggesting adding negations in general, just this specific one.

So, I think what you actually want is something like this, assuming we will add a supports tag to the spec (which I warmed up to quite a bit in the past weeks):

• By default, no assumption about online/offline behavior can be made, GNOME Software simply doesn't know
• App author adds a <internet/> tag to requires: App needs internet of the given speed and will be useless without (in case of requires)
• App author adds a (new!) <offline/> element to a supports tag: App can work in offline mode and doesn't need internet

The "offline" tag could even be integrated with the "internet" tag somehow, for example <internet offline="yes"/> meaning offline-mode is supported, while <internet firstrun="yes"/> for the "only needed for initialization / caching" case (not entirely happy with this naming, so it's really just a brainstorming idea).

[pwithnall]

The "offline" tag could even be integrated with the "internet" tag somehow, for example <internet offline="yes"/> meaning offline-mode is supported, while <internet firstrun="yes"/> for the "only needed for initialization / caching" case (not entirely happy with this naming, so it's really just a brainstorming idea).

That could work, and would avoid the confusing possibility of an author adding <internet/> and <offline/>.

So that would give us:

<requires><internet/></requires>

Requires internet connectivity to work.

<requires><internet offline="yes"/></requires>

Does not use the internet at all.

<recommends><internet/></recommends>

Recommends having an internet connection, but it’s not mandatory.

<recommends><internet offline="yes"/></recommends>

Not sure what this means. Maybe: normally doesn’t use an internet connection but if it’s available the app will be slightly better? i.e. A weaker version of <recommends><internet offline="no"/></recommends>.

---

This still feels a bit odd to me. In particular <requires><internet offline="yes"/></requires>, which reads to me as “this requires the internet, lol no it actually doesn’t use it at all”.

How about setting the value of the <internet/> element to a well-known keyword? Appstream seems to not use that style of element much, so you might not be keen, but I’ll suggest anyway:

<requires><internet>always</internet></requires><!-- Always needs the internet -->
<requires><internet>offline-only</internet></requires><!-- Never uses the internet -->
<requires><internet>first-run</internet></requires><!-- Requires the internet on first run -->
<recommends><internet>always</internet></recommends><!-- Would work but a lot worse if there was no internet -->
<recommends><internet>offline-only</internet></recommends><!-- Never uses the internet -->
<recommends><internet>first-run</internet></recommends><!-- Can make use of the internet on first run but doesn’t require it -->

We could add a speed_mbitps="2" attribute to be used in the cases where the component can use internet, to specify a recommended minimum speed.

[ximion]

This still feels a bit odd to me. In particular , which reads to me as “this requires the internet, lol no it actually doesn’t use it at all”.

When revisiting this issue I also immediately stumbled over that and my initial thought was "this combination sounds like a logic error". So I think this definitely should be <internet allows_offline="yes" />

How about setting the value of the element to a well-known keyword? Appstream seems to not use that style of element much, so you might not be keen, but I’ll suggest anyway:

This style is actually used for <control/>: https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#tag-relations-control - so there is a precedent, and this would actually be easy to implement too.

I actually think your suggestion, including the value names and explanation, would make the most sense here!
If you want to create a PR against the specification, please do! :-) (Otherwise I will get to it eventually)
Do you have a concrete application for the speed limit? I think that would be a rarely used feature and it will be very hard for clients to actually measure that and take action based on the speed requirement set...

By the way, since we have so many of these requirements for software centers to implement now, I am planning to add API for that to libappstream which will try to test for as many of these as the library can (so, no screen size stuff as libappstream must not depend on a GUI) and delegate the rest to a delegate function that the software center can implement. Otherwise we will soon duplicate a crazy amount of code between KDE Discover, GNOME Software and Elementary AppCenter, and software centers may even interpret these tags differently.

[cassidyjames]

Inspired both by the GNOME Annual General Meeting at GUADEC and my work with GNOME and Flatpak apps in an offline deployment, I came to file this exact issue. 😄 We constantly hit the issue where a deployment partner requests using an app, and we later discover that app requires an Internet connection at some point either for perpetual functionality or for a one-time download on launch.

So for us at the Endless OS Foundation, being able to query Flathub apps or at least give our deployment partner a badge/kudos to look for would lead to fewer issues with getting apps that require connectivity.

Identifying and promoting these apps would also seem to be in line with GNOME's goals of local-first computing, so there may be wider interest than just from the Endless OS side. But to do that, we need the right metadata.

[cassidyjames]

Another angle to approach this would be from the GNOME kudos side, which appears not to be standardized or documented in AppStream itself. But I could see kudos for local-first or offline-capable being a thing that exists.

[pwithnall]

Another angle to approach this would be from the GNOME kudos side, which appears not to be standardized or documented in AppStream itself. But I could see kudos for local-first or offline-capable being a thing that exists.

Kudos are deprecated in gnome-software and we’re moving away from them and just using standard appstream instead :)

[cassidyjames]

@pwithnall TIL! In that case, it's more rationale to include something like "offline capable" and/or "Internet required" to AppStream, imho.

[ximion]

That has always been more reasonable ;-) A proper generally applicable solution is so much better than using a custom, project-specific hack.
I still think @pwithnall 's last suggestion is the most readable and unambiguous one, so what this needs now is a specification draft and implementation.... ;-)

[pwithnall]

Do you have a concrete application for the speed limit? I think that would be a rarely used feature and it will be very hard for clients to actually measure that and take action based on the speed requirement set...

The concrete application I had in mind was games. Some of them (busy online multiplayer games) are unusable without a certain amount of bandwidth and latency.

I’ve added wording and support for a bandwidth_mbitps attribute in my PR for this (#421), but you can drop it if you want.

Ready for review: #421