Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Links to IzzyOnDroid repo? #302

Closed
IzzySoft opened this issue Aug 18, 2023 · 16 comments
Closed

Links to IzzyOnDroid repo? #302

IzzySoft opened this issue Aug 18, 2023 · 16 comments
Assignees

Comments

@IzzySoft
Copy link

The IzzyOnDroid Repo is the largest 3rd-party F-Droid repo with currently about 1.100 apps listed. It can be easily added to the official F-Droid client (and is even pre-configured with several others like NeoStore). Maybe you can add links to it as well where FOSS apps are available which are for some reason NOT (yet?) at F-Droid.org? If you wish to filter for eligible candidates: the repo features a custom anti-feature NonFreeComp (app contains non-free component(s)) for apps you'd rather wish to skip. Apps not carrying that AF should meet your criteria (as well as F-Droid's).

Disclosure: I'm the one running that repo.

Disclosure 2: I'm also one of the F-Droid maintainers.

Answering the not-yet-asked question: Some authors wanted their apps in my repo and not at F-Droid, for different reasons. And some apps are difficult to build by F-Droid and thus are not (yet) listed there. My repository basically follows the same guidelines as F-Droid's, with 3 exceptions: it does not build from source but uses the APKs provided by the developers, it accepts apps which are almost-but-not-fully meet F-Droid's inclusion criteria (those apps are marked NonFreeComp as they have a very limited amount of non-free libraries) – and there's a size limit as my repo runs on personal resources.

I gladly answer any questions when there are some left. Thanks in advance!

@offa
Copy link
Owner

offa commented Aug 18, 2023

Thanks for your suggestion @IzzySoft.

My reasons for not including 3rd party repos so far:

  • Trustworthiness of unofficial repos (eg. who maintains it, security, integrity of packages?)
  • How to ensure user privacy?
  • How to ensure that only FOSS packages are listed and avoid proprietary not slipping in (eg. with an update)?
  • Avoid repo cluttering
  • Avoid one-project repos or providing just a few entries
  • Require additional setup steps
  • How long will they be available?
  • Simplicity (if not FOSS only)
  • Are packages updated on new versions?

The official repo makes things easy: There's not much to care.

However, your repo is some kind of special as it's somewhat known in the community already (and of course your role 😄). So in this case I do not have a strong opinion in either direction.

those apps are marked NonFreeComp as they have a very limited amount of non-free libraries

Is the label available on the client too?

@IzzySoft
Copy link
Author

My reasons for not including 3rd party repos so far

Reasonable questions to ask – so let me be reasonable as well and try to answer them.

Trustworthiness of unofficial repos (eg. who maintains it, security, integrity of packages?)

My repo contains a details link which addresses this. In short: I maintain it, security measures are in place as outlined behind the link (automated scans by VT and my own library scanner, the latter is also used by F-Droid.org; results of both are clearly shown when visiting the repo with a web browser – unfortunately there's no way to transport that to the index itself; integrity of packages is ensured by "pinning" the hashes of their signing keys (AllowedSigningKeys; if an APK comes with a key not explicitly configured for it, it won't make it into the index until clearly resolved – I can detail on this if needed).

How to ensure user privacy?

No cookies, no JavaScript, no trackers. Webserver logs are rotated to /dev/null (just kept long enough to make sure any issues can be resolved should they arise). That's a principle applying to all my websites btw.

How to ensure that only FOSS packages are listed and avoid proprietary not slipping in (eg. with an update)?

Same library scanner again – and see NonFreeComp above. Other than a changed signature, such an APK might (shortly) make it to the index (only with app updates, not when an app is newly added) but will be dealt with immediately (e.g. the affected version removed, updates disabled until resolved). As already indicated, apps marked NonFreeComp you most likely want to skip.

Oh, and if an app which didn't have NonFreeComp set before suddenly adds something proprietary? My updater will yell at me screaming "fix that or I'll annoy you until your brain leaves through your ears!" 🙈 Honestly: I have a checker in place running at the end of each update cycle, reporting any deviations immediately by mail to me.

Avoid repo cluttering

Understandable. My argument on this is that my repo is the largest 3rd-party repo by far (as pointed out, ~1.1k apps) with only F-Droid.org itself being larger (currently ~4.2k apps) and "number 3" having far less than 100 apps (at least from the repos I know).

Avoid one-project repos or providing just a few entries

Yupp. Does not apply to mine I'd say, as outlined already.

Require additional setup steps

They are minimal, depending on the client used. With NeoStore, my repo simply must be enabled. With the official repo, just scan the QR code to add it (spoiler: the official client will soon™ support the QR Code scan directly) or type in the address by hand. But admitted, yes: it's an additional step.

How long will they be available?

"They" being the apps in the repo, or the repo itself? I cannot "guarantee" for either. But given that my repo exists since … oof, 2015/2016 and is integral part of my "suit", I certainly hope it stays around the next years 😉

Simplicity (if not FOSS only)

As I don't know what exactly you mean here, I must guess: Yes, all apps are "FOSS". I only accept apps with their source openly available, covered by a FOSS license approved by OSI/FSF (there are a handful older apps with unclear license which I'll probably "weed out" soon – those were from the very early days when I was still trying to get things working). As with F-Droid itself, licenses are clearly shown. Already said, but relevant here: FOSS vs F/LOSS. Some apps have few non-libre components, see NonFreeComp. These are "accepted to a degree" (see the linked details) and clearly marked NonFreeComp.

Are packages updated on new versions?

Only if the authors/developers provide such 🙈 But yes: my updater runs once a day to check the corresponding repositories and fetch the updates. For some apps which somehow "fell dormant" (no updates for a year), updates are only checked monthly. Very few apps require manual updates, which I try to check once a month as well (luckily those don't have frequent updates). And several apps simply no longer receive updates as their development ceased, their repository was archived or gone entirely. So again, pretty much the same handling as on F-Droid.org, just that updates in my repo are usually a little faster as I don't build apps from their sources myself but simply fetch the developers' APKs.

Note: Should an app no longer receive updates, that's clearly pointed out as well. Either with the app's description (if the source still exists but development has come to a stand-still or the repo was even archived – or via the standard NoSourceSince anti-feature F-Droid.org itself uses as well if the repo is gone or an app turned proprietary.

Is the NonFreeComp label available on the client too?

Depends on which client we speak of here. The official client shows it as it shows the official anti-features (at least when using the latest version). NeoStore shows it but currently without any description, just stating something like "unknown anti-feature" (NeoStore still uses index-v1 but will be updated to use index-v2 soon, where custom anti-features are supported). I cannot speak for the other clients as I didn't test them.

However, your repo is some kind of special as it's somewhat known in the community already (and of course your role 😄).

I cannot deny either of this, thanks 😆 I hope my explanations help you decide. Being a maintainer at F-Droid.org as well, I aim for maximum transparency and try to implement as much of the "safe-guards" F-Droid offers (like the AllowedSigningKeys mentioned above, which to my knowledge my repo is the only one applying it to ALL apps inside).


Should you have any additional questions, I try to answer them to my best knowledge. You find more details including most of the code etc. used with my repo here by the way – where people also can recommend apps I should take in, report issues or make other suggestions.

@IzzySoft
Copy link
Author

IzzySoft commented Aug 18, 2023

PS: You might ask why I permit NonFreeComp at all. So just to clarify: the idea is not to legitimate the use of proprietary libraries in FOSS apps. It's rather to motivate developers to go "full FOSS" and eliminate the few remaining "F-Droid blockers" as well, to then move on to F-Droid.org when they finally meet the inclusion criteria there. This approach is not always successful, but quite often. Almost 500 apps have moved "through" my repo that way. More than 400 are solely on F-Droid.org now, the others are still in my repo as the authors asked me to keep them there. Seeing there are 356 apps with anti-features in my repo currently, that means it was (and is) a 50:50 chance – which I'd say is definitely worth it.

Should an app go the other direction (i.e. add more proprietary stuff than it initially came with when added to my repo) – see above. At one point updates will be disabled with the authors informed – and if it cannot be helped, the app is unlisted from my repo (which unfortunately had to do more than once already, but luckily not too often; more like it was just dependencies slipping in unnoticed, and the developers fixed that so the app could remain and receive updates again).

Again you see, pretty much like it would happen on F-Droid.org 😉 There the corresponding build would be disabled (i.e. "no update"), and later either newer builds were fixed or the app would be archived. Only that my archive is /dev/null

@offa
Copy link
Owner

offa commented Aug 18, 2023

Thanks for your detailed and extensive answer!

Just to be clear, my concerns are about 3rd party repos in general, not specific targeting yours :-).

"They" being the apps in the repo, or the repo itself?

The repo itself, but applies to apps too. Your repo wont have issues with that 😉.

As I don't know what exactly you mean here,

Sorry, some of my points are written more like bullet points without much context 🙈.

If all entries are FOSS it's simple to stay FOSS only; if not, one has to figure out on it's own.
F-Droid is a no-brainer: Select whatever you see and you'll get FOSS. The other extreme would be e.g. Play Store: There are FOSS packages around (at least I guess), but figuring those out might be complicated.

It's rather to motivate developers to go "full FOSS" and eliminate the few remaining "F-Droid blockers" as well, to then move on to F-Droid.org when they finally meet the inclusion criteria there. This approach is not always successful, but quite often.

Indeed, thank you very much for doing this! 😄


You have answered all concerns and I don't have any objections 👍.

So what's to do now?

  • Add links to entries
  • Update contribution guide
  • Link example – is https://apt.izzysoft.de/fdroid/index/apk/<app id> ok, or is there another / preferred format?
  • Add a link to Getting more

I'll create a PR for those.

@IzzySoft
Copy link
Author

Just to be clear, my concerns are about 3rd party repos in general, not specific targeting yours :-).

Understood that from the start – but I was applying for my repo 😊

Play Store: There are FOSS packages around (at least I guess), but figuring those out might be complicated.

That's putting it mildly. Some link to their git repos (usually Github) so it's "easy" to verify – others just "claim" to be FOSS (fat chance to go searching and find out)…

You have answered all concerns and I don't have any objections 👍.

Yay, thanks!

And yes, you can use that link syntax – or simply the same as with F-Droid: https://apt.izzysoft.de/packages/<applicationId>/ is set up to be compatible here (originally so that app links shared from the F-Droid client know where to link to). Both work. I made a few mistakes in the "early days", today I would not have put /apk/ in the path but rather /app/ 🙈

So what's to do now?

if I can help with details, just let me know. I certainly could e.g. produce a link of package names available in my repo, connected with their source location (that one is easy: grep "Source:" *.yml) and maybe filter out "unwanted" candidates (no longer updated, having NonFreeComp etc: for app in $(grep "Source:" *.yml); do; if (has_what_we_do_not_want) continue; etc pp; echo "$name $repo"; done – something like that; maybe better going by the database a la SELECT name, category, shortdesc, source .. where no_bad_things=1;). If you want such a list that is.

@offa offa self-assigned this Aug 20, 2023
offa added a commit that referenced this issue Aug 20, 2023
offa added a commit that referenced this issue Aug 22, 2023
@offa
Copy link
Owner

offa commented Aug 22, 2023

I'm closing here, there are PRs with links in the pipeline already, contribution guide is updated 👍

Feel free to reopen at any time if there's still something missing.

Thanks again! 😄

@offa offa closed this as completed Aug 22, 2023
@IzzySoft
Copy link
Author

And to you! Great to see the list growing 🥳

@shuvashish76
Copy link
Contributor

@IzzySoft Little bit offtopic question, is there a way to contribute to your repo list for categories of apps?
The limitation of this list is we can't assign multiple tags/categories to it. I was thinking if I could contribute to maintain categories for your repo apps so that users can filter more directly from their choice of client. Tagging is the best way to sort apps similar to what Calibre does for sorting books.
I use other sources as reference for creating categories though not strictly followed.

Or you don't allow new* categories to sync your categories with official F-Droid list of categories? In that case I think I'll create a discussion on F-Droid though little hope for such change, because of different opinions.

@IzzySoft
Copy link
Author

@shuvashish76 All categories available at F-Droid are available in my repo, too – plus some more. So if it works with F-Droid.org, my list is even more granular. As meanwhile custom categories are possible, I'm open to suggestions – but I wouldn't add too many of them (usually a category should at least have about 10 apps using it to make sense).

@shuvashish76
Copy link
Contributor

usually a category should at least have about 10 apps

Yes that does makes sense, users can use keyword specific searches in their choice of F-Droid clients anyway. 👍

@shuvashish76
Copy link
Contributor

Izzy has done really great job by compiling all the sources to a single list. I propose to add the https://android.izzysoft.de/applists/perms to "Tutorials and Guides" section as "Permissions by IzzySoft", @offa what do you think?

@IzzySoft
Copy link
Author

Note: Thanks to @shuvashish76 pointing it out to me, the mentioned list was just brought up to date (removal of dead links, adding fresh ones, updating risk ratings of perms according to OWASP, and some more).

@offa
Copy link
Owner

offa commented Feb 26, 2024

@offa what do you think?

👍 Added, thank you

And of course shout-out to @IzzySoft for doing the actual work 🎉.

@shuvashish76
Copy link
Contributor

shuvashish76 commented Apr 1, 2024

Info: Small comparison F-Droid vs Izzy repo.
image

Article by IzzyOnDroid: https://www.kuketz-blog.de/android-apps-auf-dem-seziertisch-eine-vertiefte-betrachtung/

@offa
Copy link
Owner

offa commented Apr 1, 2024

Thanks for sharing! 👍

@IzzySoft
Copy link
Author

IzzySoft commented Apr 4, 2024

Note that article triggered some activity at F-Droid finally, so the red fields for "Intent Filter" and "Self-Updater" might need to be changed into yellow "partials" (only at app inclusion, as they were just implemented a few days ago with F-Droid's Issuebot – which to my knowledge only runs automatically on RFP, i.e. Requests For Packaging of new apps, and manually on merge requests for new apps if someone remembers to trigger that). Not sure how reliable Issuebot is, as it seems to miss results from some modules every now and then; e.g. for my apk library scanner, I didn't see any results for months (and it would always report if it ran, and simply state if it didn't find anything – but not even that is mentioned in any reports for quite a while now).

More background in a different version of that blog article here: Ramping up security: additional APK checks are in place with the IzzyOnDroid repo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants