New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up own F-Droid repo to push urgent updates to users #1981

Open
AltNico opened this Issue Jan 1, 2019 · 3 comments

Comments

Projects
None yet
4 participants
@AltNico
Copy link

AltNico commented Jan 1, 2019

At 35C3 there was a F-Droid meetup where @theScrabi showed up and explained your need at NewPipe to sometimes quickly push updates to F-Droid. This is totally understandable and we at F-Droid would really like to offer quicker updates through our infrastructure, but at the moment we can't improve publication time without reducing F-Droid's security standards.

To give some insights: as you all know, at F-Droid everything gets built from source. Unlike Google Play, this means that the building process isn't decentralized at developers' infrastructure, but we rather have a central place where everything is built. This is a great target for people to infiltrate F-Droid's infrastructure and get onto hundreds of thousands of people's devices. Therefore, we have quite high security standards here and run builds only on fully trusted hardware, maintained by fully trusted F-Droid members.

There are plans to decentralize F-Droid's build infrastructure to let builds run on multiple semi-trusted servers run by semi-trusted F-Droid members, but those plans require a lot of work and aren't reachable within the next months. Anyway, what's needed for them are apps to be built reproducibly. So most of the work proposed in this issue needs to be done anyway when F-Droid starts to offer quicker updates.

What I propose to NewPipe (and any other app that wants to distribute quicker updates for apps installed through F-Droid) mainly consists of two parts:

  • Set up a F-Droid repo where you publish official builds of NewPipe that can be reproducibly built by F-Droid
  • Include F-Droid's update channels library into NewPipe to inform users about urgent updates

Let me explain this a little bit further.

People around F-Droid worked hard to make reproducible builds possible. What this means is that F-Droid tries to build an app and then compares this build with the app provided by upstream. If those builds match, F-Droid publishes the upstream .apk. As of now, there are two reproducibly built apps in F-Droid: Öffi thanks to @schildbach's and Briar thanks to @grote's work.

This alone doesn't solve F-Droid's slow publication time yet. It still needs the same amount of time to publish apps, in some cases it could even take more time if there are problems with apps' reproducibility.

However, one important aspect changed: the apps are now signed by upstream, meaning upstream can publish updates which can be installed seamlessly by users.

Here, F-Droid's update channels come into play: if F-Droid for whatever reason needs too long to publish an update or upstream wants to publish an urgent update almost instantly, that update channel library can be used to notify users about an available update that's so far not yet in F-Droid's official repo, but which can be installed from upstream's repo.

The implementation and UX would be like this:

  • On the app's first run, users can opt-in to get notified by upstream about urgent updates.
  • The app then regularly checks if there's an urgent update available by upstream. This can be a simple JSON file inside NewPipe's source code repo which contains the version of the app that every user should at least have.
  • If that version is newer than the version installed on the user's device, F-Droid's update channel runs and notifies the user about an update. The user can then click on that notification which results in the update being downloaded. It can then be installed with another click, opening Android's package manager. The UX is the same like inside F-Droid's app, only that it's inside NewPipe and doesn't touch F-Droid at all.

Be aware that implementing F-Droid's update channel could result in the app being kicked out of Google Play. This isn't a problem for NewPipe but I state it here for everyone else reading this. Even if the mechanism is deactived, an app that includes update channel's code violates Google Play's guidelines. For example, Öffi got kicked out of Google Play for including a deactived code which would show a donation button to its users.

By using F-Droid's update channels, you also get another feature for free: when people install .apks from F-Droid without installing the F-Droid app, they normally don't get updates for it. With that library, they would get notified about updates and could install it conveniently with some taps on the popping up notification.

Of course you can also use this F-Droid repo of NewPipe for more cool stuff. People can add it to F-Droid like any other repo and you could push beta and nightly builds to that repo, with those builds automatically checking for updates and proposing them to users all the time.

Some more interesting links:

  • At verification.f-droid.org you can see if your app is already reproducible and what's missing in case it's not. Just search for NewPipe and watch out for the build's date.
  • The build scripts of Briar and Öffi. Note the Binaries entry and the use of disorderfs at Briar's side. This is an essential hack because as of now, it's not possible to build Android apps reproducibly with standard Android SDK tools! (This was reported to Google several months ago, but so far they didn't react.)
  • On GitLab F-Droid's nightly repo is hosted.

The F-Droid documentation also contains a lot of useful information:

I'm CCing @eighthave and @Bubu as they were also at the meetup and possibly know more about my proposed stuff. Also, so far nobody has done this. I'm proposing it to NewPipe because it doesn't have that problem with being on Google Play and could serve as an example app for other apps. This is of course also interesting for Conversations, where @iNPUTmice attended the F-Droid meetup, but they have the problem of being on Google Play and so far don't publish official builds outside of Google Play.


@schildbach

This comment has been minimized.

Copy link

schildbach commented Jan 1, 2019

Thanks for this extensive writeup! I'd like to add that Bitcoin Wallet is reproducable as well and it's interesting for the fact that it uses the backwards-compatible way with dual signatures. Existing users will get updates signed by F-Droid; new installs will get the developer signed version.

The configuration metadata can be found in this directory: https://gitlab.com/fdroid/fdroiddata/tree/master/metadata/de.schildbach.wallet/signatures

@eighthave

This comment has been minimized.

Copy link
Contributor

eighthave commented Jan 2, 2019

@uniqx and I did a lot of work to get the multi-signature support working, happy to hear that it is appreciated

@licaon-kter

This comment has been minimized.

Copy link

licaon-kter commented Jan 19, 2019

This is the right time...

gkeegan added a commit to gkeegan/NewPipe that referenced this issue Jan 21, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment