Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FOSS build flavour for F-Droid #770

Closed
Bubu opened this issue Oct 9, 2017 · 54 comments
Closed

FOSS build flavour for F-Droid #770

Bubu opened this issue Oct 9, 2017 · 54 comments

Comments

@Bubu
Copy link

Bubu commented Oct 9, 2017

I just tried upgrading the app on f-droid, but I noticed that the recent releases (2.0.2 and 2.1) introduced crashlytics resp. google play services, which are closed source libraries.

Could you provide a gradle build flavour that doesn't use these libraries so the app can be updated on F-Droid again?

@andrewnenakhov
Copy link
Member

Can you contact me over email? info@xabber.com, got some questions for you

@Bubu
Copy link
Author

Bubu commented Oct 10, 2017

email sent

@cl0ne
Copy link

cl0ne commented Jan 15, 2018

Any news on upgrade on f-droid?

@andrewnenakhov
Copy link
Member

We'll attend to it once we'll fix some lags in modified interface.

@Human
Copy link
Contributor

Human commented Mar 19, 2018

@Bubu Other than a foss build flavor that doesn't include Google Play Services or Crashlytics, are there any other requirements?

@Bubu
Copy link
Author

Bubu commented Mar 19, 2018

@Human No, I don't think so.

@Human
Copy link
Contributor

Human commented Mar 19, 2018

@Bubu Where is the check done? On the resulting APK? You just run 'zipinfo' (or equivalent) on it to look for something like com.crashlytics and reject the build if it's found?

@Bubu
Copy link
Author

Bubu commented Mar 19, 2018

@Human There are some checks performed on the build.gradle files before building the app. See mostly here: https://gitlab.com/fdroid/fdroidserver/blob/master/fdroidserver/scanner.py#L65 (blacklist) and here: https://gitlab.com/fdroid/fdroidserver/blob/master/fdroidserver/scanner.py#L101 (whitelist of maven repos)

@Human
Copy link
Contributor

Human commented Mar 19, 2018

@Bubu My understanding is that a build flavor can customize how the project is built and what ends up in the APK, but it doesn't change what's in the sources. If the sources are scanned before even trying a build, I think it'll always fail the check here because several build flavors do use Google Play Services and Crashlytics, requiring the google repo and the crashlytics aar to be in the gradle files. So, even if a foss build flavor didn't end up putting them into the APK, the sources still reference them.

Because each build flavor does get its own src subdirectory, in theory, the foss flavor could have its own version of whatever java sources used google or Crashlytics blobs, but this would be akin to maintaining a fork or a special branch of the main project, since it'd always be behind the main build flavors and require periodic updating. And if your checks don't ignore proprietary blobs found in other src subdirectories, this would fail anyway.

@Bubu
Copy link
Author

Bubu commented Mar 19, 2018

@Human The scanner is pretty dumb but it does recognize the flavorCompile directives and only produces errors on those matching the flavour we are building.

The rest that ends up being in build.gradle but is not actually required for that flavour is mostly removed via a prebuild=sed -i -e '/fabric/d' build.gradle command in the fdroid metadata file or something similar to this.

This is done for a lot of apps that use a foss/non-foss flavour, you'll find plenty of examples, i.e. here: https://gitlab.com/fdroid/fdroiddata/blob/master/metadata/com.gh4a.txt#L91

@Human
Copy link
Contributor

Human commented Mar 20, 2018

@Bubu I can do some more investigation, but the offending libraries are referenced via classpath and not compile. There appears to be no flavor-specific classpath prefix.

@Bubu
Copy link
Author

Bubu commented Mar 20, 2018

Right, that is one of the cases where the fdroid recipe would sed out that line. It would just be worth testing that this fdroid/foss flavour indeed keeps working when these lines are removed/commented out.

@Human
Copy link
Contributor

Human commented Mar 22, 2018

@Bubu Do I also need to remove the Facebook library? com.facebook.android:facebook-android-sdk:4.31.0 I didn't find any facebook checks in the code you linked to earlier.

@Bubu
Copy link
Author

Bubu commented Mar 22, 2018

@Human That library seems to be open source: https://github.com/facebook/facebook-android-sdk

So there is no problem in including that into an fdroid build.

@Human
Copy link
Contributor

Human commented Mar 23, 2018

@Bubu Thanks. There's another snag I'm working through. There are several keys and secrets for 3rd-party login mechanisms stubbed out in ./xabber/src/dev/res/values/social_keys.xml. For dev builds, the app will build and run, but anything relying on these keys will fail. For prod builds, it is my understanding that core project members have a copy of the fleshed-out file locally. Since the foss build flavor is supposed to be "prod minus closed-source stuff" the missing keys and secrets are a problem.

You might want to ask a core project member for permission to maintain a copy of that key file in your build environment. Otherwise, the f-droid version will lack the login API features available to the Play Store version.

@Bubu
Copy link
Author

Bubu commented Mar 23, 2018 via email

@Human
Copy link
Contributor

Human commented Mar 23, 2018

@Bubu All of the keys and secrets are used in BaseLoginActivity.java The APIs are 3rd-party authentication to create / access a "Xabber account." Facebook, Google, and Twitter are supported in the code base. (The Github auth is stubbed out and nonfunctional at the moment.)

I have not personally tried any of the features requiring a Xabber account, and I haven't explored how deeply the integration goes. I don't know yet if it's something that could be turned on or off with a flag, or if the notion of a "Xabber account" is pervasive throughout the app. I'd have to investigate further.

In parallel with my investigation, would you find out if the core dev(s) would be ok with f-droid having access to the keys and secrets in some capacity? If they were to say yes, that might be simpler and faster than conditionally hiding the relevant UI.

@Human
Copy link
Contributor

Human commented Mar 23, 2018

@Bubu It's actually pretty simple to just bypass the splash screen that asks you how you want to log in, so if there's no way to get the 3rd-party credentials, it can be disabled at the root.

@andrewnenakhov
Copy link
Member

There are only two possible solutions to that problem right now:

  • Disable those features in the Foss build.

@Bubu we planned to disable social auth features in F-droid build for exactly that reason. If keys issued to us by FB&GOOG&Twitter are used in the wild, it can be used by anyone under our credentials, which is bad.

@andrewnenakhov
Copy link
Member

I would also ask to wait a bit more before building for F-droid, we're in the process of fixing some important bugs that plague current version, it'd be better to move to F-droid once we're finished with this phase.

@Human
Copy link
Contributor

Human commented Mar 23, 2018

@andrewnenakhov If you let me know when you'd be comfortable having an F-droid build, I can merge and continue the work. At the moment, I have a handle on all the required issues except for a clean way to avoid including crashlytics code. @Bubu The solution for that may have to be pre-editing the relevant lines out.

@gerroon
Copy link

gerroon commented Jun 2, 2018

Hi

Is there any update on this build work? I realize this is some work, I am just trying to see when we can have the latest version in F-droid

thanks

@andrewnenakhov
Copy link
Member

@gerroon sorry for delays. We're currently very overloaded making exciting new features like very nice and user-friendly media sending and also combating a significant amount of technical debt that surfaced in android version of Xabber. So knowing of some severe issues that resurfaced while introducing new functionality we have 2 options: to build from an already obsolete version of Xabber or to add some great features, fix everything, make it stable and then make a build for f-droid. We choose the latter.

@gerroon
Copy link

gerroon commented Jun 3, 2018

@andrewnenakhov

That is fair enough. I realized it must be a lot of work already. Take your time, Xabber is one of the most loaded Xmpp client already.

thanks

@IzzySoft
Copy link

May I ask what is the status here? F-Droid users certainly would be very excited for the exciting features as well – if they one day were available at F-Droid 😉

Reference: issue 1184 at fdroiddata

@IzzySoft
Copy link

Well, no answer after half a year doesn't exactly raise hope. Have you given up, and we should mark the app as "no more updates" at F-Droid, or even move it to archive? It's way behind there, the last available version being v2.0.1 from 5/2017, i.e. being more than 2 years old. @andrewnenakhov ? You've had a lot of releases since. Any chance we get this tackled?

@Human
Copy link
Contributor

Human commented Jul 18, 2019

Maybe forking is the right solution?

@IzzySoft
Copy link

@Human of course you can fork, fix and provide a PR. I can't (well, I can fork, but then I'm stuck – not being an Android developer and all that). To clarify why I chimed in: I'm one of the F-Droid maintainers, walking through our issue queue and trying to figure what is stuck and where. I do that about 2 or 3 times a year for house-keeping.

So no offense meant, but may I be so bold and ask the status again – given that I cannot fix the code myself? 😃

@andrewnenakhov
Copy link
Member

@IzzySoft @Human we're still committed to doing this. Just really busy :(

we initially planned to do lots of improvements and roll it out all together with our server and ios version and web version but we ran into some problems with app performance and stability, fixing this took a lot of time. Maybe we'll not try to roll out everything at once and will just release a copy of what we have on Google Play. Hopefully, next week.

@IzzySoft
Copy link

Hi @andrewnenakhov – that sounds great, thanks! Looking forward to it. Feel free to ping me if I should update the issue on our end to get some packager to take a look (when there is a fitting tag, that is). "Next week" is more than I dared hoping for!

@Str4tocaster
Copy link
Contributor

Hello @IzzySoft
I am currently working on version of Xabber Android for F-Droid.
I have a question about com.google.android.gms:play-services-safetynet dependency.
Have F-Droid restriction on this dependency or not? We use recaptcha from this packet to signup process in our cloud service.

@Bubu
Copy link
Author

Bubu commented Jul 24, 2019

@Str4tocaster I'm pretty sure nothing under com.google.android.gms:* is free software. So it cannot be included in F-Droid.

@IzzySoft
Copy link

Apart from that, ReCaptcha is a nightmare for users, not only privacy-wise. Whenever I encounter it, I close the tab/app/whatever straight away. I'm tired of marking buses, store-fronts, water hydrants … over and over, only to finally be allowed to check a box (last time I gave in, I gave up at round 10). I'd strongly recommend dropping that.

@andrewnenakhov
Copy link
Member

@IzzySoft well it depends on how it set up and how many attempts occur with similar parameters. In our case for most registered users it doesn't appear even once, acting invisibly. However, some users did report that on some devices it just fails to load, making registration impossible. IN this case we suggest registering with our (or any other server) website to obtain an account.

Also, for now, we decided to do just the same for F-droid build: when user presses on 'sign up' button we'll simply open a system browser pointing to registration url.

@IzzySoft
Copy link

Meaning, the fdroid flavor would come without ReCaptcha (and without any other GMS/proprietary dependencies), right? Then it should be fine (pending approval from @Bubu – I'm not familiar with the "build stuff").

@andrewnenakhov
Copy link
Member

Yes, without ReCaptcha

@Str4tocaster
Copy link
Contributor

@Bubu can you check branch feature/fdroid for compliance F-Droid requirements? I tried to remove all proprietary dependencies. The build variant for F-Droid is openProdRelease.

@Bubu
Copy link
Author

Bubu commented Jul 25, 2019

Looks good from a short glance.

@Str4tocaster
Copy link
Contributor

@Bubu What our next steps to release Xabber on F-Droid?

@andrewnenakhov
Copy link
Member

@Bubu what should we do next? Just wait?

@Bubu
Copy link
Author

Bubu commented Aug 13, 2019

I have no time to do anything here, sorry.

@IzzySoft
Copy link

@Rudloff could you maybe pickup where Bubu left?

@Rudloff
Copy link

Rudloff commented Aug 20, 2019

I tried to build a7da670 with the open variant and it seems to work as expected.
Now, we would need a stable tag that includes this variant.

@IzzySoft
Copy link

Thanks Pierre!

@andrewnenakhov your turn again, answering the "what should we do next": ^^ 😃

@Str4tocaster
Copy link
Contributor

@Rudloff thanks for testing! We will prepare release build and make a stable tag soon.

@licaon-kter
Copy link

licaon-kter commented Sep 16, 2019

Should this be a new app: com.xabber.android like on Google Play?

Or the old org.xabber.androiddev ?

/LE Any news on #857 ? I still see connections even before trying to sign up or in... :(

@Str4tocaster
Copy link
Contributor

Str4tocaster commented Oct 7, 2019

@Rudloff
@IzzySoft
The stable tag https://github.com/redsolution/xabber-android/releases/tag/v2.6.6 is here.
We are ready to return at F-Droid!

@licaon-kter
Copy link

@Str4tocaster and some answers?

@Str4tocaster
Copy link
Contributor

@licaon-kter
com.xabber.android

@licaon-kter
Copy link

@Str4tocaster
Copy link
Contributor

@licaon-kter Thanks! And when Xabber appear at F-Droid?

@licaon-kter
Copy link

The current cycle can be see here: https://f-droid.org/wiki/index.php?title=Special:RecentChanges&days=7&from=&hidebots=0&hideanons=1&hideliu=1&limit=500

And scroll a bit and see it's already built: https://f-droid.org/wiki/page/com.xabber.android/lastbuild_644

So now we wait for sign & publish ;)

@licaon-kter
Copy link

Done... in F-Droid Client pull to refresh ;)

@Str4tocaster
Copy link
Contributor

@licaon-kter Thanks a lot for help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants