Please consider publishing AndroidCelestia on F-Droid

[rojoweko]

Hi,

please consider publishing the AndroidCelestia app on F-Droid
(https://www.f-droid.org/), an alternative app store exclusively for
free and open source software (FOSS). Via F-Droid you would provide a
growing community of privacy concerned Android users, who do not want to
use Goggle's Play Store, with your great App! As far as I know
AndroidCelestia is developed under the terms of the GPL and if it also does not
depend on proprietary libraries, that cannot be stripped without
disabling major functionalities, the most important requirements for a
F-Droid release are met.

You can file a request for packaging via this link:
https://gitlab.com/fdroid/rfp/issues/new

Thanks in advance! ;-)

[levinli303]

while it is always nice to have Celestia in a new app store, it would also mean that we have to publish to multiple app stores which would mean more effort. And also I saw that fdroid works differently from other app stores that they build the app by themselves.

You are very welcome to file the issue for requesting for packaging by yourself. The build instruction can be found here: https://github.com/levinli303/Celestia/wiki/Development

[ClockGen]

@levinli303 Hello, I could work on packaging Celestia to F-Droid. Do I understand correctly that you're ok with it being published, but don't want to maintain the F-Droid version? If that's the case, I can add a note in description, saying that this is an unofficial version and may lag behind updates. Also, it's usually possible to set up automatic update checking from F-Droid side so updates will be automatically checked out and built (although, with a delay).

Either way, I quickly checked the source code, there's a few concerns:
We prefer to not have apps with trackers in F-Droid, is it ok if we patch out or disable these two libraries on F-Droid side

AndroidCelestia/app/build.gradle

Line 142 in b39a44d

implementation "com.microsoft.appcenter:appcenter-analytics:${appCenterSdkVersion}"

during build time? (If there was a switch inside the app that'd allow to turn off analytics, we could simply instead set this switch to false during build time, but I couldn't find one).

The link you posted above says that it's needed to download prebuilt libraries from the CI, however I checked it, and it appears it downloads and builds dependencies with each build? (that's even better, because we need to build everything from source on F-Droid side anyway).

[levinli303]

@ClockGen hi, I'm ok with what you proposed. please be sure to checkout the repos with the commits as specified in the release notes.

I plan to add a different flavor in build which disables AppCenter in code, so you can build for that flavor instead of patching the code. The dependency might still be checked in but disabled in code. I'll update this issue when I add the flavor. Please do not make a release until this change is made.

the c dependencies (libpng, lua, etc...) are pre-built libraries downloaded from azure artifacts. I would suggest you do the same though you can still find build scripts for all the libraries here https://dev.azure.com/CelestiaProject/Celestia/_git/apple-android (build_android.sh)

I suppose f-droid distributes .apk files, will it be signed, and if so with what key?

[ClockGen]

please be sure to checkout the repos with the commits as specified in the release notes.

Hm, might be tricky to implement. Usually fdroidserver checks out git tags only (with automatic builds) and expects code to be available in one repo, so it'd be hard to somehow track external components, or it'll require manual intervention with new builds. Is it possible to add Celestia, CelestiaContent and CelestiaLocalization as git submodules instead?

the c dependencies (libpng, lua, etc...) are pre-built libraries downloaded from azure artifacts. I would suggest you do the same though you can still find build scripts for all the libraries here https://dev.azure.com/CelestiaProject/Celestia/_git/apple-android (build_android.sh)

F-Droid has a requirement to build apps completely from source (with exceptions for build environment, like android SDK/NDK, artifacts from maven repos or packages from buildserver repos, which is debian bullseye, etc), so we'll need to build them from source anyway.

I suppose f-droid distributes .apk files, will it be signed, and if so with what key?

F-Droid usually builds and signs apps with its own key, however recently F-Droid began shifting towards reproducible builds https://f-droid.org/docs/Reproducible_Builds/.
F-Droid buildserver will build the apk from source, then retrieve the apk from upstream (in this case from this repo releases page) and compare them with diffoscope. If APKs match, then the upstream APK, signed with author key, is published. That'd require you to also publish the new flavor APK as well.

This wouldn't work if we had to patch the source code (because the APKs would differ), but now that you plan to make a separate flavor, this could be enabled, although, very tricky, considering how big the project is and that it's almost entirely native code (compiled libraries need to be identical and there's a lot of variation even from small things like path where it's compiled, etc, there's a list of common problems in the article I linked above).
Reproducible builds sometimes fail verification though and require manual intervention (there's lots of things we can't track, for example you'd decide to upgrade NDK version, or bump versions of third party prebuilt libraries, etc), just building and signing the apk with F-Droid key would be easier for automatic updates, but it depends on how you want to proceed of course.

[levinli303]

I'll suggest keeping a fork or a separate branch for f-droid builds so we can work around a few issues. Every time a release is made, you can merge the changes into that fork or branch. There should be few changes between the fork and the main branch:

1. submodules set up (Celestia/CelestiaContent/CelestiaLocalization/apple-android)
2. changes in scripts pointing to the path of the submodules instead
3. remove AppCenter integration (this can be done in main branch via flavor)
4. add a separate step to build the native dependencies from apple-android repo

as for signing, I'm ok if it is signed by f-droid with its own key. but we probably should mention somewhere that this version and Google Play version has different signatures so users would need to uninstall the existing version if they want to switch between the two.

[ClockGen]

A separate fork would require manual updates and merges, but if reproducible builds aren't needed then it gives us a lot of freedom to patch these things during the build on F-Droid side, f-droid allows to run custom commands during various stages of build, (here's an example of a complex build).

In current situation I think we can set up completely automatic builds and don't require human interaction at all (f-droid will automatically build new releases after you tag them in this repo), these steps can be done on f-droid buildserver:

2. changes in scripts pointing to the path of the submodules instead
4. add a separate step to build the native dependencies from apple-android repo

For this step:

1. submodules set up (Celestia/CelestiaContent/CelestiaLocalization/apple-android)

we don't need to set up a separate repo with submodules either, I think the easiest approach would be is that you attach a text file to new github releases, the file would contain commits for every repo, something like this:

Celestia=ef9c24e
CelestiaContent=27d9de3
CelestiaLocalization=e7ff7d0

Files in releases can be accessed statically, so we can parse this file during the build and fetch repos accordingly.

Most of these things will be done on F-Droid side only, so if no new problems will arise, we only need appcenter-free flavor and the text file I mentioned above from your side.

Another thing, F-Droid usually uses fastlane structures to obtain descriptions, their translations and screenshots for apps (some reference links: one two and an example).
With fastlane, app description, banner, screenshots, etc are hosted in your repo and F-Droid fetches it, so you can change it yourself. Without fastlane, app description is hardcoded in app metadata in this repo, so if you'd want to change something, you'd need to open a merge request. If added, F-Droid will check for fastlane at repository tags, so it'd only show up if you tag a new release. Providing screenshots and app banner without fastlane is also not possible.
If you agree, I can write it and open a PR in this repo.

[levinli303]

great. I'll make a txt for releases after 1.6.9. but note that the build scripts for dependencies are hosted on azure not GitHub https://dev.azure.com/CelestiaProject/Celestia/_git/apple-android, CelestiaContent is under CelestiaProject not celestiamobile, so we need to agree on the format.

as for fastlane, I certainly would not be against it given that it could help make distribution easier for Play too.

[ClockGen]

but note that the build scripts for dependencies are hosted on azure not GitHub

I'm gonna have to adapt build scripts for F-Droid buildserver anyway so I won't fetch them directly.

CelestiaContent is under CelestiaProject not celestiamobile, so we need to agree on the format.

I can just parse the file and git clone needed repos during the build (same will be done for deps), something like this:

export CELESTIA_HASH=$(sed -nr 's/^Celestia=(.*)$/\1/p' versions.txt)
export CELESTIA_CONTENT_HASH=$(sed -nr 's/^CelestiaContent=(.*)$/\1/p' versions.txt)
export CELESTIA_LOC_HASH=$(sed -nr 's/^CelestiaLocalization=(.*)$/\1/p' versions.txt)
cd ..
git clone https://github.com/celestiamobile/Celestia
git clone https://github.com/CelestiaProject/CelestiaContent
git clone https://github.com/celestiamobile/CelestiaLocalization
cd Celestia && git reset --hard $CELESTIA_HASH && cd -
cd CelestiaContent && git reset --hard $CELESTIA_CONTENT_HASH && cd -
cd CelestiaLocalization && git reset --hard $CELESTIA_LOC_HASH && cd -

(Note that this is just pseudocode, I didn't test it yet).

Also it'd be better if you add the versions file directly to this repository, I thought that just attaching them to github releases would be enough, and it could be retrieved per git tag with this link:
https://github.com/celestiamobile/AndroidCelestia/releases/download/%v/versions.txt
Where %v can be substituted by fdroidserver with versionName. Unfortunately the choice of substitutions is limited, and I then realized that your git tags look different than versionName, so this wouldn't work.

[levinli303]

added an unofficial flavor here: #74

[ClockGen]

Thanks, I'll start working on trying to build it on my buildserver, this may take some time.

[ClockGen]

I adapted the build process to run on fdroidserver. For now it's in my private branch: https://gitlab.com/fruitsnack/fdroiddata/-/commit/8fd9e07b5b496c0fb263fd34f7d79992efb7fff9. Also, I couldn't find any donation or translation links for Celestia, so if you have them I can add them to metadata.

I also opened a PR that adds Fastlane and expected versions.txt file #76.

The build recipe is not complete, it builds and Celestia works fine, however I hardcoded commit hashes for now, until you merge the PR that will add versions.txt. After that I'll modify it to parse that file. For builds to be built correctly, please update that file with correct hashes before tagging a new version in the future.

Autoupdates also work fine, so fdroid will correctly pick up new versions once you tag them (checkupdate on gitlab CI failed because currently tagged versionCode is lower than in commit that's checked out).

[ClockGen]

@levinli303 I adapted the build recipe to use versions.txt from #79, it builds fine. I'll wait for you to merge develop into main and tag a new version (for fastlane) then I'll open a merge request in fdroiddata.

[levinli303]

the next release should be sometime later in March. I'll let you know.

[ClockGen]

I opened a draft MR in fdroiddata: https://gitlab.com/fdroid/fdroiddata/-/merge_requests/14599.

[levinli303]

@ClockGen I plan to release 1.7.0 from the develop branch one of these days, letting you know now in case you want to test based on that.

[levinli303]

@ClockGen 1.7.0 is released on Google Play now.

[ClockGen]

Hello, sorry for no response, I'll try to adapt the build recipe this weekend. It's still gonna take some time for MR review by F-Droid maintainers and some more time for the build cycle as well.