[Feature]: F-Droid

[anonimo82]

Feature description

This is a suggestion more than a request: you could distribute AndroidIDE on F-Droid.

What version of AndroidIDE you're using?

v2.1.1

Duplicate issues

• This issue has not been reported yet.

Code of Conduct

• I agree to follow this project's Code of Conduct

[itsaky]

I'll publish it soon. But this is a duplicate of #419.

[anonimo82]

Sorry, please forgive me

[matan-h]

the issue "#419" is not found . is there an updated link?

[yzqzss]

Is there any progress?
:)

[itsaky]

I've tried to publish AndroidIDE to F-Droid multiple times, but it always fails to determine the version of AndroidIDE from the source code.

However, we might consider creating a custom F-Droid repository which can be used to publish AndroidIDE. After that, you'll be able to add that repository to the F-Droid client.

[yzqzss]

I've tried to publish AndroidIDE to F-Droid multiple times, but it always fails to determine the version of AndroidIDE from the source code.

Could you paste out your f-droid-data .yml build file? Maybe I or someone else can help. :)

[itsaky]

I've tried to publish AndroidIDE to F-Droid multiple times, but it always fails to determine the version of AndroidIDE from the source code.

Could you paste out your f-droid-data .yml build file? Maybe I or someone else can help. :)

Unfortunately, I don't have the files now. Sorry.

However, the "fastlane" directory in the repository contains some basic information about the project which you can use to create the metadata if you want.

Edit:
You should also note that the AndroidIDE project now uses product flavors to separate the assets and reduce the APK size. I don't know if F-Droid supports product flavors (with same package name).

[linsui]

You should also note that the AndroidIDE project now uses product flavors to separate the assets and reduce the APK size. I don't know if F-Droid supports product flavors (with same package name).

Do you mean this? https://github.com/AndroidIDEOfficial/AndroidIDE/blob/b40d52e6cf23d01a14cef7a6a6a3ce1345167ddc/common/src/main/java/com/itsaky/androidide/app/configuration/CpuArch.kt

[itsaky]

You should also note that the AndroidIDE project now uses product flavors to separate the assets and reduce the APK size. I don't know if F-Droid supports product flavors (with same package name).

Do you mean this? https://github.com/AndroidIDEOfficial/AndroidIDE/blob/b40d52e6cf23d01a14cef7a6a6a3ce1345167ddc/common/src/main/java/com/itsaky/androidide/app/configuration/CpuArch.kt

Not specifically, but it's related. I'm talking about product flavors which are configured for AndroidIDE here.

I've tried to publish AndroidIDE to F-Droid multiple times, but it always fails to determine the version of AndroidIDE from the source code.

The version name and version code of AndroidIDE is calculated dynamically, based on the type of commits after the previous Git tag. This is achieved using the Nyx tool. I'm not sure, but I think this might be the cause that the version names are not properly inferred by F-Droid.

[linsui]

Not specifically, but it's related. I'm talking about product flavors which are configured for AndroidIDE here.

This is supported but F-Droid requires different version codes for different abis.

The version name and version code of AndroidIDE is calculated dynamically, based on the type of commits after the previous Git tag. This is achieved using the Nyx tool. I'm not sure, but I think this might be the cause that the version names are not properly inferred by F-Droid.

The version name and version code can be overrided manully anyway. But for auto update we need to get it from somewhere. E.g., you can put a text file in besides the apk in github release with the version name and version code.

[itsaky]

This is supported but F-Droid requires different version codes for different abis.

For this, I suppose we can present a number to the existing version code for each product flavor. For example version code 270 could be prepended with 1 so it becomes 1270. Is this acceptable? Do they have any documentation for this?

The version name and version code can be overrided manully anyway. But for auto update we need to get it from somewhere. E.g., you can put a text file in besides the apk in github release with the version name and version code.

If we just need to write the version name and version code of the APK to a file, then yes, we can do that.

I still need more information about this. Is it possible for you to contact me on telegram? Or via email (see my profile for the address)?

I'll reopen this issue as #419 has been deleted for some reason.

[linsui]

For this, I suppose we can present a number to the existing version code for each product flavor. For example version code 270 could be prepended with 1 so it becomes 1270. Is this acceptable? Do they have any documentation for this?

We support extending the version code with 1 so it becomes 2701.

I still need more information about this. Is it possible for you to contact me on telegram? Or via email (see my profile for the address)?

I don't use telegram. I thought it's easier to discuss here but I can send you an email if you prefer that. :)

[itsaky]

I don't use telegram. I thought it's easier to discuss here but I can send you an email if you prefer that. :)

It's okay, we can discuss here.

This is supported but F-Droid requires different version codes for different abis.
The version name and version code can be overrided manully anyway. But for auto update we need to get it from somewhere. E.g., you can put a text file in besides the apk in github release with the version name and version code.

Can you point me to the documentation where these things are mentioned (if there are any)? I can try to submit again with the new information.

[linsui]

There is not too much document about this (maybe I should write some about this). Basically everything is in https://f-droid.org/docs/Build_Metadata_Reference/. You can change the source code in prebuild so you can replace the version generator with a static code, ($$VERSION$$ and $$VERCODE$$).

[itsaky]

Hey @linsui, Is there a way make fdroid build command ignore certain maven repositories? Specifically, I want to add the following repository to ignore list :

https://repo.gradle.org/gradle/libs-releases/

This repository is required for the Gradle Tooling API which we use to run Gradle tasks in the IDE.

Here is the metadata file com.itsaky.androidide.yml :

Categories:
  - Development
License: GPL-3.0-only
WebSite: https://androidide.com
SourceCode: https://github.com/AndroidIDEOfficial/AndroidIDE
IssueTracker: https://github.com/AndroidIDEOfficial/AndroidIDE/issues
Translation: https://crowdin.com/project/androidide
Changelog: https://github.com/AndroidIDEOfficial/AndroidIDE/releases
Donate: https://androidide.com/donate/

AutoName: AndroidIDE

RepoType: git
Repo: https://github.com/AndroidIDEOfficial/AndroidIDE

Builds:
  - versionName: v2.6.1-beta
    versionCode: 261
    commit: 83a33ac3ee4bc03cd41a0f07d059f54adad5862e
    subdir: app
    sudo:
      - apt-get update
      - apt-get install -y openjdk-17-jdk-headless
      - update-java-alternatives --jre-headless -a
    init: chmod +x ../scripts/setup_fdroid_build.sh && ../scripts/setup_fdroid_build.sh $$VERSION$$ $$VERCODE$$
    gradle:
      - arm64-v8a
      - armeabi-v7a
      - x86_64
    scanignore:
      - build-logic/
      - testing/resources/
      - app/src/main/assets/data/common/android.jar
      - app/src/main/assets/data/common/gradle-wrapper.zip
      - app/src/arm64-v8a/assets/data/arm64-v8a/aapt2
      - app/src/armeabi-v7a/assets/data/armeabi-v7a/aapt2
      - app/src/x86_64/assets/data/x86_64/aapt2
      - subprojects/framework-stubs/libs/android.jar
      - termux/termux-app/src/main/cpp/bootstrap-*.zip
    gradleprops:
      - ide.build.fdroid=true

AutoUpdateMode: Version +-fdroid %v
UpdateCheckMode: HTTP
VercodeOperation:
  - 100 * %c + 1
  - 100 * %c + 2
  - 100 * %c + 3
UpdateCheckData: https://androidide.com/functions/fdroid-version-check.php|versionCode=(.*)|.|versionName=v(.*)

I haven't been able to perform the actual build yet, because of the repository error. I looked into the build metadata reference, but it doesn't seem to contain any information related to this issue (or at least I didn't find anything).

[linsui]

We shouldn't ignore it. But you can use scanignore to ignore the file. Generally we should build those files in scanignore from source.

[linsui]

You can only build 1 apk in 1 build block. So you need to add 3 build blocks with different flavor, version name and version code.

[linsui]

How do you get the aapt2 binary? They are packaged in google maven repo. We can use that.

[itsaky]

We shouldn't ignore it. But you can use scanignore to ignore the file. Generally we should build those files in scanignore from source.

When building with fdroidserver, it complains about .class files in the build-logic directory (which are compiled from source). And the testing/resources project contains a test project which is used for testing purposes. That is why these two directories are added to scanignore. Other files are required at runtime, so they need to be ignored. However, I'll try not to include them in the codebase for future versions.

You can only build 1 apk in 1 build block. So you need to add 3 build blocks with different flavor, version name and version code.

Thanks, I'll add Build blocks for each flavor. For new updates, do I need to update these Build blocks manually?

How do you get the aapt2 binary? They are packaged in google maven repo. We can use that.

We actually build it from source here. The one published on Google Maven Repo is not usable on Android (as it is built for Linux x86_64).

[linsui]

When building with fdroidserver, it complains about .class files in the build-logic directory (which are compiled from source). And the testing/resources project contains a test project which is used for testing purposes.

This can be added to scandelete.

For new updates, do I need to update these Build blocks manually?

They can be auto updated. You can add a list in VercodeOperation.

We actually build it from source here.

Oh, that's good. Then we can build them from source.

[itsaky]

They can be auto updated. You can add a list in VercodeOperation.

Is the existing VercodeOperation correct? We have 3 flavors named after supported ABIs arm64-v8a, armeabi-v7a and x86_64.

Oh, that's good. Then we can build them from source.

Well, if we build the platform tools along with AndroidIDE, it would increase repository's size a lot. Building aapt2 requires cloning frameworks/base which is about 1GB even for a shallow clone. And there are many other repositories that need to be cloned, just to build those tools.

I can use Termux's approach of handling these things (for aapt2 as well as bootstrap packages). They build and publish the bootstrap packages in the termux-packages repository on GitHub, then download those archives when building the project. Is this acceptable on F-Droid?

[linsui]

the existing VercodeOperation correct? We have 3 flavors named after supported ABIs arm64-v8a, armeabi-v7a and x86_64.

That's correct.

I can use Termux's approach of handling these things (for aapt2 as well as bootstrap packages). They build and publish the bootstrap packages in the termux-packages repository on GitHub, then download those archives when building the project. Is this acceptable on F-Droid?

I thought this should be good. It's always better to build it from source but it maybe too difficult. We can have a try later. 1 GB is not a big problem for F-Droid. You can add it in a submodule or we can handle it with srclibs.

[itsaky]

That's correct.

Thanks!

I thought this should be good. It's always better to build it from source but it maybe too difficult. We can have a try later. 1 GB is not a big problem for F-Droid. You can add it in a submodule or we can handle it with srclibs.

I meant that it could be a problem for people trying to build the project locally. Adding it to a module would still be a problem for such cases as everything will be cloned anyway. At the moment, I have no idea how to use Build.srclibs field/property.

[itsaky]

I'll probably go ahead with the submodules approach for building aapt2 if F-Droid really prefers that, but for the bootstrap packages, I can either keep them in scanignore (as it is now) or I can follow Termux's approach of downloading them at build time. Building the bootstrap packages within source is not a good idea as it requires a lot of time to setup and build.

Do you think we can publish this to FDroid as it is now, then work on these things after the release is published? I need to ask this because AndroidIDE v2.7.0-beta has been finalized and adding those changes will take some time to implement properly.

[linsui]

I meant that it could be a problem for people trying to build the project locally. Adding it to a module would still be a problem for such cases as everything will be cloned anyway. At the moment, I have no idea how to use Build.srclibs field/property.

Then maybe srclibs should be used instead. You need to add your repo in srclibs/android-ide-platform-tools.yml first then add android-ide-platform-tools@v34.0.4 in the build metadata. Then you can build it and copy the binary.

I'll probably go ahead with the submodules approach for building aapt2 if F-Droid really prefers that, but for the bootstrap packages, I can either keep them in scanignore (as it is now) or I can follow Termux's approach of downloading them at build time. Building the bootstrap packages within source is not a good idea as it requires a lot of time to setup and build.

Building them from source is prefered but difficult. Currently we don't build the Termux binaries, either. I thought it's OK to use the Termux's approach for now.

Do you think we can publish this to FDroid as it is now, then work on these things after the release is published? I need to ask this because AndroidIDE v2.7.0-beta has been finalized and adding those changes will take some time to implement properly.

I thought we still need to test and debug the build metadata. I'm not sure how long it will take but given that Android IDE is a very complicant project I thought it's will take some time to get everything done. So if you are going to publish a new release then maybe you should do that first and we can try to publish the next version.

[itsaky]

Then maybe srclibs should be used instead. You need to add your repo in srclibs/android-ide-platform-tools.yml first then add android-ide-platform-tools@v34.0.4 in the build metadata.

Thanks. Will do.

Then you can build it and copy the binary.

Looking at the existing srclibs and metadata of other applications that use srclibs, I think that the FDroid build server will simply clone the specified srclib repo then the instructions in the build metadata's prebuild will build or copy the required files. Is that correct?

I thought we still need to test and debug the build metadata. I'm not sure how long it will take but given that Android IDE is a very complicant project I thought it's will take some time to get everything done.

I'm trying to build the project using fdroidserver so I can verify everything works fine, at least for my local installation. The build works fine as of now (with the binaries included in the source). However, I've configured the version names incorrectly in the metadata, so it's showing some error about expected version names AFTER the build process. I'll try to get everything right so that it won't be a hassle for you guys to review and test the merge request.

So if you are going to publish a new release then maybe you should do that first and we can try to publish the next version.

I'll publish v2.7.0-beta now and try to publish v2.7.1-beta to FDroid after all these changes.

[linsui]

Looking at the existing srclibs and metadata of other applications that use srclibs, I think that the FDroid build server will simply clone the specified srclib repo then the instructions in the build metadata's prebuild will build or copy the required files. Is that correct?

The srclib will be cloned into $$android-ide-platform-tools$$ and you can build it from there. Please do build in build.

I'm trying to build the project using fdroidserver so I can verify everything works fine, at least for my local installation. The build works fine as of now (with the binaries included in the source). However, I've configured the version names incorrectly in the metadata, so it's showing some error about expected version names AFTER the build process. I'll try to get everything right so that it won't be a hassle for you guys to review and test the merge request.

That's great! Then the only problem is the scanignore. I'm not sure if we should consider those files as "assests". And maybe we should also build the Gradle Tooling API plugin since it's from a maven repo we don't allow or we should allow it since it's from gradle directly (even in the same repo https://github.com/gradle/gradle/tree/master/platforms/ide/tooling-api). Are the android.jar from the sdk directly?

I'll publish v2.7.0-beta now and try to publish v2.7.1-beta to FDroid after all these changes.

So please open an MR when the metadata is realy, then we can discuss those problem there.

[itsaky]

The srclib will be cloned into $$android-ide-platform-tools$$ and you can build it from there. Please do build in build.

Got it, thanks!

That's great! Then the only problem is the scanignore. I'm not sure if we should consider those files as "assests". And maybe we should also build the Gradle Tooling API plugin since it's from a maven repo we don't allow or we should allow it since it's from gradle directly (even in the same repo https://github.com/gradle/gradle/tree/master/platforms/ide/tooling-api).

In my personal opinion, allowing it would be better choice, after checking the repository for problematic libraries of course. However, building that module from source would be difficult since it depends on other modules from Gradle's source. It is also worth noting that the gradle-tooling-api module's pom file says that it only depends on org.slf4j:slf4j-api i.e. it has a single dependency.

Are the android.jar from the sdk directly?

Yes. I can update the codebase to copy it directly from the current (installed) Android SDK, if FDroid allows that. Or do you have something else in mind?

[linsui]

In my personal opinion, allowing it would be better choice, after checking the repository for problematic libraries of course. However, building that module from source would be difficult since it depends on other modules from Gradle's source. It is also worth noting that the gradle-tooling-api module's pom file says that it only depends on org.slf4j:slf4j-api i.e. it has a single dependency.

Unfortunately Gradle itself is problematic so we need to be careful here... It uses non-free plugin in its codebase. https://github.com/gradle/gradle/blob/master/settings.gradle.kts#L12

Yes. I can update the codebase to copy it directly from the current (installed) Android SDK, if FDroid allows that. Or do you have something else in mind?

Then we just need to add them to scandelete and copy them from installed SDK in build.

[itsaky]

Unfortunately Gradle itself is problematic so we need to be careful here... It uses non-free plugin in its codebase. https://github.com/gradle/gradle/blob/master/settings.gradle.kts#L12

I did some tests, and it looks like we can configure a repository but only use it for a specific module. Example :

dependencyResolutionManagement {
  repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
  repositories {
    ...
    maven {
      url = uri("https://repo.gradle.org/gradle/libs-releases/")
      content {
        includeModule("org.gradle", "gradle-tooling-api")
      }
    }
  }
}

If this defined, it seems like Gradle will use it to only resolve that dependency. So, if you define another dependency that is published on the same repository :

implementation("org.gradle:gradle-build-operations:8.5")

the build fails with the following error :

* What went wrong:
Execution failed for task ':app:collectArm64-v8aReleaseDependencies'.
> Could not resolve all files for configuration ':app:arm64-v8aReleaseRuntimeClasspath'.
   > Could not find org.gradle:gradle-build-operations:8.5.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/org/gradle/gradle-build-operations/8.5/gradle-build-operations-8.5.pom
       - https://repo.maven.apache.org/maven2/org/gradle/gradle-build-operations/8.5/gradle-build-operations-8.5.pom
       - https://s01.oss.sonatype.org/content/repositories/snapshots/org/gradle/gradle-build-operations/8.5/gradle-build-operations-8.5.pom
       - https://jitpack.io/org/gradle/gradle-build-operations/8.5/gradle-build-operations-8.5.pom
     Required by:
         project :app

However, updating gradle-tooling-api to a newer version with the same restrictions (includeModule), works fine :

I don't know if this is something FDroid can accept, but it's worth considering for our issue at least.

[itsaky]

@linsui As Gradle is licensed under Apache License 2.0, I can try to re-publish the gradle-tooling-api module to Maven Central under our group name. FDroid accepts the Maven Central repository, so it shouldn't be an issue. What do you think?

[linsui]

As Gradle is licensed under Apache License 2.0, I can try to re-publish the gradle-tooling-api module to Maven Central under our group name. FDroid accepts the Maven Central repository, so it shouldn't be an issue. What do you think?

A workground but totally OK. 😂

For items in scanignore:

      - build-logic/ # Move to scandelete
      - testing/resources/ # It's OK to keep it here or move to scanelete
      - app/src/main/assets/data/common/android.jar # Copy from $$SDK$$
      - app/src/main/assets/data/common/gradle-wrapper.zip # Not sure about this.
      - app/src/arm64-v8a/assets/data/arm64-v8a/aapt2 # Build from source
      - app/src/armeabi-v7a/assets/data/armeabi-v7a/aapt2 # Build from source
      - app/src/x86_64/assets/data/x86_64/aapt2 # Build from source
      - subprojects/framework-stubs/libs/android.jar # Copy from $$SDK$$
      - termux/termux-app/src/main/cpp/bootstrap-*.zip # Download at runtime?

It seems the tooling-api and the wrapper jars are shipped in the gradle binary tarball. Maybe we can simply copy them from there.

[itsaky]

It seems the tooling-api and the wrapper jars are shipped in the gradle binary tarball. Maybe we can simply copy them from there.

Didn't think of that, but yeah, totally doable. We won't have to publish the module then.

  - termux/termux-app/src/main/cpp/bootstrap-*.zip # Download at runtime?

Yes, already working on it.

Apart from these, I've been facing some weird issues. For example, with the fdroidserver installation on a server (Ubuntu 22.04, DigitalOcean Droplet), when building the application (or when gradlew clean is run by fdroidserver`), it fails with an error saying that the SDK location cannot be determined. However, building on the local machine (with fdroidserver) works fine.

Also, when building the platform tools on the server, the $$SDK$$ and $$NDK$$ values were relative to the fdroiddata directory, which caused some problems with the build.

The commands for building aapt2 were :

cd $$android-ide-platform-tools$$

# Set CMake version
cmake_version=3.28.1
cmake_name=cmake-$cmake_version-linux-x86_64

# Download CMake
wget https://github.com/Kitware/CMake/releases/download/v$cmake_version/$cmake_name.tar.gz
tar xf $cmake_name.tar.gz

# Get source files
python3 get_source.py --tag=platform-tools-34.0.4

# Patch source files
find patches -name "*.patch" -exec git apply {} \;

# Build protobuf
cd src/protobuf
ln -sf $(realpath ../googletest) third_party/googletest
cmake -GNinja -Dprotobuf_BUILD_TESTS=OFF
ninja -j$(nproc --all)
cd -

# Build aapt2
ln -sf $(realpath ./src/googletest/googletest) ./src/boringssl/src/third_party/googletest
./build.py \
  --ndk=/build/\$ANDROID_HOME/ndk/26.1.10909125 \ # Had to hardcode this, due to previously mentioned error.
  --build=$(pwd)/build \
  --api=34 \
  --abi=arm64-v8a \
  --protoc=$(pwd)/src/protobuf/protoc \
  --cmake=$cmake_name/bin/cmake \
  --target=aapt2

Is it okay to download CMake like that?

[linsui]

Is it okay to download CMake like that?

You can build cmake from source. See https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/is.xyz.vcmi.yml#L768 for example. :)

[itsaky]

Is it okay to download CMake like that?

You can build cmake from source. See https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/is.xyz.vcmi.yml#L768 for example. :)

Okay.

[itsaky]

@linsui

So finally, the metadata is complete now. The GitLab Pipelline build failed, because "execution took longer than 1h0m0s seconds". However, the build for arm64-v8a flavor had succeeded by that time and the build for armeabi-v7a was being started.

I'll create the merge request on GitLab now so you can review.

[linsui]

That's great!

[itsaky]

A merge request has been opened.

[itsaky]

The merge request has been merged and AndroidIDE should be available on F-Droid soon!

[linsui]

It has been published. :) The website will be updated soon. You can add this somewhere.

[<img src="https://fdroid.gitlab.io/artwork/badge/get-it-on.png"
    alt="Get it on F-Droid"
    height="80">](https://f-droid.org/packages/com.itsaky.androidide)

[itsaky]

It has been published. :) The website will be updated soon. You can add this somewhere.

[<img src="https://fdroid.gitlab.io/artwork/badge/get-it-on.png"
    alt="Get it on F-Droid"
    height="80">](https://f-droid.org/packages/com.itsaky.androidide)

Thanks!