[Draft] [Android] Architecture revamp

[AenBleidd]

Starting from Android API 28 Google add some restrictions to native executables: https://android-review.googlesource.com/c/platform/system/sepolicy/+/804149
Basically this means that we have no ability to download and run project executables anymore.

In order to be up-to-date and secure, @davidpanderson and me decided to change the architecture of BOINC Manager for Android. This is the only way I see to continue our mission.

The main change is next: now BOINC Manager for Android will include all project executables inside the bundle ( https://developer.android.com/guide/app-bundle/ ).

This ticket is an entry point and collection of another tickets that should cover main workflow and its edge cases.

Additional information: https://developer.android.com/guide/playcore/feature-delivery

Release critical

• [Draft] [Android] Remove assets directory #3562 - Remove assets directory
• [Draft] [Android] Switch from all_projects_list.xml to static list of supported projects #3563 - Switch from all_projects_list.xml to static list of supported projects
• [Draft] [Android] AccountManager should add only supported projects from static projects list #3564 - AccountManager should add only supported projects from static projects list
• [Draft] [Android] Remove ability to add projects manually #3565 - Remove ability to add projects manually
• [Draft] [Android] Convert APK to Bundle #3566 - Convert APK to Bundle
• [Draft] [Android] Contact projects to inform about upcoming changes and request binaries to include in APK #4088 - Contact projects to inform about upcoming changes and request binaries to include in APK
• [Draft] [Android] Investigate how to move project executables to RO folder #4089 - Investigate how to move project executables to RO folder
• [Draft] [Android] Increase Target API level to 29 #4091 - Increase Target API level to 29

Release non-critical

[davidpanderson]

This was a joint decision of Vitalii and me. Also: this architecture will make it feasible to have a BOINC client for iOS.

[davidpanderson]

To clarify further: this issue should be discussed by the BOINC community, including the projects that offer Android apps. Vitalii and I think this is probably the best approach (as opposed, for example, to trying to work around the new restrictions, or using older APIs).

[adamradocz]

To clarify further: this issue should be discussed by the BOINC community, including the projects that offer Android apps. Vitalii and I think this is probably the best approach (as opposed, for example, to trying to work around the new restrictions, or using older APIs).

Agree.

This was a joint decision of Vitalii and me. Also: this architecture will make it feasible to have a BOINC client for iOS.

Finally :).

[TheAspens]

This change by Google is unfortunate for BOINC. This will be a much harder model to work with. Thanks for thinking through alternatives and beginning the discussion.

Can you write up a explanation about how this will work from the project perspective? When a project has new versions of its binaries how does it update them? What will be the update frequency? Is there a way to have each project's binaries as a "sub-bundle" so that the overall downloaded is based on only the projects the user is actually contributing to (and so that updates to one projects binaries doesn't trigger a re-download of everything for everyone)?

[AenBleidd]

I can't answer these question right now because not everything is clear for me either. From what I can see now this will be done similar to this: 1 - When project has some executables ready to test or ready to release, they should inform me. I build a new APK. Debug version of it could be used by project for testing. This step could be automated. 2 - I suppose that it makes sense to make an update as often as project ready to ship updated binaries. Hopefully, this will not happen too often. 3 - Each project will be in a separate bundle, so the user will download only that binaries that are used by projects attached. I suppose upgrade will be in the same way but I'm not 100% sure about this. Best regards, Vitalii Koshura пн, 6 апр. 2020 г. в 16:30, Kevin Reed <notifications@github.com>:

…

This change by Google is unfortunate for BOINC. This will be a *much* harder model to work with. Thanks for thinking through alternatives and beginning the discussion. Can you write up a explanation about how this will work from the project perspective? When a project has new versions of its binaries how does it update them? What will be the update frequency? Is there a way to have each project's binaries as a "sub-bundle" so that the overall downloaded is based on only the projects the user is actually contributing to (and so that updates to one projects binaries doesn't trigger a re-download of everything for everyone)? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#3561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYVTINVLAXE5DPRF5SN3SLRLHRO3ANCNFSM4MBODKVA> .

[truboxl]

Some really hardcore discussion over at termux/termux-app#1072
Whatever Termux does can be an insightful lesson to BOINC

[talregev]

This was a joint decision of Vitalii and me. Also: this architecture will make it feasible to have a BOINC client for iOS.

@AenBleidd
Vitalii, I have a better suggestion for android architecture.
You should separate the projects binary and boinc client to different apps in google store.
What that I mean: That boinc client will be one app in google store. when the client will download the project files that is not binaries, (xml and data files) he will know which binary app he can download from google play store. and ask from the user (With one or two button push) to download it. That boinc client can update in the time is needed and binary will update separately from boinc client and will be in project responsibility and speed.
I am not sure if this suggestion can do in IOS. But definitely can do in google play.
For example like google translate. It will not come with all languages like Japanese (Before it was also for Hebrew), and it download data or additional files for this language and it update separately from google translate. (I guess it update more quickly).

So please take this suggestion if you want more flexibility in boinc client.

[AenBleidd]

@talregev, what about scheduling? How can one application (boinc client in this case) send a command to run another application (project application)?

[talregev]

@AenBleidd
I am not sure how to do it, But I know that opencv for android in the previous versions done it.
In the previous opencv you compile your program to some shell, and when you run your program it ask you to download another app (opencv manager) then your program can use opencv library and all functionality. I am not sure how they do it. But it open source and we can look and understand.

[talregev]

How can one application (boinc client in this case) send a command to run another application (project application)

I guess there is some (not one) solutions, and need to think about this.

[scitor]

This sounds bad, have you considered using WebAssembly for the binary part? Since running a webview is nothing unusual for an app (in fact most apps don't do much more), the js engines are very capable nowadays.
The projects binaries would have to be changed, but supposedly not so much as to compile them into the apk. Plus you would gain essentially universal compatibility, even on older devices (at least those able to run js/wa).

[talregev]

@scitor Do you know how to do it?
Can you give me an example?

[scitor]

Well, I'm not a C++ programmer, but there are surely some resources one could look into.. I could imagine the science binaries aren't that much dependent on OS/arch, so the only thing really changing would be communication with parent and file system access (I suppose). If CryptoMiners can do it, why can't my chrome smash childhood cancer too? :D
I'm just seeing this is already being discussed here.

[AenBleidd]

This sounds bad, have you considered using WebAssembly for the binary part? Since running a webview is nothing unusual for an app (in fact most apps don't do much more), the js engines are very capable nowadays.
The projects binaries would have to be changed, but supposedly not so much as to compile them into the apk. Plus you would gain essentially universal compatibility, even on older devices (at least those able to run js/wa).

It's a big change and it will not solve the main issue: binaries still should be put inside APK. So I see no real benefit of doing this.

[scitor]

[...] binaries still should be put inside APK. [...]

If that's the goal, then yes, webviews don't solve that problem. I thought of it as being better having one flexible client where you can plug-in the parts you want, but thats just preference...

[darthnithin]

also would allow web browsers

[efmer]

What about making an app for every project.
Android allows to start other app e.g. a project. Click me
Communication between the projects and the BOINC client APP is easy.
This will make it much easier for projects to test and deploy on their own time.

Like this

1. Core BOINC client APP.
2. Project APP 1,2,3,4,5
3. GUI

[AenBleidd]

For not BOINC can run multiple projects simultaneously and even multiple tasks from the same project.

• Will this solution with multiple applications work the same?
• Also, I'd prefer to not ask user to install any other APKs to run projects using BOINC (I know we can install APK from the BOINC APK itself but anyway).

The first question is the most important

[AenBleidd]

And why GUI and client should be different APKs?

[talregev]

@AenBleidd
The suggestion of @efmer is similar to my suggestion.
I also think like you that the gui and the client should be the same app, the the other projects app should be separate.
In my design the project app just download their correspond binary, and client (With gui) have access and can run them.

• Also, I'd prefer to not ask user to install any other APKs to run projects using BOINC (I know we can install APK from the BOINC APK itself but anyway).

If we going to this direction, we must ask from the user at the first time to download the project app. by link via boinc app itself.

[AenBleidd]

@talregev please read my question in Discord chat. Because I'm not sure your and @efmer design will ever work as we want

[talregev]

@AenBleidd
I just found a way to do it:
https://stackoverflow.com/a/58748468

[AenBleidd]

And this method is already described in this particular ticket

[AenBleidd]

I decided not to continue with this change. Mainly because it will make a big impact on UX and not significantly smaller output.

[talregev]

We never go back to android market?

[AenBleidd]

We never go back to android market?

We will but in a different way

[talregev]

Can you tell me about the different way?

[AenBleidd]

Not now. When time comes - I'll create a story for that.

It was decided a while ago to not continue with this story, I just had no time and desire before to close all tickets that are not needed anymore.

[talregev]

You should close it as close as not planned. (and not close and complete like you did).

[AenBleidd]

This is a new functionality and not yet on mobile. In any case, I mostly rely on tags.