Add support for network based locations

[vanitasvitae]

I'm using Microg UnifiedNLP on hammerhead, but it does not provide me with locations (It should use OpenCellID/Mozilla Location Services to calculate my location offline from a database based on cell IDs). I get SecurityExceptions every few minutes. Can you add support for this?

Here is a log:

02-17 18:59:35.308 17609 17609 I SignalStrengthTracker: Removing old signal strength entries from DB.
02-17 18:59:35.767 24807 25051 E NlpLocationProvider: Exception
02-17 18:59:35.767 24807 25051 E NlpLocationProvider: java.lang.SecurityException: need INSTALL_LOCATION_PROVIDER permission, or UID of a currently bound location provider
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at android.os.Parcel.readException(Parcel.java:1620)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at android.os.Parcel.readException(Parcel.java:1573)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at android.location.ILocationManager$Stub$Proxy.reportLocation(ILocationManager.java:1281)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at com.android.location.provider.LocationProviderBase.reportLocation(LocationProviderBase.java:136)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at org.microg.nlp.location.BackendFuser.updateLocation(BackendFuser.java:101)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at org.microg.nlp.location.BackendFuser.update(BackendFuser.java:86)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at org.microg.nlp.location.ThreadHelper.run(ThreadHelper.java:77)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:423)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:278)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:270)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
02-17 18:59:35.767 24807 25051 E NlpLocationProvider:   at java.lang.Thread.run(Thread.java:818)

[vanitasvitae]

As far as I understood so far, AOSP needs UnifiedNLP to be build into the system/signed with the build key of the rom. Would it be an option to add this to copperheadOS?

[vanitasvitae]

As far as I understood so far, AOSP needs UnifiedNLP to be build into the system/signed with the build key of the rom. Would it be an option to add this to copperheadOS?

[christoph-buente]

Network based location would be awesome. I also installed µg UnifiedNlp (no GAPPS) https://f-droid.org/repository/browse/?fdid=com.google.android.gms but it does not seem to be allowed to register in the system. According to the selfcheck it says:
"Your system does not support this UnifiedNlp packacke. Either install a matching package or a compatibility Xposed module."

And

"The system did not bind the UnifiedNlp service. If you just installed UnifiedNlp you should try to reboot this device."

Rebooting does not help. Any chance this can be supported in future versions?

[christoph-buente]

Network based location would be awesome. I also installed µg UnifiedNlp (no GAPPS) https://f-droid.org/repository/browse/?fdid=com.google.android.gms but it does not seem to be allowed to register in the system. According to the selfcheck it says:
"Your system does not support this UnifiedNlp packacke. Either install a matching package or a compatibility Xposed module."

And

"The system did not bind the UnifiedNlp service. If you just installed UnifiedNlp you should try to reboot this device."

Rebooting does not help. Any chance this can be supported in future versions?

[mar-v-in]

As far as I understood so far, AOSP needs UnifiedNLP to be build into the system/signed with the build key of the rom. Would it be an option to add this to copperheadOS?

There are two ways to add support to a ROM for network location providers.
One is, as you mentioned, to sign the UnifiedNlp package with the build key of the ROM. It's not necessary for it to be part of the system partition, so Copperhead could still make it optional when choosing this way (but would be required to build/sign it theirself and upload the apk somewhere).

It's also possible to allow third party network location providers. This is the "default" way, as Google is third party from manufacturer position. This is done by adding a new package to the config_locationProviderPackageNames array resource (defined in frameworks/base/core/res/res/values/config.xml). Usually, ROMs add com.google.android.gms there, which allows the usage of the Google location provider (which is often installed via GAPPS package). This is why the F-Droid download for ROMs without GAPPS has this package name ;). If you don't want to allow Google, but only UnifiedNlp, there is a separate package with package name org.microg.nlp which can also be added to said config option to allow usage of UnifiedNlp.

I don't think there are relevant security implications from allowing third party location providers: The only thing you can do due to this is to spoof the location, nothing really to fear about, IMHO.

[mar-v-in]

As far as I understood so far, AOSP needs UnifiedNLP to be build into the system/signed with the build key of the rom. Would it be an option to add this to copperheadOS?

There are two ways to add support to a ROM for network location providers.
One is, as you mentioned, to sign the UnifiedNlp package with the build key of the ROM. It's not necessary for it to be part of the system partition, so Copperhead could still make it optional when choosing this way (but would be required to build/sign it theirself and upload the apk somewhere).

It's also possible to allow third party network location providers. This is the "default" way, as Google is third party from manufacturer position. This is done by adding a new package to the config_locationProviderPackageNames array resource (defined in frameworks/base/core/res/res/values/config.xml). Usually, ROMs add com.google.android.gms there, which allows the usage of the Google location provider (which is often installed via GAPPS package). This is why the F-Droid download for ROMs without GAPPS has this package name ;). If you don't want to allow Google, but only UnifiedNlp, there is a separate package with package name org.microg.nlp which can also be added to said config option to allow usage of UnifiedNlp.

I don't think there are relevant security implications from allowing third party location providers: The only thing you can do due to this is to spoof the location, nothing really to fear about, IMHO.

[thestinger]

I won't be signing an application without understanding the security implications and I won't poke holes in the security model chosen by Android as a general rule. They choose to allow only first-party location services, and CopperheadOS isn't going to be changing that. UnifiedNlp would only be signed if it respected that security model itself, and I don't think it does. Spoofing location information can be seen as a security issue. This isn't on the table right now because I don't have time to devote to forking UnifiedNlp.

[thestinger]

I won't be signing an application without understanding the security implications and I won't poke holes in the security model chosen by Android as a general rule. They choose to allow only first-party location services, and CopperheadOS isn't going to be changing that. UnifiedNlp would only be signed if it respected that security model itself, and I don't think it does. Spoofing location information can be seen as a security issue. This isn't on the table right now because I don't have time to devote to forking UnifiedNlp.

[mar-v-in]

UnifiedNlp exposes the location provider functionality to third party modules, which once activated can spoof the location. If the ability to spoof locations (which is built into AOSP in developer settings anyway) is a security issue for you, then I guess UnifiedNlp is nothing for you.

If - long-term - you plan to build a custom, more restrictive location provider into CopperheadOS, e.g. based on Mozilla Location Services, I'd be glad to help out, just drop me a message.

[mar-v-in]

UnifiedNlp exposes the location provider functionality to third party modules, which once activated can spoof the location. If the ability to spoof locations (which is built into AOSP in developer settings anyway) is a security issue for you, then I guess UnifiedNlp is nothing for you.

If - long-term - you plan to build a custom, more restrictive location provider into CopperheadOS, e.g. based on Mozilla Location Services, I'd be glad to help out, just drop me a message.

[thestinger]

It would be fine if it was opt-in per app, but the way UnifiedNlp does it isn't really acceptable to me. The user should have to explicitly whitelist each app, and the configuration shouldn't simply store it by app name. It needs to be by name + signature.

[thestinger]

It would be fine if it was opt-in per app, but the way UnifiedNlp does it isn't really acceptable to me. The user should have to explicitly whitelist each app, and the configuration shouldn't simply store it by app name. It needs to be by name + signature.

[thestinger]

And I'd need to be very confident in UnifiedNlp if I was going to sign it, which is going to require a time investment to figure out if the way it works fits into the desired security model, etc.

I'm very wary of opening up any security holes that are not present in stock Android. That would defeat the purpose of the project. Having F-Droid included as a privileged application is already crossing over that line. It's something that needs to be considered with great care and not taken for granted.

[thestinger]

And I'd need to be very confident in UnifiedNlp if I was going to sign it, which is going to require a time investment to figure out if the way it works fits into the desired security model, etc.

I'm very wary of opening up any security holes that are not present in stock Android. That would defeat the purpose of the project. Having F-Droid included as a privileged application is already crossing over that line. It's something that needs to be considered with great care and not taken for granted.

[mar-v-in]

1. The user currently needs to actively activate each app in settings before it is used first time. Is there anything else you expect as "whitelisting"?
2. Currently signature is not stored with the app, but that's indeed a good idea i'll add in the next release.

[mar-v-in]

1. The user currently needs to actively activate each app in settings before it is used first time. Is there anything else you expect as "whitelisting"?
2. Currently signature is not stored with the app, but that's indeed a good idea i'll add in the next release.

[thestinger]

@mar-v-in That's what I want it to do, but I need to take a look at it to make sure it's doing it the way I want. Just don't have time to take on all of this stuff that people want...

All I'm really saying is that I'm not comfortable signing something without knowing much about it, but that's the route I'd want to take for this. We can have our own F-Droid repository with stuff like this.

[thestinger]

@mar-v-in That's what I want it to do, but I need to take a look at it to make sure it's doing it the way I want. Just don't have time to take on all of this stuff that people want...

All I'm really saying is that I'm not comfortable signing something without knowing much about it, but that's the route I'd want to take for this. We can have our own F-Droid repository with stuff like this.

[mar-v-in]

Sure thing and I fully understand your position - I won't put my signature under anything I don't know either. Just wanted to make sure I understand your ideas right.

[mar-v-in]

Sure thing and I fully understand your position - I won't put my signature under anything I don't know either. Just wanted to make sure I understand your ideas right.

[christoph-buente]

Hey, thx for picking up on this. Please don't judge the book by it's cover. I think UnifiedNlp is worth a code audit and would provide real value to the users without sacrificing security.

My two cents: UnifiedNlp does not provide anything except an interface for network based location geocoders. So first step is: the user would need install UnifiedNlp actively. And then the user would need to install a backend for network based locations. There are several available in f-droid right now. Some would be not preferred (like the Apple one). But on the detail page of this packages it is made very clear, that this piece of software is made to track the user. And then there are others like GSMLocationNlpBackend which can be loaded locally and provides a secure way of GSM based positioning without being tracked.

Not sure i can provide any help with this issue but glad to help.

Thx, Christoph

[christoph-buente]

Hey, thx for picking up on this. Please don't judge the book by it's cover. I think UnifiedNlp is worth a code audit and would provide real value to the users without sacrificing security.

My two cents: UnifiedNlp does not provide anything except an interface for network based location geocoders. So first step is: the user would need install UnifiedNlp actively. And then the user would need to install a backend for network based locations. There are several available in f-droid right now. Some would be not preferred (like the Apple one). But on the detail page of this packages it is made very clear, that this piece of software is made to track the user. And then there are others like GSMLocationNlpBackend which can be loaded locally and provides a secure way of GSM based positioning without being tracked.

Not sure i can provide any help with this issue but glad to help.

Thx, Christoph

[xmikos]

Just to let you know, if you install OpenGApps, network location doesn't work too. So it is not only problem with microG UnifiedNLP, but also with network location provider from official Google Apps.

[xmikos]

Just to let you know, if you install OpenGApps, network location doesn't work too. So it is not only problem with microG UnifiedNLP, but also with network location provider from official Google Apps.

[thestinger]

The best solution for Google Play would be the target files solution you proposed. I really wouldn't want to place any trust in Google Play in the default builds.

[thestinger]

The best solution for Google Play would be the target files solution you proposed. I really wouldn't want to place any trust in Google Play in the default builds.

[z0id]

Any update on this? Is there a way to get UNlp to work on Copperhead?
I just installed the OS and I see there's no way to get location service while indoors, which is a pretty important feature imho.
Cheers!

[z0id]

Any update on this? Is there a way to get UNlp to work on Copperhead?
I just installed the OS and I see there's no way to get location service while indoors, which is a pretty important feature imho.
Cheers!

[thestinger]

No update on this yet. Not planned in the near future.

[thestinger]

No update on this yet. Not planned in the near future.

[xmikos]

I didn't have time to finish my scripts for Copperhead OS customization yet :-( Too much work, other FOSS projects, etc. Once finished, it will allow you to download official build target files, customize it (add microG, GApps, seSuperuser, etc.) and re-sign it with your own key (this way you can still have locked bootloader for better security from "evil-maid" type of attacks). But I will not have time to work on it again till august :-/

[xmikos]

I didn't have time to finish my scripts for Copperhead OS customization yet :-( Too much work, other FOSS projects, etc. Once finished, it will allow you to download official build target files, customize it (add microG, GApps, seSuperuser, etc.) and re-sign it with your own key (this way you can still have locked bootloader for better security from "evil-maid" type of attacks). But I will not have time to work on it again till august :-/

[thestinger]

A problem with this is that we aren't supposed to be marking the OS as having a network location service if there isn't one. It causes a test failure in the CTS and perhaps other issues. It may be necessary to stop doing that and then it wouldn't be possible to bundle one after the fact.

[thestinger]

A problem with this is that we aren't supposed to be marking the OS as having a network location service if there isn't one. It causes a test failure in the CTS and perhaps other issues. It may be necessary to stop doing that and then it wouldn't be possible to bundle one after the fact.

[thestinger]

This isn't a planned feature at the moment. Supplemental location information may happen at some point but it's not actually going to be network-based and it wouldn't be based on anything requiring a bunch of user choice / input.

[thestinger]

This isn't a planned feature at the moment. Supplemental location information may happen at some point but it's not actually going to be network-based and it wouldn't be based on anything requiring a bunch of user choice / input.

[mar-v-in]

re "requiring a bunch of user choice / input", UnifiedNlp allows a ROM to provision it's configuration. If UnifiedNlp + a backend is installed you can have it running straight from the beginning without user input. When UnifiedNlp resides in /system/priv-app it will also hide it's launcher icon and show it in the Settings->Location screen instead.

I know this to be used in at least one commercial-grade, end-user, AOSP-based aftermarket firmware (not sure if it passes CTS though). The power-users are still able to configure custom backends and others have a nice network-based location service running. The pre-configured location backend btw is a custom development by the firmware developer that uses their servers. This is exactly what UnifiedNlp was developed for: adding a unified API for network location providers, even proprietary ones, but still allowing the user to actually choose.

Also I wonder how you want to provide supplemental location information that is not network-based, even GPS is sort of network based and the only way I can thing of to not be network-based is not automatic -> ask the user for his current location (which was, btw, realized as a UnifiedNlp backend as well).

But if "user choice" is a problem for you, any approach based on UnifiedNlp is probably wrong.

[mar-v-in]

re "requiring a bunch of user choice / input", UnifiedNlp allows a ROM to provision it's configuration. If UnifiedNlp + a backend is installed you can have it running straight from the beginning without user input. When UnifiedNlp resides in /system/priv-app it will also hide it's launcher icon and show it in the Settings->Location screen instead.

I know this to be used in at least one commercial-grade, end-user, AOSP-based aftermarket firmware (not sure if it passes CTS though). The power-users are still able to configure custom backends and others have a nice network-based location service running. The pre-configured location backend btw is a custom development by the firmware developer that uses their servers. This is exactly what UnifiedNlp was developed for: adding a unified API for network location providers, even proprietary ones, but still allowing the user to actually choose.

Also I wonder how you want to provide supplemental location information that is not network-based, even GPS is sort of network based and the only way I can thing of to not be network-based is not automatic -> ask the user for his current location (which was, btw, realized as a UnifiedNlp backend as well).

But if "user choice" is a problem for you, any approach based on UnifiedNlp is probably wrong.

[thestinger]

CopperheadOS won't be sending PII like location information to a server by default or encouraging users to do it via opt-out or even opt-in prompting. It's also not going to be adding non-trivial features aimed only at power users right now. That doesn't leave any room to include this feature as described in the issue. It doesn't rule out supplementing the GPS with other information, but it would have to be from local databases and it would need to be a very lean app not adding significant attack surface or user-facing complexity. It's not in the cards right now. It can be revisited in the future but there's no point of leaving this issue open because it's far from what we would actually want.

[thestinger]

CopperheadOS won't be sending PII like location information to a server by default or encouraging users to do it via opt-out or even opt-in prompting. It's also not going to be adding non-trivial features aimed only at power users right now. That doesn't leave any room to include this feature as described in the issue. It doesn't rule out supplementing the GPS with other information, but it would have to be from local databases and it would need to be a very lean app not adding significant attack surface or user-facing complexity. It's not in the cards right now. It can be revisited in the future but there's no point of leaving this issue open because it's far from what we would actually want.

[thestinger]

If an issue isn't specific / actionable enough to be tagged as a project for contributors and I don't plan on working on it then it's being closed. That's just how the issue tracker is going to be used. It's not going to be used to track ideas unless they can be broken down into something we would accept. I don't want people thinking that we would accept pull requests for stuff like this when there's a whole lot of thought that needs to go into it before touching it, and in all likelihood the needs will not align well with existing projects working on it. There's much higher priority stuff to be worked on. The features aren't even all ported over to Nougat yet and it has been 4 months...

[thestinger]

If an issue isn't specific / actionable enough to be tagged as a project for contributors and I don't plan on working on it then it's being closed. That's just how the issue tracker is going to be used. It's not going to be used to track ideas unless they can be broken down into something we would accept. I don't want people thinking that we would accept pull requests for stuff like this when there's a whole lot of thought that needs to go into it before touching it, and in all likelihood the needs will not align well with existing projects working on it. There's much higher priority stuff to be worked on. The features aren't even all ported over to Nougat yet and it has been 4 months...

[spacekitteh]

Maybe tag it as "wish list (manpower)" or something? That way it's easier to find in the future.

[spacekitteh]

Maybe tag it as "wish list (manpower)" or something? That way it's easier to find in the future.

[thestinger]

There's the far-future label for that but this is more than that. Bundling stuff not matching up with our priorities doesn't make sense, especially if it's more than we would want to take over maintaining. Including F-Droid, Silence and Etar was already something that took a lot of thought and it has risk. What if Silence stops being maintained? Probably wouldn't do that again particularly since people don't really seem to be using it for encrypted texting due to lack of popularity.

[thestinger]

There's the far-future label for that but this is more than that. Bundling stuff not matching up with our priorities doesn't make sense, especially if it's more than we would want to take over maintaining. Including F-Droid, Silence and Etar was already something that took a lot of thought and it has risk. What if Silence stops being maintained? Probably wouldn't do that again particularly since people don't really seem to be using it for encrypted texting due to lack of popularity.

[thestinger]

For system-level code, we need to have a full understanding of everything that's included beyond AOSP. It needs to be simple and minimal with a focus on security / robustness / maintainability above features / flexibility. Need to be able to fully take over maintenance at any point. F-Droid privileged extension is very minimal, F-Droid not so much, but it's pretty much a hard requirement for us. This is far from that.

[thestinger]

For system-level code, we need to have a full understanding of everything that's included beyond AOSP. It needs to be simple and minimal with a focus on security / robustness / maintainability above features / flexibility. Need to be able to fully take over maintenance at any point. F-Droid privileged extension is very minimal, F-Droid not so much, but it's pretty much a hard requirement for us. This is far from that.

[z0id]

Just wanted to add that UnifiedNLP has backends that provide offline location support using precompiled base-stations/WiFi APs databases.

[z0id]

Just wanted to add that UnifiedNLP has backends that provide offline location support using precompiled base-stations/WiFi APs databases.

[Rudd-O]

A solution that meets all of thestinger's requirements:

1. Bundle UnifiedNLP (org.microg.nlp) with CopperheadOS images, but DO NOT bundle any location providers for it. This leaks no location data, produces no upstream network requests, and does not compromise users' privacy in any way.
2. Document in the Copperhead site what people can do in order to get network location enabled. Suggest in the documentation that the most privacy-friendly option is the offline GSM database location provider; point out other location providers. All of these are available on F-Droid and should be installable. Point out in the docs how to enable these location providers once they have been installed.

I would go as far as to say that CopperheadOS should just include the offline GSM database location provider by default, and enabled, as this particular location provider (a) does nothing if a database has not been downloaded (b) does not do any network requests that leak or are problematic from a privacy standpoint.

I believe the only thing that is missing is thestinger's condition that the code be reviewed. thestinger, would you trust someone else's review?

[Rudd-O]

A solution that meets all of thestinger's requirements:

1. Bundle UnifiedNLP (org.microg.nlp) with CopperheadOS images, but DO NOT bundle any location providers for it. This leaks no location data, produces no upstream network requests, and does not compromise users' privacy in any way.
2. Document in the Copperhead site what people can do in order to get network location enabled. Suggest in the documentation that the most privacy-friendly option is the offline GSM database location provider; point out other location providers. All of these are available on F-Droid and should be installable. Point out in the docs how to enable these location providers once they have been installed.

I would go as far as to say that CopperheadOS should just include the offline GSM database location provider by default, and enabled, as this particular location provider (a) does nothing if a database has not been downloaded (b) does not do any network requests that leak or are problematic from a privacy standpoint.

I believe the only thing that is missing is thestinger's condition that the code be reviewed. thestinger, would you trust someone else's review?

[thestinger]

would you trust someone else's review?

If they've clearly done a good job.

Regardless, I'm unwilling to bundle more code without substantial help maintaining the existing code. It's already too much so nothing like this is happening without significant contributions.

[thestinger]

would you trust someone else's review?

If they've clearly done a good job.

Regardless, I'm unwilling to bundle more code without substantial help maintaining the existing code. It's already too much so nothing like this is happening without significant contributions.