Mozilla Location Service is retiring #2237
Comments
|
We had also AppleWifiNlpBackend, could you please merge it in microG? |
|
Interesting... is there a natural successor to it? |
|
Apple does not allow us to access their location service. Also, in contrast to most Google APIs we're using, usage is not strictly required for interoperability, so we don't have a legal basis for using their API in most jurisdictions. Unfortunately there is no natural successor I'm aware of. However there are also other open source projects (e.g. geoclue, the location service of Linux desktop) affected, so I'm hoping for some coordinated effort to bring a similar open source service to life. |
|
Can we just continue to run MLS in some jurisdiction that isn't concerned about patents? |
|
This would sort of kill microG for me, can we just use Apple's NLP? It was always more accurate anyway? |
|
As I mentioned in the /e/ OS Community I hope something better than MLS is used because ever since microG went to 0.3.0 every app but maps (Magic Earth) is not accurate at all and is off by 50-100 miles. I agree with using Apple's NLP. Unless a good accurate open source NLP is found. |
|
Even if we have to put Apple's NLP behind a radio button with a big disclaimer "Apple doesn't want you to do this, you're on your own"
…-------- Original Message --------
On 14 Mar 2024, 12:53 pm, ToughDBlue wrote:
As I mentioned in the /e/ OS Community I hope something better than MLS is used because ever since microG went to 0.3.0 every app but maps (Magic Earth) is not accurate at all and is off by 50-100 miles. I agree with using Apple's NLP. Unless a good accurate open source NLP is found.
—
Reply to this email directly, [view it on GitHub](#2237 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AB4ZXESVFVT54IBEXNC7QTDYYGMUHAVCNFSM6AAAAABEVCZHJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJXGM4TCNJTGQ).
You are receiving this because you commented.Message ID: ***@***.***>
|
It is all about the data. The more users contribute data to the service, the more precise the results become. |
I understand but when I had microG 0.2.27 it all worked great. How about going back to that NLP method? |
|
As I wrote above:
As a last resort option we could of course have that included and put a big warning that using it is a breach in ToS and Apple can sue your for financial compensation if you used that feature when trying to enable it, but if we can figure out somehting else I would definitely prefer that. If you have problems with location in current versions of microG I suggest to enable local location learning in settings and then use an app that requests both GPS and network based location. As soon as you get a GPS fix, it would then remember the correct location of nearby wifi networks and use it instead of MLS, so MLS' data quality doesn't matter anymore. |
Understandable, don't want to breach any TOS. Didn't know that. I'd rather something else be figured out as well. |
|
SatStat would do the trick. Also apps that use Google Maps API of microG often do request both locations through microG's API.
|
That works! Although my weather app says "Current Location" and not the actual city and state. The map in the weather is correct but again just says "Current Location" and not city and state. I have Nominatim enabled. Is there a way to reset Nominatim? |
|
I feel that the loss of UnifiedNLP together with this makes for an unacceptable step back. @mar-v-in I know there were technical reasons behind dropping UnifiedNLP but consider that it was working fine or well enough not to notice the issues for many, including on newer versions of Android, and that there was a small but relatively healthy "ecosystem" of backends and not just Apple, for instance I was an avid user of offline backends, and the GSM Location backend wasn't limited to "learning" but could download cell databases for local use. That was amazing and given I use mostly open source applications, I do lose a lot of the usefulness of microG when network location goes away. |
This was intended to be a feature in the new implementation as well (+ importing wifi and cell databases from file). With the learning code, there already is the logic to handle location data once it's in the database, we "just" lack the downloading/importing.
I think there might be a bias here. As you said, you've been mostly working with offline backends, which are faster in processing and less likely to run into timeouts or similar issues than online backends. I am convinced that the single process approach and centralized code will in the long run improve the results, but I do recognize that short term there are downsides as long as we lack some important features (with download/import of data being one of the major lacking features). |
|
The way that I see it, one of the primary purposes for which people use their smartphones is navigation. Personally, I'd say navigating is a more common use case for a smartphone than voice calls. If they don't have a fast, accurate location provider (ie if all they have is GPS), they can't navigate with their phone. The lock-on time is too high and it's not usable effectively indoors or in dense cities. If people can't navigate with their phone on MicroG, they won't use MicroG on their phone. Losing the ability to provide navigation - via whatever means - isn't just a feature regression, it's an existential threat to MicroG usage. It's important to do things the right way, but it's also important to make sure the project has a healthy enough base of users that it survives. |
|
Does anyone know how accurate OpenCellID and other downloadable databases actually are? Maybe if their TOS allows we could add a menu to download their data (maybe on a country by country basis). |
Downloading can be worked around for some (I e.g. do that on my Linux machine, using FastLacellsGenerator, which includes MLS and OpenCellID, and then push the database to all my devices – also saves me from downloading and generating it multiple times). Easiest way would probably be supporting that SQLite database directly. Not sure if the corresponding Backend could be used stand-alone to download and generate the database on-device for those who need it – but having it supported would be a good first step IMHO. Btw: I saw that TowerCollector now supports custom MLS URLs for upload, so there might be some "aggregators" for that, too. Not sure if they could be accessed the way microG currently accesses MLS, though. But an offline database is what some of us would definitely like to have anyway 😜 |
Yeah I've been using microG now for a while, however having to mess about whenever attempting to order an Uber is becoming too much. Sandboxed GMS is looking a lot better in terms of quality of life. I really don't want to have to deal with downloading local towers etc, I just want a decent enough NLP provider (Apple or Google) |
|
I vote for return UnifiedNLP. Or someone self-hosted database server. |
This doesn't really solve the privacy issues; this would push people to Google and Apple for Location Services, weakening the privacy/FOSS projects. That way you can create a public database but have it be relatively private; someone revealing their SSID won't have the same effect on their location privacy as it usually has; of course there's other public databases like Wigle that can make this an issue, but they seem very bad and outdated that accuracy is... eh. As you would only really know the mac and SSID if you know where the network is anyway; of course you wouldn't hash individual properties as that doesn't really give you many benefits The issue with such an approach rather than just publishing raw data, is that protocol ossification can be an issue. One solution to this is allow users to backup their contribution data and never delete it; so in the case of an upgrade to the network, one can re-upload new data. I think something like this can also have value in a closed ecosystem, which would have better privacy, as in a map it is still very possible to do some analysis of contributed points and dates and things; I personally think that a closed system that uses a private protocol is better than a completely open, or than closed but not private to the operator; with a closed system with a private protocol, we could potentially move the data in a private way to other projects; and data analysis is harder. TowerCollector is an interesting project, the maintainer is busy ATM and it's not his focus, so a fork to include WiFi, Bluetooth data, etc and be compatible with such a protocol would be needed. This would be a lot of work and research; so it's very likely something like this isn't actually developed; it's still interesting to debate this, especially what improvements can be made to location services overall :P
/e/foundation seems to be offering to do that (thankfully!) |
It would push me to using the local databases I was using before UnifiedNLP was retired. Can't speak for others of course. I also wonder: how many independent developers are probably going to want to be contributing to this repository, with PRs that must rightly not break things and be approved, as opposed to tinkering by writing their own module in their own repository, maybe using a simple existing module as a template, and publishing it there? I think that was a powerful aspect of the approach. |
|
FYI, my plan for the next version is to add:
|
Will that include the cell database as used by LocalGSM – and maybe auto-detect if a fresh copy has been placed in a defined location (as LocalGSM does)? Idea is to easily distribute an existing database to multiple devices for offline use (in addition to or instead of online checks like Ichnaea). |
And @rtsisyk offered the same on Mozilla's thread on this. There doesn't seem to be any shortage of people/organizations willing to do the hosting. |
|
FYI, all of us and a bunch of others are in communication with each other. We're aiming to ensure there is a single successor of MLS endorsed and supported by all the libre software projects involved, including microG |
|
Relatedly, one issue I've had myself and seen many reports of with the internal UNLP implementation in recent GmsCore versions beyond the Mozilla db staleness: Just enabling any of the Mozilla UNLP sources at all seems to "pollute" GPS results to the point that even with a good GPS lock geolocation is flaky or completely anomalous. You can see this with SatStat on the map page where the blue UNLP "circle" and/or pin is constantly turning grey ("stale data") and/or flickering on/off constantly even when the device is stationary. Seems to me that a good GPS lock should override UNLP inputs especially if the UNLP inputs are extremely unstable. I can open a separate issue on this if necessary. Just seems that this should be fixed along with enabling new MLS hosts, or else I think it will undermine the utility of those as well. |
|
The network location provider is (and should be) independent of GPS and having a GPS fix and the results from GPS are not considered in it. The blue circle in SatStat displays the network location provider, the red circle shows the GPS location. There is also the fused location, merged from both. This should use GPS once GPS is available and network location if not. The flickering of the blue location is probably caused by the update interval. MicroG only provides an update if updated data exists, which due to rate limiting might be less often then SatStat's requested interval. |
|
Perhaps my analysis of the problem was not correct. But the observation that any enabling of the Mozilla UNLP sources often results in completely anomalous geolocation results is consistently demonstrable. I have talked to various people from all around the world that were getting results like placing them in the middle of the Atlantic ocean or something, until they disabled the Mozilla UNLP data sources in the microG utility. After disabling the Mozilla switches, they got good geolocation results. I actually just saw the Fused location provider in SatStat for the first time after acquiring a Pixel7Pro I installed CalyxOS on. Prior to that, I had never seen that data source displayed in SatStat on my older devices at all. (Some of which are running older versions of LOS, some CarbonROM. All using Qualcomm SoCs.) |
|
The fused provider is not available on all devices, that's why there is also a fused location API in Play Services that is also provided by microG; that's why some apps need microG to display locations, they use the Play Services fused location provider. If erroneous network location data claims to be more accurate then GPS, it might be preferred over GPS data. I've never seen this as a relevant issue in practice, but it could explain the behavior you described. Indeed this could be improved in microG's fused provider by making low accuracy GPS still override high accuracy network location if the network location is off by too far from the GPS. |
|
If it's helpful I could try to go back to the people who observed that behaviour and ask them to re-enable the Mozilla switches and collect logs or something. I will try it on my devices again and check. (FWIW: I disabled/uninstalled the Mozilla UNLP backend ~2 years ago due to the data staleness in a city that might be the most extensively geomapped in the world: San Francisco, California) |
I just tested this and can confirm it doesn't place me way out of where I actually am anymore. Can someone ELI5 what the "Remember GPS Location" option does? |
When an app requests both GPS and network location (like e.g. SatStat and apps with a map using Google Fused location API) and this option is enabled, microG will store a record of the GPS location and the information used for network location at that point in time in a local database. If later, for the same network information, a location is requested, the local database is used instead of requesting from Mozilla. There are a bunch of restrictions, like GPS locations being ignored if their accuracy is too low and the local database entries first having to accumulate some certainty (by having multiple occurrences of the same network), so its effect might not be visible immediately. The feature is disabled by default as the contents of the database can be used to recover a user's partial location history (if they use GPS). |
In order to get accurate geolocation results I've had to disable Mozilla services since microG went to versions higher than 0.2.27. Even enabling "Remember GPS location" puts me far away from where I'm really at. |
|
microG gained the ability to customize MLS URL recently commit, but this nightly build isn't updated til now. |
Now you just need another MLS clone service. Afaik that doesn't exist yet. I know @mar-v-in is involved in that planning by the FOSS community as stated above, as a stopgap before building something better from the ground up. (The last significant public discussion of that I know of went down in flames, 'nuff said about that for now.) But even if/when such a clone MLS service goes online, and even if it bootstraps its database by importing the existing MLS database, that data is so stale as to be (IMHO) pretty close to useless in most regions of the world. The microG users I support are starting to lose patience after standalone UNLP backends were deprecated in May 2023. Coarse location (eg for country-level S/W license checks etc) can usually be done trivially via GeoIP. IMHO what we need a UNLP service for is for more precise location when GPS is not available or feasible, and to "jumpstart" a relatively high-accuracy location while waiting for a GPS fix. Cell towers are typically too far apart to be useful for that, the MLS WiFi/BT data was never public and the whole field is polluted by a gaggle of patent trolls dying to sue anyone that tries to field any sort of RF-emitter-based geolocation service. (Skyhook [now part of Qualcomm], Geoscope, Enovsys, etc) Building a new online db of RF emitters from scratch will take years to get decent global coverage. There needs to be a robust new set of incoming data inputs to make it feasible, and there's no way that the small microG userbase can make a useful dent in that, it will have to come from a LOT more endpoints. |
I only noticed everything in passing and would like to add something to your statement. I hope the FOSS community is already working on it :) Unfortunately I am too stupid to program, but I have a lot of funny ideas in stock. |
|
@Sapiosenses , may be use downloadable OpenCellID database for self-hosted backend. For now I use proprietary (push server proprietary anyway) Google endpoint |
As was announced at mozilla/ichnaea#2065, Mozilla will retire MLS soon. The final deadline for third parties seems to be set to June 12.
The text was updated successfully, but these errors were encountered: