No setting to disable: download sends unique user tracking UUID & number of days since install

[sneak]

OsmAnd/OsmAnd/src/net/osmand/plus/download/DownloadOsmandIndexesHelper.java

Lines 273 to 281 in 22e40f1

	if (nd > 0) {
	strUrl += "&nd=" + nd;
	}
	strUrl += "&ns=" + ctx.getAppInitializer().getNumberOfStarts();
	try {
	strUrl += "&aid=" + ctx.getUserAndroidId();
	} catch (Exception e) {
	e.printStackTrace();
	}

This appears to send the number of days since install, as well as a unique identifier (getUserAndroidId), to the index server when fetching indices, without respect for the telemetry preference setting.

This is a data leak that allows for a user's travel history to be tracked by the server, as these requests include client IP and a unique tracking identifier, and client IP is coarse (city-level) geolocation. This means the server can see the various cities the given userIosId travels to as the client IP changes over time.

This spyware feature also exists in the iOS version, and fails to respect the user's consent choice there as well. It seems unlikely that this is an accident.

osmandapp/OsmAnd-iOS#2115

[gy2256]

Any follow up from the code maintainer?

[scaidermern]

Any follow up from the code maintainer?

See osmandapp/OsmAnd-iOS#2115

[vshcherb]

UUID is anonymized randomly generated key per installation, cause this key is not associated with any advertisement id or not transferred to any other 3rd party or even used, this key doesn't require consent. As you mentioned the IP could provide more information cause it's location specific.

This approach helps to the server monitor fair usage of resources.

[vshcherb]

Some thing to add:

1. it's in privacy policy (https://osmand.net/help-online/privacy-policy/#3-the-information-company-collects)
2. We don't want to use IP as a counter, so we use randomly generated UUID which preserves the privacy much more than IP itself
3. You don't need to use OsmAnd Services if you don't agree with Terms of Use , you can download or build maps yourself.

[zander]

There is a strong disconnect between the tone and even some accusations and the actual issue being reported.

First of all, connecting to the server is only something that happens during download. OSMAnd is an offline map and downloads happen rarely. For heavy users less than once a month, for many much less than that. Additionally, all such downloads are user-initiated, they have to explicitly press a download button. Nothing that would be useful to build a profile of any specific user.

Allegations of travel data and such being leaked are thus really not in evidence.

The data that is being shared is not possible to connect to any personally identifying information. Notice that google explains that since some 5 years the getUserAndroidId is unique per installed application. So any other application the user uses (on the same phone, or not) will not share the UUID. So on top of the fact that not much of a profile can be made at all, there is no way to connect such to any actual personal info.

Does anyone actually have any real issues with the existing code?

For the record, I don't have any relation to the company behind OSMAnd, never received anything from them (other than FOSS software) and I'm just an open source developer standing up for common sense.

[vshcherb]

I think there are 2 direction of discussions: consent and identification.

1. I don't think consent is needed, same as you don't need consent that your IP is used to download maps. Second it's not a website where you read information, here you use resources, so in that sense by downloading data you agree to fair use of resources (Terms of Service) and in that case you agree that you don't download too many resources especially if you are not paid user for example

It would be completely different case if program would send statistics in background that you're using the application and here is an IP.

To summarize: it's probably needs to be more clarified in Terms of Service but it doesn't require a consent.

2. User identification. We switched from IP to UUID to calculate how many active users & how often update exactly to solve the server load issue and don't use personal data. For example we don't require email to identify user and don't track IP exactly cause IP could provide location specific information. Random UUID doesn't identify anything if it's not connected to any other services or other private information. We don't share UUID with any other applications so they also couldn't track that UUID even if these apps have personal information.

It's completely incorrect to say that app could view travel history, I would say it's total nonsense cause the server only views what and when maps were downloaded in contrast of other apps that provide Internet services.

---

Last but not least.

The websites require to ask consent only for information they gather for example if you search an address on website and website doesn't store these addresses or don't connect them to IP (address is essentially publicly available information), consent is not needed. If the website will start storing information in the table [IP, Search], then consent is needed. So it's all about how the data is processed and used and not only what's being transmitted.

[vshcherb]

Just to provide another point of view on this topic. As a Service provider I need a unique identifier to predict the load on servers (especially once maps are updated), so in the end it would say:

"Download maps won't be provided if user doesn't agree to create a random key for the installation".

Does it makes sense then to implement at all ? Cause in the end it will end up, if you to download - you need a consent which to me is actually totally clear if you don't want to know what & when you downloaded - download or generate maps yourself and import to the app. App won't send to the server what you've imported yourself!

[zander]

@sneak

you claimed that;

to track the travel history of that user via client-ip geolocation.

Considering that this is an offline-map which doesn't normally do downloads, how do you support your claims? If you continue to make big claims, please provide actual evidence.

[nemobis]

Il 14/08/22 13:57, Jeffrey Paul ha scritto:

No, this doesn't make sense at all, as I can distribute a forked client that sends no ID, or a random UUID on every single HTTP request

In which case the client will be blocked from downloading the datasets, no? Sounds like the point.

[zander]

@sneak

Please stop suggesting that the UUID is somehow private or anonymous. With client IP geolocation it can identify users by travel history.

Maybe the problem lies that nobody explained in technical details what it means for OSMAnd to be an offline map.

It means that the application, when started, does not cause any network traffic at all. You can travel and use the map daily, including searches for point-of-interests and doing routing, without ever causing a single download. Without, specifically, leaking that uuid.

OSMAnd, in other words, is perfectly useful and allows people to use the app for months or years without ever triggering the download you have noticed sends an UUID. People can download all their data at home or at work, regardless of where they travel later.

Sneak: you are wrong. It is time to admit you came to wrong conclusions and time to publicly show you are able to learn when presented with new facts. Thank you.

[saddy001]

Let me summarize a bit.

• To keep the server load tolerable the admins want to limit the per-user-map-downloads
• To achieve that, OsmAnd creates an UUID with Java.util.UUID.randomUUID() (https://tutorialspoint.com/java/util/java_util_uuid.htm)
• The server logs all requests containing the UUID and the user's IP and stores it indefinitely
• The UUID is used to block too frequent downloads

Consent
I am no lawyer, but I would look at it by means of GDPR. However, I would agree with the accepted answer here that there is no consent required.

Concerns that come to mind
Although I believe the claims that the UUID/IPs are never shared with any third party, I think it would be better to not have to trust anyone. Maybe one day the server gets hacked or the admins are forced to hand over all data to law enforcement.
Indeed I don't like the idea that all my IP history gets into the wrong hands. IP adresses are not anonymous and can be assigned to individuals in many ways. Flourishing VPN services show that people understood that.

Ideas for compromises
Sadly, I have no idea that takes no development effort. Although I believe that the effort would be manageable. Here are two:

• UUID could be re-generated e.g. when more than 2 weeks have passed. For that, OsmAnd would need to store an additional timestamp together with the UUID
• Records could be deleted on a regular basis on the server-side
• @scaidermern had another one

[saddy001]

The uuid does not limit anything. Anyone can make as many requests as they want with a new uuid each time, defeating this limit.

This is true. But it limits the average user case, where the user does not manipulate the request just to download some maps and I guess, this comprises almost 100% of all users.

[vshcherb]

@saddy001 - agree that server has no control but practically it's unsolvable issue cause indeed anything could happen on the server side (i.e. hacked or rights transferred). In the end it would be a liable action or reputation loss.

• We gather general statistics by 1 month mostly though I would tend agree to regenerate every 3 months cause then we could generate quarter reports
• We also need to spend time on server side to correctly clean the data and leave just summary reports without any private information. For that matter we need to do proper automation and standardize reports a bit .

---

I think these are doable actions and if we're fine with them, I will start planning implementation. Although it might not be too much development effort but I want to implement carefully so in summary reports it will be comparing "apple to apples".

[scaidermern]

This is true. But it limits the average user case, where the user does not manipulate the request just to download some maps and I guess, this comprises almost 100% of all users.

I intend to distribute a fork of this application with all unique IDs randomized on every request. Your assumption will not remain accurate.

You can do this of course. However, how many users do you expect to use your fork? I bet the above number will still remain close to 100%.

Also there is nothing stopping a single someone from making a high request load to the server with random IDs, rendering the server data collection useless.

Blocking via IP is still possible unless you intend to perform a DDOS... Let's go back to the topic, shall we?

[realpixelcode]

it's in privacy policy

Do you really think anyone actually reads it?

we use randomly generated UUID which preserves the privacy much more than IP itself

A unique identifier that, in contrast to the IP address, can't be easily anonymised by the user and that also persists across IP addresses “preserves the privacy”? Surely, Google uses those Android ad IDs because they're so incredibly anonymous.

You don't need to use OsmAnd Services if you don't agree with Terms of Use

Besides the fact that privacy laws are a thing, this is incredibly detrimental to say, considering that OsmAnd has always presented itself as a privacy-friendly alternative to commercial services. Telling people not to use it if they care about privacy is something I'd only expect from Google or Microsoft.

you can download or build maps yourself

This is not an option for probably 99% of users.

I agree that this unique ID is not as bad as the issue's author claimed it to be. However, I also think that OsmAnd should not be designed in a way that could theoretically enable behavioural tracking.

Would simply re-generating the ID at regular intervals really be too difficult? In my opinion, it would be such an easy way to improve everyone's privacy and to calm anyone who's worried.

[mnalis]

Please stop suggesting that the UUID is somehow private or anonymous. With client IP geolocation it can identify users by travel history. It is PII.

@sneak Yes, UUID is PII. But so is IP address itself according to GDPR. So even if there were no UUID at all, PII would still be transmitted and stored by OsmAnd servers. Thanks for pointing to this issue, but I think it is now time to stop beating a dead horse (we get it: current OsmAnd situation is not perfect, but they seem to recognize the problem and look willing to improve on it, so let's try to not paint them as villains worse than Google or Microsoft - for the time being, eh?) and concentrate on how to best address this issue - in an actionable way that is acceptable both to privacy-oriented users and OsmAnd server admins, while also following GDPR and other applicable laws.

My suggestions:

• randomize said UUID often - e.g. as suggested by @scaidermern in No setting to disable: download sends UUID & number of days since install OsmAnd-iOS#2115 (comment)
• change privacy policy at https://osmand.net/help-online/privacy-policy/#3-the-information-company-collects to be both correct and up-to-date, and with an explanation why something is being done (e.g. something akin to ... when downloading maps from within OsmAnd, your IP address and randomly generated unique identifier are transmitted to OsmAnd servers in order to facilitate download and prevent server load issues & abusive behaviour, as well as generate map popularity statistics. That unique identifier is regenerated every XXXXX days/hours/xxxx. If you download maps manually outside of OsmAnd, the unique identifier will never be sent to OsmAnd servers nor elsewhere...). While updating it, please also take notice of editing plugin and its set of additional privacy issues. Nice example of IMHO clear and useful privacy policy is at https://github.com/streetcomplete/StreetComplete/blob/v45.2/app/src/main/res/values/strings.xml#L314-L336 ; one may want to pick ideas/text from there.
• also, what does Settings / Analytics / Maps downloaded change when turned off? I mean, by the nature of transferring the map the OsmAnd servers would know what maps are downloaded from the app, regardless of this flag. Does this also share the information about maps downloaded outside of OsmAnd? Or does it collect, but then not count/process information about maps downloaded from users with this turned off? I.e. what is exactly the difference between on and off there?
• actually link to said privacy policy from within OsmAnd app in Help menu. At least I can't seem to find it there.

[aschrijver]

Just a showerthought with my morning coffee in hand, as I bumped into this issue, but couldn't you hash the IP + UUID in similar way passwords are hashed? That way at least what's in the server logs would be useless to malicious hackers should they manage to obtain it.

[mtvec]

Since the client needs to be trusted anyway for this whole scheme to work, how about having the client itself keep track of the number of downloads? It could then either send this number to the server, which would use it to decide whether or not to allow the download, or just refuse to initiate the download itself.

[saddy001]

actual anti-abuse measures implemented in the server

Can you elaborate a little on how that might be accomplished?

[vshcherb]

Major actionable points:

1. UUID will be regenerated (every 2-3 months)
2. Make sure logs are truncated / deleted, so only summary left
3. Privacy policy will be updated and more descriptively about (Fair Use of OsmAnd Online Services and UUID)

Major information point:

1. Providing UUID is part of Terms of Usage of OsmAnd Online Services such as downloading maps. UUID or account id for OsmAnd PRO will always be part of request
2. OsmAnd Online Services are not unique way to use OsmAnd Client application. Maps could be downloaded or generated manually. So there is no issue that OsmAnd Client hiddenly transmits private data
3. There is no any indication / facts / points provided that UUID + Downloaded Map can somehow track user or even more identify real person. Identification is a clear procedure that UUID will be connected to real data such as Name / Surname / Living Address / Working Address / Photo. If IP is corrupted and doesn't provide enough privacy then issue is not related to UUID.

---

That's my basic understanding for now. Please leave comments in the nearest future if I'm missing something and that issue will be frozen to not create any further never ending discussions!

[mtvec]

@vshcherb As far as I can tell, using a UUID does not provide any benefit for OsmAnd compared to the approach I mentioned above (simply storing a counter in the client). On the contrary, using a UUID seems to make things more difficult (regeneration, clearing logs, updating the privacy policy). It also makes tracking users at least theoretically more easy.

Could you maybe clarify why this approach is not being considered? Am I missing something?

[vshcherb]

Except that we couldn't count of how many unique users per month / quarter were active, exactly why uid was introduced to give a sum by day / month.

[mtvec]

I'm sorry for my ignorance but why do you need this information? I wonder how useful it is because, for example, I'm a very active user but almost never update my maps so I won't show up in these statistics.

If you really want to know how many unique users download maps, couldn't the number of unique IP addresses be used as a decent approximation? Not that I necessarily like the idea of IP addresses being stored but at least this is information the servers get anyway.

[mnalis]

Just a quick additional information:
If OsmAnd B.V. or their users are situated in European Union (and they are), they will need to follow the GDPR.

• @sneak note that user consent is only one of six allowable ways for obtaining legal basis for data processing
• the OsmAnd B.V. might decide to use some other legal basis for data processing. In any case, they should consult a GDPR specialist and spell out in their Privacy Policy exactly what Personally Identifiable Information (like UUID or username, or IP addresses) is being collected, and specify in advance all purposes for which such data will be used, as well as other things (will that data be shared with third parties, stored/processed outside of EU etc.). If they later want to use PII for another purpose, they must re-obtain additional consent (or other legal basis) - they're not allowed to reuse previously collected data for new purpose retroactively. There are many things that need to be done correctly, which is why they should consult a GDPR specialist (disclaimer: I am not one so it is not an ad for myself; but I had to go thru the same pains, so I am informed above average in the matter). There is a checklist they might want to go through to verify if they did everything they need to.
• While having UUID might be required for some uses (e.g. all registered users), I would suggest seriously considering options that require it as little as possible, especially for "anonymous" users. Any data collected has a chance to leak / be misused, so not collecting it in the first place would be smarter than having to invest resources and money to deal with protecting it adequately, and dealing with a possible breaches (and GDPR fines) later. Also, OSM crowd in particular are often much more biased to be privacy concerned (or they'd likely be using Google Maps in the first place) so it would be good business idea to try not to alienate them unnecessarily.

[mnalis]

This misunderstands what identification means.
It's irrelevant that you don't know the user's name; you have your own name for them - a UUID.

@sneak is correct here. See for example this Law SE question "Are auto-generated identifiers PII if they cannot be linked to an identity?"

[realpixelcode]

Especially noteworthy (regarding the Law SE question):

Key observations:

• Singling out a data subject already counts as identification, meaning that it isn't necessary to infer their real-world identity!

[...]

[vshcherb]

I agree that I need a proper legal advise and in the end consent or any other legally correct form of user getting informed.

Completely another topic whether Map downloads Service could be provided without identifying how many users and potentially prevent out of service issues.

In the end that needs to be decided by OsmAnd BV, so I'm not going to take decision on my onw, cause it has legal and business implications.

Note: the issue will be frozen to stop message flooding and updates will be sent to it to get everybody informed.

[vshcherb]

Summary

The issue has bee fixed in Android / iOS and already available in nightly builds and will be released in 4.3. What's been done by sorted by priority:

1. It's possible to disable that UUID sending in Privacy Global Settings as the result the traffic could be deprioritized when lots of maps are downloaded i.e. in the beginning of the month.
2. UUID is rotated every 3 months so there is no user profile built for a longer history
3. ToS with explanation has been updated https://osmand.net/help-online/terms-of-use/
4. There is a clear indication On first screen after installation and a link to ToS.

As a result I'm closing the issue to make it completed. Thanks for reporting it so we could resolve that subject.

[realpixelcode]

Great! Thanks for this improvement, highly appreciated! 👍

[nemobis]

Il 18/10/22 03:23, Jeffrey Paul ha scritto:

You simply can't send any sort of tracking identifier for a user unless the user consents to it

This is clearly false: for instance no specific explicit consent is needed for the transmission of the IP address that comes along with every TCP/IP request, when it's necessary to perform the action requested by the user. Otherwise you'd end up in a catch 22 situation where you need previous consent to allow the user to download the privacy policy where they'll learn how to provide consent.

[OddSobriquet]

when it's necessary to perform the action requested by the user

If you are able to opt out and the outcome of the request does not change compared to if you opted-in, it isn't necessary to perform said action.

From how I understood the use of the uuid it is a non-essential ID this is exactly how it works and it means it is not technically required to provide the service which does indeed mean, it needs to be opt in, not opt out.

[mnalis]

If you are able to opt out and the outcome of the request does not change compared to if you opted-in, it isn't necessary to perform said action.

As I understood the Summary above, the outcome does change. In particular, your download request (in times of heightened load) might be slowed down significantly (or maybe even delayed for some time or completely refused until retried later when load is lower).

As s side note, there are other privacy-related options that can be enabled/disabled at that screen, that you might want to check, as well other related system-wide options (i.e. Google's "Improve Location Accuracy" setting, enabled by default on Android phones, leaks hugely bigger amount of privacy and location data, and has much worse privacy record than OsmAnd. Not to mention that if you used Google Play app store to download any of the apps, you are also leaking more privacy data daily than this UUID setting would if you left it in "on" position forever).

Not to say this isn't important, but before starting worrying about default setting of this UUID setting, one at minimum should be running de-googlified ROM on their phone and using exclusively some privacy oriented app-store dedicated to providing non-proprietary free software only instead of Google one (or Apple's, but there is no way to avoid using IOS on iPhone devices AFAIK, so you are out of luck there in any case). Some people do, and thus they might want to suggest (to their app store!) to change that default (for that app store users), yes.

For Google Play and (Apple store) crowd who (willingly or unknowingly, regardless) continues to surrender their privacy daily anyway, I do not see much point in changing the OsmAnd UUID default, as it only would result in their OsmAnd experience getting worse, with no noticeable privacy improvement at all.

[s38b35M5]

I think it's unfortunate to see the completely appropriate and respectful comments by @sneak that shed light on the importance of this issue are marked as abuse.

As this issue is linked directly to the anti-features section on F-Droid and other repos, its at least poor optics.