Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identify heavy app tile users by User-Agent #371

Closed
pnorman opened this issue Mar 3, 2020 · 13 comments
Closed

Identify heavy app tile users by User-Agent #371

pnorman opened this issue Mar 3, 2020 · 13 comments
Labels
service:tiles The raster map on tile.openstreetmap.org

Comments

@pnorman
Copy link
Collaborator

pnorman commented Mar 3, 2020

The OWG could use help identifying these high-volume User Agents. We can contact them if necessary, but could use help with the identifying.

I'll be editing this post to keep the information consolidated, and hiding messages as I integrate them.

High usage app User-Agents

Daily hits User-Agent Contact
2560964 GDAL WMS driver (http://www.gdal.org/frmt_wms.html)
2483953 ArcGIS Client Using WinInet
1836191 Maperitive/2.4.3
1146725 cgeo https://github.com/cgeo/cgeo/issues
684191 sunriseinnovations.WIS.beta https://www.sunriseinnovations.ie/
503447 Nemo Outdoor https://www.keysight.com/main/contactInformation.jspx?nid=-11143.0.00&lc=eng&cc=US
391522 ir.pakcharkh.bdood/19 https://cafebazaar.ir/app/ir.pakcharkh.bdood?l=en
379844 com.taxiadmins.client https://play.google.com/store/apps/details?id=com.taxiadmins.client
377245 uz.smartlab.itaxi.client https://play.google.com/store/apps/details?id=uz.smartlab.itaxi.client
371303 GeliosMobile/98 CFNetwork/1121.2.2 Darwin/19.3.0 https://www.geliossoft.com/contacts
350733 com.rsm.muoversiaroma https://play.google.com/store/apps/details?id=com.rsm.muoversiaroma
335876 staticmaps@1.1.4 (Node.js v12.14.1)
334280 com.qualteh.mobtaxi.android.client https://play.google.com/store/apps/details?id=com.qualteh.mobtaxi.android.client
317557 OruxMaps v.7.5.8 GP https://play.google.com/store/apps/details?id=com.orux.oruxmapsDonate
310438 com.helleniccomms.makedonia.driver https://play.google.com/store/apps/details?id=com.helleniccomms.makedonia.driver
300515 in.kipo https://play.google.com/store/apps/details?id=in.kipo
290211 it.Beta80Group.whereareu https://play.google.com/store/apps/details?id=it.Beta80Group.whereareu
287639 1.71.7
281412 OfflineMaps/3.4 (om86; ar; +https://www.offline-maps.net) https://play.google.com/store/apps/details?id=net.psyberia.offlinemaps
277342 Other
272811 GeoTiler/0.14.3
266006 ArcGISRuntime-Android/100.2.1 (Android 5.1; armeabi-v7a; SAMSUNG-SM-T365)
264563 RT3000host
261361 com.tiskel.terminal
259940 OruxMaps v.7.4.22 https://play.google.com/store/apps/details?id=com.orux.oruxmapsDonate
254269 de.schildbach.oeffi https://schildbach.de/
244848 com.helleniccomms.glyfada.driver
231130 Nokia (Qt) Graphics Dojo 1.0
224615 ArcGIS.JavaSE.10.1.1
224314 at.tbirn.lfkapp/21230501
216037 com.caynax.sportstracker/20601
214502 com.mivselltech.mobilesales2
206937 com.taxi505.driver/135
204403 ru.taxomet.tadriver
197363 com.softlogics.client

Contacted

Daily hits User-Agent Contact Notes
870173 pl.itaxi.driver https://itaxi.pl/english/
576751 net.uztaxi.driver https://play.google.com/store/apps/details?id=net.uztaxi.driver
498789 OSMDroid/2.1 (its; rutaxi 3.28.0) https://play.google.com/store/apps/details?id=com.its.rto
482874 com.helleniccomms.mercedes.driver https://play.google.com/store/apps/details?id=com.helleniccomms.mercedes.driver
469157 ru.taximaster.www https://play.google.com/store/apps/details?id=ru.taximaster.www
453734 com.arobs.trackgps https://play.google.com/store/apps/details?id=com.arobs.trackgps
409986 com.helleniccomms.asteras.driver https://play.google.com/store/apps/details?id=com.helleniccomms.asteras.driver

Newly blocked UAs

Daily hits User-Agent Contact
1784800 Where my children/2.0.9 (iPhone; iOS 13.3.1; Scale/2.00)
1730899 Where my children/2.0.9 (iPhone; iOS 13.3.1; Scale/3.00)
1530710 nossoonibusjp.android.crosswalk https://www.jampabus.com.br/
711132 br.com.concisoti.potybus https://play.google.com/store/apps/details?id=br.com.concisoti.potybus&hl=en_US
784944 com.soft373.taptaxi https://taptaxi.su/
770529 com.kradac.ktxcore https://kradac.com/
709664 ru.crowdsystems.topcontrol.knd https://crowdsystems.ru/en
700755 C# TilesDownloader
688465 okhttp/4.2.2
543909 kc_android
507506 Android
497577 MapDownloader
486335 /n software IPWorks HTTP/S Component - www.nsoftware.com https://www.nsoftware.com/company/contact.aspx
324496 Where my children/2.0.9 (iPhone; iOS 13.3; Scale/2.00)
285467 Where my children/2.0.9 (iPhone; iOS 13.3; Scale/3.00)
241033 Where my children/2.0.9 (iPhone; iOS 12.4.5; Scale/2.00)
221438 okhttp/3.12.2
213015 Microsoft-ATL-Native/12.00

Total requests without referer: 529910207

@scaidermern

This comment has been minimized.

@dorotheakazazi
Copy link

com.helleniccomms.mercedes.driver
com.helleniccomms.makedonia.driver
com.helleniccomms.asteras.driver
..
com.helleniccomms.mercedes.driver is the Android app TaxiClub Driver https://play.google.com/store/apps/details?id=com.helleniccomms.mercedes.driver, contact: helleniccomms@gmail.com
..
com.helleniccomms.asteras.driver is the Android app ΡΑΔΙΟΤΑΞΙ ΑΣΤΕΡΑΣ https://play.google.com/store/apps/details?id=com.helleniccomms.asteras.driver, contact: support@asteras1.gr

@pnorman anything hellenic* is Greek-related, so if you want a Greek translation of the text when contacting them, let me know.

@scaidermern
Copy link

scaidermern commented Mar 6, 2020

/n software IPWorks HTTP/S Component - www.nsoftware.com is obviously https://www.nsoftware.com/company/contact.aspx although I'm not sure where this software uses a map.

ir.pakcharkh.bdood/19 is the Android app بیدود (BDOOD) https://cafebazaar.ir/app/ir.pakcharkh.bdood?l=en

GeliosMobile/98 CFNetwork/1121.2.2 Darwin/19.3.0 is the Android/iOS app GeliosMobile by GeliosSoft https://www.geliossoft.com/contacts, screenshots with OSM-based maps can be found at https://apps.apple.com/th/app/geliosmobile/id999837601#?platform=ipad

OfflineMaps/3.4 (om86; ar; +https://www.offline-maps.net) is the Android app All-In-One Offline Maps https://play.google.com/store/apps/details?id=net.psyberia.offlinemaps

GeoTiler/0.14.3 are users of the GeoTiler library https://github.com/wrobell/geotiler

com.tiskel.terminal is a taxi management app by Tiskel https://www.tiskel.com/produkty?lang=en

com.helleniccomms.glyfada.driver is the Android app Radiotaxi Glyfadas Driver https://play.google.com/store/apps/details?id=com.helleniccomms.glyfada.driver

@tastrax
Copy link

tastrax commented Mar 11, 2020

I suspect that ArcGIS.JavaSE.10.1.1 is ArcGIS Runtime SDK for Android development

https://topic.alibabacloud.com/a/an-overview-of-the-arcgis-runtime-sdk-for-android-development-note-arcgis-runtime-sdk_1_12_30779938.html

@tastrax
Copy link

tastrax commented Mar 11, 2020

Maperitive/2.4.3 is likely http://maperitive.net/docs/

@Lineflyer
Copy link

Hello openstreetmap-team,

we (the c:geo team) were made aware about this issue recently.
c:geo is an opensource free geocaching app for Android.

Some months ago we changed the c:geo app in a way that it provides a dedicated user-agent (instead of some default set by the mapsforge library) when requesting tiles to allow identification for your team. See this posting and the corresponding issue for details. AFAICS there has also been some contact to your team at this point of time.

From your table above I do now see, that we are under the top 5 of tile requests (as far as I understand your table), which I never would have expected...
While we are just a small opensource team, without any commercial background, I would still like to ask, how you evaluate the situation of OSM tile usage by c:geo, and whether we can do anything to improve the load situation on your side or any other measure we can take to help you? Ideas and constructive discussion is always welcome.

@pnorman
Copy link
Collaborator Author

pnorman commented Mar 12, 2020

c:geo is identifying itself with a User-Agent, is open-source and a related activity, mentions OSM when you select the OpenStreetMap Carto basemap, and isn't using it as a default, so we're not looking at blocking it at this time. As our needs and load changes, this could change but we would attempt to contact you if it wasn't urgent.

Could you change the "OSM: Map" to "OpenStreetMap Standard" or something that has the full name?

From your table above I do now see, that we are under the top 5 of tile requests (as far as I understand your table), which I never would have expected...

This is surprising to me too. I'll open up an issue on your tracker with some ideas.

@Firefishy
Copy link
Member

Offline mass down of tiles is not permitted, even by Open Source apps.

@pnorman
Copy link
Collaborator Author

pnorman commented Mar 22, 2020

Thanks everyone for helping identify the big users. We've been able to identify the big users and reach out to them. The remaining uses are into the long tail of small users, and if they stop being small users, we've got this to look back on and get their contact information.

I'm going to close this, noting that there's about half a dozen apps where we're giving them time to respond before blocking.

@moving-bits
Copy link

moving-bits commented Apr 10, 2020

@pnorman
Beside other means of reducing the load, like increased usage of offline map files, is there a way to make the MapsForge lib use a different tile server (eg cyclosm.openstreetmap.fr) to spread the load?

sorry, wrong repo - will redirect to mapsforge repo

@hbogner
Copy link

hbogner commented May 5, 2020

Maperitive/2.4.3 is likely http://maperitive.net/docs/

Yes it is. I used it few years ago.

@Symbian9
Copy link

Maperitive/2.4.3 is likely http://maperitive.net/docs/

Yes it is.

No, http://maperitive.net/docs/ is Maperitive's docs pages.

Contact page (mail list) is here:

@hbogner
Copy link

hbogner commented May 20, 2020

Or contact the author directly: https://twitter.com/breki74

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
service:tiles The raster map on tile.openstreetmap.org
Projects
None yet
Development

No branches or pull requests

9 participants