After update to LOS 16, no quests are loaded via WiFi, only sometimes it works via mobile data

[Moini]

• Start app in flight mode, with WiFi on
• Select to search for quests
• See no red bar, but get notification that it looks for new quests
• After some time, get a notification that quests couldn't be loaded

It sometimes works via mobile data, but of course takes a lot longer. it often stops after about 20% of the red bar.

The maps are updated normally via WiFi.

Versions affected
Lineage OS 16 (build from July 1st), StreetComplete 12.1 from fdroid

Not affected
Lineage OS 15 (it worked before I upgraded the OS on my phone)

[matkoniecz]

Is it possible that you or someone else is using Overpass API on the same Wifi network?

I noticed exactly the same behavior with multiple phones running scans for quests as the same time - it is caused by rate limiting of Overpass allowing only one query at time from a single IP.

It may explain why mobile data scan works (as it is using a separate IP).

What is displayed at http://overpass-api.de/api/status during failing scans (use the same WIFI network as phone)?

Is http://overpass-api.de/api/status page accessible from your phone?

Are you maybe on a trip to Russia? (see #1389)

[Moini]

Hi, and thanks for replying!

The very first time I loaded that web page after starting the search in StreetComplete, it showed 1 currently running query (and a growing red bar). Then it stopped growing, and told me that slots were available in 47 and 57 seconds (or so).

I'm not in Russia, and there's no other device in my network that looks at openstreetmap, or anything related (as far as I know...). The only running app that is related to OSM is this one (I hope).

When the phone isn't looking for new quests, there is no request registered on that page.

[matkoniecz]

I admit that I have no idea what may be going wrong here, things that I considered would block quest downloads only sometimes or completely block overpass.

Maybe @westnordost will have an idea what is going wrong.

If you would be interested in doing that - capturing logs is very likely to help.

https://wiki.lineageos.org/logcat.html has an explanation how it can be done

"With a computer" method can be also done by running Android Studio and looking at logcat tab.

This may be quite time-consuming to setup this things if you have neither adb nor Android Studio, nor a rooted phone. But in case that you have one of this things then it is quite likely that logs will help to reveal what is going wrong.

BTW, look at logs before posting them in a public - it is possible that some other apps will log some private data (StreetComplete logs will for example mention location where scan is running).

[Moini]

No problem, I have adb installed - I need it for backups, file transfer and for installing the OS.

What I did:

• Wake up phone
• StreetComplete was still open
• center on current location
• select to search for quests here
• after 2 sec, get info that it didn't work
• retry
• nothing happens
• retry
• nothing happens

What took longest was finding where the relevant part starts and redacting any addresses that are unrelated to this. Unfortunately, my nextcloud seems to have dirtied the log a bit, let me know if it's still possible to use the attached log.

[log removed, see linked gist below]

(maybe the next version of streetcomplete could write its own log files which are accessible to the user? Some other apps I have do that.)

[matkoniecz]

Thanks!

Note that in line 347, 587 of published log and maybe other scan location is logged - not sure whatever you consider it as private or not (log may be useful so consider redacting just location if you prefer to not publish it).

Are there any problems with WIFI visible in other applications? Have you tried on other WIFI network?

Can you open http://overpass-api.de/api/interpreter?data=[out%3Ajson][timeout%3A25]%3Bnode(1)%3Bout%3B web page from your phone?

It should display page with

{
  "version": 0.6,
  "generator": "Overpass API 0.7.55.7 8b86ff77",
  "osm3s": {
    "timestamp_osm_base": "2019-07-04T14:05:01Z",
    "copyright": "The data included in this document is from www.openstreetmap.org. The data is made available under ODbL."
  },
  "elements": [

{
  "type": "node",
  "id": 1,
  "lat": 51.4779481,
  "lon": -0.0014863
}

  ]
}

---

beyond this line is a wild handwaving, with some chance of a productive handwaving

Maybe app is buggy in some way, maybe LOS 16 has WIFI regression (?)

Or maybe there are hardware issues on a specific phone/router that appeared now? (I have weird connection issues on phone that I now use for app testing - though focused on receiving calls, that were almost certainly caused by hardware problems)

From quick search I found only https://gitlab.com/LineageOS/issues/android/issues/371 with summary in https://gitlab.com/LineageOS/issues/android/issues/371#note_181502938 (but is should not be SC specific).

in log relevant part appears to be

07-04 15:24:13.813 21314 23967 E QuestDownload: de.westnordost.osmapi.common.errors.OsmConnectionException: java.net.ConnectException: Failed to connect to overpass-api.de/2a01:4f8:110:502c::2:443

and later

07-04 15:25:07.654 21314 23967 E QuestDownload: de.westnordost.osmapi.common.errors.OsmConnectionException: java.net.SocketTimeoutException: timeout

so it has problems to connect for some reason but I was unable to spot it what may cause it

with full available stack and more in https://gist.github.com/matkoniecz/24e782dd304081f170910043b6f77495 but redacted (full log may be still useful for @westnordost - maybe reason for this problem can be spotted)

[westnordost]

I can not add anything more to this than @matkoniecz already noted. The app can simply not connect to the overpass server.
If this persists, it sounds like this is rather an issue of LineageOS. This app just uses the basic Java HttpURLConnection class to make HTTP requests.

[Moini]

Okay, hmm... too bad, I'd have preferred this to be a more limited issue :-(

Thank you very much, @westnordost and @matkoniecz.

When I click the link, the json reply looks just like the one you get, only a different location.

When I reload it and also retry getting new quests in StreetComplete, I hit the rate limit and get corresponding info from the api.

So, uhm. There are other apps that have problems connecting, and are losing connection, and then stop (up)loading files randomly, namely nextcloud, and I also had this once with AntennaPod yesterday.

It works well for rocket.chat, fennec, and fdroid works, too.

StreetComplete is the only app where it doesn't work at all, although the amount of data to be transferred is tiny, I suppose. Maybe due to the unfortunate combo between the rate limit and the dropping connection.

Is there a way to delay or retry connecting when it fails? To detect if connection is available before trying to connect? A message to show to the user that is more specific? A longer timeout setting?

(there are many apps where I would love to be able to set the timeout..., as they might then even work with mobile data, with just basic bandwidth of a few kb/s)

(the location is semi-private, anyone can see my address on my website - 'Impressumspflicht'... - but thanks a lot for cleaning, I will remove the link to the attachment from my own post above)

Thank you again!

[matkoniecz]

Is there a way to delay or retry connecting when it fails? To detect if connection is available before trying to connect?

It is already doing this

[westnordost]

Is there a way to delay or retry connecting when it fails? To detect if connection is available before trying to connect? A message to show to the user that is more specific? A longer timeout setting?

FYI, the default timeout is currently set to 45s. This might be a problem on its own, because I recently found out that download of certain quest types can take quite long.

[matkoniecz]

So that is why #1010 had unexplained timeouts after 45 seconds!

I am making PR that will increase timeouts to default 180 seconds (query running on Overpass server blocks any further calls from the same IP so it should not harm anyone and should help with expensive queries and flaky connection).

[matkoniecz]

I was in a place with a poor internet (connection via long-distance wireless then via satellite) and I was able to both

• reproduce this issue
• confirm that stop timeouting after 45 seconds [ready for review] #1472 fixed it, at least in case where it was caused by Internet working but with unusually high lag

[Moini]

Cool :) Thank you for doing so extensive testing, and for fixing it, too, @matkoniecz !