iOS - HTTP mode - loses connection without error - fiddly fix

[TonyFeestneus]

I have been using OwnTracks for a few years now and it is a great app. I use it to publish my location via http to a home server to track my whereabouts. Ever since I started using the app, every few weeks the connection gets lost. I can fix it by changing the HTTP URL to a faulty one, publish my location, get errors, put back the same URL as there has always been, publish my location et voila: fixed.

This has become a nuissance. Is this a known problem?

[ckrey]

We heard of similar occasional problems using MQTT. Both MQTT and HTTP share the same network API NSURLSession, so the problem might be related.

Do you use plain HTTP or HTTPS? If HTTPS, which kind of certificates do you use?

[TonyFeestneus]

HTTPS, the server I uses, uses Let's Encrypt certificates. Do you think it chokes on there? I do not think so.

It is always one or the other phone which stops working, and then the other just keeps on working without needing to interfere.

[jpmens]

I doubt it's the certificates.

You say home server, is this actually a server in your home which you address via a dynamic DNS entry, and if so, would it be possible for you to use a different provider?

[TonyFeestneus]

Hi, Yes it is via a DDNS entry (ddns.net). You'd think another one won't have the issue? Why is the issue on one phone only and not on both?

[jpmens]

I missed the part with "the other phone".

[TonyFeestneus]

np, but what could be the issue then. And why does it not throw errors when it is clearly not getting a correct status from the server?

Can I do anything on the server end to return a better status for each access?

[jpmens]

Have you been able to determine, when one of the phones stops publishing, whether there really is nothing arriving at the server? (HTTP server logs)?

Also is there any way you could try and track whether, when a phone stops publishing, whether a change in the DDNS name has ocurred? (I am assuming your IP changes occasionally - ignore if that's not the case.)

Also, you write I can fix it by changing the HTTP URL to a faulty one. Have you attempted swiping the app out when publishing is not possible, and restarting? Does that solve the issue?

And finally, for now, please state iPhone model, iOS version, and we'll assume lastest OwnTracks version. :-)

[TonyFeestneus]

• Yep, determined through the server logs (and making the recipient server send me a mail upon every access and receiving nothing)
• My IP does not change occasionally, only when the whole house power is lost. Once in a couple of years, has happened only thrice before so thought the DDNS could come in handy but is actually not necessary
• Yep, have closed the app, really closed it, restart phone etc, only tries to gain contact when I change the URL. And when I change it back it works
• iPhone 12 mini, also iPhone SE, iOs 13-14-15 and latest OT version (now 14.2.2)

Thanks :)

[ckrey]

We have setup a "home server" with dynamic DNS here (also via ddns.net)
We cannot reproduce the problem even when force changing the IPv4 address.
We do get meaningful error messages on the status screen of the app.

When the problem occurs, what does the status screen show?

If it encounters a problem, it will display the error until it tries to reconnect. The reconnect happens after 1,2,4,8,16,... seconds.

[TonyFeestneus]

I will update when the next time occurs. In the status screen no errors will show, it just shows "inactive". Even after force publishing the location.

Will be continued.

Are you sure no one has encountered this before? I have had it happen numerous times

[TonyFeestneus]

Now I get an error on both devices, unrelated or different to the problems I encountered before, but since I finally see an error code I thought to chime in.

The error I get now is error HTTP response 500. If I go to the browser, I can perfectly visit the url I have set. This happens since last night on both devices, no certificates or anything has changed in the meantime.

[jpmens]

HTTP status codd 500 is being caused server-side, and you'll have to see in your server logs what the issue is. Maybe some PHP upgrade or other configuration change which might be causing this?

[ckrey]

The app POSTs messages to the URL. A browser typically uses the GET method.

[TonyFeestneus]

So after trying a faulty URL and switching it back, I received the request at my server (I have printed out the json request here), but it keeps saying "connecting" until finally getting a timeout 1001 error. All the while it publishes location like below a couple times.

batt: 52,
lon: changedforprivacy,
acc: 1414,
bs: 1,
p: 100.765,
vac: 141,
lat: changedforprivacy,
topic: 'owntracks/user/C1EA41E5-71DE-4changed',
conn: 'm',
tst: 1641568095,
alt: 10,
_type: 'location',
tid: 'me' }

So I can confirm server does work and receive the request (again, nothing changed between here and last night).

[jpmens]

That's not JSON ...

Be that as it may, I don't think we've yet asked you what softare your OwnTracks devices are actually speaking to. Can you please tell us?

[TonyFeestneus]

Raspberry Pi with Espress server

[jpmens]

Do you mean Express (with an 'x'), so a home-grown program based on NodeJS?

If so, you're going to have to dig into whatever that code is doing; the HTTP code 500 originates from that. While it might be possible that the server is borking because of some strange data we're giving it, the culprit for the code 500 is on the server side.

[TonyFeestneus]

Yes Express sorry. Did you also read I am not getting the 500 error anymore? And do receive the incoming request?

[TonyFeestneus]

The thing is, making a POST request from another source does not resolve in a 500 error. The server is up and running and receiving requests.

So what HAS changed?

[jpmens]

Lead us through what your Express code does. Which JSON elements it’s expecting in the OwnTracks payload and what it does with the data.

[jpmens]

Another thing you can do to test is use curl to POST a minimal OwnTracks payload to your server. The following, run on your Raspi should do the trick (pppp is the TCP port number your Express server is listening on):

$ curl -v -d ‘{"_type":"location","lat":11,"lon":22}’ http://localhost:pppp

It’d help if you could show us the output of that, please.

[TonyFeestneus]

* Rebuilt URL to: ‘lat:11’/
* Failed to convert ‘lat to ACE; string contains a disallowed character
* Could not resolve host: ‘lat
* Closing connection 0
curl: (6) Could not resolve host: ‘lat
* Rebuilt URL to: ‘lon:22’/
* Failed to convert ‘lon to ACE; string contains a disallowed character
* Could not resolve host: ‘lon
* Closing connection 1
curl: (6) Could not resolve host: ‘lon
* Rebuilt URL to: http://localhost:8080/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#2)
> POST / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Length: 20
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 20 out of 20 bytes
< HTTP/1.1 302 Found
< X-Powered-By: Express
< Location: https://localhost/
< Vary: Accept
< Content-Type: text/plain; charset=utf-8
< Content-Length: 40
< Date: Fri, 07 Jan 2022 19:04:40 GMT
< Connection: keep-alive
< 
* Curl_http_done: called premature == 0
* Connection #2 to host localhost left intact

[TonyFeestneus]

I was originally using the php endpoint to send a mqtt message locally using, yes, shell_exec. I found out that it i stop using shell_exec, OwnTracks manages to connect without error 500.

So stripped the php file, and am now listening with the Express js file to incoming requests, verifying, and then handling the rest. The php file where OwnTracks posts to, is just empty. It works, but owntracks keeps coming up with timeouts 1001 even though it now works.

How? Why?

[ckrey]

OwnTracks is watiting for a HTTP response 200.
Otherwise there will be a timeout.

[jpmens]

I don’t think that curl command was correctly issued in the shell; please pay close attention to the distinct quotes.

Regarding your “how”s and “why”s, that’s a good question. I think you are assuming we know what the code you are running looks like, but we do not. I for one do not understand the combination of PHP and Express …

So you accepted the HTTP post in PHP and then ran a shell command to produce MQTT? Why?

Then you say the php file is “stripped” and Express.js is listening to HTTP. What then, does the php file do?

If Express.js is handling your HTTP POST requests, please ensure it responds with an HTTP 200 as @ckrey points out.

[TonyFeestneus]

@ckrey: I was giving it a 200 response through php but tried it via express server settings now and that fixes the problem!!

@jpmens: at first, I accepted the HTTPS POST in PHP, did some calculations with the data and had it send a MQTT message locally. Why? Because I did not know of any other way to get a MQTT message locally upon outside request.
I have changed my code to an empty PHP file and just detect the request of the file through the Express server (in runtime), I can fetch the same data and calculate with it. Then have it send the local mqtt message.

Seems to work fine now, thanks again guys. Although it does not change the fact it is very strange for non-changed code to stop working overnight, and requests to be succesfully made for months in a row and then be stuck. Perhaps it will keep on working now.

[jpmens]

Glad you got it going.