IOS Immich mobile app fails to connect over 4G via Tailscale (works on WiFi and in browser)

[AlexT59]

The bug

Hello,

First of all, thank you for the great work on Immich!

Context: I’m using Tailscale to access my home server remotely, with Caddy as a reverse proxy to manage access to various applications running on my NAS.

Issue: When I attempt to connect to Immich via the mobile app over a 4G connection, the connection fails immediately during the server ping. This issue does not occur when my phone is connected via WiFi (whether at home or on another remote WiFi network).

Interestingly, when accessing Immich through a web browser on the same mobile device, it works correctly over both 4G and WiFi.

In all cases, Tailscale was active.

The OS that Immich Server is running on

TrueNAS-SCALE-24.10-RC.2

Version of Immich Server

v1.118.2

Version of Immich Mobile App

v1.118.0

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

N/A, was installed using Immich App of TrueNas

Your .env content

N/A

Reproduction steps

1. Enable Tailscale on the NAS and mobile device.
2. Connect to Immich using the mobile app while on WiFi. The connection is successful.
3. Switch the mobile connection to 4G with Tailscale still enabled.
4. Attempt to connect to Immich through the mobile app over 4G. The connection fails.

Relevant log output

Tailscale (when it works on WiFi)

2024/10/27 09:54:59 magicsock: adding connection to derp-8 for [+Hov1]
2024/10/27 09:54:59 magicsock: 2 active derp conns: derp-8=cr0s,wr0s derp-18=cr6h52m0s,wr2m0s
2024/10/27 09:54:59 derphttp.Client.Recv: connecting to derp-8 (lhr)
2024/10/27 09:54:59 magicsock: disco: node [+Hov1] d:242883f05df49ead now using 192.168.1.187:41641 mtu=1360 tx=6d2b653aa6f2
2024/10/27 09:54:59 magicsock: derp-8 connected; connGen=1

Caddy (when it works on WiFi)

{"level":"debug","ts":1730022899.6709793,"logger":"events","msg":"event","name":"tls_get_certificate","id":"3dcf55de-b6f3-45dc-a278-81b2241ff703","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4866,4867,49195,49199,49196,49200,52393,52392,49161,49171,49162,49172,156,157,47,53],"ServerName":"photo.ts.nasalex.fr","SupportedCurves":[29,23,24],"SupportedPoints":"AA==","SignatureSchemes":[1027,2052,1025,1283,2053,1281,2054,1537,513],"SupportedProtos":null,"SupportedVersions":[772,771],"RemoteAddr":{"IP":"127.0.0.1","Port":51376,"Zone":""},"LocalAddr":{"IP":"127.0.0.1","Port":443,"Zone":""}}}}
...
{"level":"debug","ts":1730022899.7635746,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.1.28:30041","duration":0.007401243,"request":{"remote_ip":"127.0.0.1","remote_port":"51402","client_ip":"127.0.0.1","proto":"HTTP/1.1","method":"GET","host":"photo.ts.nasalex.fr","uri":"/api/server/config","headers":{"User-Agent":["Dart/3.5 (dart:io)"],"Accept-Encoding":["gzip"],"X-Forwarded-For":["127.0.0.1"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["photo.ts.nasalex.fr"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","server_name":"photo.ts.nasalex.fr"}},"headers":{"X-Immich-Cid":["vu9gco6k"],"Content-Type":["application/json; charset=utf-8"],"Content-Length":["314"],"Etag":["\"13a-MAAz6zZgWF2/QWiMLiI3/FegTzw\""],"Date":["Sun, 27 Oct 2024 09:54:59 GMT"],"Connection":["keep-alive"],"Keep-Alive":["timeout=5"],"X-Powered-By":["Express"]},"status":200}

Tailscale (when it fails on 4G)

2024/10/27 09:55:52 CreateEndpoint error for 100.125.128.92:50387 -> 100.68.1.28:443: connection was refused

Caddy (when it fails on 4G)

{"level":"debug","ts":1730022952.4722066,"logger":"http.stdlib","msg":"http: TLS handshake error from 127.0.0.1:57476: EOF"}

Immich App (when it fails on 4G)

ApiException 400: Socket operation failed: GET /server/ping (Inner exception: ClientException with SocketException: Connection reset by peer (OS Error: Connection reset by peer, errno = 54), address = photo.ts.nasalex.fr, port = 50923, uri=https://photo.ts.nasalex.fr/api/server/ping)

#0      IOClient.send (package:http/src/io_client.dart:154)
<asynchronous suspension>
#1      BaseClient._sendUnstreamed (package:http/src/base_client.dart:93)
<asynchronous suspension>
#2      ApiClient.invokeAPI (package:openapi/api_client.dart:101)
<asynchronous suspension>
#3      ServerApi.pingServer (package:openapi/api/server_api.dart:492)
<asynchronous suspension>
#4      Future.timeout.<anonymous closure> (dart:async/future_impl.dart:963)
<asynchronous suspension>
#5      ApiService._isEndpointAvailable (package:immich_mobile/services/api.service.dart:106)
<asynchronous suspension>
#6      ApiService._resolveEndpoint (package:immich_mobile/services/api.service.dart:87)
<asynchronous suspension>
#7      ApiService.resolveAndSetEndpoint (package:immich_mobile/services/api.service.dart:69)
<asynchronous suspension>
#8      LoginForm.build.getServerLoginCredential (package:immich_mobile/widgets/forms/login/login_form.dart:104)
<asynchronous suspension>

Immich server

no logs

Additional information

• The issue only occurs with the Immich app on 4G, while other applications (e.g., Nextcloud) work correctly under the same network conditions on 4G.
• Access to Immich via a web browser over 4G with Tailscale enabled works without issues.

Please let me know if you need more informations or if I need to do other tests to help you

[bo0tzz]

This seems like an issue in your network configuration, possibly a firewall blocking things or such.

[AlexT59]

If it's a network configuration issue, how do you explain that I can access Immich using a web brower (Safari and Chrome) but not using the app on the same device, same config ?

[bo0tzz]

I don't use or know the nuances of tailscale etc, so I don't have an explanation for that, but the errors seem to pretty clearly be with the stuff inbetween and not with Immich (as it works fine on wifi, and doesn't have any knowledge about or interaction with network differences like this)

[AlexT59]

Just tested on an Adnroid device and the app successfully conect to the server.
I think it's related to my iPhone but I absolutly don't know where to look for that specific problem as Tailscale works with other apps like Nextcloud...