Error message connect: connection refused after some minutes running PhotoPrism

[adymorz]

After some minutes of running photoprism, it starts to show errors in the UI (either "no x found" or "connect: connection refused"). There are no error messages in the MariaDB log during that state. Connecting to the DB using the mysql CLI on the MariaDB docker container is still possible. The log of PhotoPrism is pasted below (using PHOTOPRISM_DEBUG set to true).

To get the system working again, I have to restart both containers (MariaDB and PhotoPrism).

I am using PhotoPrism® Plus Build 220901-f493607b0, connected to a MariaDB server 10.9.3-MariaDB-1:10.9.3+maria~ubu2204. Both run each in a Docker container on an Unraid system. The system has 32 GB memory in total, all storages used are far away from being full.

Docker is running in version 20.10.18. Unraid is on 6.11.1.

There is no swap space configured, because there is no built in option to do that in Unraid and I do not know how to savely enable it. I think it should not be required because of the installed 32 GB of memory.

The originals folder already contains 228G of image files. Because I filled in a lot of images in a short time I can not tell if the problems started at some point importing images.

I went though the troubleshooting guide but found no hint what could cause the connection errors.

Could you please give me some advice?
How can I debug even further?

1st log ("metadata: running face recognition" => "dial tcp 10.1.0.10:3306: connect: connection refused")

time="2022-10-19T19:20:39+02:00" level=info msg="webdav: /originals/ enabled, waiting for requests"
time="2022-10-19T19:20:39+02:00" level=info msg="webdav: /import/ enabled, waiting for requests"
time="2022-10-19T19:20:39+02:00" level=debug msg="server: successfully initialized [486.781µs]"
time="2022-10-19T19:20:39+02:00" level=info msg="server: listening at 0.0.0.0:2342"
time="2022-10-19T19:20:49+02:00" level=debug msg="server: GET /api/v1/ws (200) [8.141114413s]"
time="2022-10-19T19:20:49+02:00" level=debug msg="server: GET /people (200) [7.088669ms]"
time="2022-10-19T19:20:49+02:00" level=debug msg="server: GET /api/v1/subjects?count=120&offset=0&q=&hidden=&order=relevance&files=1&type=person (200) [1.399663ms]"
time="2022-10-19T19:20:50+02:00" level=debug msg="server: GET /api/v1/t/f523d85f0b0ca861286cdfe8ef9459e90f60ad24-0dc06c26e1d2/d616f0fb/tile_320 (200) [11.497021ms]"
time="2022-10-19T19:20:50+02:00" level=debug msg="server: GET /api/v1/t/9ceab484ea68a41e7b206813a0bc2df05771deb1-1280d01631d9/d616f0fb/tile_320 (200) [6.015354ms]"
time="2022-10-19T19:20:50+02:00" level=debug msg="server: GET /api/v1/t/1d98f88f28920b73b7745dbac83822c153a27880-11f0b820e15e/d616f0fb/tile_320 (200) [6.194959ms]"
time="2022-10-19T19:20:50+02:00" level=debug msg="server: GET /api/v1/t/d43a188218a3e6e6b254d728de9c8a8a11d12c6c-0a10861fb151/d616f0fb/tile_320 (200) [6.299375ms]"
time="2022-10-19T19:20:50+02:00" level=debug msg="server: GET /api/v1/t/c4fed90f700374f026c3edd3f51edfd13384f3e1-14d08309f0ef/d616f0fb/tile_320 (200) [6.275079ms]"
time="2022-10-19T19:20:50+02:00" level=debug msg="server: GET /api/v1/t/67f7b3c672f7a8ba07f0dae772f4d94dca31bb25-11609a19710f/d616f0fb/tile_320 (200) [2.729949ms]"
time="2022-10-19T19:20:50+02:00" level=debug msg="server: GET /api/v1/t/0fc7f99a337537d2dc74507ae75f0c8f36d8d734-0bd0d70c7109/d616f0fb/tile_320 (200) [5.470067ms]"
time="2022-10-19T19:30:49+02:00" level=debug msg="server: GET /api/v1/config (200) [14.64501ms]"
time="2022-10-19T19:35:39+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-19T19:35:39+02:00" level=warning msg="sync: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-19T19:35:39+02:00" level=error msg="faces: markers: dial tcp 10.1.0.10:3306: connect: connection refused (purge orphans) (remove orphan markers)"
time="2022-10-19T19:35:39+02:00" level=warning msg="share: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-19T19:35:39+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (fix references)"
time="2022-10-19T19:35:39+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (create subjects)"
time="2022-10-19T19:35:39+02:00" level=error msg="faces: dial tcp 10.1.0.10:3306: connect: connection refused (resolve ambiguous subjects)"
time="2022-10-19T19:35:39+02:00" level=warning msg="dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-19T19:35:39+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-19T19:35:39+02:00" level=warning msg="metadata: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-19T19:40:49+02:00" level=debug msg="server: GET /api/v1/config (200) [13.849417ms]"
time="2022-10-19T19:50:28+02:00" level=debug msg="server: GET /api/v1/ws (200) [29m38.504065995s]"
time="2022-10-19T19:50:28+02:00" level=debug msg="server: GET /people (200) [914.918µs]"
time="2022-10-19T19:50:28+02:00" level=debug msg="server: GET /api/v1/subjects?count=120&offset=0&q=&hidden=&order=relevance&files=1&type=person (400) [877.618µs]"
time="2022-10-19T19:50:28+02:00" level=debug msg="server: GET /static/icons/app/1024.png (200) [827.96µs]"
time="2022-10-19T19:50:39+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-19T19:50:39+02:00" level=warning msg="sync: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-19T19:50:39+02:00" level=error msg="faces: markers: dial tcp 10.1.0.10:3306: connect: connection refused (purge orphans) (remove orphan markers)"
time="2022-10-19T19:50:39+02:00" level=warning msg="share: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-19T19:50:39+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (fix references)"
time="2022-10-19T19:50:39+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (create subjects)"
time="2022-10-19T19:50:39+02:00" level=error msg="faces: dial tcp 10.1.0.10:3306: connect: connection refused (resolve ambiguous subjects)"
time="2022-10-19T19:50:39+02:00" level=warning msg="dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-19T19:50:39+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-19T19:50:39+02:00" level=warning msg="metadata: dial tcp 10.1.0.10:3306: connect: connection refused"
[GIN-debug] POST   /api/v1/batch/photos/delete --> github.com/photoprism/photoprism/internal/api.BatchPhotosDelete.func1 (4 handlers)
[GIN-debug] POST   /api/v1/batch/albums/delete --> github.com/photoprism/photoprism/internal/api.BatchAlbumsDelete.func1 (4 handlers)
[GIN-debug] POST   /api/v1/batch/labels/delete --> github.com/photoprism/photoprism/internal/api.BatchLabelsDelete.func1 (4 handlers)
[GIN-debug] GET    /api/v1/svg/photo         --> github.com/photoprism/photoprism/internal/api.GetSvg.func1 (4 handlers)
[GIN-debug] GET    /api/v1/svg/raw           --> github.com/photoprism/photoprism/internal/api.GetSvg.func2 (4 handlers)
[GIN-debug] GET    /api/v1/svg/file          --> github.com/photoprism/photoprism/internal/api.GetSvg.func3 (4 handlers)
[GIN-debug] GET    /api/v1/svg/video         --> github.com/photoprism/photoprism/internal/api.GetSvg.func4 (4 handlers)
[GIN-debug] GET    /api/v1/svg/label         --> github.com/photoprism/photoprism/internal/api.GetSvg.func5 (4 handlers)
[GIN-debug] GET    /api/v1/svg/portrait      --> github.com/photoprism/photoprism/internal/api.GetSvg.func6 (4 handlers)
[GIN-debug] GET    /api/v1/svg/folder        --> github.com/photoprism/photoprism/internal/api.GetSvg.func7 (4 handlers)
[GIN-debug] GET    /api/v1/svg/album         --> github.com/photoprism/photoprism/internal/api.GetSvg.func8 (4 handlers)
[GIN-debug] GET    /api/v1/svg/broken        --> github.com/photoprism/photoprism/internal/api.GetSvg.func9 (4 handlers)
[GIN-debug] GET    /api/v1/svg/uncached      --> github.com/photoprism/photoprism/internal/api.GetSvg.func10 (4 handlers)
[GIN-debug] GET    /api/v1/status            --> github.com/photoprism/photoprism/internal/api.GetStatus.func1 (4 handlers)
[GIN-debug] GET    /api/v1/errors            --> github.com/photoprism/photoprism/internal/api.GetErrors.func1 (4 handlers)
[GIN-debug] DELETE /api/v1/errors            --> github.com/photoprism/photoprism/internal/api.DeleteErrors.func1 (4 handlers)
[GIN-debug] POST   /api/v1/feedback          --> github.com/photoprism/photoprism/internal/api.SendFeedback.func1 (4 handlers)
[GIN-debug] PUT    /api/v1/connect/:name     --> github.com/photoprism/photoprism/internal/api.Connect.func1 (4 handlers)
[GIN-debug] GET    /api/v1/ws                --> github.com/photoprism/photoprism/internal/api.Websocket.func1 (4 handlers)
[GIN-debug] GET    /s/:token                 --> github.com/photoprism/photoprism/internal/api.Shares.func1 (4 handlers)
[GIN-debug] GET    /s/:token/:share          --> github.com/photoprism/photoprism/internal/api.Shares.func2 (4 handlers)
[GIN-debug] GET    /s/:token/:share/preview  --> github.com/photoprism/photoprism/internal/api.SharePreview.func1 (4 handlers)
[GIN-debug] HEAD   /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] GET    /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PUT    /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] POST   /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PATCH  /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] DELETE /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] OPTIONS /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] MKCOL  /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] COPY   /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] MOVE   /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] LOCK   /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] UNLOCK /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PROPFIND /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PROPPATCH /originals/*path          --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] HEAD   /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] GET    /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PUT    /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] POST   /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PATCH  /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] DELETE /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] OPTIONS /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] MKCOL  /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] COPY   /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] MOVE   /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] LOCK   /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] UNLOCK /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PROPFIND /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PROPPATCH /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
time="2022-10-19T19:56:13+02:00" level=debug msg="server: GET /api/v1/ws (200) [5m45.143413284s]"
time="2022-10-19T19:56:13+02:00" level=debug msg="server: GET /people (200) [1.322923ms]"
time="2022-10-19T19:56:13+02:00" level=debug msg="server: GET /api/v1/subjects?count=120&offset=0&q=&hidden=&order=relevance&files=1&type=person (400) [850.754µs]"

2nd log ("closing bad idle connection: connection reset by peer")

time="2022-10-20T17:49:48+02:00" level=debug msg="photos: found 60 results for public:true dist:20 year:0 month:0 quality:3 camera:0 lens:0 count:60 order:newest merged:true [5.645856ms]"
time="2022-10-20T17:49:48+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=0&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=newest&q=&public=true&quality=3 (200) [16.277917ms]"
time="2022-10-20T17:49:48+02:00" level=debug msg="photos: found 60 results for public:true dist:20 year:0 month:0 quality:3 camera:0 lens:0 count:60 offset:60 order:newest merged:true [5.956188ms]"
time="2022-10-20T17:49:48+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=60&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=newest&q=&public=true&quality=3 (200) [18.157799ms]"
time="2022-10-20T17:49:48+02:00" level=debug msg="photos: found 60 results for public:true dist:20 year:0 month:0 quality:3 camera:0 lens:0 count:60 offset:120 order:newest merged:true [6.043739ms]"
time="2022-10-20T17:49:48+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=120&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=newest&q=&public=true&quality=3 (200) [16.409629ms]"
time="2022-10-20T17:53:09+02:00" level=debug msg="server: GET /api/v1/ws (200) [3m47.985526176s]"
time="2022-10-20T17:53:09+02:00" level=debug msg="server: GET /browse (200) [1.604361ms]"
[mysql] 2022/10/20 17:53:09 packets.go:122: closing bad idle connection: connection reset by peer
[mysql] 2022/10/20 17:53:09 connection.go:158: driver: bad connection
[mysql] 2022/10/20 17:53:09 packets.go:122: closing bad idle connection: connection reset by peer
[mysql] 2022/10/20 17:53:09 connection.go:158: driver: bad connection
time="2022-10-20T17:53:09+02:00" level=warning msg="search: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-20T17:53:09+02:00" level=debug msg="api-v1: abort /api/v1/photos with code 400 (ungültige anfrage)"
time="2022-10-20T17:53:09+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=0&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=newest&q=&public=true&quality=3 (400) [1.03818ms]"
[mysql] 2022/10/20 17:53:09 packets.go:122: closing bad idle connection: connection reset by peer
time="2022-10-20T17:53:09+02:00" level=debug msg="server: GET /static/icons/app/1024.png (200) [177.623µs]"
time="2022-10-20T17:53:36+02:00" level=debug msg="server: GET /api/v1/ws (200) [27.288698915s]"
time="2022-10-20T17:53:36+02:00" level=debug msg="server: GET /browse (200) [1.317099ms]"
time="2022-10-20T17:53:37+02:00" level=warning msg="search: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-20T17:53:37+02:00" level=debug msg="api-v1: abort /api/v1/photos with code 400 (ungültige anfrage)"
time="2022-10-20T17:53:37+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=0&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=newest&q=&public=true&quality=3 (400) [1.061332ms]"
time="2022-10-20T17:55:25+02:00" level=debug msg="server: GET /api/v1/config (200) [14.210364ms]"

3rd log ("metadata: running face recognition" => "dial tcp 10.1.0.10:3306: connect: connection refused")

time="2022-10-21T08:45:09+02:00" level=debug msg="places: found 3,105 results for public:true quality:3 count:500000 [28.318223ms]"
time="2022-10-21T08:45:09+02:00" level=debug msg="server: GET /api/v1/geo?count=500000&offset=0&q=&public=true&quality=3 (200) [42.704729ms]"
time="2022-10-21T08:45:09+02:00" level=debug msg="server: GET /api/v1/t/21ffdbffde06232fea03023a7439230608f852a4/d616f0fb/tile_50 (200) [5.486818ms]"
time="2022-10-21T08:45:09+02:00" level=debug msg="server: GET /api/v1/t/8f8468f9c55c71abf8c4ea0cc6e76a5cf7557261/d616f0fb/tile_50 (200) [5.225759ms]"
time="2022-10-21T08:45:10+02:00" level=debug msg="server: GET /static/plugins/maplibre-gl-rtl-text/v0.2.3/maplibre-gl-rtl-text.js (200) [2.56376ms]"
time="2022-10-21T08:54:35+02:00" level=debug msg="server: GET /api/v1/config (200) [15.03545ms]"
time="2022-10-21T08:54:54+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T08:54:54+02:00" level=warning msg="sync: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T08:54:54+02:00" level=error msg="faces: markers: dial tcp 10.1.0.10:3306: connect: connection refused (purge orphans) (remove orphan markers)"
time="2022-10-21T08:54:54+02:00" level=warning msg="share: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T08:54:54+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (fix references)"
time="2022-10-21T08:54:54+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (create subjects)"
time="2022-10-21T08:54:54+02:00" level=error msg="faces: dial tcp 10.1.0.10:3306: connect: connection refused (resolve ambiguous subjects)"
time="2022-10-21T08:54:54+02:00" level=warning msg="dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T08:54:54+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-21T08:54:54+02:00" level=warning msg="metadata: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:04:35+02:00" level=debug msg="server: GET /api/v1/config (200) [14.555203ms]"
time="2022-10-21T09:09:54+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T09:09:54+02:00" level=warning msg="sync: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:09:54+02:00" level=error msg="faces: markers: dial tcp 10.1.0.10:3306: connect: connection refused (purge orphans) (remove orphan markers)"
time="2022-10-21T09:09:54+02:00" level=warning msg="share: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:09:54+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (fix references)"
time="2022-10-21T09:09:54+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (create subjects)"
time="2022-10-21T09:09:54+02:00" level=error msg="faces: dial tcp 10.1.0.10:3306: connect: connection refused (resolve ambiguous subjects)"
time="2022-10-21T09:09:54+02:00" level=warning msg="dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:09:54+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-21T09:09:54+02:00" level=warning msg="metadata: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:14:35+02:00" level=debug msg="server: GET /api/v1/config (200) [14.593433ms]"
time="2022-10-21T09:24:35+02:00" level=debug msg="server: GET /api/v1/config (200) [14.950434ms]"
time="2022-10-21T09:24:54+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T09:24:54+02:00" level=warning msg="sync: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:24:54+02:00" level=warning msg="share: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:24:54+02:00" level=error msg="faces: markers: dial tcp 10.1.0.10:3306: connect: connection refused (purge orphans) (remove orphan markers)"
time="2022-10-21T09:24:54+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (fix references)"
time="2022-10-21T09:24:54+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (create subjects)"
time="2022-10-21T09:24:54+02:00" level=error msg="faces: dial tcp 10.1.0.10:3306: connect: connection refused (resolve ambiguous subjects)"
time="2022-10-21T09:24:54+02:00" level=warning msg="dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:24:54+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-21T09:24:54+02:00" level=warning msg="metadata: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:34:35+02:00" level=debug msg="server: GET /api/v1/config (200) [15.103771ms]"
time="2022-10-21T09:38:10+02:00" level=warning msg="search: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T09:38:10+02:00" level=debug msg="api-v1: abort /api/v1/photos with code 400 (ungültige anfrage)"
time="2022-10-21T09:38:10+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=0&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=oldest&q=&public=true&quality=3 (400) [1.147767ms]"

4th log ("closing bad idle connection: connection reset by peer")

time="2022-10-21T16:54:30+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=120&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=oldest&q=&public=true&quality=3 (200) [49.963572ms]"
[mysql] 2022/10/21 17:04:30 packets.go:122: closing bad idle connection: connection reset by peer
time="2022-10-21T17:04:30+02:00" level=debug msg="server: GET /api/v1/config (200) [14.164293ms]"
time="2022-10-21T17:07:59+02:00" level=debug msg="server: GET /api/v1/ws (200) [13m29.086158155s]"
time="2022-10-21T17:07:59+02:00" level=debug msg="server: GET /browse (200) [6.62363ms]"
time="2022-10-21T17:07:59+02:00" level=warning msg="search: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T17:07:59+02:00" level=debug msg="api-v1: abort /api/v1/photos with code 400 (ungültige anfrage)"
time="2022-10-21T17:07:59+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=0&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=oldest&q=&public=true&quality=3 (400) [1.446831ms]"

5th log ("metadata: running face recognition" => "closing bad idle connection: connection reset by peer")

time="2022-10-21T17:57:30+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T17:57:30+02:00" level=debug msg="faces: found no orphan markers [30.85608ms]"
time="2022-10-21T17:57:30+02:00" level=debug msg="markers: found no invalid references [49.537484ms]"
time="2022-10-21T17:57:30+02:00" level=debug msg="markers: found no missing subjects [333.445µs]"
time="2022-10-21T17:57:37+02:00" level=debug msg="faces: found no ambiguous subjects [6.42267982s]"
time="2022-10-21T17:57:37+02:00" level=debug msg="faces: found no clusters to be merged [7.900987ms]"
time="2022-10-21T17:57:37+02:00" level=debug msg="faces: skipped clustering"
time="2022-10-21T17:57:37+02:00" level=debug msg="faces: found no new faces [9.09842ms]"
time="2022-10-21T17:57:37+02:00" level=debug msg="faces: found no unmatched markers"
time="2022-10-21T17:57:37+02:00" level=debug msg="faces: updated 0 markers, recognized 0 faces, 0 unknown [38.170158ms]"
time="2022-10-21T17:57:37+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-21T17:57:37+02:00" level=debug msg="moments: analyzing 26505 photos and 192 videos, with threshold 15"
time="2022-10-21T17:57:37+02:00" level=debug msg="moments: 357 albums saved as yaml files"
time="2022-10-21T17:57:37+02:00" level=debug msg="index: updating counts"
time="2022-10-21T17:57:37+02:00" level=debug msg="counts: updated 0 places [17.967824ms]"
time="2022-10-21T17:57:37+02:00" level=debug msg="counts: updated 0 subjects [17.740805ms]"
time="2022-10-21T17:57:38+02:00" level=debug msg="counts: updated 0 labels [47.379273ms]"
time="2022-10-21T17:57:38+02:00" level=debug msg="index: updating covers"
time="2022-10-21T17:57:38+02:00" level=debug msg="covers: updated 0 albums [675.496µs]"
time="2022-10-21T17:57:38+02:00" level=debug msg="covers: updated 0 folders [24.17499ms]"
time="2022-10-21T17:57:38+02:00" level=debug msg="covers: updated 0 months [10.676024ms]"
time="2022-10-21T17:57:38+02:00" level=debug msg="covers: updated 0 labels [52.010358ms]"
time="2022-10-21T17:57:38+02:00" level=debug msg="covers: updated 0 subjects [8.452823ms]"
time="2022-10-21T17:59:57+02:00" level=debug msg="server: GET /api/v1/ws (200) [6m17.58422265s]"
time="2022-10-21T17:59:57+02:00" level=debug msg="server: GET /browse (200) [999.519µs]"
[mysql] 2022/10/21 17:59:57 packets.go:122: closing bad idle connection: connection reset by peer
[mysql] 2022/10/21 17:59:57 connection.go:158: driver: bad connection
[mysql] 2022/10/21 17:59:57 packets.go:122: closing bad idle connection: connection reset by peer
[mysql] 2022/10/21 17:59:57 connection.go:158: driver: bad connection
time="2022-10-21T17:59:57+02:00" level=warning msg="search: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T17:59:57+02:00" level=debug msg="api-v1: abort /api/v1/photos with code 400 (ungültige anfrage)"
time="2022-10-21T17:59:57+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=0&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=oldest&q=&public=true&quality=3 (400) [1.352686ms]"
[mysql] 2022/10/21 17:59:57 packets.go:122: closing bad idle connection: connection reset by peer
time="2022-10-21T17:59:57+02:00" level=debug msg="server: GET /static/icons/app/1024.png (200) [111.967µs]"
[GIN-debug] MKCOL  /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] COPY   /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] MOVE   /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] LOCK   /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] UNLOCK /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PROPFIND /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)
[GIN-debug] PROPPATCH /import/*path             --> github.com/photoprism/photoprism/internal/server.WebDAV.func2 (5 handlers)

6th log ("metadata: running face recognition" => "dial tcp 10.1.0.10:3306: connect: connection refused")

time="2022-10-21T18:03:26+02:00" level=info msg="webdav: /originals/ enabled, waiting for requests"
time="2022-10-21T18:03:26+02:00" level=info msg="webdav: /import/ enabled, waiting for requests"
time="2022-10-21T18:03:26+02:00" level=debug msg="server: successfully initialized [445.302µs]"
time="2022-10-21T18:03:26+02:00" level=info msg="server: listening at 0.0.0.0:2342"
time="2022-10-21T18:09:57+02:00" level=debug msg="server: GET /api/v1/config (200) [66.057364ms]"
time="2022-10-21T18:18:26+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T18:18:26+02:00" level=debug msg="faces: found no orphan markers [26.621806ms]"
time="2022-10-21T18:18:26+02:00" level=debug msg="markers: found no invalid references [48.023581ms]"
time="2022-10-21T18:18:26+02:00" level=debug msg="markers: found no missing subjects [274.116µs]"
time="2022-10-21T18:18:33+02:00" level=debug msg="faces: found no ambiguous subjects [6.544901841s]"
time="2022-10-21T18:18:33+02:00" level=debug msg="faces: found no clusters to be merged [7.677821ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="faces: skipped clustering"
time="2022-10-21T18:18:33+02:00" level=debug msg="faces: found no new faces [9.195885ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="faces: found no unmatched markers"
time="2022-10-21T18:18:33+02:00" level=debug msg="faces: updated 0 markers, recognized 0 faces, 0 unknown [34.514871ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-21T18:18:33+02:00" level=debug msg="moments: analyzing 26505 photos and 192 videos, with threshold 15"
time="2022-10-21T18:18:33+02:00" level=debug msg="moments: 357 albums saved as yaml files"
time="2022-10-21T18:18:33+02:00" level=debug msg="index: updating counts"
time="2022-10-21T18:18:33+02:00" level=debug msg="counts: updated 0 places [17.839095ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="counts: updated 0 subjects [17.952123ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="counts: updated 0 labels [48.808469ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="index: updating covers"
time="2022-10-21T18:18:33+02:00" level=debug msg="covers: updated 0 albums [677.903µs]"
time="2022-10-21T18:18:33+02:00" level=debug msg="covers: updated 0 folders [24.199343ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="covers: updated 0 months [11.306155ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="covers: updated 0 labels [53.964609ms]"
time="2022-10-21T18:18:33+02:00" level=debug msg="covers: updated 0 subjects [8.698336ms]"
time="2022-10-21T18:19:57+02:00" level=debug msg="server: GET /api/v1/config (200) [65.484024ms]"
time="2022-10-21T18:29:57+02:00" level=debug msg="server: GET /api/v1/config (200) [14.731004ms]"
time="2022-10-21T18:33:26+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T18:33:26+02:00" level=error msg="faces: markers: dial tcp 10.1.0.10:3306: connect: connection refused (purge orphans) (remove orphan markers)"
time="2022-10-21T18:33:26+02:00" level=warning msg="sync: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T18:33:26+02:00" level=warning msg="share: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T18:33:26+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (fix references)"
time="2022-10-21T18:33:26+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (create subjects)"
time="2022-10-21T18:33:26+02:00" level=error msg="faces: dial tcp 10.1.0.10:3306: connect: connection refused (resolve ambiguous subjects)"
time="2022-10-21T18:33:26+02:00" level=warning msg="dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T18:33:26+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-21T18:33:26+02:00" level=warning msg="metadata: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T18:39:57+02:00" level=debug msg="server: GET /api/v1/config (200) [14.089362ms]"
time="2022-10-21T18:48:26+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T18:48:26+02:00" level=warning msg="sync: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T18:48:26+02:00" level=warning msg="share: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T18:48:26+02:00" level=error msg="faces: markers: dial tcp 10.1.0.10:3306: connect: connection refused (purge orphans) (remove orphan markers)"
time="2022-10-21T18:48:26+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (fix references)"
time="2022-10-21T18:48:26+02:00" level=error msg="markers: dial tcp 10.1.0.10:3306: connect: connection refused (create subjects)"
time="2022-10-21T18:48:26+02:00" level=error msg="faces: dial tcp 10.1.0.10:3306: connect: connection refused (resolve ambiguous subjects)"
time="2022-10-21T18:48:26+02:00" level=warning msg="dial tcp 10.1.0.10:3306: connect: connection refused"

7th log ("metadata: running face recognition" => "closing bad idle connection: connection reset by peer")

time="2022-10-21T19:00:52+02:00" level=debug msg="server: GET /api/v1/config (200) [67.170584ms]"
time="2022-10-21T19:05:49+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T19:05:49+02:00" level=debug msg="faces: found no orphan markers [28.315595ms]"
time="2022-10-21T19:05:49+02:00" level=debug msg="markers: found no invalid references [48.879813ms]"
time="2022-10-21T19:05:49+02:00" level=debug msg="markers: found no missing subjects [264.504µs]"
time="2022-10-21T19:05:55+02:00" level=debug msg="faces: found no ambiguous subjects [6.439656473s]"
time="2022-10-21T19:05:55+02:00" level=debug msg="faces: found no clusters to be merged [7.927001ms]"
time="2022-10-21T19:05:55+02:00" level=debug msg="faces: skipped clustering"
time="2022-10-21T19:05:55+02:00" level=debug msg="faces: found no new faces [9.67985ms]"
time="2022-10-21T19:05:55+02:00" level=debug msg="faces: found no unmatched markers"
time="2022-10-21T19:05:55+02:00" level=debug msg="faces: updated 0 markers, recognized 0 faces, 0 unknown [34.395823ms]"
time="2022-10-21T19:05:55+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-21T19:05:56+02:00" level=debug msg="moments: analyzing 26505 photos and 192 videos, with threshold 15"
time="2022-10-21T19:05:56+02:00" level=debug msg="moments: 357 albums saved as yaml files"
time="2022-10-21T19:05:56+02:00" level=debug msg="index: updating counts"
time="2022-10-21T19:05:56+02:00" level=debug msg="counts: updated 0 places [17.923087ms]"
time="2022-10-21T19:05:56+02:00" level=debug msg="counts: updated 0 subjects [17.997183ms]"
time="2022-10-21T19:05:56+02:00" level=debug msg="counts: updated 0 labels [48.849449ms]"
time="2022-10-21T19:05:56+02:00" level=debug msg="index: updating covers"
time="2022-10-21T19:05:56+02:00" level=debug msg="covers: updated 0 albums [540.769µs]"
time="2022-10-21T19:05:56+02:00" level=debug msg="covers: updated 0 folders [24.348765ms]"
time="2022-10-21T19:05:56+02:00" level=debug msg="covers: updated 0 months [10.504739ms]"
time="2022-10-21T19:05:56+02:00" level=debug msg="covers: updated 0 labels [50.70521ms]"
time="2022-10-21T19:05:56+02:00" level=debug msg="covers: updated 0 subjects [8.316966ms]"
time="2022-10-21T19:10:52+02:00" level=debug msg="server: GET /api/v1/config (200) [70.658509ms]"
time="2022-10-21T19:20:49+02:00" level=debug msg="metadata: running face recognition"
time="2022-10-21T19:20:49+02:00" level=debug msg="faces: found no orphan markers [25.722521ms]"
time="2022-10-21T19:20:49+02:00" level=debug msg="markers: found no invalid references [47.435737ms]"
time="2022-10-21T19:20:49+02:00" level=debug msg="markers: found no missing subjects [398.128µs]"
time="2022-10-21T19:20:52+02:00" level=debug msg="server: GET /api/v1/config (200) [67.919057ms]"
time="2022-10-21T19:20:55+02:00" level=debug msg="faces: found no ambiguous subjects [6.41347363s]"
time="2022-10-21T19:20:55+02:00" level=debug msg="faces: found no clusters to be merged [7.623514ms]"
time="2022-10-21T19:20:55+02:00" level=debug msg="faces: skipped clustering"
time="2022-10-21T19:20:55+02:00" level=debug msg="faces: found no new faces [8.515406ms]"
time="2022-10-21T19:20:55+02:00" level=debug msg="faces: found no unmatched markers"
time="2022-10-21T19:20:55+02:00" level=debug msg="faces: updated 0 markers, recognized 0 faces, 0 unknown [31.188489ms]"
time="2022-10-21T19:20:55+02:00" level=debug msg="metadata: starting routine check"
time="2022-10-21T19:20:56+02:00" level=debug msg="moments: analyzing 26505 photos and 192 videos, with threshold 15"
time="2022-10-21T19:20:56+02:00" level=debug msg="moments: 357 albums saved as yaml files"
time="2022-10-21T19:20:56+02:00" level=debug msg="index: updating counts"
time="2022-10-21T19:20:56+02:00" level=debug msg="counts: updated 0 places [17.948625ms]"
time="2022-10-21T19:20:56+02:00" level=debug msg="counts: updated 0 subjects [17.726916ms]"
time="2022-10-21T19:20:56+02:00" level=debug msg="counts: updated 0 labels [49.507199ms]"
time="2022-10-21T19:20:56+02:00" level=debug msg="index: updating covers"
time="2022-10-21T19:20:56+02:00" level=debug msg="covers: updated 0 albums [596.512µs]"
time="2022-10-21T19:20:56+02:00" level=debug msg="covers: updated 0 folders [24.217066ms]"
time="2022-10-21T19:20:56+02:00" level=debug msg="covers: updated 0 months [10.645949ms]"
time="2022-10-21T19:20:56+02:00" level=debug msg="covers: updated 0 labels [51.87722ms]"
time="2022-10-21T19:20:56+02:00" level=debug msg="covers: updated 0 subjects [8.553244ms]"
time="2022-10-21T19:29:17+02:00" level=debug msg="server: GET /api/v1/ws (200) [38m24.763663392s]"
time="2022-10-21T19:29:17+02:00" level=debug msg="server: GET /browse (200) [862.538µs]"
[mysql] 2022/10/21 19:29:17 packets.go:122: closing bad idle connection: connection reset by peer
[mysql] 2022/10/21 19:29:17 connection.go:158: driver: bad connection
[mysql] 2022/10/21 19:29:17 packets.go:122: closing bad idle connection: connection reset by peer
[mysql] 2022/10/21 19:29:17 connection.go:158: driver: bad connection
time="2022-10-21T19:29:17+02:00" level=warning msg="search: dial tcp 10.1.0.10:3306: connect: connection refused"
time="2022-10-21T19:29:17+02:00" level=debug msg="api-v1: abort /api/v1/photos with code 400 (ungültige anfrage)"
[mysql] 2022/10/21 19:29:17 packets.go:122: closing bad idle connection: connection reset by peer
time="2022-10-21T19:29:17+02:00" level=debug msg="server: GET /api/v1/photos?count=60&offset=0&merged=true&country=&camera=0&lens=0&label=&year=0&month=0&color=&order=oldest&q=&public=true&quality=3 (400) [1.696562ms]"

[lastzero]

10.1.x.x is not a default internal Docker IP range, so I would check your DNS and network configuration first.

[adymorz]

Thanks for your answer.

The same setup works for nextcloud and another mariadb instance without issues for weeks without rebooting one of the two docker containers. If it would be something with the network setup, it would cause troubles imediatily or for all docker containers.

What do you think about the message metadata: running face recognition showing in 5 of 7 cases just before the errors start to arise?

[lastzero]

I don't know why running a query should crash mariadb or kill the network connection completely. That does not make sense.

[adymorz]

I will set log_warnings=2 in the mariadb container to find out, if there are timeouts occuring on the mariadb side as described in https://mariadb.com/docs/reference/mdb/system-variables/wait_timeout/

[adymorz]

Since I use the host name in the environment variable PHOTOPRISM_DATABASE_SERVER instead of the fixed IP, the connection issues are gone, while still using a non default network config. I have no idea why this happens, but I am happy to have a flawlessly working photorpism app now.

[graciousgrey]

Good to hear that it is working now :)

[lastzero]

We have updated our Troubleshooting Checklists to provide specific advice on using fixed IP addresses for hosts other than localhost:
https://docs.photoprism.app/getting-started/troubleshooting/mariadb/#cannot-connect