Container is marked as unhealthy

[zandhaas]

Trilium Version

0.59.1

What operating system are you using?

Other Linux

What is your setup?

Local (no sync)

Operating System Version

Linux DS918 4.4.180+ #42962 SMP Fri Feb 10 16:32:29 CST 2023 x86_64 GNU/Linux synology_apollolake_918+

Description

After the container upgrade on march the first Trillium is working as expected but in Portainer the container is marked "unhealthy"

Error logs

No USER_UID specified, leaving 1000
No USER_GID specified, leaving 1000
App db version is 213, while db version is 212. Migration needed.
DB size: 1540 KB
Trusted reverse proxy: false
App HTTP server starting up at port 8080
{
"appVersion": "0.59.1",
"dbVersion": 213,
"syncVersion": 29,
"buildDate": "2023-02-28T23:39:34+01:00",
"buildRevision": "9eb3075f65af98bd8c4a8ca8cc241ae2ae05bfa9",
"dataDirectory": "/data",
"clipperProtocolVersion": "1.0",
"utcDateTime": "2023-03-01T11:04:19.733Z"
}
CPU model: Intel(R) Celeron(R) CPU J3455 @ 1.50GHz, logical cores: 4 freq: 1501 Mhz
Listening on port 8080
Created backup at /data/backup/backup-before-migration.db
Attempting migration to version 213
Migration with JS module
Becca (note cache) load took 22ms
Migration to version 213 has been successful.
Becca (note cache) load took 9ms
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 25ms)
[session-file-store] Deleting expired sessions
Optimizing database
Optimization finished in 0ms.
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 11ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 6ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 6ms)
[session-file-store] Deleting expired sessions
Created backup at /data/backup/backup-daily.db
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 7ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 7ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 6ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 10ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 9ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 7ms)
[session-file-store] Deleting expired sessions
Optimizing database
Optimization finished in 0ms.
Slow query took 31ms: SELECT COUNT(1) FROM notes
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 66ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 62ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 7ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 16ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 12ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 6ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 6ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 7ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 8ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 7ms)
[session-file-store] Deleting expired sessions
Optimizing database
Optimization finished in 3ms.
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 10ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 6ms)
[session-file-store] Deleting expired sessions
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 120ms)
WARNING: Wrong password from 172.29.0.1, rejecting.
Generated CSRF token q38RD4Xb-PbXy22vYPD8xLi-sVh7ZCGLd5MU with secret _csrf=ePGY4C8cZ0kfOrXELBJ0HK90; Path=/
200 GET /api/options with 6525 bytes took 4ms
200 GET /api/tree with 11160 bytes took 6ms
Keyboard action showNoteInfo found in database, but not in action definition.
Keyboard action showLinkMap found in database, but not in action definition.
Keyboard action focusOnAttributes found in database, but not in action definition.
Keyboard action toggleZenMode found in database, but not in action definition.
200 GET /api/keyboard-actions with 11935 bytes took 9ms
200 GET /api/script/widgets with 2 bytes took 2ms
websocket client connected
200 GET /api/keyboard-shortcuts-for-notes with 2 bytes took 3ms
200 POST /api/tree/load with 6038 bytes took 6ms
200 GET /api/tree?subTreeNoteId=9b4QOEeKqq06 with 5207 bytes took 5ms
200 GET /api/tree?subTreeNoteId=fDa52IxJiN9k with 2458 bytes took 4ms
200 GET /api/note-map/fDa52IxJiN9k/backlink-count with 11 bytes took 3ms
200 GET /api/notes/fDa52IxJiN9k with 1300 bytes took 4ms
200 POST /api/tree/load with 18350 bytes took 3ms
200 GET /api/script/startup with 2 bytes took 3ms
Caught interrupt/termination signal. Exiting.
No USER_UID specified, leaving 1000
No USER_GID specified, leaving 1000
DB size: 1540 KB
Trusted reverse proxy: false
App HTTP server starting up at port 8080
{
"appVersion": "0.59.1",
"dbVersion": 213,
"syncVersion": 29,
"buildDate": "2023-02-28T23:39:34+01:00",
"buildRevision": "9eb3075f65af98bd8c4a8ca8cc241ae2ae05bfa9",
"dataDirectory": "/data",
"clipperProtocolVersion": "1.0",
"utcDateTime": "2023-03-02T10:03:26.689Z"
}
CPU model: Intel(R) Celeron(R) CPU J3455 @ 1.50GHz, logical cores: 4 freq: 1501 Mhz
Listening on port 8080
Becca (note cache) load took 21ms
Table counts: notes: 79, note_revisions: 14, branches: 79, attributes: 133, etapi_tokens: 0
All consistency checks passed with no errors detected (took 39ms)
Generated CSRF token ssqGDIsx-V04AehUxcyvM5UO7KQvzPid2RI4 with secret undefined
304 GET /api/options with 6525 bytes took 5ms
304 GET /api/tree with 11160 bytes took 5ms
Keyboard action showNoteInfo found in database, but not in action definition.
Keyboard action showLinkMap found in database, but not in action definition.
Keyboard action focusOnAttributes found in database, but not in action definition.
Keyboard action toggleZenMode found in database, but not in action definition.
304 GET /api/keyboard-actions with 11935 bytes took 9ms
304 GET /api/script/widgets with 2 bytes took 2ms
websocket client connected
304 GET /api/keyboard-shortcuts-for-notes with 2 bytes took 5ms
200 POST /api/tree/load with 6038 bytes took 6ms
304 GET /api/tree?subTreeNoteId=9b4QOEeKqq06 with 5207 bytes took 3ms
304 GET /api/tree?subTreeNoteId=fDa52IxJiN9k with 2458 bytes took 3ms
200 POST /api/tree/load with 18350 bytes took 5ms
304 GET /api/notes/fDa52IxJiN9k with 1300 bytes took 4ms
304 GET /api/note-map/fDa52IxJiN9k/backlink-count with 11 bytes took 2ms
304 GET /api/script/startup with 2 bytes took 3ms
Slow 204 PUT /api/options with 0 bytes took 32ms
200 GET /api/notes/Y4aJpBeo1ly4 with 618 bytes took 4ms
200 GET /api/note-map/Y4aJpBeo1ly4/backlink-count with 11 bytes took 5ms
200 GET /api/notes/9b4QOEeKqq06 with 431 bytes took 5ms
200 GET /api/note-map/9b4QOEeKqq06/backlink-count with 11 bytes took 4ms
204 PUT /api/options with 0 bytes took 5ms
204 PUT /api/options with 0 bytes took 4ms
200 GET /api/notes/4BKQscvDctEe with 431 bytes took 3ms
200 GET /api/note-map/4BKQscvDctEe/backlink-count with 11 bytes took 2ms
200 GET /api/note-map/1AVCsbMaelAv/backlink-count with 11 bytes took 3ms
204 PUT /api/options with 0 bytes took 5ms
200 GET /api/notes/1AVCsbMaelAv with 966 bytes took 3ms
200 POST /api/notes/relation-map with 1177 bytes took 5ms
304 GET /api/notes/9b4QOEeKqq06 with 431 bytes took 3ms
304 GET /api/notes/9b4QOEeKqq06 with 431 bytes took 3ms
200 GET /api/notes/aCzFg2NTw45j with 442 bytes took 4ms
Slow 204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 12ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 5ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 6ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 5ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 3ms
204 PUT /api/notes/1AVCsbMaelAv/data with 0 bytes took 4ms
200 GET /api/notes/PjC9AaqmE7Pj with 435 bytes took 2ms
200 GET /api/health-check with 15 bytes took 2ms
200 GET /api/health-check with 15 bytes took 2ms
200 POST /api/tree/load with 4826 bytes took 3ms
204 PUT /api/options with 0 bytes took 9ms
200 POST /api/tree/load with 7860 bytes took 4ms
200 GET /api/note-map/_options/backlink-count with 11 bytes took 3ms
200 GET /api/tree?subTreeNoteId=_hidden with 25205 bytes took 4ms
204 PUT /api/branches/root__hidden/expanded/1 with 0 bytes took 2ms
200 GET /api/tree?subTreeNoteId=_options with 7859 bytes took 4ms
200 GET /api/app-info with 251 bytes took 2ms
No response

[jcvara]

Same here.

Can add to this that Portainer shows the error "sh: can't open 'DockerHealthcheck.sh': No such file or directory" on the container health report.

Edit: I've also found a difference in the latest image:

On the image layers, previous images referenced this in layer 18:

18 | 0 B | HEALTHCHECK &{["CMD-SHELL" "sh DockerHealthcheck.sh"] "0s" "0s" "0s" '\x00'}

Yet latest image references this:

18 | 0 B | HEALTHCHECK &{["CMD-SHELL" "node docker_healthcheck.js"] "0s" "0s" "10s" '\x00'}

Hope this tidbit is useful.

Edit 2: Going through the changelog, perhaps it's related to #3582

[zadam]

Hello, could you provide output from this command?

docker inspect [CONTAINER_NAME] --format "{{json .State.Health }}"

Do you have some specific setup? TLS configured in trilium, different port etc.?

[jcvara]

Output: https://pastebin.com/P4Duj0R0

I have Trilium behind Traefik, using the default port (8080). I'm not using TLS in the container, certs are managed by Traefik.

Since the update the container runs but it doesn't get routed by Traefik, so it's not accessible.

[xenolithis]

I had this experience as well with a similar rev proxy setup. turned the normal port config back on, tested, turned off and it is working now without having a host exposed port. 🤷️

edit: healthcheck seems fine as well

[zandhaas]

docker inspect [CONTAINER_NAME] --format "{{json .State.Health }}"

{"Status":"unhealthy","FailingStreak":2528,"Log":[{"Start":"2023-03-03T08:10:34.870633994+01:00","End":"2023-03-03T08:10:35.215146694+01:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"},{"Start":"2023-03-03T08:11:05.411672947+01:00","End":"2023-03-03T08:11:05.480141764+01:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"},{"Start":"2023-03-03T08:11:35.535185153+01:00","End":"2023-03-03T08:11:35.61732857+01:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"},{"Start":"2023-03-03T08:12:05.677653081+01:00","End":"2023-03-03T08:12:05.75205972+01:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"},{"Start":"2023-03-03T08:12:35.784502134+01:00","End":"2023-03-03T08:12:35.855438693+01:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"}]}

I'm using the nginx-pm as reverse proxy. Except the unhealty state tillium is working OK over here.
What I saw in the container that there is no DockerHealthcheck.sh but a DockerHealthcheck.js

[jcvara]

I am not privy to Docker and Traefik internal workings, but I cannot find the Trilium container on the list of containers in Traefik's dashboard, so I have the theory that somehow the unhealthy status either prevents the propagation of the container's labels and Traefik can't pick them up, or Traefik is refusing to route Trilium's container due to the healthcheck.

I've checked everything thrice, config hasn't changed and both Traefik and Trilium containers are on the same network. I've tried restarting and recreating both containers to no avail.

I doubt it's pertinent, but just in case I'm running the official Traefik v2.2 image.

Edit: Further investigation led me to find this Traefik issue that mentions how Traefik won't list nor route containers until their status is 'healthy': traefik/traefik#7842

So, Trilium is supposed to be working OK, even though the healthcheck fails, but I can't access it due to this Traefik feature.

Edit 2: Workaround:

I mounted /usr/src/app/DockerHealthcheck.sh as a bind, with the contents:

#!/bin/sh
exit 0;

And completely bypassed the healthcheck. Now Traefik is picking up the container and routing it. Haven't found any other problems so far, everything is working as expected.

[zadam]

There was a change and trilium 0.59 which now uses docker_healtcheck.js instead of the old DockerHealthcheck.sh.

But this is correctly set up in the Dockerfile in the published images as visible here.

So I'm confused where does the docker finds the reference to now non-existing DockerHealthcheck.sh.

[jcvara]

Indeed, that is the conclusion I arrived at, and also tried unsuccessfully to find where the reference to DockerHealthcheck.sh could be.

I don't have much experience building Docker images, could it be possible that there's some Docker build cache that wasn't flushed before building the latest image? Just throwing stuff at the wall here...

[zadam]

The docker hub image is the source of truth and it lists the correct healthcheck, so the cache would have to be on the clients. I don't know either, are you using the images directly from dockerhub or perhaps re-built from somewhere else?

[zandhaas]

It has something to do with the upgrade process.
I update my containers with WATCHTOWER.

I just removed the trillium container and installed it again via docker-compose.
After I did this the trillium container shows as healthy again. So it seems something is "cached" during an upgrade with Watchtower.

[jcvara]

I am using the images directly from dockerhub, and I upgrade manually by recreating the container, pulling the latest image. I manage my containers with Portainer, tho, and will try deleting and creating it again from scratch and see what the result is in a few hours, will edit with the results.

[ghost]

@jcvara Did it work for you? I am having the same issue. I am running Portainer on Debian and after updating several containers I am getting a health alert on the Trilium container.

sh: can't open 'DockerHealthcheck.sh': No such file or directory

I have removed the container (left volume intact to reuse) and reimplemented it by pulling down a fresh image from dockerhub. The alert is still present and indicating that the container is not healthy. I am able to access the container with no issue on local or external networks (npm by jlesage if that matters) and am about ready chaulk it up to a false positive that I can mute, but I would like to get it out of my logs if possible.

[jcvara]

@Dauntless-One Unfortunately life's caught up with me something fierce and I haven't had the time yet to run some more tests, but Trilium is indeed working correctly, and the healthcheck failure is a false positive, so I believe it can be safely ignored.

If you'd like to get it out of your logs you might want to take a look at the workaround I posted in this comment to temporarily bypass the issue until it's resolved.

[adidgit]

It has something to do with the upgrade process. I update my containers with WATCHTOWER.

I just removed the trillium container and installed it again via docker-compose. After I did this the trillium container shows as healthy again. So it seems something is "cached" during an upgrade with Watchtower.

That worked for me. Thanks!

Before i was getting:

 docker inspect trilium --format "{{json .State.Health }}"
{"Status":"unhealthy","FailingStreak":22354,"Log":[{"Start":"2023-03-27T09:07:52.934013695+03:00","End":"2023-03-27T09:07:53.114561524+03:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"},{"Start":"2023-03-27T09:08:23.120759478+03:00","End":"2023-03-27T09:08:23.316958797+03:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"},{"Start":"2023-03-27T09:08:53.323587151+03:00","End":"2023-03-27T09:08:53.492402216+03:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"},{"Start":"2023-03-27T09:09:23.499901561+03:00","End":"2023-03-27T09:09:23.679616848+03:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"},{"Start":"2023-03-27T09:09:53.686295068+03:00","End":"2023-03-27T09:09:53.860301458+03:00","ExitCode":2,"Output":"sh: can't open 'DockerHealthcheck.sh': No such file or directory\n"}]}

After I redeployed stack all was ok:

 docker inspect trilium --format "{{json .State.Health }}"
{"Status":"healthy","FailingStreak":0,"Log":[{"Start":"2023-03-27T09:16:01.995733214+03:00","End":"2023-03-27T09:16:02.283028077+03:00","ExitCode":0,"Output":"STATUS: 200\n"},{"Start":"2023-03-27T09:16:32.28992297+03:00","End":"2023-03-27T09:16:32.600708838+03:00","ExitCode":0,"Output":"STATUS: 200\n"},{"Start":"2023-03-27T09:17:02.607217557+03:00","End":"2023-03-27T09:17:02.961354871+03:00","ExitCode":0,"Output":"STATUS: 200\n"},{"Start":"2023-03-27T09:17:32.967317237+03:00","End":"2023-03-27T09:17:33.240452386+03:00","ExitCode":0,"Output":"STATUS: 200\n"},{"Start":"2023-03-27T09:18:03.246793492+03:00","End":"2023-03-27T09:18:03.536929354+03:00","ExitCode":0,"Output":"STATUS: 200\n"}]}

[d03j]

I am having the same problem with 60.4 (unhealthy container but otherwise working normally).

Log extract:

00:29:35.990 204 POST /api/sync/finished with 0 bytes took 5ms 00:29:37.036 Slow query took 166ms: SELECT entityName, entityId, hash FROM entity_changes WHERE isSynced = 1 A ND entityName != 'note_reordering' 00:29:37.123 Content hash computation took 253ms 00:29:37.129 Slow 304 GET /api/sync/check with 13633 bytes took 481ms 00:29:39.968 Table counts: notes: 376, note_revisions: 656, branches: 379, attributes: 925, etapi_tokens: 0 00:29:40.072 All consistency checks passed with no errors detected (took 105ms) 00:40:19.047 Slow query took 237ms: SELECT value FROM options WHERE name = 'initialized' 00:40:19.073 DB size: 9564 KB 00:40:44.764 Trusted reverse proxy: false 00:40:45.024 App HTTPS server starting up at port 8080 00:40:45.026 { "appVersion": "0.60.4", "dbVersion": 214, "syncVersion": 29, "buildDate": "2023-06-19T23:26:50+02:00", "buildRevision": "5905950c17791ce0eb278e010c2c8b3450fdb447", "dataDirectory": "/home/node/trilium-data", "clipperProtocolVersion": "1.0", "utcDateTime": "2023-07-25T00:40:18.794Z" } 00:40:45.052 CPU model: Intel(R) Atom(TM) CPU 330 @ 1.60GHz, logical cores: 4 freq: 1599 Mhz 00:40:45.221 Listening on port 8080 00:40:45.582 Slow query took 323ms: SELECT noteId, title, type, mime, isProtected, dateCreated, dateModified, utcDateCreated, utcDateModified FROM notes WHERE isDeleted = 0 00:40:45.789 Slow query took 86ms: SELECT branchId, noteId, parentNoteId, prefix, notePosition, isExpanded, utcDateModified FROM branches WHERE isDeleted = 0 00:40:46.015 Slow query took 172ms: SELECT attributeId, noteId, type, name, value, isInheritable, position, utcDateModified FROM attributes WHERE isDeleted = 0 00:40:46.144 Slow query took 21ms: SELECT name, value, isSynced, utcDateModified FROM options 00:40:46.178 Becca (note cache) load took 922ms 00:40:50.331 Slow query took 34ms: SELECT COUNT(1) FROM notes 00:40:50.402 Slow query took 64ms: SELECT COUNT(1) FROM note_revisions 00:40:50.452 Slow query took 45ms: SELECT COUNT(1) FROM attributes 00:40:50.455 Table counts: notes: 376, note_revisions: 656, branches: 379, attributes: 925, etapi_tokens: 0 00:40:50.566 Slow query took 106ms: SELECT noteId as entityId FROM notes LEFT JOIN entity_changes ON entity_ch anges.entityName = 'notes' AND entity_changes.entityId = noteId WHERE entity_changes.id IS NULL 00:40:51.129 Slow query took 562ms: SELECT id, entityId FROM entity_changes LEFT JOIN notes ON entityId = note Id WHERE entity_changes.isErased = 0 AND entity_changes.entityName = 'notes' AND noteId IS NULL 00:40:51.179 Slow query took 44ms: SELECT noteId as entityId FROM note_contents LEFT JOIN entity_changes ON en tity_changes.entityName = 'note_contents' AND entity_changes.entityId = noteId WHERE entity_changes.id IS NULL 00:40:51.237 Slow query took 58ms: SELECT id, entityId FROM entity_changes LEFT JOIN note_contents ON entityId = noteId WHERE entity_changes.isErased = 0 AND entity_changes.entityName = 'note_contents' AND noteId IS NULL 00:40:51.310 Slow query took 35ms: SELECT noteRevisionId as entityId FROM note_revision_contents LEFT JOIN entity_changes ON entity_changes.entityName = 'note_revision_contents' AND entity_changes.entityId = noteRevisionId WHERE entity_changes.id IS NULL 00:40:51.438 Slow query took 111ms: SELECT branchId as entityId FROM branches LEFT JOIN entity_changes ON entity_changes.entityName = 'branches' AND entity_changes.entityId = branchId WHERE entity_changes.id IS NULL 00:40:51.463 Slow query took 24ms: SELECT id, entityId FROM entity_changes LEFT JOIN branches ON entityId = branchId WHERE entity_changes.isErased = 0 AND entity_changes.entityName = 'branches' AND branchId IS NULL 00:40:51.583 Slow query took 114ms: SELECT attributeId as entityId FROM attributes LEFT JOIN entity_changes ON entity_changes.entityName = 'attributes' AND entity_changes.entityId = attributeId WHERE entity_changes.id IS NULL 00:40:51.713 Slow query took 129ms: SELECT id, entityId FROM entity_changes LEFT JOIN attributes ON entityId = attributeId WHERE entity_changes.isErased = 0 AND entity_changes.entityName = 'attributes' AND attributeId IS NULL 00:40:51.861 Slow query took 135ms: SELECT name as entityId FROM options LEFT JOIN entity_changes ON entity_changes.entityName = 'options' AND entity_changes.entityId = name WHERE entity_changes.id IS NULL 00:40:51.915 Slow query took 25ms: SELECT DISTINCT notes.noteId FROM notes LEFT JOIN branches ON notes.noteId = branches.noteId AND branches.isDeleted = 0 WHERE notes.isDeleted = 0 AND branches.branchId IS NULL 00:40:51.952 Slow query took 22ms: SELECT notes.noteId, notes.type, notes.mime FROM notes JOIN note_contents USING (noteId) WHERE isDeleted = 0 AND isProtected = 0 AND content IS NULL 00:40:52.069 Slow query took 98ms: SELECT DISTINCT name FROM attributes 00:40:52.114 All consistency checks passed with no errors detected (took 1819ms) 00:44:08.244 Slow 304 GET /api/setup/status with 59 bytes took 117ms 00:44:09.555 Slow 200 POST /api/login/sync with 55 bytes took 56ms 00:44:11.085 Slow query took 81ms: INSERT OR REPLACE INTO note_revisions ( noteRevisionId, noteId, type, mime, title, isProtected, utcDateLastEdited, utcDateCreated, utcDateModified, dateLastEdited, dateCreated ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) 00:44:11.109 Slow query took 22ms: INSERT OR REPLACE INTO entity_changes ( entityName, entityId, hash, isErased, changeId, componentId, instanceId, isSynced, utcDateChanged ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) 00:44:11.170 Slow query took 37ms: INSERT OR REPLACE INTO entity_changes ( entityName, entityId, hash, isErased, changeId, componentId, instanceId, isSynced, utcDateChanged ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) 00:44:11.214 Slow query took 27ms: INSERT OR REPLACE INTO entity_changes ( entityName, entityId, hash, isErased, changeId, componentId, instanceId, isSynced, utcDateChanged ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
docker inspect:

docker inspect trilium-notes --format "{{json .State.Health }}" {"Status":"unhealthy","FailingStreak":4066,"Log":[{"Start":"2023-07-26T21:57:42.569536947+10:00","End":"2023-07-26T21:57:43.827862779+10:00","ExitCode":1,"Output":"ERROR\n"},{"Start":"2023-07-26T21:58:14.021484917+10:00","End":"2023-07-26T21:58:15.264360583+10:00","ExitCode":1,"Output":"ERROR\n"},{"Start":"2023-07-26T21:58:45.363658513+10:00","End":"2023-07-26T21:58:46.62338893+10:00","ExitCode":1,"Output":"ERROR\n"},{"Start":"2023-07-26T21:59:16.795939943+10:00","End":"2023-07-26T21:59:18.013492642+10:00","ExitCode":1,"Output":"ERROR\n"},{"Start":"2023-07-26T21:59:48.147555692+10:00","End":"2023-07-26T21:59:49.532985576+10:00","ExitCode":1,"Output":"ERROR\n"}]}