Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Request failed with status code 502 #601

Closed
ajoshuasmith opened this issue Sep 7, 2022 · 25 comments
Closed

Request failed with status code 502 #601

ajoshuasmith opened this issue Sep 7, 2022 · 25 comments
Labels
bug Something isn't working needs triage Bug that needs triage from maintainer

Comments

@ajoshuasmith
Copy link

I upgraded to the latest version v1.27.0_37-dev from v1.26.0_36-dev and I received this error when accessing the GUI.

Any potential troubleshooting steps that I can follow?

Error code 500
Request failed with status code 502
Verbose

Request failed with status code 502,AxiosError,ERR_BAD_RESPONSE,[object Object],[object XMLHttpRequest],[object Object]
@ajoshuasmith ajoshuasmith added bug Something isn't working needs triage Bug that needs triage from maintainer labels Sep 7, 2022
@bo0tzz
Copy link
Member

bo0tzz commented Sep 7, 2022

Try clearing your cookies/site data. I believe this may be a cached version of the frontend you're getting.

@ajoshuasmith
Copy link
Author

ajoshuasmith commented Sep 7, 2022 •
edited

I tried another browser and cleared

image

@alextran1502
Copy link
Contributor

Please provide the requested info so we can help you troubleshoot

@alextran1502
Copy link
Contributor

Can you also the server log and the microservices log?

@ajoshuasmith
Copy link
Author

Ubuntu 20.04 VM running via Unraid
docker-compose is not modified from what is provided.
Custom .env -
env.txt

Apologies for my ignorance - could you let me know where those logs would be located?

Thank you sir!

@alextran1502
Copy link
Contributor

You can do docker logs <container-name> :)

@ajoshuasmith
Copy link
Author

Now that is simple :D
microservices log.txt

server logs.txt

@alextran1502
Copy link
Contributor

Ok I will need the database logs as well. Those containers cannot connect to the database, I think there is a database migration issue.

Did you use any tag in your docker-compose file other than release?

@ajoshuasmith
Copy link
Author

I don't believe so - everything defaulted.

immich postgres.txt

@alextran1502
Copy link
Contributor

Hmm somehow you have no space left on the device

2022-09-07 10:43:40.469 UTC [1] FATAL: could not write lock file "postmaster.pid": No space left on device

@meichthys
Copy link

meichthys commented Sep 7, 2022 •
edited

I'm also seeing this when deploying via docker-compose:
image

Here's the logs in microservices:

[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [NestFactory] Starting Nest application...
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] DatabaseModule dependencies initialized +47ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] JwtModule dependencies initialized +0ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] ConfigModule dependencies initialized +7ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 7  - 09/07/2022, 9:40:04 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 7  - 09/07/2022, 9:40:09 PM   ERROR [TypeOrmModule] Unable to connect to the database. Retrying (1)...
Error: getaddrinfo EAI_AGAIN immich
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:71:26)
[Nest] 7  - 09/07/2022, 9:40:17 PM   ERROR [TypeOrmModule] Unable to connect to the database. Retrying (2)...
Error: getaddrinfo EAI_AGAIN immich
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:71:26)
[Nest] 7  - 09/07/2022, 9:40:25 PM   ERROR [TypeOrmModule] Unable to connect to the database. Retrying (3)...
Error: getaddrinfo EAI_AGAIN immich

and in postgress:

PostgreSQL init process complete; ready for start up.
2022-09-07 21:36:30.222 UTC [1] LOG:  starting PostgreSQL 14.5 (Debian 14.5-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-09-07 21:36:30.222 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2022-09-07 21:36:30.222 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2022-09-07 21:36:30.231 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-09-07 21:36:30.243 UTC [61] LOG:  database system was shut down at 2022-09-07 21:36:30 UTC
2022-09-07 21:36:30.250 UTC [1] LOG:  database system is ready to accept connections

@meichthys
Copy link

Is this line correct in the docker-compose? It seems to be pointing to the wrong image?

immich/docker/docker-compose.yml

Line 19 in 853a65a

image: altran1502/immich-server:release

@alextran1502
Copy link
Contributor

Is this line correct in the docker-compose? It seems to be pointing to the wrong image?

immich/docker/docker-compose.yml

Line 19 in 853a65a

image: altran1502/immich-server:release

It is the correct image. Have you tried to restart your whole VM?

@alextran1502
Copy link
Contributor

@meichthys Can you guys try to bring the containers down with docker-compose down then update docker-compose pull immich-web immich-server and finally docker-compose up again?

@ajoshuasmith
Copy link
Author

ajoshuasmith commented Sep 7, 2022 •
edited

Hmm somehow you have no space left on the device

2022-09-07 10:43:40.469 UTC [1] FATAL: could not write lock file "postmaster.pid": No space left on device

I am now seeing no space left on the device doing that pull for the new version. 150GB allocated to this VM with only this running off of it. It's looking like an issue with my VM.

*Photos are stored on a share elsewhere

image

df -h

image

@alextran1502
Copy link
Contributor

You are using 23G out of 23G of your root filesystem, you will need to expand your drive.
Screen Shot 2022-09-07 at 18 22 14

This might help, please proceed with caution https://askubuntu.com/questions/116351/increase-partition-size-on-which-ubuntu-is-installed

@alextran1502
Copy link
Contributor

@ItsMeAPizza you can do docker images and remove the image that has no tags with docker rmi <image-id> to save space. Using docker means that you occasionally have to perform docker system prune

@ajoshuasmith
Copy link
Author

I am more than likely just going to rebuild the VM. Is there a way to point immich to the data folder that it created? Not a big deal to me if it's not possible since I can just re-upload everything.

Immich Data
image

Appreciate all the assistance @alextran1502!

@alextran1502
Copy link
Contributor

You will need to bring the Postgres mounted volume over to preserve the data in the database as well

@meichthys
Copy link

meichthys commented Sep 8, 2022 •
edited

@meichthys Can you guys try to bring the containers down with docker-compose down then update docker-compose pull immich-web immich-server and finally docker-compose up again?

@alextran1502 I've tried repulsing the images without any difference. My guess is that it has something to do with the .eng file which I've loaded into portainer. I'm thinking that portainer is storing the environment variables in a different location than what the images are expecting.

With most docker projects I can just dump the environment variables into portainer with no issue, but something seems different here.

@bo0tzz
Copy link
Member

bo0tzz commented Sep 8, 2022

@meichthys, the service is trying to connect to your database on the dns name immich

Unable to connect to the database. Retrying (1)...
Error: getaddrinfo EAI_AGAIN immich

Are you sure that's correct? Make sure to check your .env file and container names.

@meichthys
Copy link

@bo0tzz Thanks a lot! That was it. My .eng had a different name than my docker compose did for the Postgres container. After fixing the .env and removing my Postgres volume, it's working now!

@alextran1502 I think we can close this now 👍

@bo0tzz bo0tzz closed this as completed Sep 8, 2022
@Malopieds
Copy link

Malopieds commented Sep 11, 2022 •
edited

Hi, I have the same issue on fresh install, on postgress logs I have:
2022-09-11 09:09:19.568 UTC [575] DETAIL: Role "postgres" does not exist.
and on microservices:

error: password authentication failed for user "postgres"
    at Parser.parseErrorMessage (/usr/src/app/node_modules/pg-protocol/dist/parser.js:287:98)
    at Parser.handlePacket (/usr/src/app/node_modules/pg-protocol/dist/parser.js:126:29)
    at Parser.parse (/usr/src/app/node_modules/pg-protocol/dist/parser.js:39:38)
    at Socket.<anonymous> (/usr/src/app/node_modules/pg-protocol/dist/index.js:11:42)
    at Socket.emit (node:events:527:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)

This happened on both custom install and one-step install, the problem isn't about space on disk, I tried clearing the cache/using other devices/browsers.
When connecting to http://:2283, i got
Request failed with status code 502,AxiosError,ERR_BAD_RESPONSE,[object Object],[object XMLHttpRequest],[object Object]

@bo0tzz
Copy link
Member

bo0tzz commented Sep 11, 2022

Try clearing the postgres data volume: docker-compose down -v (or, if you mounted it to a path, delete the folder manually). Then bring everything back up. That should recreate the database and user.

@Malopieds
Copy link

Thanks, it worked like a charm !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working needs triage Bug that needs triage from maintainer
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@meichthys @bo0tzz @alextran1502 @Malopieds @ajoshuasmith