[Bug] 502 Bad Gateway error

[Triskae]

Describe the bug
After the 0.4.9 update, I cannot access to my apps. Since I have a 502 Bad Gateway error on the 2/app_management/web/appgrid endpoint.

A clear and concise description of what the bug is.

To Reproduce

Steps to reproduce the behavior:

1. Install the 0.4.9
2. See the error

Expected behavior

A clear and concise description of what you expected to happen.

Screenshots

If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

 - OS: [e.g. iOS] Ubuntu
 - Browser [e.g. chrome, safari] Chromium
 - Version [e.g. 22]: Chromium v123.0.6312.105

System Time

Run timedatectl and share the output

               Local time: mar. 2024-05-21 13:55:26 CEST
           Universal time: mar. 2024-05-21 11:55:26 UTC
                 RTC time: mar. 2024-05-21 11:55:26
                Time zone: Europe/Paris (CEST, +0200)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

Logs

Run following command to collect corresponding logs:

sudo journalctl -xef -u casaos-gateway

mai 21 13:55:11 astra-server casaos-gateway[51466]: 2024/05/21 13:55:11 http: proxy error: EOF
mai 21 13:55:16 astra-server casaos-gateway[51466]: 2024/05/21 13:55:16 http: proxy error: EOF
mai 21 13:55:21 astra-server casaos-gateway[51466]: 2024/05/21 13:55:21 http: proxy error: EOF
mai 21 13:55:26 astra-server casaos-gateway[51466]: 2024/05/21 13:55:26 http: proxy error: EOF
mai 21 13:55:32 astra-server casaos-gateway[51466]: 2024/05/21 13:55:32 http: proxy error: EOF
mai 21 13:55:37 astra-server casaos-gateway[51466]: 2024/05/21 13:55:37 http: proxy error: EOF


sudo journalctl -xef -u casaos-user-service

mai 21 13:52:36 astra-server casaos-user-service[50903]: [GIN] 2024/05/21 - 13:52:36 | 200 |     548.796µs |   192.168.1.163 | GET      "/v1/users/current"
mai 21 13:52:36 astra-server casaos-user-service[50903]: [GIN] 2024/05/21 - 13:52:36 | 200 |     509.225µs |   192.168.1.163 | GET      "/v1/users/current/custom/widgets_config"

sudo journalctl -xef -u casaos-local-storage

mai 21 13:52:28 astra-server casaos-local-storage[51531]: 2024-05-21T13:52:28.077+0200        error        failed to publish event to message bus        {"status": "404 Not Found", "responseError": "json: unsupported type: func() (io.ReadCloser, error)", "func": "service.(*notifyServer).SendNotify", "file": "/home/runner/work/CasaOS-LocalStorage/CasaOS-LocalStorage/service/notify.go", "line": 32}
mai 21 13:52:36 astra-server casaos-local-storage[51531]: 2024-05-21T13:52:36.103+0200        error        failed to publish event to message bus        {"status": "404 Not Found", "responseError": "json: unsupported type: func() (io.ReadCloser, error)", "func": "service.(*notifyServer).SendNotify", "file": "/home/runner/work/CasaOS-LocalStorage/CasaOS-LocalStorage/service/notify.go", "line": 32}


sudo journalctl -xef -u casaos-app-management

mai 21 13:55:58 astra-server casaos-app-management[51701]: net/http.(*conn).serve(0xc00170c900, {0x2f6fbc0, 0xc000e3dce0})
mai 21 13:55:58 astra-server casaos-app-management[51701]:         /opt/hostedtoolcache/go/1.21.9/x64/src/net/http/server.go:2009 +0x5f4
mai 21 13:55:58 astra-server casaos-app-management[51701]: created by net/http.(*Server).Serve in goroutine 1
mai 21 13:55:58 astra-server casaos-app-management[51701]:         /opt/hostedtoolcache/go/1.21.9/x64/src/net/http/server.go:3086 +0x5cb


sudo journalctl -xef -u casaos.service

mai 21 13:57:00 astra-server casaos[51727]: 消息来了，message：{"type":"ping"}
mai 21 13:57:03 astra-server casaos[51727]: /bin/bash -c source /usr/share/casaos/shell/helper.sh ;CatNetCardState enp2s0
mai 21 13:57:03 astra-server casaos[51727]: /bin/bash -c source /usr/share/casaos/shell/helper.sh ;CatNetCardState wlp3s0
mai 21 13:57:08 astra-server casaos[51727]: /bin/bash -c source /usr/share/casaos/shell/helper.sh ;CatNetCardState enp2s0
mai 21 13:57:08 astra-server casaos[51727]: /bin/bash -c source /usr/share/casaos/shell/helper.sh ;CatNetCardState wlp3s0

Additional context

Add any other context about the problem here.
I tried to reboot the system, I ran the install command, but all of this did not solve the issue.

[andreasringdal]

+1 with these symptoms

Attempted both system reboot and casaos user service restart
Stopped all running containers (except portainer). Result unchanged.
Related log files
journalctljournalctl -xeu casaos-app-management --no-pager.txt

Some highlites

May 30 22:58:27 docker3 casaos-app-management[5698]:         /opt/hostedtoolcache/go/1.21.9/x64/src/net/http/server.go:3086 +0x5cb
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.440+0200        error        failed to load compose file        {"error": "no configuration file provided: not found", "path": "/data/compose/3/docker-compose.yml", "func": "service.(*ComposeService).List", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/service/compose_service.go", "line": 184}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.492+0200        error        extension `x-casaos` not found        {"func": "service.(*App).StoreInfo", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/service/app.go", "line": 18}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.493+0200        error        extension `x-casaos` not found        {"func": "service.(*App).StoreInfo", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/service/app.go", "line": 18}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.494+0200        error        failed to get store info        {"error": "extension `x-casaos` not found", "composeAppID": "obico-server", "func": "v2.composeAppsWithStoreInfo.func1", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/route/v2/compose_app.go", "line": 645}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.495+0200        error        extension `x-casaos` not found        {"func": "service.(*App).StoreInfo", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/service/app.go", "line": 18}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.495+0200        error        extension `x-casaos` not found        {"func": "service.(*App).StoreInfo", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/service/app.go", "line": 18}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.500+0200        error        error while getting appstore compose app        {"error": "lstat /var/lib/casaos/appstore/7c98580ba862e556368981f3cd3afa87: no such file or directory", "func": "service.(*AppStoreManagement).ComposeApp", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/service/appstore_management.go", "line": 391}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.500+0200        info        app not found in any appstore        {"id": "postgresql", "func": "service.(*AppStoreManagement).ComposeApp", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/service/appstore_management.go", "line": 400}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024-05-30T22:58:28.500+0200        info        Using default appstore        {"func": "service.(*AppStoreManagement).ComposeApp", "file": "/home/runner/work/CasaOS-AppManagement/CasaOS-AppManagement/service/appstore_management.go", "line": 407}
May 30 22:58:28 docker3 casaos-app-management[5698]: 2024/05/30 22:58:28 http: panic serving 127.0.0.1:60842: runtime error: invalid memory address or nil pointer dereference
May 30 22:58:28 docker3 casaos-app-management[5698]: goroutine 35033 [running]:
May 30 22:58:28 docker3 casaos-app-management[5698]: net/http.(*conn).serve.func1()

[Triskae]

I'm afraid to uninstall and reinstall, I don't want to lose evrything ...

[Triskae]

Okay, tried to uninstall and reinstall, and same problem !!

[andreasringdal]

Other changes: the server (Debian) has updated docker to 26.1 roughly at the same time.
Some API features where removed from API in 26 and 25. I do not know if any of them are used by Casaos
https://docs.docker.com/engine/deprecated/

Status Feature Deprecated Remove
Deprecated Unauthenticated TCP connections v26.0 v27.0
Deprecated Container and ContainerConfig fields in Image inspect v25.0 v26.0
Deprecated Deprecate legacy API versions v25.0 v26.0
Removed Container short ID in network Aliases field v25.0 v26.0
Deprecated IsAutomated field, and "is-automated" filter on docker search v25.0 v26.0
Removed logentries logging driver v24.0 v25.0
Removed OOM-score adjust for the daemon v24.0 v25.0
Removed Buildkit build information v23.0 v24.0

[Mahoo12138]

Same here, how to rollback to the old version?

[Mahoo12138]

Same here, how to rollback to the old version?

I try to upgraded some services after uninstalling casaos that can resolve this issues :

casaos-uninstall
wget -qO- https://get.casaos.io > install.sh
vim install.sh
./install.sh

Here are the changes:

...
"${CASA_DOWNLOAD_DOMAIN}IceWhaleTech/CasaOS-Gateway/releases/download/v0.4.9-alpha2/linux-${TARGET_ARCH}-casaos-gateway-v0.4.9-alpha2.tar.gz"
...
"${CASA_DOWNLOAD_DOMAIN}IceWhaleTech/CasaOS-AppManagement/releases/download/v0.4.9-alpha7/linux-${TARGET_ARCH}-casaos-app-management-v0.4.9-alpha7.tar.gz"
...

[Triskae]

Any news ?
I'm moving to another solution, cause I'm unable to manage my containers ...

[andreasringdal]

@Triskae downgrading to 0.4.7 workded for me. Hopefully solved in future releases.
Adding portainer in addition to casaos is recommended for more advanced management of containers.

[Triskae]

Thanks a lot ! I switched to dockge ... maybe one day I'll switch back to CasaOS