panic at sqlx-postgres

[oluceps]

May 07 10:52:12 hastur systemd[1]: Started atticd.service.
May 07 10:52:12 hastur atticd[2380750]: Attic Server 0.1.0 (release)
May 07 10:52:12 hastur atticd[2380750]: Running migrations...
May 07 10:52:12 hastur atticd[2380750]: Starting API server...
May 07 10:52:12 hastur atticd[2380750]: Listening on [::]:8083...
May 07 10:52:25 hastur atticd[2380750]: thread 'main' panicked at /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/sqlx-postgres-0.7.3/src/message/parse.rs:31:13:
May 07 10:52:25 hastur atticd[2380750]: assertion failed: self.param_types.len() <= (u16::MAX as usize)
May 07 10:52:25 hastur atticd[2380750]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
May 07 10:52:26 hastur systemd[1]: atticd.service: Main process exited, code=exited, status=101/n/a
May 07 10:52:26 hastur systemd[1]: atticd.service: Failed with result 'exit-code'.
May 07 10:52:26 hastur systemd[1]: atticd.service: Consumed 18.979s CPU time, received 48.8M IP traffic, sent 33.0M IP traffic.
May 07 10:52:36 hastur systemd[1]: atticd.service: Scheduled restart job, restart counter is at 8.

Postgresql db runs well, sometimes attic cache works but mostly reported 502 bad gateway.

atticd service turns into loaded few secs after restart.

● atticd.service
     Loaded: loaded (/etc/systemd/system/atticd.service; enabled; preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Tue 2024-05-07 10:54:04 UTC; 5s ago
    Process: 2381688 ExecStart=/nix/store/6k266xrd64mmasw6qipiyyspzlcw2v7s-attic-server-0.1.0/bin/atticd >
   Main PID: 2381688 (code=exited, status=101)
         IP: 49.0M in, 33.1M out
         IO: 0B read, 0B written
        CPU: 19.550s

[mweinelt]

Seeing the same issue, here is the full backtrace. Happens on 4dbdbee.

Attic is using a local PostgreSQL instance via UDS:

[database]
url = "postgresql:///atticd?host=/run/postgresql"

Attic Server 0.1.0 (release)
Running migrations...
Starting API server...
Listening on [::1]:3080...
thread 'main' panicked at /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/sqlx-postgres-0.7.3/src/message/parse.rs:31:13:
assertion failed: self.param_types.len() <= (u16::MAX as usize)
stack backtrace:
   0:     0xaaaac160d890 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h75dbf07dc864c0bd
   1:     0xaaaac12ea4e4 - core::fmt::write::h0c7806eb64ac0461
   2:     0xaaaac15fa118 - std::io::Write::write_fmt::ha53e993f45f43425
   3:     0xaaaac160e3f0 - std::sys_common::backtrace::print::hb1055ea5fa282e57
   4:     0xaaaac160dfe0 - std::panicking::default_hook::{{closure}}::h9bc2adbd2b89efe9
   5:     0xaaaac160eae8 - std::panicking::rust_panic_with_hook::h976eed390f682de4
   6:     0xaaaac160e55c - std::panicking::begin_panic_handler::{{closure}}::h98a1c1b557857ce3
   7:     0xaaaac160e4f0 - std::sys_common::backtrace::__rust_end_short_backtrace::h70cf8b31899cdba4
   8:     0xaaaac160e4e8 - rust_begin_unwind
   9:     0xaaaac0b18488 - core::panicking::panic_fmt::h0971f4e715269fcb
  10:     0xaaaac0b185f8 - core::panicking::panic::hc12375156bef5000
  11:     0xaaaac159ed98 - <sqlx_postgres::message::parse::Parse as sqlx_core::io::encode::Encode>::encode_with::h79ba913a2712567e
  12:     0xaaaac14c9548 - sqlx_postgres::connection::executor::<impl sqlx_postgres::connection::PgConnection>::run::{{closure}}::h56770bc5a81e6fe1
  13:     0xaaaac14c7864 - sqlx_postgres::connection::executor::<impl sqlx_core::executor::Executor for &mut sqlx_postgres::connection::PgConnection>::fetch_many::{{closure}}::{{closure}}::h51ff32aaa35b017b
  14:     0xaaaac14c7218 - <sqlx_core::ext::async_stream::TryAsyncStream<T> as futures_core::stream::Stream>::poll_next::hb507818a0face454
  15:     0xaaaac14f9cc4 - <futures_util::stream::try_stream::try_filter_map::TryFilterMap<St,Fut,F> as futures_core::stream::Stream>::poll_next::hd0f4c94ba45b9f7f
  16:     0xaaaac14f9b54 - <futures_util::stream::try_stream::try_collect::TryCollect<St,C> as core::future::future::Future>::poll::h30c21c196cca80a0
  17:     0xaaaac14f9928 - sqlx_core::query::Query<DB,A>::execute::{{closure}}::h5a5e5140afb99b25
  18:     0xaaaac14efbec - sea_orm::driver::sqlx_postgres::SqlxPostgresPoolConnection::execute::{{closure}}::{{closure}}::h2f372ffabb662e09
  19:     0xaaaac14ecc04 - <sea_orm::database::db_connection::DatabaseConnection as sea_orm::database::connection::ConnectionTrait>::execute::{{closure}}::{{closure}}::had193f554662a7fc
  20:     0xaaaac14ec110 - <sea_orm::database::db_connection::DatabaseConnection as sea_orm::database::connection::ConnectionTrait>::execute::{{closure}}::hde3d828308cf0005
  21:     0xaaaac0ccc814 - sea_orm::executor::delete::exec_delete_only::{{closure}}::h2f11eeae8aed11f1
  22:     0xaaaac0b89594 - attic_server::gc::run_reap_orphan_chunks::{{closure}}::{{closure}}::hb67183c1c24e84c3
  23:     0xaaaac0b8f6e4 - attic_server::gc::run_garbage_collection_once::{{closure}}::{{closure}}::h89dc1c913277d145
  24:     0xaaaac0b8e7c0 - attic_server::gc::run_garbage_collection_once::{{closure}}::h201d39b472dc0c67
  25:     0xaaaac0b890d0 - attic_server::gc::run_garbage_collection::{{closure}}::hf384f18fd405ecc2
  26:     0xaaaac0d09f18 - atticd::main::{{closure}}::hdd9d907e342fb490
  27:     0xaaaac0cd7ef8 - <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h01cc9cc1e4ddc5a5
  28:     0xaaaac0c8aff0 - tokio::runtime::park::CachedParkThread::block_on::h28bc328c6d984544
  29:     0xaaaac0d07b8c - atticd::main::h2c0e4f4e02060529
  30:     0xaaaac0c082f8 - std::sys_common::backtrace::__rust_begin_short_backtrace::h2372091c7ad13400
  31:     0xaaaac0d1362c - main
  32:     0xffff94f4c500 - __libc_start_call_main
  33:     0xffff94f4c5d8 - __libc_start_main@@GLIBC_2.34
  34:     0xaaaac0b32730 - _start
  35:                0x0 - <unknown>

[cole-h]

Sounds similar to #115 (which has a PR attempting to fix it: #116)

[mweinelt]

Thanks, Cole! I applied it and will report back if it fixes the issue for me.

[mweinelt]

Has been stable for 19 hours, I think the fix is good.

[girlbossceo]

Ran into this issue too with postgresql via UNIX socket + S3, unfortunately destroyed my caches before stumbling upon this issue and the PR fix. I'm going to safely assume the PR would have fixed this though.

url = "postgresql://attic:<password>@%2Frun%2Fpostgresql/attic"

Attic Server 0.1.0 (release)
thread 'main' panicked at /opt/attic/.cargo/registry/src/index.crates.io-6f17d22bba15001f/sqlx-postgres-0.7.3/src/message/parse.rs:31:13:
assertion failed: self.param_types.len() <= (u16::MAX as usize)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[cole-h]

#116 was merged, so I believe this can be closed now.