Odd pool timeout errors when using sqlx::test

[samtay]

I'm observing something really odd, but it requires a bit of context.

app context

I'm working on an API server at work (via the rocket framework) for which there are a lot of existing tests that have kind of a jerry-rigged strategy: each test creates a randomized postgres schema, sets the search path to the new schema, and runs migrations & tests within that schema (as a kind of crude isolation method). It's been around since before my time, and while it is quite clunky and awkward at times, for our nascent API server it mostly gets the job done and is pretty quick to execute.

worker threads

I'm working on creating a new binary, separate from the rocket app, that runs long lived worker threads. One of these runs a sqlx PgListener and performs tasks on incoming notifications. So my tests for this worker will start off with something like tokio::task::spawn(run_worker_forever()), and then perform various actions and ensure that the worker is behaving as expected. Furthermore since this worker will ultimately be allowed to run on more than one node at a time, I have a concurrency test that spawns more than one, and also runs multiple db actions concurrently (just as a way of testing that my worker doesn't drastically affect the web server, no deadlocks, behaves atomically, etc.).

oddity

I was excited to try out the new sqlx::test attribute that just landed, especially since the worker stuff is pretty separate from the rest of the application and existing test infrastructure, it seemed serendipitous to try it out. Mostly, it's going great. However when I run cargo test PATTERN to target just these new worker tests, they only pass about half of the time. The other half, I get these spurious pool timed out while waiting for an open connection errors.

The really odd thing is that these failures do not happen if I run all of the tests via cargo test. I've literally run them in a loop, printing exit codes, and left my computer for a while. cargo test consistently passes, and cargo test PATTERN fails half the time.

Can anyone think of a reason for this to happen? If not I can try to come up with a minimum reproducible example but it might be tough to replicate.