Expected Behavior
Files from the same actor should not step on each other if the actor is re-invoked before the previous files have been cleaned up.
Actual Behavior
Unsure if concurrency of requests is the actual cause of this error, but plausible.
When load testing Substrate in a cpu-starved environment (especially a low-end machine), seeing errors like this:
rpc error: code = Internal desc = while resetting actor dirs: while deleting bundle dir: %w: unlinkat /var/lib/ateom-gvisor/actors/0c5207f6-5ffa-4d6b-9e9e-f9d2476a108c/bundles/glutton/rootfs/usr/share/zoneinfo/right: directory not empty
Steps to Reproduce the Problem
- Spin up substrate
- Create a lot of workers per-vCPU on a single node
- Run a glutton load test that 100% saturates these workers without oversubscribing (may need to tweak wait time between invocations)
Specifications
Expected Behavior
Files from the same actor should not step on each other if the actor is re-invoked before the previous files have been cleaned up.
Actual Behavior
Unsure if concurrency of requests is the actual cause of this error, but plausible.
When load testing Substrate in a cpu-starved environment (especially a low-end machine), seeing errors like this:
rpc error: code = Internal desc = while resetting actor dirs: while deleting bundle dir: %w: unlinkat /var/lib/ateom-gvisor/actors/0c5207f6-5ffa-4d6b-9e9e-f9d2476a108c/bundles/glutton/rootfs/usr/share/zoneinfo/right: directory not empty
Steps to Reproduce the Problem
Specifications