multiple tests turns my computer into an oven

[ekeren]

I tried this:

Use the following

bring postgres;

let db = new postgres.Database(name: "test", pgVersion: 15) as "RDS: Postgres";

test "main1" { }
test "main2" { }
test "main3" { }
test "main4" { }

wing run

This happened:

The result are 5 postgres docker containers (and any other thing I put there) even if I don't run the tests.

% docker ps
CONTAINER ID   IMAGE         COMMAND                  CREATED         STATUS         PORTS                                         NAMES
600609072e39   postgres:15   "docker-entrypoint.s…"   4 minutes ago   Up 4 minutes   0.0.0.0:33360->5432/tcp, :::33360->5432/tcp   wing-container-01HW83RGBS9TVN117XXZDMZ7A2
df44e5d9339b   postgres:15   "docker-entrypoint.s…"   4 minutes ago   Up 4 minutes   0.0.0.0:33359->5432/tcp, :::33359->5432/tcp   wing-container-01HW83RG3GBCGK6BCE17S4A69H
a6d8378ff6a9   postgres:15   "docker-entrypoint.s…"   4 minutes ago   Up 4 minutes   0.0.0.0:33358->5432/tcp, :::33358->5432/tcp   wing-container-01HW83RFTZ4N17ENJ6CT7RBW79
611dac5b8327   postgres:15   "docker-entrypoint.s…"   4 minutes ago   Up 4 minutes   0.0.0.0:33356->5432/tcp, :::33356->5432/tcp   wing-container-01HW83RFEG7RFTPQ1B4Q47FEVY
60f3f365d526   postgres:15   "docker-entrypoint.s…"   4 minutes ago   Up 4 minutes   0.0.0.0:33357->5432/tcp, :::33357->5432/tcp   wing-container-01HW83RFBFFA8VDAVZY2X33CP2

I expected this:

To only load these containers when needed

Is there a workaround?

No response

Anything else?

No response

Wing Version

0.70.27

Node.js Version

No response

Platform(s)

No response

Community Notes

• Please vote by adding a 👍 reaction to the issue to help us prioritize.
• If you are interested to work on this issue, please leave a comment.

[skyrpex]

The simulator (in test mode) creates every resource for every test, that's why there are 5 containers. It does for a reason: to be able to execute many tests in parallel, with isolated data. But... we might want to change that behavior since it's overkill in some situations (e.g. user only wants to execute one single test).

On the Console's perspective, #6876 might address this issue. We're experimenting with the console test runner: the test simulator is only going to start when a test is going to be run.

[skyrpex]

@Chriscbr what if the simulator didn't create multiple environments for every test, but they all shared the same environment? This has some benefits and enables some features:

• If the console wants to run tests in isolation, we can just create multiple simulators
• Doesn't seem useful, but we can run multiple tests using the same environment, if we ever wanted to
• Running a single test would be more efficient: the simulator will create NumberResources x 1 instead of NumberResources x NumberOfTests
• The simulator code will be simplified, I believe

[Chriscbr]

The result are 5 postgres docker containers (and any other thing I put there) even if I don't run the tests.

@skyrpex Is it possible for the Wing Console to avoid starting test-simulator until the user clicks "run tests" or "run" on a particular test?

what if the simulator didn't create multiple environments for every test, but they all shared the same environment?

If I'm understanding your idea, then when I run wing test in my terminal, then the Wing CLI would also have to do the same thing and it would have to create N simulators for my N tests (to ensure all tests are running isolated) and it would be responsible for pooling all of their logs/traces/results together.

I think the idea is starting to make sense to me after thinking on it for a while -- it might just be a bit of work to make sure simulators each get their own state directory and such. The only other issue I can think of is that now if two resources emit the same log/trace, the only difference would be that they're running on different simulators -- so every trace or log would also have to be associated with a simulator ID, which is a bit of extra bookkeeping to add everywhere. But it's probably doable.

I was thinking of how else we could solve your problem (where running 1 test in the Wing Console causes more resources to be simulated than needed) -- as an alternative, the simulator could be more tightly integrated with the testing system, and expose an API for running a single test (which would start the simulator, and somehow only start the resources that are needed(?)). But this direction sounds more complex / brittle if I had to be honest.

[skyrpex]

Is it possible for the Wing Console to avoid starting test-simulator until the user clicks "run tests" or "run" on a particular test?

Yes, the problem is that users are allowed to run/enqueue tests as they wish, and concurrency errors will arise. For example:

• User runs test 1
• While test 1 is running, user runs test 2
• Test 1 finishes, test 2 still running. User runs test 1 again
• Test 1 will then fail because we didn't reload the simulator state. We can't reload the simulator because it will mess up with test 2, which is still running

That's why we would very much prefer to handle multiple simulators from the Console. We can instantiate one simulator per running test, and stop it after the test was run.

[...] But this direction sounds more complex / brittle if I had to be honest.

Yeah, I think the simplest approach is to let the console and the wing cli to handle the creation of multiple simulators. It should be a straightforward process, though...

[monadabot]

Congrats! 🚀 This was released in Wing 0.79.15.