First end to end test

[stchris]

How to review this: run make e2e. You can run the visual debugger by running PWDEBUG=1 make e2e.

TODOS

• figure out the "disable host check" thing
• make CI work
• reduce the amount of events this gets triggered by
• perhaps parametrize the BASE_URL in CI
• have the test clean up after itself
• use authentication instead of single user mode
• have the tests run on separate (configurable) ElasticSearch & Postgres DBs
• prefix test-created resources with [e2e]
• fix the timestamp prefix when waiting for the ui container

[tillprochaska]

I added a few more comments below, but nothing important, feel free to ignore if you disagree :).

Also didn’t mention it before, but I tested this on my machine and make e2e did all the magic and it worked out of the box. (Although the test was failing, for the same reason it fails in CI).

One more suggestion, I often find it helpful to open a shell in the container then run commands directly in the container, because that sometimes makes it a faster to iterate, so instead of running make e2e repeatedly, when I’m actively working on tests, I’d prefer to run make e2e-shell (as a convenience), and then run make e2e-local in the container. Would it be ok if I added that to the Makefile?

[stchris]

@tillprochaska I was looking at adding the e2e-shell target, but the e2e container does not have make installed. Would you like me to add it?

Just for comparison: here's my workflow when doing iterations during writing a test: I bring up Aleph with make services and then run make e2e-local inside the aleph virtualenv. The big advantage is that you can use the visual debugger and codegen tools this way.

[tillprochaska]

I bring up Aleph with make services and then run make e2e-local inside the aleph virtualenv.

Just to make sure I’m not misunderstanding you: You run make e2e-local in the api container? Or directly on your host machine?

The big advantage is that you can use the visual debugger and codegen tools this way.

Yep, makes sense. I was going to look into doing this while running Playwright in Docker, but I assume this probably requires some fairly complex configuration, so probably not worth it.

Let’s just ignore my comment about this for now. I’ll try to figure out a workflow that works well for me first, and then I can always add something to the Makefile later :)

[stchris]

Just to make sure I’m not misunderstanding you: You run make e2e-local in the api container? Or directly on your host machine?

I'm running it on my host machine.

[tillprochaska]

Oh, just noticed one more thing. When running the tests locally, they write to my local dev database and ES index -- I think it would be good to use a test database and index here as well, so it can be deleted/reset easily. For our current Python tests, we override the settings here:

https://github.com/alephdata/aleph/blob/main/aleph/tests/util.py#L32
https://github.com/alephdata/aleph/blob/main/aleph/tests/util.py#L29

[stchris]

Oh, just noticed one more thing. When running the tests locally, they write to my local dev database and ES index -- I think it would be good to use a test database and index here as well, so it can be deleted/reset easily. For our current Python tests, we override the settings here:

https://github.com/alephdata/aleph/blob/main/aleph/tests/util.py#L32 https://github.com/alephdata/aleph/blob/main/aleph/tests/util.py#L29

I'll take this as an item into the discussion next week. To me it was practical at times to be able to open up my local Aleph installation and check what the tests have done, but having a clean env makes sense to me.

[Rosencrantz]

Might be good to have some cleanup in place for when the tests finish running?