Revamp e2e test suite to isolate each configuration

[jviotti]

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

[github-actions]

Benchmark Index (community)

Details

Benchmark suite	Current: c237d4a	Previous: 8a21e62	Ratio
Add one schema (0 existing)	46 ms	43 ms	1.07
Add one schema (100 existing)	456 ms	405 ms	1.13
Add one schema (1000 existing)	4337 ms	4108 ms	1.06

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

Benchmark Index (enterprise)

Details

Benchmark suite	Current: c237d4a	Previous: 8a21e62	Ratio
Add one schema (0 existing)	42 ms	55 ms	0.76
Add one schema (100 existing)	422 ms	454 ms	0.93
Add one schema (1000 existing)	3922 ms	4242 ms	0.92

This comment was automatically generated by workflow using github-action-benchmark.

[cubic-dev-ai]

3 issues found across 285 files

Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="test/e2e/headless/hurl/list.hurl">

<violation number="1" location="test/e2e/headless/hurl/list.hurl:130">
P2: Incorrect JSONPath index repeats entry 0 and skips validating `entries[1].baseDialect`.</violation>
</file>

<file name="test/e2e/common.mk">

<violation number="1" location="test/e2e/common.mk:21">
P1: Ensure `down` always runs even when tests fail; current `all` flow skips cleanup on failure.</violation>
</file>

<file name=".github/actions/e2e/action.yml">

<violation number="1" location=".github/actions/e2e/action.yml:20">
P1: Action inputs are interpolated into a shell command without validation, enabling command injection/path traversal via `suite` or `edition` values.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[augmentcode]

🤖 Augment PR Summary

Summary: Revamps the E2E suite so each server configuration runs in its own self-contained Docker build/compose setup, reducing cross-suite coupling.

Changes:

• Replaces the monolithic sandbox composite action with a parameterized e2e composite action.
• Updates CI to run separate E2E suites for empty, headless, html, chaos, and an enterprise-only suite.
• Introduces per-suite folders under test/e2e/ with their own Dockerfile, compose.yml, and test assets.
• Adds a shared test/e2e/common.mk harness to standardize compose lifecycle + Hurl/Playwright execution.
• Reworks root Makefile targets to align with the new suite layout and simplify local execution.
• Moves Playwright UI tests under the HTML suite and reuses them for the enterprise suite.

Technical Notes: Suites build an image derived from one that pre-indexes a suite-specific one.json; Hurl is executed in CI via a pinned container image for consistent tooling.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 2 suggestions posted.

Comment augment review to trigger a new review at any time.

[jviotti]

TODO:

• Somehow unify enterprise and html a bit more
• See if we can move enterprise specific tests to ./enterprise