Skip to content

Implements an e2e test infra#277

Merged
arcanis merged 6 commits intomainfrom
mael/e2e-tests
Apr 10, 2026
Merged

Implements an e2e test infra#277
arcanis merged 6 commits intomainfrom
mael/e2e-tests

Conversation

@arcanis
Copy link
Copy Markdown
Member

@arcanis arcanis commented Apr 10, 2026

Working on setting up full e2e tests for popular third-party packages.


Note

Medium Risk
Adds a new GitHub Actions E2E pipeline that runs arbitrary shell tests in a Docker sandbox and can auto-create/update GitHub issues on main, plus a small change to Python venv PYTHONPATH construction; misconfiguration could cause CI noise or incorrect env behavior.

Overview
Adds a new E2E GitHub Actions workflow that discovers tests/e2e/*.sh, builds and artifacts yarn-bin, runs each test in parallel inside a custom Docker action sandbox, uploads per-test logs/status artifacts, and on main automatically creates/updates e2e-failure GitHub issues with log tails.

Introduces local e2e tooling (scripts/e2e/run.sh and yarn run e2e) plus initial smoke and django e2e scripts. Updates zpm’s python command to include site-packages subdirectories in PYTHONPATH, and tweaks IslandDependencyProvider::fetch_dependencies to avoid holding a RefCell borrow across subsequent work.

Reviewed by Cursor Bugbot for commit 7363b59. Bugbot is set up for automated code reviews on this repo. Configure here.

}

entries.join(&separator.to_string())
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PYTHONPATH includes package subdirs, shadowing standard library

High Severity

build_site_packages_pythonpath adds every non-hidden subdirectory of site-packages to PYTHONPATH. Since PYTHONPATH entries are prepended to sys.path before the standard library, package directories like site-packages/django/ become top-level import sources. This causes Django's internal submodules (e.g., django/http/, django/test/) to shadow Python standard library modules of the same name. For the Django e2e test, import http would resolve to django/http/ instead of the stdlib http, breaking http.server used by Django's dev server.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 1317c7d. Configure here.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 10, 2026

⏱️ Benchmark Results

gatsby install-full-cold

Metric Base Head Difference
Mean 2.435s 2.455s +0.81% ⚠️
Median 2.431s 2.452s +0.86% ⚠️
Min 2.381s 2.406s
Max 2.512s 2.494s
Std Dev 0.029s 0.023s
📊 Raw benchmark data (gatsby install-full-cold)

Base times: 2.405s, 2.450s, 2.431s, 2.512s, 2.407s, 2.450s, 2.470s, 2.442s, 2.420s, 2.408s, 2.389s, 2.434s, 2.398s, 2.478s, 2.381s, 2.432s, 2.456s, 2.463s, 2.422s, 2.468s, 2.471s, 2.412s, 2.425s, 2.439s, 2.410s, 2.421s, 2.428s, 2.431s, 2.460s, 2.430s

Head times: 2.446s, 2.406s, 2.428s, 2.454s, 2.441s, 2.444s, 2.456s, 2.493s, 2.483s, 2.494s, 2.483s, 2.441s, 2.474s, 2.450s, 2.465s, 2.488s, 2.444s, 2.479s, 2.433s, 2.445s, 2.461s, 2.425s, 2.469s, 2.475s, 2.454s, 2.432s, 2.430s, 2.417s, 2.447s, 2.481s


gatsby install-cache-and-lock (warm, with lockfile)

Metric Base Head Difference
Mean 0.366s 0.370s +1.12% ⚠️
Median 0.369s 0.371s +0.67% ⚠️
Min 0.355s 0.355s
Max 0.381s 0.381s
Std Dev 0.008s 0.006s
📊 Raw benchmark data (gatsby install-cache-and-lock (warm, with lockfile))

Base times: 0.376s, 0.372s, 0.369s, 0.371s, 0.371s, 0.376s, 0.371s, 0.374s, 0.370s, 0.375s, 0.374s, 0.373s, 0.369s, 0.381s, 0.369s, 0.368s, 0.359s, 0.355s, 0.357s, 0.363s, 0.370s, 0.363s, 0.364s, 0.356s, 0.356s, 0.355s, 0.358s, 0.360s, 0.355s, 0.355s

Head times: 0.355s, 0.362s, 0.370s, 0.368s, 0.361s, 0.360s, 0.359s, 0.373s, 0.374s, 0.366s, 0.371s, 0.372s, 0.374s, 0.371s, 0.366s, 0.381s, 0.371s, 0.368s, 0.371s, 0.378s, 0.369s, 0.375s, 0.378s, 0.377s, 0.374s, 0.373s, 0.373s, 0.377s, 0.371s, 0.366s

};

if !file_type.is_dir() {
continue;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Symlinked packages excluded from PYTHONPATH construction

Medium Severity

build_site_packages_pythonpath uses entry.file_type() which does not follow symlinks — is_dir() returns false for symlinks to directories. The venv linker in venv.rs creates symlinks for PackageData::Local and Reference::Link packages via fs_symlink. Those symlinked entries under site-packages will be silently skipped and never added to PYTHONPATH, so Python won't be able to resolve imports from locally-linked packages whose importable name differs from the directory structure.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 92e9043. Configure here.

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 4 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1404947. Configure here.

@arcanis arcanis merged commit 2281b5c into main Apr 10, 2026
19 checks passed
@arcanis arcanis deleted the mael/e2e-tests branch April 10, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant