Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use disk cache to run wd tests #1970

Merged
merged 5 commits into from
May 10, 2024
Merged

Use disk cache to run wd tests #1970

merged 5 commits into from
May 10, 2024

Conversation

garrettgu10
Copy link
Collaborator

This PR does a few things:

  1. It updates the pyodide bucket to one that includes a file called all_wheels.zip in its GitHub release.
  2. It uses this file to create a bazel repo that contains all the wheels
  3. It passes a path to the bazel repo as --disk-cache-dir when the test is being run.

Open questions:

  1. Does this work on Windows
  2. Does this break tests on edgeworker somehow

Why not just download the wheels from R2? The main reason is that the R2 bucket is not accessible from our build infrastructure.

@garrettgu10 garrettgu10 requested review from a team as code owners April 4, 2024 20:58
@dom96
Copy link
Collaborator

dom96 commented Apr 5, 2024

The main reason is that the R2 bucket is not accessible from our build infrastructure.

Why is that? Can't we make the R2 bucket accessible from there?

WORKSPACE Show resolved Hide resolved
build/pyodide_bucket.bzl Outdated Show resolved Hide resolved
samples/pyodide-langchain/worker.py Outdated Show resolved Hide resolved
@garrettgu10 garrettgu10 force-pushed the ggu/disk-cache-tests branch 2 times, most recently from 5de9c3d to b1d765c Compare April 16, 2024 17:08
@garrettgu10 garrettgu10 force-pushed the ggu/disk-cache-tests branch 3 times, most recently from f52e195 to e0ef78b Compare May 3, 2024 17:28
garrettgu10 and others added 3 commits May 7, 2024 11:41
Comment on lines 9 to 10
(name = "aiohttp", pythonRequirement = "aiohttp"),
(name = "ssl", pythonRequirement = "ssl"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(name = "aiohttp", pythonRequirement = "aiohttp"),
(name = "ssl", pythonRequirement = "ssl"),

Copy link
Contributor

@hoodmane hoodmane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @garrettgu10!

@hoodmane hoodmane merged commit 33dce06 into main May 10, 2024
10 checks passed
@hoodmane hoodmane deleted the ggu/disk-cache-tests branch May 10, 2024 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants