Use file loader without local cache in the upload server #755
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
csegarragonz
commented
May 23, 2023
| @@ -221,6 +221,76 @@ TEST_CASE_METHOD(UploadTestFixture, "Test upload and download", "[upload]") | |||
| } | |||
| } | |||
|
|
|||
| TEST_CASE_METHOD(UploadTestFixture, | |||
There was a problem hiding this comment.
This test fails against the current main branch
csegarragonz
changed the title
csegarragonz
force-pushed
the
upload-no-cache
branch
4 times, most recently
from
7a62cc5
to
2420adb
Compare
csegarragonz
force-pushed
the
upload-no-cache
branch
from
14597fe
to
d80851b
Compare
csegarragonz
commented
May 30, 2023
| @@ -49,17 +55,17 @@ std::vector<uint8_t> MachineCodeGenerator::hashBytes( | |||
| } | |||
|
|
|||
| std::vector<uint8_t> MachineCodeGenerator::doCodegen( | |||
| std::vector<uint8_t>& bytes, | |||
| const std::string& fileName, | |||
There was a problem hiding this comment.
These two arguments in the argument list were not used, so I remove them.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The behaviour of the upload server should be that, irrespective of whether the
user/functionhas been uploaded before, the WASM bytes and the machine code are re-generated and uploaded to S3.To achieve this goal, at some point in the past we introduced the file loader without local cache. As a reminder, the default behaviour of the file loader is to cache files in the local filesystem.
However, we were only using the file loader without cache to upload the WASM bytes, but not to generate the machine code. A
MachineCodeGeneratorinstance takes aFileLoaderin its constructor which, before, was configured to use the local cache.This means that, even though the WASM file was uploaded, the generated machine code (which was later on uploaded to S3) corresponded to a potentially old WASM file. This was causing issues and making the
quick-starttests fail. (Why this was not caught in the GHA is a different story that I cover in #746).In this PR I fix this issue by explicitly giving
getMachineCodeGeneratoraFileLoader&and add a regression test. I also update theFunctionLoaderTestFixtureto run upload tests for different WASM VMs.