Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove WriteFile and move it into CopyFile #204

Merged
merged 5 commits into from Dec 13, 2021
Merged

Remove WriteFile and move it into CopyFile #204

merged 5 commits into from Dec 13, 2021

Conversation

bolshoytoster
Copy link
Contributor

Should fix #195

  • It still uses filesytem.Split but just to get the directory name to create it if it doesn't exist.

  • It opens the file with os.OpenFile and set the file permissions there instead of after copying.

  • It removes the file if io.Copy returns an error.

  • It doesn't need error handling on toFile.Close as it only returns an error if it's called multiple times.

- I've removed the WriteFile and put it into CopyFile.

- It still uses filesytem.Split but just to get the directory name to create it if it doesn't exist.

- It opens the file with os.OpenFile and set the file permissions there instead of after copying.

- It removes the file if io.Copy returns an error.

- It doesn't need error handling on toFile.Close as it only returns an error if it's called multiple times.
@vercel
Copy link

vercel bot commented Dec 12, 2021

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/vercel/turbo-site/3R4mnhZPU4TScGGwjYhfqinmMHbb
✅ Preview: https://turbo-site-git-fork-bolshoytoster-main.vercel.sh

@jaredpalmer
Copy link
Contributor

This is roughly 2-3x slower on my benchmarks

@bolshoytoster
Copy link
Contributor Author

bolshoytoster commented Dec 13, 2021

@jaredpalmer

This is roughly 2-3x slower on my benchmarks

I'm not sure how it could be that much slower, it's practically the same apart from:

  • filepath.Split instead of path.Split - Maybe this is it, but I doubt it would affect the speed this much.
  • os.OpenFile instead of ioutil.TempFile - ioutil.TempFile is just a wrapper for os.OpenFile

It could be that this one removes the file if there's an error copying, wheras the old implementation didn't bother.

If you're on windows the old one could be running quicker due to the bug this is meant to fix.

This one also avoids the need for creating a temporary file and renaming it.

@jaredpalmer
Copy link
Contributor

Run ./turbow.sh run build test and see for yourself .

I will formally benchmark again tomorrow

@bolshoytoster
Copy link
Contributor Author

If I don't figure out what caused it it would probably be best to revert the commit and just replace path.Split with filepath.Split.

The only change is changing path.Split -> filepath.Split
@bolshoytoster
Copy link
Contributor Author

I’ve reverted merging CopyFile and WriteFile and the benchmarks are just about the same as the original.

@jaredpalmer jaredpalmer merged commit 06cea44 into vercel:main Dec 13, 2021
sokra pushed a commit that referenced this pull request Oct 25, 2022
This is a very early version of the next-dev test runner. I'm opening this early to get thoughts from folks re: the direction of the design and implementation.

Fixes #204 

Currently it:
* Discovers integration test fixtures from the filesystem. Right now these are expected to be single files that get bundled and will eventually include assertions. This is powered by the test-generator crate, which allows us not to have to manually enumerate each case. We could consider using this for the node-file-trace tests as well.
* Starts the dev server on a free port and opens a headless browser to its root. The browser control is implemented with the https://crates.io/crates/chromiumoxide crate, which expects Chrome or Chromium to already be available.

Eventually it will:
* [x] Implement a minimal test environment loaded in the browser so that assertions can be run there from bundled code.
* [x] Report back the results of these assertions to rust, where we can pass/fail cargo tests with those results.

In the future it could:
* Possibly include snapshot-style tests to assert on transformed results. This could be in the form of fixture directories instead of files cc @jridgewell
* Support expressing special configuration of turbopack in a fixture, possibly as another file in the fixture directory.
* [x] ~Possibly support distributing tests to a pool of open browsers instead of opening and closing for each test.~

Test Plan: See next PRs
sokra pushed a commit that referenced this pull request Oct 25, 2022
Co-authored-by: Henry Heffernan <henryheffernan@gmail.com>
jridgewell pushed a commit to vercel/next.js that referenced this pull request Mar 10, 2023
This is a very early version of the next-dev test runner. I'm opening this early to get thoughts from folks re: the direction of the design and implementation.

Fixes vercel/turbo#204 

Currently it:
* Discovers integration test fixtures from the filesystem. Right now these are expected to be single files that get bundled and will eventually include assertions. This is powered by the test-generator crate, which allows us not to have to manually enumerate each case. We could consider using this for the node-file-trace tests as well.
* Starts the dev server on a free port and opens a headless browser to its root. The browser control is implemented with the https://crates.io/crates/chromiumoxide crate, which expects Chrome or Chromium to already be available.

Eventually it will:
* [x] Implement a minimal test environment loaded in the browser so that assertions can be run there from bundled code.
* [x] Report back the results of these assertions to rust, where we can pass/fail cargo tests with those results.

In the future it could:
* Possibly include snapshot-style tests to assert on transformed results. This could be in the form of fixture directories instead of files cc @jridgewell
* Support expressing special configuration of turbopack in a fixture, possibly as another file in the fixture directory.
* [x] ~Possibly support distributing tests to a pool of open browsers instead of opening and closing for each test.~

Test Plan: See next PRs
sokra pushed a commit to vercel/next.js that referenced this pull request Mar 13, 2023
This is a very early version of the next-dev test runner. I'm opening this early to get thoughts from folks re: the direction of the design and implementation.

Fixes vercel/turbo#204 

Currently it:
* Discovers integration test fixtures from the filesystem. Right now these are expected to be single files that get bundled and will eventually include assertions. This is powered by the test-generator crate, which allows us not to have to manually enumerate each case. We could consider using this for the node-file-trace tests as well.
* Starts the dev server on a free port and opens a headless browser to its root. The browser control is implemented with the https://crates.io/crates/chromiumoxide crate, which expects Chrome or Chromium to already be available.

Eventually it will:
* [x] Implement a minimal test environment loaded in the browser so that assertions can be run there from bundled code.
* [x] Report back the results of these assertions to rust, where we can pass/fail cargo tests with those results.

In the future it could:
* Possibly include snapshot-style tests to assert on transformed results. This could be in the form of fixture directories instead of files cc @jridgewell
* Support expressing special configuration of turbopack in a fixture, possibly as another file in the fixture directory.
* [x] ~Possibly support distributing tests to a pool of open browsers instead of opening and closing for each test.~

Test Plan: See next PRs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants