Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows support #149

Open
retep998 opened this issue Oct 19, 2017 · 12 comments
Open

Windows support #149

retep998 opened this issue Oct 19, 2017 · 12 comments

Comments

@retep998
Copy link
Member

@retep998 retep998 commented Oct 19, 2017

There's a large number of crates out there which are Windows specific, so it is essential that crater be able to test those crates.

CLI usage on Windows

  • #267: Have Crater successfully build on windows
  • #275: Run cargo check on Appveyor
  • #280: Run cargo build on Appveyor
    • C dependencies needs to be downloaded in the Appveyor image
  • Implement and test all the functions in src/native/windows.rs
    • #332: It would be nice to isolate all platform-specific code into a native module
  • Create the Docker image used by Crater to test crates
  • Successfully execute a demo run on Windows
  • Run cargo test on Appveyor
    • The test suite needs to be working on Windows
  • Successfully execute a full run on Windows
    • This probably needs to be done on a server, full runs takes ages

Craterbot support for Windows

  • Add a way to categorize agents by platform
  • Add support for scheduling runs on multiple platforms at the same time
  • Add support for rendering multiple platforms together in the report
  • Ensure the agent works on Windows

Reference material

@aidanhs aidanhs mentioned this issue Oct 20, 2017
2 of 2 tasks complete
@aidanhs
Copy link
Member

@aidanhs aidanhs commented Oct 20, 2017

Some unordered thoughts:

  • there are a lot of assumptions of linux in Crater, like joining paths with format!("{}/{}"). I'm guilty of being lazy and doing this recently :( Not a big problem, 'just' a bunch of papercuts
  • isolation is essential so we need to use Docker (or similar). I have a lot of experience with Docker on Linux, but none yet on Windows (and it's not too easy for me to get access), so I have a some questions:
    • is Docker on Windows fast enough at starting containers to be able to do start a container for every operation (generate-lockfile,test) on crates?
    • what environment do Windows crates assume? I'm vaguely aware of nanoserver - is that 'full' enough? But what about libraries that create GUIs on Windows, are they supported? For Linux we install a ton of random dependencies to help out crates that may require random libraries - do we need similar on Windows, e.g. the MSVC toolchain or whatever?
    • are all the Docker features we use supported (mainly volumes off the top of my head)? Are there important differences to be aware of?
  • more generally, I develop on Windows about once every 3 months, and have a commensurate amount of experience with useful things like scripting - I know the basics of a Windows dev environment, but little more, so a committed volunteer with Windows experience may be necessary
    • relatedly, we have crater linux machines. I know a bit about Linux admin, but nothing about Windows

There's nothing insurmountable, but Linux issues are higher yield for me at the moment. I will keep this tracking issue in mind though to at least try and stop the gap from getting worse.

It's clearly important to make sure we get to Windows at some point, but I don't know how long it's going to take - any help on this front would be much appreciated!

@retep998
Copy link
Member Author

@retep998 retep998 commented Oct 20, 2017

nanoserver would not work. It is so extremely stripped down that I have my doubts over whether even libstd would work.

Windows Server 2016 has containers which use Docker. I don't know how fast they are but it is certainly something we could benchmark. There is plenty of documentation though so anyone who wants to work on this can fairly easily learn what they need to know. https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/

As long as we have VC++ installed and the environment configured using vcvars, then that should take care of almost all crates which are capable of working.

@joshtriplett
Copy link
Member

@joshtriplett joshtriplett commented Jun 4, 2018

Following up on this: given how much we rely on crater to detect potential breakage, we really do need this. Would it make sense to try to seek out some help from an appropriate Windows developer? If someone popped up willing to do this work, would the crater developers have time to work with them to get this in place? If so, it might make sense to put out a call for help here.

@pietroalbini
Copy link
Member

@pietroalbini pietroalbini commented Jul 12, 2018

Added a bunch of tasks that needs to be done at the top of the issue. Appveyor with cargo check is also going to be up soon (#275) so at least we know Crater compiles on Windows.

Now that Craterbot is sort of running adding proper Windows support is going to be feasible, we "just" need to get an agent running on a Windows machine. Unfortunately I have almost no experience developing on Windows, but I'm happy to review PRs if someone wants to do the work.

@ecstatic-morse
Copy link
Contributor

@ecstatic-morse ecstatic-morse commented Feb 9, 2019

I've been doing a bit of work on this, so I thought I should post my progress here:

I've published a proof-of-concept Docker container which installs the requirements for rustc on Windows as well as vcpkg, a package manager for native code libraries. I learned some lessons while provisioning the host system (an Windows 10 Pro Azure VM), which are recorded in NOTES.md, perhaps these will be useful when provisioning a Windows machine for crater runs.

I've had some trouble building crater on the host , but was able to get far enough to encounter what I believe to be a rustup bug. Once this is resolved, I'll try get the minicrater runs passing with my changes to the native module.

There's a single failing unit test on Windows (it expects the result to be a path using "/" as a separator) which I will submit a fix for at some point :).

@pietroalbini
Copy link
Member

@pietroalbini pietroalbini commented Feb 9, 2019

Whoa, thank you a lot for working on this! ❤️

I've published a proof-of-concept Docker container which installs the requirements for rustc on Windows as well as vcpkg, a package manager for native code libraries.

Great! Can you send a PR to rust-lang/crates-build-env?

I learned some lessons while provisioning the host system (an Windows 10 Pro Azure VM), which are recorded in NOTES.md, perhaps these will be useful when provisioning a Windows machine for crater runs.

It would be awesome to get this into the docs on the repo.

@ecstatic-morse
Copy link
Contributor

@ecstatic-morse ecstatic-morse commented Feb 18, 2019

Another progress update:

The modifications in #399 along with the new docker image are enough to get the unit tests passing and minicrater working (modulo a bug where Docker on Windows does not report whether a container was killed due to an OOM condition).

I'm planning to do a Windows crater run on the top 100 to find some more bugs, as well as write an AppVeyor script in the next few days.

@ecstatic-morse
Copy link
Contributor

@ecstatic-morse ecstatic-morse commented Feb 20, 2019

@pietroalbini I thought I had seen an issue where you explained how to do a top 100 crater run, but I can't seem to find it now. Am I imagining things? Is this easy to do?

Also, the latest commit to #399 run the unit tests on AppVeyor. Once rust-lang/crates-build-env#7 is merged, I'll enable the minicrater tests. Unfortunately, I believe that actually running containers requires an AppVeyor machine with nested virtualization. One exists, but I don't know if it's available in the free tier.

@pietroalbini
Copy link
Member

@pietroalbini pietroalbini commented Feb 20, 2019

@pietroalbini I thought I had seen an issue where you explained how to do a top 100 crater run, but I can't seem to find it now. Am I imagining things? Is this easy to do?

Just pass --crate-select=top-100 instead of --crate-select=demo. small-random (20 random crates) also exists.

Also, the latest commit to #399 run the unit tests on AppVeyor. Once rust-lang/crates-build-env#7 is merged, I'll enable the minicrater tests.

🎉

Unfortunately, I believe that actually running containers requires an AppVeyor machine with nested virtualization. One exists, but I don't know if it's available in the free tier.

I'll try to figure if we can enable nested virtualization on our account, thanks for pointing this out.

bors added a commit that referenced this issue Jul 25, 2019
Support crater runs on Windows

Resolves #149.

This allows builds to run on [the newly added docker image](rust-lang/crates-build-env#5). In order to run successfully, you'll need to correctly [configure the Docker host](https://github.com/rust-lang/crater/blob/master/docs/agent-machine-setup-windows.md) ~~and `crater`'s `work` directory should have an ACL to enable full-access by the `"Authenticated Users"` group. This allows the container to write to the newly created `target` directories.~~

~~The changes in the first commit don't belong in the finished product: Windows' permissions are [fundamentally different than the POSIX ones](https://en.wikipedia.org/wiki/Access_control_list) so I need to think more about how to change the current API to be more cross-platform. However, `.exe`s are executable by default on Windows and user remapping isn't done within the container anyways, so my changes were enough to this PR up and running.~~

~~I couldn't get OpenSSL to link properly when building crater on Windows, despite setting `OPENSSL_DIR`, so I've temporarily reverted "switch from ring to openssl" to let me run tests. This should be un-reverted before this is merged.~~

~~The last commit points to a self-published version of the docker image, not a `rust-ops` published one. An official version can be published once I get [CI up and running](rust-lang/crates-build-env#5 (comment)) for the Windows image.~~

The concerns listed above have all been resolved. The ACL change seems to have been obsoleted by later versions of Docker on Windows.
@ecstatic-morse
Copy link
Contributor

@ecstatic-morse ecstatic-morse commented Jul 25, 2019

Now that #399 is merged, crater runs are possible on Windows. Work to build as many crates as possible by adding dependencies to crates-build-env-windows is ongoing. There's also a few outstanding issues with Docker for Windows that need to be worked around (see #426 and this feature request).

The next step is to make craterbot aware of alternate platforms. @pietroalbini I think this should go in a new issue? Both @rylev and I would need some mentoring instructions to implement this.

@pietroalbini
Copy link
Member

@pietroalbini pietroalbini commented Jul 28, 2019

Opened an issue with rough intructions to make craterbot aware of other platforms: #430

Ping me on that issue or Discord if y'all need mentoring.

@ecstatic-morse
Copy link
Contributor

@ecstatic-morse ecstatic-morse commented Sep 5, 2019

I opened a PR (#450) with an initial implementation of #430. It doesn't address scheduling for multiple platforms simultaneously or merging reports between platforms, however.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

5 participants