Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NixOS in Windows Store for Windows Subsystem for Linux (WSL) #30391

Open
ThomasMader opened this issue Oct 13, 2017 · 42 comments

Comments

Projects
None yet
@ThomasMader
Copy link
Contributor

commented Oct 13, 2017

Nix already runs in Ubuntu via WSL afaik.
I think it would make sense to add an official NixOS image to the Windows Store.
Arch Linux is trying already. (microsoft/WSL#2253)

Are there any showstoppers?

@bencoman

This comment has been minimized.

Copy link

commented Oct 15, 2017

This would be super cool. I'd like to move to using NixOS, but I can't give up my Windows machine, and I don't want to run NixOS in a virtual machine. What I'd really like to try doing is a continuous integration system using NixOS based build management to kick off compilation on the Windows side.

#28365 is an impediment, but drilling through the linked issue I saw an update yesterday from a MS staffer that they have been working on it.

Also I'm not sure if NixOS/nix#1203 is the same or different root issue.

@PierreR

This comment has been minimized.

Copy link
Contributor

commented Oct 25, 2017

By the way, I have a permssion denied just after the creation of /nix when I try to install nix on WSL
nix

Any idea ?

@jtojnar

This comment has been minimized.

Copy link
Contributor

commented Oct 25, 2017

@PierreR I also had this problem. I solved it by downloading the archive and copying the files to the directory manually.

@PierreR

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2017

@jtojnar Thanks ! Indeed if I run each problematic mv myself, it is going to work.

Any idea where the issue is coming from. I would like to streamline that process but it is going to be a bit cumbersome if such issue lingers on.

@PierreR

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2017

Just for the record I have hit microsoft/WSL#1671. Not only WSL seems to be unbearably slow, it would also fail strangely with this error:

error: setting synchronous mode: disk I/O error

My feeling is that WSL is too beta and adding an official NixOS image to the Windows Store might be premature.

@jtojnar

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2017

@PierreR disk I/O error is NixOS/nix#1203

@PierreR

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2017

@jtojnar yes it is. Thanks ! Everything works fine for me except for the sluggishness.

@CMCDragonkai

This comment has been minimized.

Copy link
Contributor

commented Nov 13, 2017

Does NixOS the distro itself require anything special beyond what Ubuntu as a distro requires? If not, then it should be possible to get NixOS (and not just nix) onto WSL!

@benley

This comment has been minimized.

Copy link
Member

commented Dec 28, 2017

NixOS the distro relies pretty heavily on systemd to run services. It would still be great to use nixos-rebuild for building the user environment and making software available, but without systemd it would be difficult to use any nixos modules that expect to run daemons or cronjobs.

@mdekstrand

This comment has been minimized.

Copy link
Contributor

commented Feb 10, 2018

NixOS the distro relies pretty heavily on systemd to run services. It would still be great to use nixos-rebuild for building the user environment and making software available, but without systemd it would be difficult to use any nixos modules that expect to run daemons or cronjobs.

Other distros will be in the same boat for any required services, but I can see this being a particular problem for Nix with the Nix Daemon.

Since WSL is intended to be a single-user environment, a NixOS that runs like single-user Nix could be what is needed to make it work.

@WSLUser

This comment has been minimized.

Copy link

commented Mar 6, 2018

I would love to see another distro in the Windows Store. It only took Kali a couple months from request to publishing into the Windows Store (which released 3/5/2017 for record). Note that you can manually run any services as most of the syscalls are supported though I'm not too familiar with NixOS to guess what specifics might not exist yet in WSL. So while systemd won't automatically run the Nix Daemon, it's entirely possibly you "may" be able to manually run it. Do an strace and place a bug on WSL if there are any errors specific to WSL. And since emphasis is on systemd running, see microsoft/WSL#534, microsoft/WSL#1579, microsoft/WSL#2530 and vote for this UserVoice

Also try to use the latest Insider as it will greatly help in discovering what issues may still exist. For instance, you're able to perform background tasks on Insider.

@ThomasMader

This comment has been minimized.

Copy link
Contributor Author

commented Mar 27, 2018

Microsoft open sourced a WSL Sample for Linux Distribution maintainers. Might be very helpful.
https://blogs.msdn.microsoft.com/commandline/2018/03/26/wsl-distro-launcher/

@WSLUser

This comment has been minimized.

Copy link

commented Apr 25, 2018

Any news in this front? I'd manually compile myself to test it out but won't have time for it for awhile. If anybody on the NIXOS team is running build 17134 SCU (pre-release still), if they could give it a go on this build, it would be appreciated.

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented May 14, 2018

I'm taking a stab at this. Initial success so far @ https://twitter.com/GeoffreyHuntley/status/995977211176927232

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented May 16, 2018

Blocked by microsoft/WSL#3207

@PerArneng

This comment has been minimized.

Copy link

commented May 25, 2018

Thanks for the work on this. Testing out linux distros on Windows is really easy now days and you can run several side by side. Getting NixOS on to the app store could be a great gateway drug to NixOS and the excellent Nix package manager.

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented Aug 8, 2018

In private discussions with Microsoft about getting NixOS into the Microsoft App Store. Have been a little overwhelmed recently but it's high on my todo list. Microsoft are waiting on me.

@bencoman

This comment has been minimized.

Copy link

commented Aug 8, 2018

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented Oct 25, 2018

Considering nixcon is on; it only makes sense to pick this back up again. I had to park it because I was overcommitted. Let's start this back up again.

screen shot 2018-10-25 at 9 18 01 pm

Next steps are as follows:

  • NixOS Foundation to sign up for a "Company" Windows Developer Account
  • NixOS Foundation to review contracts and execute them
  • Obtain testing plan from Microsoft
  • Obtain publishing and release plan from Microsoft
  • Productionise the NixOS build of install.tar.gz and embed into an appx.
  • Distribute the appx to microsoft
  • Party
@matthewbauer

This comment has been minimized.

Copy link
Member

commented Nov 3, 2018

What's the structure of install.tar.gz? I couldn't find that documented. Also, someone using WSL might want to add a package for "wslu" which has some nice things for interacting between Linux and Windows:

https://github.com/wslutilities/wslu

@zimbatm

This comment has been minimized.

Copy link
Member

commented Nov 3, 2018

@ghuntley do you have a URL for the NixOS Foundation to review the documents?

@hypersw

This comment has been minimized.

Copy link

commented Nov 25, 2018

Thanks for pushing NixOS towards running over WinNT! That would be handy.

The roadmap above is much about putting NixOS into the appstore. Which is quite separate a task from making NixOS runnable on WSL. I'd pretty much do with a ZIP which I'd download from nix site and install for me, not sure I even like the idea of mouseclicking it out of the fancy store instead. Especially if we have to wait for the legal stuff just for the sake of some GUI install scenario.

What's the technical status of running NixOS on WSL? Is it still blocked by the question marks and pipes in the paths? If not, are there any distributions we could try and run right away?

@matthewbauer

This comment has been minimized.

Copy link
Member

commented Nov 27, 2018

What's the technical status of running NixOS on WSL? Is it still blocked by the question marks and pipes in the paths? If not, are there any distributions we could try and run right away?

I don't have WSL, but in theory, it should be possible now using the "containerTarball" (nixos-system-x86_64-linux.tar.xz).

You can manipulate this into an install.tar.gz file and get it to run (and make a PR to get it in future release). There will most likely be bugs encountered doing this though. wslu looks to be a good resource to base the install.tar.gz on.

EDIT: It looks like WSL doesn't support systemd! So we will need to do a little more work to get this to work. How would we run the Nix daemon?

Another question that remains to be solved: how can we get Nix to detect that it can run Windows binaries? Nix supports "extra-platforms" so we can reuse that if there were an easy way to run a builder in "windows" mode. Can't find any sort of syscall/API to do this though.

@WSLUser

This comment has been minimized.

Copy link

commented Nov 28, 2018

The proprietary init does the windows interop calls to the apis. Far as daemons go, as I stated before, you dont need systemd to run services. In Debian-based distros, you just do service blah start. For example ssh. Want it to start automatically? Not supported out of box yet but you can add to your bashrc, zshrc, etc. as workaround

@matthewbauer matthewbauer added this to the 19.03 milestone Dec 13, 2018

@wmertens

This comment has been minimized.

Copy link
Contributor

commented Mar 31, 2019

Now granted, WSL should support all POSIX filenames, and microsoft/WSL#3207 is definitely a bug on their side. However, would it not be relatively simple to replace all NTFS-invalid characters in Nix store path names with _?

Names clashing seems very unlikely since the input hash is prepended. There may be some derivations that have illegal characters in the package content filenames, those can be handled on a case-by-case basis.

Alternative is to wait for NixOS/rfcs#17 ;-)

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

Current status:

  • [still waiting on nixos foundation to sign legal paperwork to allow distribution into the app store and to register a corporate account]
  • systemd issues potentially resolved by today's announcement at build//
  • following up with WSL product managers about getting access to WSL2 bits (under NDA if need be)

See https://devblogs.microsoft.com/commandline/announcing-wsl-2/

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

Developer agreement is at https://docs.microsoft.com/en-us/legal/windows/agreements/app-developer-agreement

Last correspondence with Microsoft below:

Hi Geoffrey and Graham,

Thanks for reaching out. It’s nice to virtually meet you. Here’s an overview of the process:

As outlined below, you’ve completed the first step by reaching out. I’ve pasted the other steps while adding additional details.

  • Agree with the WSL team on a testing and publishing plan
  • We have some basic manual tests that we conduct such as testing networking. Are there any specific compatibility concerns you have for NixOS running on WSL?
  • The other part of publishing is that we ask you to send us your appx file. This will be put on a special Store CDN that will allow customers to install your app offline. You can read more on what that would look like
  • Complete any required paperwork
  • When you sign up as a company for a Windows Developer Account you will need to sign an App Developer Agreement. Please take a look at that agreement and let us know if it meets your legal requirements. There are a couple other documents in the aforementioned link, so please take a look at those as well.
  • Sign up for a "Company" Windows Developer Account https://developer.microsoft.com/en-us/store/register. note: this can take a week or more since you'll be required to confirm your organization's identity with an independent verification service via email and/or telephone.

Once the above is complete, you can submit your app and stage it for publication. There are a couple things that happen in the background with the Store that can take a little while depending on the volume of submissions they have at the time.

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

Next steps in theory are:

  • obtain WSL2 bits
  • ???cook a nothing out of the ordinary virtual machine via CI???
  • ???wrap virtual machine with WSl2 bits and package as appx [will require windows CI]???
  • nixos foundation sign legal agreements and register for corporate developer account
@puffnfresh

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

What does "ordinary" look like? Does that include systemd?

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

That's the missing piece of technical detail. Discussing now with Microsoft product managers.

Not sure if we need to cook an ova or they take a tarball of a freshly installed system (WSL1) and then inject it into a VHD. (WSL2)

Not sure if they invoke grub when booting this VHD.

Not sure if initd/systemd is invoked or how to get WSL2 to invoke the equivalent.

@WSLUser

This comment has been minimized.

Copy link

commented May 7, 2019

There's no plan to implement systemd at this time. That's what was said at their MS Build booth when I asked. But init rc scripts work just fine. You can do that now without WSL2.

@matthewbauer

This comment has been minimized.

Copy link
Member

commented May 7, 2019

Yeah, NixOS has an option to disable stage 1 with boot.isContainer. We probably need to rework stage-2 a bit to avoid running systemd directly, but also keeping as much of the activation scripts as possible. The script is here: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/boot/stage-2-init.sh. @cleverca22 has worked on some custom init scripts through not-os, which may be useful.

At the same time, providing an installer with just multi-user Nix installed would be super useful in its own right! We don't necessarily need to go full NixOS WSL for this to be useful. It sounds like we'll need to figure out how to automatically start the daemon correctly. Single user will probably work without any modifications!

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

Some more technical information can be found at https://devblogs.microsoft.com/commandline/shipping-a-linux-kernel-with-windows/ that's the best info that exists at this stage.

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

It sounds like we'll need to figure out how to automatically start the daemon correctly.

I agree.

Single user will probably work without any modifications!

It does and it's what we should aim for.

https://twitter.com/GeoffreyHuntley/status/1007607578933714945

@tycho01

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

They're releasing a 2.0 but I presume that shouldn't affect making NixOS available there.

@edolstra

This comment has been minimized.

Copy link
Member

commented May 7, 2019

@ghuntley To clarify, why does the foundation need to register as an App Developer? I assume you already have the ability to upload apps to the store?

@vatsalyagoel

This comment has been minimized.

Copy link

commented May 10, 2019

@edolstra It isn't recommended to ship an official distribution of an application under an individual's account. It brings other legal issues. Ownership of accounts and just general branding.

If he were to publish NixOS under his own account he would either have to give up ownership of his developer account to NixOS foundation or the ownership of NixOS distro on windows is handed over to Geoff

@edolstra

This comment has been minimized.

Copy link
Member

commented May 10, 2019

Okay, I've registered an account for the NixOS Foundation (currently pending verification). I've also reserved "NixOS" as an app name. What's the next step?

@matthewbauer matthewbauer modified the milestones: 19.03, 19.09 May 27, 2019

@ghuntley ghuntley referenced this issue Jun 13, 2019

Draft

[WIP] Windows Subsystem for Linux #63049

1 of 10 tasks complete
@ghuntley

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2019

WSL2 was released today and is now available for folks who are on the Windows 10 fast ring.

https://devblogs.microsoft.com/commandline/wsl-2-is-now-available-in-windows-insiders/

Okay, I've registered an account for the NixOS Foundation (currently pending verification). I've also reserved "NixOS" as an app name.

Thank-you for registering the account and reserving the application name @edolstra.

What's the next steps?

I'd love to team up with a couple folks who are more experienced with nixos internals and ship #63049 which was raised moments ago. Stop on by the pull-request, say howdy and I'll grant you push access to the pull-request branch.

@AndrewMorsillo

This comment has been minimized.

Copy link
Contributor

commented Jun 24, 2019

It looks like there is a way to run systemd on WSL2 https://github.com/arkane-systems/genie although I'm not sure if that is useful

@ghuntley

This comment has been minimized.

Copy link
Contributor

commented Jun 26, 2019

@AndrewMorsillo useful. Thanks for sharing, was unaware!

@ghuntley ghuntley referenced this issue Jun 26, 2019

Open

<3 #8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.