Would it be possible to make your lxss directory more easily accessible from Windows (i.e give it its own icon/drive in Windows Explorer)

[alanosman]

I don't really know where to suggest feature ideas, but thought I would post it here.

I would really like to see Lxss made more easily accessible from Windows Explorer. I think one of the powerful things we can do is operate using our Windows tools on linux code. That provides the best of both worlds. When you bury Lxss deep inside AppData, it makes it harder to find and work on.

thx

[fpqc]

@alanosman lol. I suggest you try doing this:
In bash:

$ touch ~/datbutt

then in cmd:

C:> notepad %localappdata%\lxss\home\yourusername\datbutt

Then just type in a letter and press ctrl s

then back in bash, do

ls ~

(hint: the file will magically disappear)

There is a reason why it's hidden right now. Win32 programs just stomp all over the NTFS extended attributes that help the magic happen.

[alanosman]

hey @fpqc - thanks for the hint. So are you saying if I touch datbutt, that everything is good.. lol

Actually - I tried what you suggested and it worked. But I guess it doesn't work at all if I have to copy the contents of say a zip file into lxss filesystem somewhere. I'd have to do that from within ubuntu.

Are there plans to add this capability. Seems like it would be really powerful to go easily bidirectional like that.

[fpqc]

They have secret projects that they say might one day make this kind of thing work. No idea how. But the topic is a known issue, by design for now, and a duplicate so I am sure the devs would appreciate if you closed this.

Read the Windows Subsystem for Linux blogpost on filesystems (and watch the video), which will make it clear why this is a tough problem.

[therealkenc]

The 'how' is pretty straightforward technically. The question is will anyone let them.

[fpqc]

@therealkenc What would that be? Mapping to the Windows ACLs directly?

[gegenschall]

@fpqc That would be so awesome, actually! Also: map Windows users to Linux users.

[fpqc]

@gegenschall One reason they're probably not doing this is that it would probably entail a much much stricter security and compatibility review. Right now, WSL runs almost like an ordinary Windows application, and since it doesn't cross security boundaries as a matter of function, it can probably have a much much faster release cycle. If we'll ever see this, I don't think it will be soon (Mapping Windows users to Linux users, that is)

[aseering]

@therealkenc -- could you elaborate? Specifically, how do you preserve the file-permissions metadata (in a way that's not a dangerous hack) in the case of a text editor modifying a file by "write temp file and rename()" ? Which, if I had to guess, would be the most common use case for direct access to the WSL directory. The Linux kernel itself doesn't even attempt to make that case Just Work(tm); it's up to the application.

I would agree if your claim was that WSL should handle files with missing metadata more gracefully. (For example, invent values based on the default system umask/etc, which is roughly analogous to what Linux itself would do.) I just don't think that's a complete solution. For example, what Linux user owns the new file?

[therealkenc]

These are not new questions. Ask anyone implementing vboxsf, or a Samba developer for the last quarter century. As for mapping Linux users to Windows users, who said they need to be mapped? 😜

[aseering]

"not new" -- indeed not :-). And the groups you've cited, will, of course, respond "there is no one right answer; it needs to be configurable." (They will also probably respond "ew, that's a mess; avoid having to rely on that precise behavior if you can, and plan to spend a while tuning the configuration to your application if you can't.")

Those examples can be configured per-mountpoint, so there's some flexibility; you can remount them, etc. This directory can't be; it's not a mount point, it's just a directory. Sure, if you control the filesystem driver, you could expose equivalent functionality regardless. But I think that's no longer trivial :-)

What I personally want is a way to mount the WSL homedir filesystem under Windows, with the ability to configure all these kinds of things per-mountpoint.

[therealkenc]

Ah but it is trivial, because they've already done the work. Okay 'trivial' is an unfair characterization. Let's use the word 'straightforward'. Anyway, save the larger conversation to next spring after WSL has gelled. It's academic right now until bigger fish are fried. Agreed, it would be nice if one could "mount the WSL homedir filesystem under Windows". In fact someone motivated can do that right now. I just haven't looked into what it would take to get TAP or something like it to redirect port 445.

Also appreciate that rootfs (nevermind rootfs/home) are not 'special' from a security standpoint. You own rootfs. All of it. The fact that editing AppData\Local\lxss\rootfs\etc\shadow with notepad.exe blows it away is not a security feature. That file is in no way more special than your Win32 Apache .htusers file.

[fpqc]

@therealkenc I thought a while ago that a straightforward way to do it might be to make it so non-root privilege pico processes run with the low-integrity sandbox security token (from internet explorer, access to AppData\LocalLow) and have sudo/su allow running as root with the user's usual token. Not sure how you could map non-root ownership for multiuser onto the sandbox token and its associated acls, but identifying root ownership and permissions with the User's regular-integrity token entries in the ACLs makes sense.

I wonder if the ACLs or low integrity framework could be set up to support low-integrity with an "owner" field where the "owner" field is a linux user created underneath the ordinary one.

[justweb1]

1. Open the lxss directory in Explorer.
2. Right click and select properties.
3. Click on the sharing tab.
4. Click on Advanced Sharing.
5. Select 'Share this Folder' and type a Share Name
6. Click on Permissions.
7. Click Add and find your user and give your user Full Control.
8. Remove the 'Everyone User'
9. Select Apply and OK.
10. In Explorer open your PC under the Network Heading and you will see your shared Linux folder.
11. Right click on it and select Map Network Drive.
12. Done.

[fpqc]

@justweb ummmm Does that respect ntfs extended attributes?

[justweb1]

@fpqc It appears so and I have not had any issues with it.

[bitcrazed]

Be warned: If you modify the contents of LXSS using Windows apps, which often don't persist, honor or update the Linux file matadata stored in extended properties, YOU ARE LILELY TO EXPERIENCE DATA LOSS AND/OR CORRUPTION.

Also, the LXSS folder is likely to move in the future.

The LXSS folder is hidden & system for a reason.

We keep saying this for a reason.

Again: Please don't modify the contents of the LXSS folder using Windows apps. Period.

[fpqc]

@justweb no, sorry it doesn't work. Try creating a file, it will not be visible.

[alanosman]

@bitcrazed (Rich) and everyone else, thanks for all your responses. Rich, what you are stating is for the current build only, right? While I'm not asking you to reveal the roadmap, I would like to know if this is a totally impossible feature or it is something that we could/would expect to see in the near future.

Having the lxss filesystem be so accessible on Windows is a wonderful opportunity to unify and streamline what teams do in development. It eliminates the constant arguments and debates that go on with developers who love to code in Linux with VIM or on a Mac using all the OpenSource tools like Ruby, Node, Mongo etc. I found that these folks treat Windows as a lost cause and our only workaround for Windows users has been to use remote servers, VMs or CygWin. To some degree, my engineers have laughed at me for not owning a Mac, because "Mac makes that so easy".

But I am stubborn and support the Windows 10 direction. So to counter, having Windows provide such seamless access to lxss would also be IMO very beneficial, because with every tool, there is a learning curve and that icon in the file system would highlight the level of deep integration between the two operating systems and demonstrate how easy it would be to work with.

If lxss was ready for production right now (and I do try test our code between your releases on it), I would make the switch from using CygWin over to Ubuntu because it would normalize the tools we use and eliminate that constant argument of "Well it's a problem because it's running on Windows". Running on CygWin does give you the ability to edit files from the outside in which I really like by accessing C:/cygwin64 folder. That has been helpful in creating bash scripts and somewhat testing them, but it is limited and problematic because full bash is not supported and the environment is not identical to what we would set up in production. It would be super nice to edit something like that in Visual Code, save and then go to your lxss terminal and run it.

I think you probably very much understand what the benefits could be, but ideally having all services we develop running on Ubuntu under Windows while using Windows based tools like Sublime, VSCode or Smartgit to manage the project and edit what we would all like to see eventually happen.

[steshaw]

@bitcrazed I'm considering buying a new (Windows?) laptop. Wondering whether the data loss/corruption problems you mention have been resolved with file sharing between Windows apps and WSL? In my case, I'd really need to be able to use at least a GUI editor like Atom/vscode in Windows to edit my source files in WSL.

[alanosman]

@steshaw - On a day to day, I have no issues working across Ubuntu (WSL) and Windows and it has come a long way since this original post. Your Windows drives are automatically mounted in /mnt (i.e. /mnt/c, /mnt/d etc) and you can open Visual Code or Atom from Windows and edit your repos which I have living on my D drive. I continue to use all my Windows tools (GitKraken, Visual Code, Elastic Search) while it executes from Bash .

I've installed NodeJS, Mongo, etc inside WSL and run our client and web server apps we are developing there. I'm not sure about Atom, but Visual Code has a flag that lets you launch and debug using WSL, so you can stay within the Bash environment but be in Windows Visual Code debugging mode. VC also lets you run a terminal to Bash. These are really nice touches and tie things together.

I really love what @bitcrazed and the team created here and there are still some things though that you need to be aware of that are missing. You don't get every feature of Ubuntu, for example, Docker will not run as a server within WSL and not all services work automatically (there is no upstart). But I believe the team is slowing getting to them and Kudo's to all the work they did here to get it this far.

[bitcrazed]

@alanosman Many thanks for sharing your workflow: I know many people who're interested in whether WSL can "cut it" in the real world are often inspired by people who're using it in real scenarios :)

@steshaw If you want to edit files using VSCode/Atom/Suiblime/Notepad++/Notepad/etc. in Windows, and build/run them in Linux, then you can do so today by storing those files on your Windows filesystem, and accessing them via your /mnt/<drive>/... "mounted drives".

Just avoid trying to navigate to your Linux filesystem (possible but difficult-for-a-v-good-reason 😉) using Windows apps/tools: We're working on improving this scenario in the future, but it's wise to avoid this scenario for now.

[steshaw]

Oh I see. Thanks, @bitcrazed. Storing my source files under /mnt/c would work for me!

[bitcrazed]

Cool.

Also, there are a TON of significant improvements in the WSL filesystem infrastructure in Insiders build 17063 that dropped today. We'll be posting details on our blog post here vs. soon: https://blogs.msdn.microsoft.com/commandline/

[fpqc]

@bitcrazed These new features are briefly explained (but with enough detail to make use of them) in this week's release notes:

https://docs.microsoft.com/en-us/windows/wsl/release-notes

It's not clear though which windows tools will destroy the lx metadata, though. Is there more information coming in that blog post?

[therealkenc]

which windows tools will destroy the lx metadata

Anything that deletes the file and creates a file in its place, as opposed to just reading and writing the file. Which is the case for some editors.

[benhillis]

There is nothing stopping editors from creating the new file with the metadata (extended attributes) of the old file.

[therealkenc]

Yes yes naturally. I alluded as much in August in a slightly different context. Probably someone is looking at adding support to libuv already; which would mean Node would have support; which would mean VS Code would have support. The question was: "which windows tools will destroy the lx metadata". Answer being more formally: everything that creates new files and doesn't (yet) support WSL metadata.

[fpqc]

@benhillis I was in particular asking if things like notepad will support it.

[tara-raj]

Following up with the Notepad team on supporting metadata and will add any updates to this thread. We're still investigating solutions to accessing Linux distro files which is rather complex so please bear with us while we explore options.

[dreamcat4]

Well 'things like notepad' is most times not notepad.exe itself. I mean... its natural to occasionaly use notepad.exe for the convenience of just immediately double-clicking to open files directly from explorer.exe.

But for any serious developement then its definately the more sophisticated code editors we are talking about. For example I use 'sublime text 3', other developers use other code editors. And there are quite a few different programs which people use. Just as an example, and unfortunately, for my own preferred editor (sublime text) - their stance is already clear: 'its not our problem, we don't want to implement this, we are not going to implement this'. (the preservation of the linux metadata).

So then!

[dreamcat4]

I mean... the only thing I can suggest here would be to have a 2nd process also watching those same files being re-saved by the source-code editor. Then after that meta-data was lost, immediately jump in and restore the metadata back to the file.

That approach could just be intended a temporary stop-gap measure. For example in the form of an independant 3rd party tool, which people use selectively on their current working development project folder(s). Its usually just some text files that have been checked out from git. An exclude pattern (like .gitignore) - to omit any build files within the same subfolder(s) that do not need to be watched. And should be ignored.

So compared to watching all of the files, which might be prohibitive from a performance standpoint, that approach might actually stand a chance of working.

Unfortunately it can lead to other problems. For example - if the NTFS file is locked by the text editor for being currently open. And it cannot be re-save by the 2nd watching process. OR if the text editor notices the file has changed on disk while it was already open. Then it will display a message saying 'do you want to reload / revert / discard the file?'. Which would occur immediately upon the user saving the file in the text ediot... a very annoying side effect indeed! I am already certain that Sublime Text would do exactly this thing.

But perhaps those issues could be gotten around. For example if the file is re-written with an idential timestamp, and all of the other file attributes, and NTFS-visible metadata, are all kept the same, including the file's checksum... I really don't know how that works sorry. Perhaps it might be possible. It would require looking into.

Eventually, you might be able to this 'all properly'. With some changes under the hood in NTFS and WSL. Like you say though: it would probably take a much longer time, compared to such an ad-hoc tool. But then we could just discard the temporary tool and stop using it anymore. Never intending for it to be the final solution.

[therealkenc]

There is nothing stopping editors from creating the new file with the metadata (extended attributes) of the old file.

@benhillis (or anyone) - What is the value of the LXATTRB field just before atime/mtime/ctime?

Getting off my chest early: "accessing Linux distro files" per above has fundamental separation of concerns problems. But let's start with the basics and worry about that later.

[benhillis]

@therealkenc - I'll have to check with @jstarks to see how he feels about documenting the structure. That seems like a prerequisite to asking third parties to persist our attributes. Agreed about the can of worms interop represents for Linux security.

[fpqc]

@benhillis I'm wondering if persistence of that metadata could somehow be added directly into the Windows API calls so if a program compiles against the newer version, it will just work. I guess the main problem is that for programs in C (where you have to allocate the memory manually) that copy the metadata into memory, they will need to store more data, so you might need something like CacheMetadataEx instead of CacheMetadata or something since the size of the data structure you're caching might change.