9p performance increase by ~10x reflected in WSL?

[i2]

Hi WSL dev team,

Recently, new patches were released to boost Linux 9p performance about 10 times! (link) I was wondering when/if these would be taken in WSL? Thank you!

Background
I have a rather peculiar setup when it comes to WSL2. I like to be absolutely minimalist. I'll provide an example:
I use the following 3 things extensively: scoop, git in my Windows and git in my WSL2. If let on their own, they would want to install git separately which leaves me with 3 different git installation files and .config to keep track of. I have managed to very nicely, just install git in my WSL2 and configure scoop and Windows to use it. The caveat is scoop and WSL git operations are very slow! I am assuming this patch will rectify a bit of the issue.

[i2]

I have good news: The changes are now merged in upstream! How long till we see it in WSL2?

[elsaco]

I did a build of linux-next kernel with the 9P patches included. Here is the result of copying a 4.3G folder from Linux to a Windows:

5.15.90.1-microsoft-standard-WSL2 kernel:

real    8m16.570s
user    0m0.431s
sys     1m23.135s

linux-next-next-20230301 kernel:

real    3m14.430s
user    0m0.238s
sys     0m29.920s

That's a 5min difference! The linux-next kernel has NET_9P_FD added to the build supporting for transport over TCP, Unix sockets and filedescriptors:

*
* Plan 9 Resource Sharing Support (9P2000)
*
Plan 9 Resource Sharing Support (9P2000) (NET_9P) [Y/n/m/?] y
  9P FD Transport (NET_9P_FD) [Y/n/m/?] (NEW)

[stevehogo]

hi @elsaco can you share your build? i want to try it on my side.

Thanks
Steve

[i2]

The reason I created this issue and asked for merge: I store my gopass passwords in WSL2 filesystem (ext4) and access them in my Windows (through browser with gopass bridge extension). Using official stable WSL2 kernel from Microsoft, any gopass inquiry takes a long time, whereas if I copy my .password-store folder to the Windows NTFS partition, same operation is almost instant!

So, I went ahead and built a kernel from master branch with 9p performance tweaks merged in, but unfortunately I am not seeing any performance gains!!!

[Gillou68310]

Can confirm it's faster, tested with this branch https://github.com/v9fs/linux/tree/ericvh/for-next

[Gillou68310]

bzImage.zip
@stevehogo

[stevehogo]

thank so much @Gillou68310

[SailorMax]

Thank you!
This kernel helped with speed of vuejs and laravel.
But has problems with startup postgis docker container! Possible, because it is still beta. Hope official release will fix it.

[i2]

@elsaco @Gillou68310 @stevehogo Can you please try accessing a Windows folder/partition with many many small files from your WSL, and see if you notice any performance gains?

Like I said, my use-case is accessing my password store (many tiny gpg files) and I haven't felt much improvement. I might be totally wrong though!

[jakubstuglik]

I have the same use-case as @i2 : many small files and unfortunately the speed on volume mounted from Windows filesystem is not improved to perceivable degree.

Interestingly: I was testing transfer speed before, on the kernel provided with Ubuntu with dd to path under Ubuntu /mnt/c (which is Windows drive), and I remember that it was very bad, like 4,8 MB/s (it was few days ago). Then today I switched to kernel provided by @Gillou68310 and tested it, and it was better - like 30-50MB/s. THEN I switched back to the old kernel (which is 5.15.90.1-microsoft-standard-WSL2) to confirm terrible speed, and tested it again and it was suddenly at my physical drive limits! But the speed of opening many files (mostly small) is still the same. I don't know what's going on here...

@Gillou68310 could you share how you build this image? I'd like to do some experiments myself but I'm not certain how to do it. I found this, do I only have to change repo from Microsoftd to (https://github.com/torvalds/linux/) and play around with branches?

[i2]

It's actually not difficult to build a WSL2 kernel. Instructions are here (It's not different than building any typical kernel actually)

[Gillou68310]

I followed this personally
https://gist.github.com/oleksis/eb6d2f1cd2a6946faefb139aa0e38c35

[Gillou68310]

Also my use case is running make from wsl on a windows folder. I got around 200 files to build and the build time has been divided by 2 for me.

[jakubstuglik]

Thank you!

[BarsMonster]

After this is widely deployed - we'd still need configurable msize to squeeze all of the throughput: #9125

[i2]

Interesting! I hope Microsoft implements it in a configurable way. I'm not sure if anyone from Microsoft WSL2 team monitors these issues?!