Cross-platform symlink

[daxpedda]

I couldn't figure out how cross-platform symlinks work. It states under features:

Good symlink handling compatible with Windows

I simply used ls -s, but the creates symlink only worked on Linux and MacOS. Using the usual shortcut creation on Windows, didn't work at all, not on Windows and, as expected, not on Linux or MacOS.

Is there something I'm missing? I couldn't find documentation on it in any case.

[nabijaczleweli]

Uhh, you make a symlink and it's there on both Unix and NT. It kinda Just Works IME. Do you have a reproducer?

I have this in one of my desktop's httpd folders, it, well, Just Works:

 Volume in drive T is TEMP
 Volume Serial Number is 5566-7788

 Directory of T:\nabijaczleweli.xyz

02/12/2021  14:44    <DIR>          .
02/12/2021  14:44    <DIR>          ..
02/12/2021  14:44    <JUNCTION>     assets [\??\p:\HTML\nabijaczleweli.github.io\assets]
02/12/2021  14:44    <JUNCTION>     CNAME [\??\p:\HTML\nabijaczleweli.github.io\CNAME]
02/12/2021  14:44    <JUNCTION>     content [\??\p:\HTML\nabijaczleweli.github.io-dev\out]
02/12/2021  14:44    <JUNCTION>     css [\??\p:\HTML\nabijaczleweli.github.io\css]
02/12/2021  14:44    <JUNCTION>     index.html [\??\p:\HTML\nabijaczleweli.github.io\index.html]
02/12/2021  14:44    <JUNCTION>     js [\??\p:\HTML\nabijaczleweli.github.io\js]
02/12/2021  14:44    <JUNCTION>     kaschism [\??\p:\HTML\kaschism\out]
02/12/2021  14:44    <JUNCTION>     keybase.txt [\??\p:\HTML\nabijaczleweli.github.io\keybase.txt]
02/12/2021  14:44    <JUNCTION>     LICENSE-CODE [\??\p:\HTML\nabijaczleweli.github.io\LICENSE-CODE]
02/12/2021  14:44    <JUNCTION>     LICENSE-CREATIVE [\??\p:\HTML\nabijaczleweli.github.io\LICENSE-CREATIVE]
02/12/2021  14:44    <JUNCTION>     old [\??\p:\HTML\nabijaczleweli.github.io\old]
02/12/2021  14:44    <JUNCTION>     pgp.asc [\??\p:\HTML\nabijaczleweli.github.io\pgp.asc]
               0 File(s)              0 bytes
              14 Dir(s)   3,977,355,264 bytes free

[daxpedda]

https://github.com/daxpedda/http-symlink-repro
http -p 20000 web
I can visit http://localhost:20000/index.html on all OSs.
I can visit http://localhost:20000/assets/test.html on Linux and MacOS, but not on Windows.

[nabijaczleweli]

Well, yeah, when you clone it it's just a regular file because Windows doesn't have normal symlinks (=> git doesn't (can't) clone them as such):

T:\http-symlink-repro>dir test
 Volume in drive T is TEMP
 Volume Serial Number is 5566-7788

 Directory of T:\http-symlink-repro\test

16/01/2022  20:33    <DIR>          .
16/01/2022  20:33    <DIR>          ..
16/01/2022  20:33                 9 assets
16/01/2022  20:33                15 index.html
               2 File(s)             24 bytes
               2 Dir(s)   3,976,990,720 bytes free

T:\http-symlink-repro>ls -l test
total 2
-rw-r--r-- 1 nabijaczleweli None  9 Jan 16 20:39 assets
-rw-r--r-- 1 nabijaczleweli None 15 Jan 16 20:39 index.html

T:\http-symlink-repro>cat test/assets
../assets

nabijaczleweli@tarta:~$ curl -SL 'http://nabijaczleweli:8003/test/assets'; echo
../assets

If you make it an actual link then it functions as such:

T:\http-symlink-repro\test>rm assets

T:\http-symlink-repro\test>mklink /J assets ..\assets
Junction created for assets <<===>> ..\assets

T:\http-symlink-repro>dir test
 Volume in drive T is TEMP
 Volume Serial Number is 5566-7788

 Directory of T:\http-symlink-repro\test

16/01/2022  20:36    <DIR>          .
16/01/2022  20:36    <DIR>          ..
16/01/2022  20:36    <JUNCTION>     assets [T:\http-symlink-repro\assets]
16/01/2022  20:33                15 index.html
               1 File(s)             15 bytes
               3 Dir(s)   3,976,990,720 bytes free

T:\http-symlink-repro>ls -l test
total 1
lrwxrwxrwx 1 nabijaczleweli None 28 Jan 16 20:36 assets -> /t/http-symlink-repro/assets
-rw-r--r-- 1 nabijaczleweli None 15 Jan 16 20:33 index.html

nabijaczleweli@tarta:~$ curl -SL 'http://nabijaczleweli:8003/test/assets' | wc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 31819  100 31819    0     0  5178k      0 --:--:-- --:--:-- --:--:-- 6214k
    160     425   31819

I can't say I understand the question at hand.

[daxpedda]

So creating Windows shortcuts through the context menu doesn't work, because it uses symbolic links. The command you used uses directory junctions, first time I heard about this concept. But this is less then ideal because git doesn't actually recognize junctions as a symlink and just pushes a copy of the whole folder.

I supposed the question is, can https support symbolic links? That way they can be uploaded to GitHub right next to a Linux symlink, which would allow for cross-platform symlinks.

[nabijaczleweli]

Huh? Right-drag -> make shortcut just makes a regular file.

mklink does have a regular symlink option (the default) and directory symlink (why these are different are beyond me), which do Just Work, as expected:

T:\temp\buks>mklink a t:\a.html
symbolic link created for a <<===>> t:\a.html

T:\temp\buks>mklink /D b t:\temp
symbolic link created for b <<===>> t:\temp

T:\temp\buks>dir
 Volume in drive T is TEMP
 Volume Serial Number is 5566-7788

 Directory of T:\temp\buks

16/01/2022  21:35    <DIR>          .
16/01/2022  21:35    <DIR>          ..
16/01/2022  21:32               582 1640783825755.LNK
16/01/2022  21:35    <SYMLINK>      a [t:\a.html]
16/01/2022  21:35    <SYMLINKD>     b [t:\temp]
05/01/2022  22:30           622,126 Perfekcyjna smierc - Helen Fields.mobi
               3 File(s)        622,708 bytes
               3 Dir(s)   3,977,027,584 bytes free

I was as surprised as anyone to find out that mklink also lets you do hardlinks, but they do work (though that in itself is unsurprising).

So AFAICT all links under NT, so long as they're links, are recognised by the rust stdlib, and work?

[daxpedda]

Huh? Right-drag -> make shortcut just makes a regular file.

I was using the Create Shortcut menu options, which creates a symbolic link. I'm not sure what you mean with "regular file", the file that it produces for me is a .lnk file.

I checked again, and I can't make symbolic links work. It shows up correctly in the file explorer though and git also recognizes it correctly. I will try a different machine tomorrow.

[nabijaczleweli]

I genuinely don't understand what you're talking about. A shortcut you make from the context menu is... a shortcut. A regular file. You can put an icon on it (this edits it). You can add arguments and change the working directory (this edits it). You can edit it directly. Unless you're running some sort of cutting-edge Windows distribution maybe (and if so – which)? dir the containing directory and post the output. If it says a size instead of one of <SYMLINK>/<SYMLINKD>/<JUNCTION> (and doesn't have the target in []s) – it's a regular file (or a directory if it says <DIR>; either way).

[daxpedda]

I'm saying that <SYMLINKD> does not work with https, but <JUNCTION> does. I'm not having any problems with how Windows perceives it, I'm having trouble with how https perceives it.

I tested it on Windows 10. As soon as I'm on a Windows machine again I will update you and upload a test repo for you to see what I mean.

[daxpedda]

So AFAICT all links under NT, so long as they're links, are recognised by the rust stdlib, and work?

Basically this is not the case for me. I don't know why. I guess whatever the conclusion here, it's not a problem in https then.

[nabijaczleweli]

Cannot reproduce this.

q:\dir>dir
 Volume in drive Q is symlinktest
 Volume Serial Number is 969F-A3A5

 Directory of q:\dir

17/01/2022  14:50    <DIR>          .
17/01/2022  14:50    <DIR>          ..
17/01/2022  14:50    <JUNCTION>     dir-JUNCTION [q:\dir]
17/01/2022  14:50    <SYMLINK>      dir-SYMLINK [.]
17/01/2022  14:50    <SYMLINKD>     dir-SYMLINKD [.]
17/01/2022  14:47            26,749 kill-mupd.log
17/01/2022  14:50    <SYMLINK>      kill-mupd.log.SYMLINK [kill-mupd.log]
17/01/2022  14:47            26,749 kill-mupd.log2
               4 File(s)         53,498 bytes
               4 Dir(s)  42,718,408,704 bytes free

q:\dir>s:\Rust-target\debug\http
Hosting "." on port 8003 without TLS and no authentication...
Ctrl-C to stop.

[2022-01-17 14:50:58] 192.168.1.109:1032 was served directory listing for \\?\Q:\dir
[2022-01-17 14:50:58]                    encoded as gzip for 343.2% ratio
[2022-01-17 14:51:01] 192.168.1.109:1032 was served directory listing for q:\dir
[2022-01-17 14:51:01]                    encoded as gzip for 347.4% ratio
[2022-01-17 14:51:04] 192.168.1.109:1032 was served directory listing for \\?\Q:\dir
[2022-01-17 14:51:04]                    encoded as gzip for 351.1% ratio
[2022-01-17 14:51:06] 192.168.1.109:1032 was served directory listing for \\?\Q:\dir
[2022-01-17 14:51:06]                    encoded as gzip for 355.1% ratio
[2022-01-17 14:51:07] 192.168.1.109:1032 was served file \\?\Q:\dir\kill-mupd.log as text/plain
[2022-01-17 14:51:07] Created temp dir $TEMP/http-Q-dir/encoded
[2022-01-17 14:51:07]                    encoded as gzip for 3278.1% ratio
[2022-01-17 14:51:10] 192.168.1.109:1032 was served file \\?\Q:\dir\kill-mupd.log as text/plain
[2022-01-17 14:51:10]                    encoded as gzip for 3278.1% ratio (cached)
[2022-01-17 14:51:12] 192.168.1.109:1032 was served file \\?\Q:\dir\kill-mupd.log2 as text/plain
[2022-01-17 14:51:12]                    encoded as gzip for 3278.1% ratio (cached)
[2022-01-17 14:51:27] Deleted temp dir $TEMP/http-Q-dir/encoded

The transcript corresponds to descending down http://nabijaczleweli:8003/dir-JUNCTION/dir-SYMLINK/dir-SYMLINKD/ then opening each kill-mupd.log* in order.

Filesystem image: symlinktest.zip (yes, it's ZSTD in a ZIP, GitHub doesn't let you upload ".zst"s for whatever reason).
Unpack, tgt-setup-lun -d "$(realpath symlinktest)" -n symlinktest as root, connect over iSCSI (I set the letter to Q, but this shouldn't matter), http dir.

[thecoshman]

Can you confirm if the issue your facing is once you clone a got repo with a link, or purely when creating links witihin Windows?

[nabijaczleweli]

Git repository checkouts of links are regular files on windows:

T:\>git clone z:\uwu\linkpsko
Cloning into 'linkpsko'...
done.

T:\>cd linkpsko

T:\linkpsko>git show
Author: наб <nabijaczleweli@nabijaczleweli.xyz>
Date:   Sun Feb 27 18:42:37 2022 +0100

    dupa

diff --git a/dupa b/dupa
new file mode 100644
index 0000000..ad3690b
--- /dev/null
+++ b/dupa
@@ -0,0 +1 @@
+dupsko
diff --git a/dupa2 b/dupa2
new file mode 120000
index 0000000..39b32e5
--- /dev/null
+++ b/dupa2
@@ -0,0 +1 @@
+dupa
\ No newline at end of file

T:\linkpsko>cat *
dupsko
dupa
T:\linkpsko>dir
 Volume in drive T is TEMP
 Volume Serial Number is 5566-7788

 Directory of T:\linkpsko

27/02/2022  18:43    <DIR>          .
27/02/2022  18:43    <DIR>          ..
27/02/2022  18:43    <DIR>          .git
27/02/2022  18:43                 7 dupa
27/02/2022  18:43                 4 dupa2
               2 File(s)             11 bytes
               3 Dir(s)   4,138,643,456 bytes free