Skip to content

tool_xattr: add support for Windows alternate data stream - #22354

Closed
vszakats wants to merge 1 commit into
curl:masterfrom
vszakats:wxattr
Closed

tool_xattr: add support for Windows alternate data stream#22354
vszakats wants to merge 1 commit into
curl:masterfrom
vszakats:wxattr

Conversation

@vszakats

@vszakats vszakats commented Jul 20, 2026

Copy link
Copy Markdown
Member

Requires a supported filesystem, i.e. NTFS.

Example:

> curl.exe https://curl.se/index.html --output test.txt --xattr --referer https://curl.se/
> cat < test.txt:Zone.Identifier

[ZoneTransfer]
HostUrl=https://curl.se/index.html
ReferrerUrl=https://curl.se/

Where newlines are CLRF, ReferrerUrl is set when using --referer
option.

Also:

  • test688: make it test --referer.
  • document user.xdg.referrer.url in --xattr man page.

Refs:
https://en.wikipedia.org/wiki/Mark_of_the_Web
https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(ADS)
https://learn.microsoft.com/en-us/windows/win32/fileio/file-streams
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c54dec26-1551-4d3a-a0ea-4fa40f848eb3

Ref: #22345


  • maybe handle the case of one-letter filenames (interpreted by drive letter by Windows).
    At least reject them, possibly expand to full path before fopen?
  • maybe implement a way to also add ZoneId=N? Maybe by using an env? That's able to set the value of N? defaulting to 3, if set but empty/invalid?
  • backtrack and drop the Zone Id thing after all.

@vszakats vszakats added cmdline tool Windows Windows-specific labels Jul 20, 2026
@vszakats

Copy link
Copy Markdown
Member Author

As normal with Windows, even if a thing looks simple, it isn't:
When using a single-letter filename, e.g. C:Zone.Identifier,
Windows interprets C: not as a filename, but a drive-letter, and
ends up creating a file named Zone.Identifier in the current
directory of drive C:.

@testclutch

This comment was marked as resolved.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Windows support for --xattr by writing Mark-of-the-Web metadata into an NTFS Alternate Data Stream (Zone.Identifier), and updates existing xattr tests to validate referer/origin metadata (with Win32-specific expectations).

Changes:

  • Implement Win32 --xattr handling by writing a Zone.Identifier ADS alongside downloaded files.
  • Extend fwrite_xattr() to accept the output filename and wire the new signature into the transfer pipeline.
  • Update xattr-related tests to include --referer and to verify Win32 vs non-Win32 metadata output formats.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/data/test644 Adjust expected xattr output for Win32 (ZoneTransfer) vs non-Win32 (user.* xattrs).
tests/data/test687 Adjust expected xattr output for Win32 (ZoneTransfer) vs non-Win32 (user.* xattrs).
tests/data/test688 Add --referer and verify referer propagation; add Win32 ZoneTransfer expectations.
src/tool_xattr.h Enable xattr support on Win32; update fwrite_xattr() signature.
src/tool_xattr.c Implement Win32 ADS writing of ZoneTransfer metadata; keep non-Win32 xattr behavior.
src/tool_operate.c Pass output filename into fwrite_xattr() to support Win32 ADS creation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/tool_xattr.c Dismissed
Comment thread src/tool_xattr.c Outdated
Comment thread src/tool_xattr.c Dismissed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Comment thread src/tool_xattr.c
Comment thread tests/data/test644
Comment thread tests/data/test687
Comment thread tests/data/test688
@github-actions github-actions Bot added the CI Continuous Integration label Jul 20, 2026
@jay

jay commented Jul 20, 2026

Copy link
Copy Markdown
Member

Environment variables can be inherited which makes them sometimes pernicious. Generally I'm not a fan of curl environment variables except for debug stuff. Global states can in some cases break scripts which could force scripts to blank it beforehand (set CURL_ZONEID=) which they shouldn't have to do. For example if CURL_ZONEID is set globally (like in System Environment) or by script and some future action does not expect it (like one script calls another script that uses curl), or it otherwise remains set and is inherited by some subsequent script, that could cause breakage. Basically it could end up adding some interactive requirement that is unexpected even after the script is done because the file has that marker.

I think an option like --output-zone-id 3 would be clearer and less likely to cause problems. Of course someone could put that in their curlrc, which we could caution against.

When using a single-letter filename, e.g. C:Zone.Identifier,
Windows interprets C: not as a filename, but a drive-letter, and
ends up creating a file named Zone.Identifier in the current
directory of drive C:.

This is tricky because in Windows a colon comes after the drive letter and before alternate data streams. So foo:bar is bar stream in file foo in the current directory of the currently selected drive, and C:foo:bar is the bar stream in the file foo in the current directory of drive C. BUT say the filename is C and you want to access the stream baz you'd have to make it a relative path first like .\C:baz and then that would access the baz stream in file C in the current directory of the currently selected drive. Yes confusing but it can be done.. probably safe to append .\ as long as the actual filename is only a single letter. The alternative way is get the full path like with GetFullPathNameW, see here for example.

@vszakats

vszakats commented Jul 20, 2026

Copy link
Copy Markdown
Member Author

I also wouldn't say I'm a big fan of envs, but for a feature like this, I'm (mildly put) not
very keen on adding then maintaining many hundreds of lines of code and documentation
for a zone-id-specific dedicated command-line option. It feels a combination of funny
(not in a good way) and wrong. It can be circumvented in any number of ways no matter
how it is implemented, and forcing it too much will break everything. Also zone id is a
Windows-specific term with Windows-specific values, hardcoding this into an option
looks overkill, and also wrong to me. E.g. on macOS the sibling feature is called
"quarantine", with an xattr named com.apple.quarantine, having 4 components, of
which the first is a 16-bit flag:
0083;abcdef01;Safari;FFFFFFFF-1234-5678-9ABC-0001001001
Where 0083 bits are partly explained here:
https://github.com/apple-oss-distributions/WebKit/blob/2a36545bf8dd89a417b724184900270fb2f523aa/Source/WebCore/PAL/pal/spi/mac/QuarantineSPI.h#L45-L50

(Besides Safari, Homebrew sets this, luckily with still an option in place to disable it.)

FWIW a command-line option can also have the same effect by pointing CURL_HOME
to a place with .curlrc having the dedicated command-line option.

I managed to solve (I think) the C:stream issue by converting to absolute path
first. That seems to work fine with C as an output filename, though I haven't
explored if there remain more corner cases.

I've since renamed the env to CURL_QUARANTINE_FLAG, to be OS/implementation
also to avoid confusion with 'zone id' as in IPv6.

If this is controversial or bad, I will just delete this feature from this PR and leave it to
someone else. I might be doing it anyway, because it seems to be a cursed thing to add
to a tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Continuous Integration cmdline tool tests Windows Windows-specific

Development

Successfully merging this pull request may close these issues.

5 participants