Invoke-WebRequest should save the file name with the name in Content-Disposition

[he852100]

How to get a file name from the server when downloading a file?

The server has a Content-Disposition property, how to get it, and is used when downloading files

Steps to reproduce

$URL=@'
https://skydrive.live.com/download?resid=A78ACCAEBB24EDD7!37948&authkey=!AKjXpDoxgONjhUw
'@

Iwr $URL -outfile ./
Or
$a=Iwr $URL -outfile ./sss

Expected behavior

Autoaddfilename
Or
PS >$a.headers
  Key                 Value
---                 -----
x-amz-id-2          {XAI956ZLVN5q12SAfXINuiiU6rUrLD0w88mtCcMt1uLcwwv…
x-amz-request-id    {3EE4C53FE4416D0E}
Date                {Fri, 24 Jan 2020 08:36:04 GMT}
ETag                {"c4ac8566c3831011ab981c9662b33183"}
Accept-Ranges       {bytes}
Server              {AmazonS3}
Last-Modified       {Wed, 18 Dec 2019 08:27:16 GMT}
Content-Disposition {attachment; filename=OpenSSH-Win32.zip}
Content-Type        {application/octet-stream}
Content-Length      {3018600}

Actual behavior

Error or null

Environment data

[he852100]

You must now specify it in a variable. When downloading large files will become difficult

[iSazonov]

If I understand correctly you want to set only target directory and use file name from Content-Disposition.
What behavior (in a script, in interactive session) should be if this header is not present?

[mklement0]

I agree that the following would make perfect sense and constitutes a simple enhancement to Invoke-WebRequest :

• Allow passing a mere directory path to Invoke-WebRequest's -OutFile parameter; e.g., . to ask the downloaded file to be placed in the current directory.

• If the -OutFile argument is indeed a directory path, look for the server-side file name in the Content-Disposition response-header field, and use that as the local file name for the downloaded file; if no such header field is present or if it contains an invalid file name, report a statement-terminating error.

This has been asked for twice before - #6618 and #6537 - but shot down on what appears to me to be flawed reasoning: this is a simple enhancement to an extant feature that makes its use more convenient, not an attempt at "feature creep".

Steve Lee (@SteveL-MSFT), given that and that this is now being asked for for the third time - can we revisit this?

[mklement0]

P.S.: Given that the file name would then be controlled by the server of origin, there's an even more pressing need to implement #3174 and to use -LiteralPath rather than -OutFile.

[he852100]

Need to get 'Content-Disposition' approach, there is currently no method.To get it you must store "iwr" as a variable. It will take up a lot of memory.

Through links, sometimes it's hard to get a file name,

[iSazonov]

There is a reference to RFC Download cmdlet PowerShell/PowerShell-RFC#124

[mklement0]

he852100 , yes, the current workaround is both inconvenient and inefficient (not sure if how the header field is parsed below is robust enough):

$url = 'https://github.com/PowerShell/Win32-OpenSSH/releases/download/v8.1.0.0p1-Beta/OpenSSH-Win32.zip'
$response = iwr $url  # stores the full file content in memory
$filename = $response.Headers.'Content-Disposition' -replace '.*\bfilename=(.+)(?: |$)', '$1'
$outDir = Convert-Path $pwd
[IO.File]::WriteAllBytes("$outDir/$filename", $response.Content)

Yes, Ilya (@iSazonov), but that is a separate matter - a dedicated downloading cmdlet with all the bells and whistles is also nice to have.

As stated, this is simply about making an existing feature more convenient - arguably, it should have worked that way from the beginning.

[iSazonov]

Michael Klement (@mklement0) The RFC comes from a fact that web cmdlets are already very complex - they have over 30 parameters! And now we discuss that -OutFile will have additional semantics. This not only opens the way to new errors (how to distinguish path and file? What is user skip final slash? It is very bad UX for script language.) and typos, but it also confuses users by complicity.

[mklement0]

they have over 30 parameters!

So it's a good thing that this enhancement doesn't require a new parameter.

how to distinguish path and file? What is user skip final slash?

The semantics would be the exact same as with Copy-Item with a file source path, for instance: if the target path refers to an existing directory, copy to that directory - whether or not there is a trailing \ or /; if the target path doesn't exist, treat it as file.

No additional complexity, just long-established and well-known patterns.

[he852100]

Ilya (@iSazonov) Can I transfer headers to a variable when I have the OutFile parameter?

You just need to provide this feature,The user determines the processing method by the variable value.

$a=iwr f outfile ./dff.vvg
$a=$a.headers xxx
rename-item xxx

[iSazonov]

I should again notice that web cmdlets is very complex. We should avoid to add more complicity. For file processing scenarios we have Mark Kraus (@markekraus) 's RFC.

[mklement0]

As noted, enhancing Invoke-WebRequest as proposed and the RFC aren't mutually exclusive.
The proposed feature adds little complexity, from what I can tell, and only makes the download feature that is already there and won't go away complete.

[mklement0]

he852100: The existing -PassThru switch provides similar functionality in conjunction with -OutFile, but it passes the entire response through - including the whole file content in property .Content.

I don't see a need for a new parameter that passes just the header through - the proposed enhancement (arguably: fix) requires no new parameters and is the most straightforward solution to the problem.