Uploading file via Invoke-RestMethod corrupts file

[Binomimus]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues.
• Refer to the FAQ.
• Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

When uploading a file to Citrix Sharefile via API the file somehow gets corrupted. I am uploading a 7-zip archive and the uploaded file is about 50% larger than the original and cannot be opened afterwards (e.g. source file is 180.930 bytes, uploaded file is 271.815 bytes). That issue does not appear when using PowerShell 7.3.5. It first appeared after upgrading to 7.4.1

Example code

$uploadUri = "###removed, see sharefile docs###"
$fileInfo = Get-Item $filepath
$fileBytes = [System.IO.File]::ReadAllBytes($fileInfo.FullName)
$fileEnc = [System.Text.Encoding]::GetEncoding('ISO-8859-1').GetString($fileBytes);
$boundary = [System.Guid]::NewGuid().ToString(); 
$LF = "`r`n";
$bodyLines = (
    "--$boundary",
    "Content-Disposition: form-data; name=`"Filedata`"; filename=`"$($fileInfo.Name)`"",
    "Content-Type: application/octet-stream$LF",
    $fileEnc,
    "--$boundary--$LF" 
) -join $LF

$uploadResult = Invoke-RestMethod -Uri $uploadUri -Method Post -ContentType "multipart/form-data; boundary=$boundary" -Body $bodyLines

The URI for uploading is generated like described in the ShareFile API documentation. See "Upload file" here: https://api.sharefile.com/docs/resource?name=Items

Expected behavior

Uploaded file is same as original

Actual behavior

Uploaded file is larger and corrupted

Error details

There is no error. From PowerShell perspective everything seems to be fine...

Environment data

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.4.1
PSEdition                      Core
GitCommitId                    7.4.1
OS                             Microsoft Windows 10.0.22631
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

[rhubarb-geek-nz]

A couple of questions, why use Invoke-RestMethod rather than Invoke-WebRequest ?

Why are you converting the byte data to string when you are saying in the content-type it is application/octet-stream? That looks like part of the corruption problem.

As an alternative using forms and PowerShell, this is an example of something I use locally

        $web = New-Object -Type 'System.Net.WebClient'

        try
        {
                $web.Headers["Authorization"] = "Basic whatevergoeshere"

                $response = $web.UploadFile('http://10.1.2.12:8080',$Path)
        }
        finally
        {
                $web.Dispose()
        }

[jborean93]

The problem stems from $fileEnc = [System.Text.Encoding]::GetEncoding('ISO-8859-1').GetString($fileBytes); which is taking the raw bytes of the file and trying to encode it into a string. This will only work for files that are actually text in the encoding specified, for binary files the conversion to a string will change the contents.

There should be two ways of doing this

• Using the -InFile parameter
• Building the multipart object and specifying it with -Body

The -InFile is certainly the simplest but I'm unsure whether it will set the content to the form data you've supplied, a quick test seems like it won't. This leaves building the multipart content yourself. I think this should work

$file = Get-Item /tmp/test.txt

$fileStream = $file.OpenRead()
$content = [System.Net.Http.MultipartFormDataContent]::new()
$content.Add(
    [System.Net.Http.StreamContent]::new($fileStream),
    "Filedata",
    $file.Name)

# Content-Type is set automatically from the FormData to
# multipart/form-data, Boundary: "..."
Invoke-RestMethod -Uri $uploadUri -Method Post -Body $content

The multipart content is definitely verbose but it allows you to use the .NET objects to build the form data needed, in this case it's including the FileStream of the file so the raw data being sent will be the raw bytes.

When looking at the raw value sent with the above I found that .NET adds a second filename entry under filename*=utf-8''text.txt and omits the Content-Type: application/octet-stream in the form data. If this is causing problems you can change the above code to the following to build that manually:

$content = [System.Net.Http.MultipartFormDataContent]::new()
$fileContent = [System.Net.Http.StreamContent]::new($fileStream)
$fileContent.Headers.ContentDisposition = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")
# Your example had quotes in your literal form-data example so I kept them here
$fileContent.Headers.ContentDisposition.Name = '"Filedata"'
$fileContent.Headers.ContentDisposition.FileName = '"{0}"' -f $file.Name
$fileContent.Headers.ContentType = 'application/octet-stream'
$content.Add($fileContent)

[Binomimus]

Thank you very much @jborean93 !
At the time of writing the code I found an example for text files and was just happy that encoding ISO-8859-1 seemed to work for zip/7z files as well ... until now. Not sure what changed in PowerShell 7.4.1 though.
The second filename seems not to cause any issues, so I am going to use your first solution. Way mor robust than building the body myself.

Thanks & greetings,
Binomimus

[microsoft-github-policy-service]

📣 Hey @Binomimus, how did we do? We would love to hear your feedback with the link below! 🗣️

🔗 https://aka.ms/PSRepoFeedback

Microsoft Forms

[Binomimus]

Thanks @rhubarb-geek-nz
I think I initially tried it this way but that's not working. I just tested again and received:
"ERROR: upload.aspx: ID='b6eb197a-8969-42fd-a62a-8ade371b7708' No files were received!" as response