Set-SFTPItem fails sometimes with ArgumentOutOfRangeException

[DarkLite1]

Description

We have a script that starts multiple jobs with Start-Job to upload or download files.

Once in a while we have a file that is not uploaded and we receive a non terminating error like this:
The background process reported an error with the following message: Unhandled exception. System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index').

What we did to find the issue

When adding Start-Transcript to Send to SFTP.ps1, which is the script that is called within the job to upload the file, we get these verbose messages:

VERBOSE: Test path '\server\d$\interface_files\moha\out'
VERBOSE: Get files in folder '\server\d$\interface_files\moha\out'
VERBOSE: Only include files with extension '.dat .txt'
VERBOSE: Create SFTP credential
VERBOSE: Start SFTP session
VERBOSE: Using SSH Key authentication for connection.
VERBOSE: Test SFTP path '/moha/out/' exists
VERBOSE: Get all SFTP files
VERBOSE: File '\server\d$\interface_files\moha\out\MO_C_FIN.txt'
VERBOSE: Rename source file
VERBOSE: Upload file '\server\d$\interface_files\moha\out\MO_C_FIN.txt.UploadInProgress'
VERBOSE: Uploading \server\d$\interface_files\moha\out\MO_C_FIN.txt.UploadInProgress to /moha/out/
VERBOSE: File uploaded
VERBOSE: Remove file
VERBOSE: Renaming /moha/out/MO_C_FIN.txt.UploadInProgress to MO_C_FIN.txt
VERBOSE: File renamed
VERBOSE: File '\server\d$\interface_files\moha\out\MO_Complet.dat'
VERBOSE: Rename source file
VERBOSE: Upload file '\server\d$\interface_files\moha\out\MO_Complet.dat.UploadInProgress'

This is the end of the transcript.

The issue

It does never reach the message Write-Verbose 'File uploaded' and the function Set-SFTPItem does not throw a terminating error although we use $ErrorActionPreference = 'Stop' at the top of the script.

The file is never uploaded and this happens only occasionally. We run 5 jobs simultaneously. We're a bit lost on why this is not going to the Catch clause or why the file is not uploaded after calling Set-SFTPItem.

Thank you for your help.

Possible root cause

It looks like Set-SFTPItem quits the entire script when something goes wrong like say the SFTP server aborting the connection during transfer.

Extra info

With downloading files by using Get-SFTPItem we never have an issue.

System info

• PowerShell 7.4.0
• Windows Server 2022
• Posh-ssh 3.1.1

[MVKozlov]

You didn't show how exactly you call your script inside the RSJob

I suspect that the error is there, because you do not use the index in a $sessionParams , and the only mention of the index inside the Set-SFTPItem is just getting a SessionId from Index thru $Global:SftpSessions

You can also try using my RSJob fork as it fixes a lot of original bugs, maybe yours is there :)

[DarkLite1]

Thank you for getting back to us.

At line 579 of this script we call the child script file with Start-Job. At line 437 of the child script file we exectue Set-SFTPItem.

Simplified view:

# Main script "Get or send SFTP file.ps1"
$actions | foreach-Object {
   $invokeParams = @{
           FilePath     = '../Send to SFTP.ps1'
           ArgumentList = ....
   }
   Start-Job @invokeParams
}

# Child script "Send to SFTP.ps1"
$params = @{
    ComputerName = $SftpComputerName
    Credential   = $sftpCredential
    AcceptKey    = $true
}
$sftpSession = New-SFTPSession @params

$sessionParams = @{
    SessionId = $sftpSession.SessionID
}

$params = @{
     Path        = $tempFile.UploadFilePath
     Destination = $SftpPath
}
Set-SFTPItem @sessionParams @params

The code of the child script is fully isolated from all other code and is running in its own PSJob. So there can be no pollution of variables due to parallelism or anything else. This code also functions fine 9 out of 10 times.

Can you tell me what you mean with not using the index or how we should use the index correctly?

[MVKozlov]

Sorry, I read -RSJob instead of -Job

about index:
you get
System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index').
but there are no parameters called 'index' here

I think you need full traceback, not just exception

[DarkLite1]

Thanks for getting back to us.

We posted the full stack trace here. It's probably related to .NET 8 but we're not knowledgeable enough to understand this in detail.

[DarkLite1]

This seems to be an issue within PowerShell itself