Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy-Item from linux to windows with ToSessopn error #4952

Open
rusnw opened this issue Sep 29, 2017 · 15 comments
Open

Copy-Item from linux to windows with ToSessopn error #4952

rusnw opened this issue Sep 29, 2017 · 15 comments
Labels
Area-FileSystem-Provider specific to the FileSystem provider Issue-Enhancement the issue is more of a feature request than a bug WG-Cmdlets-Management cmdlets in the Microsoft.PowerShell.Management module WG-Engine-Providers built-in PowerShell providers such as FileSystem, Certificates, Registry, etc. WG-Remoting PSRP issues with any transport layer

Comments

@rusnw
Copy link

rusnw commented Sep 29, 2017

Steps to reproduce

Create 2 file:

dd if=/dev/zero of=/root/30KbFile bs=30K count=1
dd if=/dev/zero of=/root/1MbFile bs=30K count=1

Script for copying of files from linux in windows:

$DestinationHost = "windows_hostname"
$username = 'admin_user'
$password = 'admin_pass'

$secureString = $password | ConvertTo-SecureString -AsPlainText -Force 
$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $username, $secureString
$Session = New-PSSession -ComputerName $DestinationHost -Credential $cred -Authentication basic

$Source = "/root/30KbFile"
$WorkDir = "C:\"
Copy-Item -Path $Source -Destination $WorkDir -Recurse -Force -ToSession $Session -Verbose

$Source = "/root/1MbFile"
$WorkDir = "C:\"
Copy-Item -Path $Source -Destination $WorkDir -Recurse -Force -ToSession $Session -Verbose

Run script

Expected behavior

Files were copied on Windows host in C:\

Actual behavior

30KbFile file was copied, 1MbFile was not copied:


VERBOSE: Performing the operation "Copy File" on target "Item: /root/30KbFile Destination: C:/CI".
VERBOSE: Performing the operation "Copy File" on target "Item: /root/1MbFile Destination: C:/CI".   
Copy-Item : Starting a command on the remote server failed with the following error message : MI_RESULT_FAILED For more information, see the about_Remote_Troubleshooting Help topic. At /root/test.ps1:14 char:1
+ Copy-Item -Path $Source -Destination $WorkDir -Recurse -Force -ToSess ...                                                                                                                                   
+ 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                                                                           
+ CategoryInfo          : InvalidOperation: (System.Management.Automation.PowerShell:PowerShell) [Copy-Item], PSRemotingTransportException                                                                    
+ FullyQualifiedErrorId : CopyFileRemoteExecutionError,Microsoft.PowerShell.Commands.CopyItemCommand                                                                                                                                                                                                                                                                                                                    Copy-Item : Failed to copy file /root/1MbFile to remote target destination.                                                                                                                                   At /root/test.ps1:14 char:1                                                                                                                                                                                   
+ Copy-Item -Path $Source -Destination $WorkDir -Recurse -Force -ToSess ...                                                                                                                                   
+ 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                                                                           
+ CategoryInfo          : WriteError: (/root/1MbFile:String) [Copy-Item], IOException                                                                                                                         
+ FullyQualifiedErrorId : CopyError,Microsoft.PowerShell.Commands.CopyItemCommand                   

Environment data

                                                                                                                                                                                                              PS /root> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta
PSEdition                      Core
GitCommitId                    v6.0.0-beta.7
OS                             Linux 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

@rusnw rusnw changed the title Copy-Item from linux to windows wint ToSessopn error Copy-Item from linux to windows with ToSessopn error Sep 29, 2017
@SteveL-MSFT SteveL-MSFT added the WG-Remoting PSRP issues with any transport layer label Sep 29, 2017
@SteveL-MSFT
Copy link
Member

SteveL-MSFT commented Sep 29, 2017

My recommendation is to install OpenSSH on Windows and use SCP

@SteveL-MSFT
Copy link
Member

SteveL-MSFT commented Sep 30, 2017

@dantraMSFT is working on getting new build of psrpclient library

@rusnw
Copy link
Author

rusnw commented Oct 5, 2017

OpenSSH in the current case will suit me as the workaround decision, but with it a creation problem DataSources 64-bit.
64 bit Datasource via ODBCCONF does not create through OpenSSH on Windows even in case of explicit specifying. It is created 32bit by DataSource in any tried case. It was the cause to refuse OpenSSH in other case.

@TravisEz13 TravisEz13 added the Issue-Enhancement the issue is more of a feature request than a bug label Apr 25, 2018
@couragecc
Copy link

couragecc commented Aug 28, 2018

Hi All,

Is this problem solved? I have the same problem:-(

@dantraMSFT
Copy link
Contributor

dantraMSFT commented Aug 28, 2018

@couragecc would you mind trying this out with the latest RC Candidate or the 6.04 release

Thanks.

@couragecc
Copy link

couragecc commented Aug 29, 2018

Here is current powershell version on my environment.

Ubuntu:
PS /opt/tivoli/tsm/client/ba/bin> $PSVersionTable

Name Value


PSVersion 6.1.0-preview.2
PSEdition Core
GitCommitId v6.1.0-preview.2
OS Linux 4.4.0-87-generic #110-Ubuntu SMP Tue Jul 18 12:55:35 UTC 2017
Platform Unix
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0

Windows:
PS C:\Users\Administrator.ICEBERG> $PSVersionTable

Name Value


PSVersion 5.1.14409.1005
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.14409.1005
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1

@couragecc
Copy link

couragecc commented Aug 29, 2018

I try installing the latest version powershell core 6.04 on Ubuntu as your suggestion, but still get the same problem on Windows remote powershell.
PS C:\Users\Administrator.ICEBERG> $o = New-PSSessionOption -SkipCACheck -SkipRevocationCheck -SkipCNCheck
PS C:\Users\Administrator.ICEBERG> $session = New-PSSession -ComputerName "192.168.137.89" -Credential $cred -Authentication basic -SessionOption $o -USESSL

PS C:\Users\Administrator.ICEBERG> Copy-Item "C:\cc\projects\workspace-trunk\cli\makefile.tsm" -Destination "/home/cc/project/" -ToSession $session -verbose
VERBOSE: Performing the operation "Copy File" on target "Item: C:\cc\projects\workspace-trunk\cli\makefile.tsm
Destination: \home\cc\project".
Copy-Item : Processing data from remote server 192.168.137.89 failed with the following error message: A general error
occurred, not covered by a more specific error code. For more information, see the about_Remote_Troubleshooting Help
topic.
At line:1 char:1

  • Copy-Item "C:\cc\projects\workspace-trunk\cli\makefile.tsm" -Destinat ...
  •   + CategoryInfo          : InvalidOperation: (System.Management.Automation.PowerShell:PowerShell) [Copy-Item], PSRe
     motingTransportException
      + FullyQualifiedErrorId : CopyFileRemoteExecutionError,Microsoft.PowerShell.Commands.CopyItemCommand
    
    

Copy-Item : Failed to copy file C:\cc\projects\workspace-trunk\cli\makefile.tsm to remote target destination.
At line:1 char:1

  • Copy-Item "C:\cc\projects\workspace-trunk\cli\makefile.tsm" -Destinat ...
  •   + CategoryInfo          : WriteError: (C:\cc\projects\...li\makefile.tsm:String) [Copy-Item], IOException
      + FullyQualifiedErrorId : CopyError,Microsoft.PowerShell.Commands.CopyItemCommand
    
    

Copy-Item : The runspace state is not valid for this operation.
At line:1 char:1

  • Copy-Item "C:\cc\projects\workspace-trunk\cli\makefile.tsm" -Destinat ...
  •   + CategoryInfo          : InvalidOperation: (System.Management.Automation.PowerShell:PowerShell) [Copy-Item], Inva
     lidRunspaceStateException
      + FullyQualifiedErrorId : CopyFileRemoteExecutionError,Microsoft.PowerShell.Commands.CopyItemCommand
    
    

Copy-Item : Cannot invoke the pipeline because the runspace is not in the Opened state. Current state of the runspace
is 'Broken'.
At line:1 char:1

  • Copy-Item "C:\cc\projects\workspace-trunk\cli\makefile.tsm" -Destinat ...
  •   + CategoryInfo          : NotSpecified: (:) [Copy-Item], InvalidRunspaceStateException
      + FullyQualifiedErrorId : System.Management.Automation.Runspaces.InvalidRunspaceStateException,Microsoft.PowerShel
     l.Commands.CopyItemCommand
    
    
    
    

@dantraMSFT
Copy link
Contributor

dantraMSFT commented Aug 29, 2018

@couragecc - the examples you are providing are the opposite of the title of the issue.

The title indicates you're copying from Linux to Windows yet the example appears to be in the opposite direction.

If you are going from Windows to Linux, I strongly recommend you use OpenSSH on both ends. It completely avoids having to use the OMI server on the target Linux machine and works quite well when using an RSA key.

For example:

$session = New-PSSession -HostName mylinuxsystem -UserName myuser -KeyFilePath ~\.ssh\myuser
Copy-Item -Path ~\myfile.txt -Destination /home/myuser -ToSession $session

@couragecc
Copy link

couragecc commented Aug 30, 2018

@dantraMSFT Thanks a lot!

One more question: Shall I use OpenSSH for Copy files from Windows (Powershell 5) to linux (Powershell core 6.0) ?(This is my scenario)

I saw some examples for both Linux and Windows side are Powershell core 6.0.

@SteveL-MSFT
Copy link
Member

SteveL-MSFT commented Aug 30, 2018

@couragecc yes, recommendation is to use OpenSSH (scp) to copy files to and from Windows/Linux. This will work faster than trying to use copy-item (currently) due to base64 encoding the payload and wrapping it in XML for WSMan transport.

@yorkshenpolestar
Copy link

yorkshenpolestar commented Nov 29, 2018

@SteveL-MSFT @dantraMSFT , for the issue: Unable copy big file from Windows to Linux when we use copy-item

SSH remote did resolve this problem. But if customer still need OMI+PSR to do this for compatibility reason, how can we resolve it? Any suggestion or tips will be greatly appreciated!

@SteveL-MSFT
Copy link
Member

SteveL-MSFT commented Dec 1, 2018

@aixiaozi the problem is that WSMan support on non-Windows hasn't been reliable. Plan is to move away from OMI client and invest in new PSRP/WSMan client library but that is a big work item and won't happen any time soon. You can also use tools like smbclient on Linux to copy file to a SMB share on Windows.

@sigv
Copy link

sigv commented Apr 5, 2020

Copy-Item from a Linux system to a Windows session still seems to not be functional for me. Additionally, microsoft/azure-pipelines-tasks#12535 has been opened last month, which also seems to imply this is not yet solved.

$Credential = Get-Credential -UserName example.com\username
$Session = New-PSSession -ComputerName destination.example.com -Credential $Credential -Authentication Negotiate
Copy-Item -Path patch.zip -Destination D:\Directory\patch.zip -ToSession $Session -Force
Remove-PSSession -Session $Session

Still results in the following generic error:

Copy-Item: Starting a command on the remote server failed with the following error message : MI_RESULT_FAILED For more information, see the about_Remote_Troubleshooting Help topic.
Copy-Item: Failed to copy file /var/atlassian/application-data/bamboo/xml-data/build-dir/123456789-123456789/patch.zip to remote target destination.

Edit: And just as in the initial report, if the patch.zip is a small file then it is copied just fine. This is only a problem with larger file sizes.


Edit 2: It does seem smbclient does get the job done very well for us. We have a functional krb5 and Samba setup on these systems, so that is presumably making this very smooth.

$ smbclient '\\destination.example.com\D$' -N -c 'put patch.zip Directory\patch.zip' 
putting file patch.zip as \Directory\patch.zip (12470.3 kb/s) (average 12470.3 kb/s)

Regardless of the above edit, it would seem preferable for us to use a single session for copying a file, then unpacking it and running a few local commands, therefore the questions still stand:
Are there any updates on WSMan transport improvements? Any flags that can be toggled as of now?

@alusvedejs
Copy link

alusvedejs commented Feb 5, 2021

4 years later the issue is still effective.

`Copy-Item –Path testfile.txt –Destination 'C:\Installables' –ToSession (New-PSSession -Credential $cred -ComputerName $server -Authentication Negotiate) -Recurse -Force -Verbose
VERBOSE: Performing the operation "Copy File" on target "Item: /home/jenkins/agent/workspace/MARKETS/Deploy/DEV/z_debug_tmp/testfile.txt Destination: C:/Installables/".

Copy-Item –Path large-5.43.40.zip –Destination 'C:\Installables' –ToSession (New-PSSession -Credential $cred -ComputerName $server -Authentication Negotiate) -Recurse -Force -Verbose
VERBOSE: Performing the operation "Copy File" on target "Item: /home/jenkins/agent/workspace/MARKETS/Deploy/DEV/z_debug_tmp/large-5.43.40.zip Destination: C:/Installables/".
Copy-Item: Starting a command on the remote server failed with the following error message : MI_RESULT_FAILED For more information, see the about_Remote_Troubleshooting Help topic.
Copy-Item: Failed to copy file /home/jenkins/agent/workspace/MARKETS/Deploy/DEV/z_debug_tmp/large-5.43.40.zip to remote target destination.`
are there good workarounds?

@thiagodpaula
Copy link

thiagodpaula commented Mar 26, 2021

Hello Everybody,

I was looking for the solution to this problem and I was able to find a website with the solution for this case.

Check the link https://serverfault.com/questions/1020068/settings-to-upload-large-file-via-winrm-and-copy-item-powershell

It describes that it is necessary to change the parameter MaxEnvelopeSizekb

First check the current value with the command get-WSManInstance -ResourceURI winrm / config | Select-Object MaxEnvelopeSizekb

Then change it to a higher value: Invoke-Command -Session $ s1 -ScriptBlock {set-WSManInstance -ResourceURI winrm / config -ValueSet @ {MaxEnvelopeSizekb = "256000"}}

close #4952

Server Fault
I´m trying to copy a large file (around 180MB) to a remote server using PowerShell Copy-Item.

Copy-Item –Path "$sourcePath" –Destination "$destPath" -Force –ToSession $session
It works for files...

@iSazonov iSazonov added Area-FileSystem-Provider specific to the FileSystem provider WG-Cmdlets-Management cmdlets in the Microsoft.PowerShell.Management module WG-Engine-Providers built-in PowerShell providers such as FileSystem, Certificates, Registry, etc. labels Dec 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-FileSystem-Provider specific to the FileSystem provider Issue-Enhancement the issue is more of a feature request than a bug WG-Cmdlets-Management cmdlets in the Microsoft.PowerShell.Management module WG-Engine-Providers built-in PowerShell providers such as FileSystem, Certificates, Registry, etc. WG-Remoting PSRP issues with any transport layer
Projects
None yet
Development

No branches or pull requests

11 participants