Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SHA256 not matching while installing docker #15

Closed
jaydeepjc opened this issue Nov 15, 2016 · 63 comments
Closed

SHA256 not matching while installing docker #15

jaydeepjc opened this issue Nov 15, 2016 · 63 comments

Comments

@jaydeepjc
Copy link

@jaydeepjc jaydeepjc commented Nov 15, 2016

I am able to find package

Find-Package -providerName DockerMsftProvider -AllVersions

Name Version Source Summary


Docker 1.12.2-cs2-ws... DockerDefault Contains the CS Docker Engine for use with Windows ...

But when I tried to install I am getting SHA256 mismatch error

PS C:> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Download size: 0MB
VERBOSE: Free space on the drive: 40384.86MB
VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from
'DockerDefault'.".

The package(s) come(s) from a package source that is not marked as trusted.
Are you sure you want to install software from 'DockerDefault'?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): A
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: Verifying Hash of the downloaded file.
WARNING: Cannot verify the file SHA256. Deleting the file.
VERBOSE: Hash verified!
WARNING: C:\Users\ADMINI1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist
Install-Package : Cannot find path
'C:\Users\ADMINI
1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip' because it does not exist.
At line:1 char:1

  • Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
  •   + CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
      + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
     eManagement.Cmdlets.InstallPackage
    
    
    

What to Do?

@jayshah-msft
Copy link
Contributor

@jayshah-msft jayshah-msft commented Nov 15, 2016

@jaydeepjc,
Are you using FIPS enabled machine?

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Nov 16, 2016

@jayshah-msft
No, I am not using FIPS enabled machine, Its simple Windows server 2016 Evalution (Desktop Experience)

@jayshah-msft
Copy link
Contributor

@jayshah-msft jayshah-msft commented Nov 23, 2016

@nampdn
Can you please run the following commands and provide the output:

Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-1-12-2-cs2-ws-beta.zip -Destination /docker.zip

Get-FileHash -Path /docker.zip -Algorithm SHA256

Please provide the output.

Thanks!

@jayshah-msft
Copy link
Contributor

@jayshah-msft jayshah-msft commented Nov 29, 2016

I see that your SHA256 is as expected and I am glad to see that your install works fine. I would still like to know why your first attempt said SHA did not match.
I will investigate on my end. Let me know if you face any further issues.

Cheers,
Jay!

@jayshah-msft
Copy link
Contributor

@jayshah-msft jayshah-msft commented Nov 29, 2016

@jaydeepjc,
Can you please try to download the file and manually check the SHA?

$destinationFolder = "Path to the destination folder"
Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-1-12-2-cs2-ws-beta.zip -Destination $destinationFolder/docker.zip
Get-FileHash -Path $destinationFolder/docker.zip -Algorithm SHA256

Thanks for your help.

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Nov 30, 2016

@jayshah-msft

I can Download and SHA is exactly the same which is in JSON file

    PS C:\> Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-1-12-2-cs2-ws-beta.zip -Destination $destinationFolder/docker.zip
   PS C:\> Get-FileHash -Path $destinationFolder/docker.zip -Algorithm SHA256

    Algorithm       Hash                                                                   Path
    ---------       ----                                                                   ----
    SHA256          FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D        C:\test\docker.zip

However when tried to install, issue still persists

    PS C:\> Install-Package -Name docker -ProviderName DockerMsftProvider

    The package(s) come(s) from a package source that is not marked as trusted.
    Are you sure you want to install software from 'DockerDefault'?
    [Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
    WARNING: Cannot verify the file SHA256. Deleting the file.
    WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-       ws-beta.zip does not exist
    Install-Package : Cannot find path
    'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip'    because it does not exist.
    At line:1 char:1
    + Install-Package -Name docker -ProviderName DockerMsftProvider
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
+ FullyQualifiedErrorId :    PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
       eManagement.Cmdlets.InstallPackage
@DmitryKakurin
Copy link

@DmitryKakurin DmitryKakurin commented Dec 6, 2016

I've had exactly the same problem at first and I've identified the reason: I had a file Get-FileHash.ps1 in my path, and it was called by powershell instead of built-in command.
Since my .ps1 was returning hash in a different format, Install-Package was thinking that hash is incorrect.
The [obvious] fix was to remove my Get-FileHash.ps1.

@jayshah-msft
Copy link
Contributor

@jayshah-msft jayshah-msft commented Dec 6, 2016

@DmitryKakurin,
Thanks for the info. Appreciate your help.
@jaydeepjc, do you have a file named Get-FileHash.ps1 in your path?

Cheers,
Jay!

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 6, 2016

@jayshah-msft

I don't have Get-FileHash.ps1 file in my path.

@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 6, 2016

@jaydeepjc, I recommend you change Get-FileHash to Microsoft.PowerShell.Utility\Get-FileHash in DockerMsftProvider.psm1. Relaunch PowerShell to see how it goes.

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 7, 2016

@jianyunt

I did the changes you suggested, still problem persists. Is there any way to stop verifying SHA256?

@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 7, 2016

You can, but not recommended.
Can you add something like the following in VerifyHashCheck()/DockerMsftProvider.psm1 line 1651, 1654?
Write-Verbose "$hash $Destination $fileHash"

Write-Verbose "$hash $Destination $fileHash $fileSha256"

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 8, 2016

I did added above lines on mentioned line numbers.

Output is as below,

PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose
VERBOSE: Importing package provider 'DockerMsftProvider'.
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Download size: 0MB
VERBOSE: Free space on the drive: 37105.95MB
VERBOSE: Downloading     https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
 C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-    ws-beta' from
'DockerDefault'.".

The package(s) come(s) from a package source that is not marked as trusted.
Are you sure you want to install software from 'DockerDefault'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: Verifying Hash of the downloaded file.
VERBOSE: FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D
C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip
WARNING: Cannot verify the file SHA256. Deleting the file.
VERBOSE: Hash verified!
WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-    beta.zip does not exist
Install-Package : Cannot find path
'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip'     because it does not exist.
 At line:1 char:1
+ Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
+ FullyQualifiedErrorId :       PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
   eManagement.Cmdlets.InstallPackage
@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 8, 2016

Please try adding these and run again. Thanks!
Write-Verbose "hash=$hash"
Write-Verbose "Destination =$Destination"
Write-Verbose "fileHash =$fileHash"
Write-Verbose "fileSha256 =$fileSha256"

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 9, 2016

Code looks like below, is it correct one which I have written?

    $fileHash = Get-FileHash -Path $Destination `
                            -Algorithm SHA256
    Write-Verbose "hash=$hash"
    Write-Verbose "Destination =$Destination"
    Write-Verbose "fileHash =$fileHash"
    
if($fileHash.Psobject.properties.name -Contains "Hash")
{
    $fileSha256 = $fileHash.Hash
    Write-Verbose "fileSha256 =$fileSha256"
}

OUTPUT is as below

 PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose
 VERBOSE: Importing package provider 'DockerMsftProvider'.
 VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
 VERBOSE: Download size: 0MB
 VERBOSE: Free space on the drive: 37091.31MB
 VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
 C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
 VERBOSE: About to download
 VERBOSE: Finished downloading
 VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
 VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from
 'DockerDefault'.".

 The package(s) come(s) from a package source that is not marked as trusted.
 Are you sure you want to install software from 'DockerDefault'?
 [Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
 VERBOSE: Containers feature is already installed. Skipping the install.
 VERBOSE: Verifying Hash of the downloaded file.
 VERBOSE: hash=FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D
 VERBOSE: Destination =C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip
 VERBOSE: fileHash =
 WARNING: Cannot verify the file SHA256. Deleting the file.
 VERBOSE: Hash verified!
 WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist
 Install-Package : Cannot find path
 'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip' because it does not exist.
 At line:1 char:1
 + Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
 + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
 eManagement.Cmdlets.InstallPackage
@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 9, 2016

fileHash is null, meaning Get-FileHash returns null. That;s the reason hash does not match.

can you please add the following?

if(-not (Test-Path -Path $Destination))
{
    Write-Verbose  "File $Destination does not exist!!!"
}
 

$fileHash = Get-FileHash -Path $Destination `
                    -Algorithm SHA256


if (-not $fileHash)
{
    Write-Verbose  "fileHash with path is null"
}
 

$fileHash = Get-FileHash -LiteralPath $Destination `
                    -Algorithm SHA256

if (-not $fileHash)
{
    Write-Verbose  "fileHash with LiteralPath is null"
} 
@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 12, 2016

Below is output

PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose
VERBOSE: Importing package provider 'DockerMsftProvider'.
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Download size: 0MB
VERBOSE: Free space on the drive: 37045.36MB
VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from
'DockerDefault'.".

The package(s) come(s) from a package source that is not marked as trusted.
Are you sure you want to install software from 'DockerDefault'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: Verifying Hash of the downloaded file.
VERBOSE: hash=FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D
VERBOSE: Destination =C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip
VERBOSE: fileHash =
VERBOSE: File C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist!!!
VERBOSE: fileHash with path is null
VERBOSE: fileHash with LiteralPath is null
WARNING: Cannot verify the file SHA256. Deleting the file.
VERBOSE: Hash verified!
WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist
Install-Package : Cannot find path
'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip' because it does not exist.
At line:1 char:1
+ Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
eManagement.Cmdlets.InstallPackage
@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 12, 2016

It looks like Get-FileHash returns null on your machine.
Run gcm get-filehash

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 13, 2016

PS C:\Users\Administrator> gcm get-filehash

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Function        Get-FileHash                                       3.1.0.0    Microsoft.PowerShell.Utility
@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 13, 2016

hmm... need to debug it.

  1. change Get-FileHash to Microsoft.PowerShell.Utility\Get-FileHash
  2. add "wait-debugger" before Microsoft.PowerShell.Utility\Get-FileHash
  3. save and relaunch powershell
  4. run the install package
  5. launch PowerShell ISE, following the debugging provider instruction to exam each variables mentioned above, such as filehash, Destination, etc.
@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 14, 2016

I debug hash , filehash, destination, filehashsha256 however doesnt get any error message
one ex is as below

PS C:\Windows\system32> Get-PSHostProcessInfo

ProcessName    ProcessId AppDomainName    MainWindowTitle                      
-----------    --------- -------------    ---------------                      
powershell          3160 DefaultAppDomain Administrator: Windows PowerShell    
powershell_ise      4720 DefaultAppDomain Administrator: Windows PowerShell ISE



PS C:\Windows\system32> Enter-PSHostProcess 3160

[Process:3160]: PS C:\Users\Administrator\Documents> Get-Runspace

 Id Name            ComputerName    Type          State         Availability   
 -- ----            ------------    ----          -----         ------------   
  1 Runspace1       localhost       Local         Opened        Available      
  2 Runspace2       localhost       Local         Opened        Available      
  3 Runspace3       localhost       Local         Opened        Available      
  4 Runspace4       localhost       Local         Opened        InBreakpoint   
  5 Runspace5       localhost       Local         Opened        Available      
  6 RemoteHost      localhost       Local         Opened        Busy           



[Process:3160]: PS C:\Users\Administrator\Documents> Debug-Runspace 4
Debugging Runspace: Runspace4
[DBG]: [Process:3160]: [Runspace4]: PS C:\Users\Administrator>> $fileHash


[DBG]: [Process:3160]: [Runspace4]: PS C:\Users\Administrator>> 
@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 14, 2016

  1. So you can confirm after executing (Press F10) Microsoft.PowerShell.Utility\Get-FileHash, it returns null, right?

  2. During the debugging can you check if Docker-1-12-2-cs2-ws-beta.zip exists?
    and run Microsoft.PowerShell.Utility\Get-FileHash c:\....Docker-1-12-2-cs2-ws-beta.zip

  3. you can run Microsoft.PowerShell.Utility\Get-FileHash on your PowerShell console no problem, right? e.g., Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe works?

  4. Just curious, add "$a=Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe" in the DockerMsftProvider.psm1 and check to see if $a is null.
    If $a is not null, get-filehash has no problem. it is likely the docker.zip does not get downloaded.

  5. Also, you may run install-package -debug -verbose ...

  6. install-package -debug -verbose -force

  7. Debugging further
    Can you add wait-debugger in line 280, e.g. right before calling DownloadPackageHelper. then F11 step into DownloadPackageHelper function.
    Watch out the code between line 1428 and 1452.
    Step into the function DownloadFile and check if the code line 1526-1538 executed.

  8. by the way, which OS you are running?
    $PSVersionTable to check your PowerShell Version.

  9. We may need a live meeting with you.

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 15, 2016

My OS IS Windows 2016 Server
I have attached the debug file.

  1. I ran wait-debugger again with f10 and found Microsoft.PowerShell.Utility module not found

  2. then ran Import-Module Microsoft.PowerShell.Utility

  3. Then ran Install-package with --debug -verbose

  4. $a= showed null value

  5. Closed poweshell and re-run InstallModule this time $a shows value

  6. Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe works fine, however when tried to check Microsoft.PowerShell.Utility\Get-FileHash c:....Docker-1-12-2-cs2-ws-beta.zip, its not there

  7. My doubt is how come hash file entry shows in verbose mode.

    PS C:\Users\Administrator> $PSVersionTable

     Name                           Value
     ----                           -----
     PSVersion                      5.1.14393.206
     PSEdition                      Desktop
     PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
     BuildVersion                   10.0.14393.206
     CLRVersion                     4.0.30319.42000
     WSManStackVersion              3.0
     PSRemotingProtocolVersion      2.3
     SerializationVersion           1.1.0.1
    

Details are in attached file

Debug.txt

@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 15, 2016

I can see two potential issues. Let’s try to isolate problems whether it is related to download zip file issue or Get-FileHash issue.
Step0:
launch a new PowerShell console, Please confirm Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe works for you.

PS C:\> get-module

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}
Script     1.2        PSReadline                          {Get-PSReadlineKeyHandler, Get-PSReadlineOption, Remove-PSReadlineKeyHandler, ...

PS C:\> Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe

Algorithm       Hash                                                                   Path
---------       ----                                                                   ----
SHA256          BA4038FD20E474C047BE8AAD5BFACDB1BFC1DDBE12F803F473B7918D8D819436       C:\Windows\System32\WindowsPowerShell\v1.0\powers...

Step1:
Close all PowerShell consoles
rename your current MicrosoftDockerProvider under C:\Program Files\WindowsPowerShell\Modules to something else
launch PowerShell
install-module MicrosoftDockerProvider -force -verbose <== this will download the module.

Step 2:
Open the DockerMsftProvider.psm1 from C:\Program Files\WindowsPowerShell\Modules\ and add the following code right after line 1647, i.e. “Write-Verbose "Verifying Hash of the downloaded file."; save it

$a=Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe

if (-not $a)
{
    Write-verbose  "Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe returns null"
}
else
{
    Write-verbose "Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe Successed! $a"
} 

Step 3:
Import-Packageprovider MicrosoftDockerProvider -force -verbose or you can simply relaunch PowerShell Console
Step 4:

Install-package  …. -verbose -force

If $a is null, it is the get-filehash issue on your machine. If $a is not null, It may be the downloading Docker-1-12-2-cs2-ws-beta.zip' issue. let’s continue.

Step 5. Remove the code added in Step 2. Add the following code right after line 1647, i.e. “Write-Verbose "Verifying Hash of the downloaded file."

        if(-not (Test-Path -Path $Destination))
       {
              Write-Verbose  "File $Destination does not exist!!!"
       }

Step6:
Import-Packageprovider MicrosoftDockerProvider -force -verbose
Install-package …. -verbose -force

Please confirm you see the message `File …. Docker-1-12-2-cs2-ws-beta.zip does not exist!!!`.
If so, meaning the Docker-1-12-2-cs2-ws-beta.zip does not get downloaded. Let’s continue.

Step 7:
Looking for “function DownloadFile” in the MicrosoftDockerProvider .psm1. Make changes as suggested below (added write-verbose in 4 places). Save it. Import-Packageprovider MicrosoftDockerProvider -force -verbose
Install-package …. -verbose -force

function DownloadFile
{
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory=$true)]
        [System.String]
        $downloadURL, 
    
    [Parameter(Mandatory=$true)]
    [System.String]
    $destination
)

try
{

Write-Verbose “downloadURL = $downloadURL”
Write-Verbose “destination = $destination”

    if(-not (CheckDiskSpace -Destination $destination -URL $downloadURL))
    {
 Write-Verbose “failed in CheckDiskSpace!!”
	return
    }

    # Download the file
    if($downloadURL.StartsWith("https://"))
    {
        Write-Verbose "Downloading $downloadUrl to $destination"
        $saveItemPath = $PSScriptRoot + "\SaveHTTPItemUsingBITS.psm1"
        Import-Module "$saveItemPath"
        $startTime = Get-Date
        Write-Verbose "About to download"
        Save-HTTPItemUsingBitsTransfer -Uri $downloadURL `
                        -Destination $destination

        Write-Verbose "Finished downloading"
        $endTime = Get-Date
        $difference = New-TimeSpan -Start $startTime -End $endTime
        $downloadTime = "Downloaded in " + $difference.Hours + " hours, " + $difference.Minutes + " minutes, " + $difference.Seconds + " seconds."
        Write-Verbose $downloadTime
    }
else
{
   Write-Verbose “downloadURL does not StartsWith https!!!”
}
catch
{
    ThrowError -CallerPSCmdlet $PSCmdlet `
                -ExceptionName $_.Exception.GetType().FullName `
                -ExceptionMessage $_.Exception.Message `
                -ExceptionObject $downloadURL `
                -ErrorId FailedToDownload `
                -ErrorCategory InvalidOperation        
}

}

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 21, 2016

get-module and Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe working properly as you mention above.
However when I rename C:\Program Files\WindowsPowerShell\Modules to C:\Program Files\WindowsPowerShell\Modules.org

I got below error

PS C:\Users\Administrator> install-module MicrosoftDockerProvider -force -verbose
install-module : The term 'install-module' is not recognized as the name of a cmdlet, function, script file,   or
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
   At line:1 char:1
    + install-module MicrosoftDockerProvider -force -verbose
    + ~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (install-module:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 21, 2016

Rename C:\Program Files\WindowsPowerShell\Modules\MicrosoftDockerProvider to C:\Program Files\WindowsPowerShell\Modules\MicrosoftDockerProvider .org

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 22, 2016

There no folder MicroSoftDockerProvider, I renamed DockerMsftProvider to DockerMsftProvider.org

then below Output

 PS C:\Users\Administrator> install-module MicrosoftDockerProvider -force -verbose
 VERBOSE: Using the provider 'PowerShellGet' for searching packages.
 VERBOSE: The -Repository parameter was not specified.  PowerShellGet will use all of the registered repositories.
 VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'.
 VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2/' and   PackageManagementProvider is
 'NuGet'.
 VERBOSE: Searching repository
 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='MicrosoftDockerProvider'' for ''.
 VERBOSE: Total package yield:'0' for the specified package 'MicrosoftDockerProvider'.
 PackageManagement\Install-Package : No match was found for the specified search criteria and module name
 'MicrosoftDockerProvider'. Try Get-PSRepository to see all available registered module repositories.
 At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1661   char:21
 + ...          $null = PackageManagement\Install-Package @PSBoundParameters
 +                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo          : ObjectNotFound: (Microsoft.Power....InstallPackage:InstallPackage) [Install- Package], Ex
    ception
 + FullyQualifiedErrorId :   NoMatchFoundForCriteria,Microsoft.PowerShell.PackageManagement.Cmdlets.InstallPackage
@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 22, 2016

install-module DockerMsftProvider -force -verbose

@jaydeepjc
Copy link
Author

@jaydeepjc jaydeepjc commented Dec 23, 2016

My Code is

    Write-Verbose "Verifying Hash of the downloaded file."
    $a=Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe
    if (-not $a)
    {
        Write-verbose  "Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe returns null"
    }
    else
    {
        Write-verbose "Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe Successed! $a"
    } 
   $fileHash = Get-FileHash -Path $Destination `
                            -Algorithm SHA256

And below is output

 PS C:\Users\Administrator> Import-Packageprovider DockerMsftProvider -force -verbose
VERBOSE: Importing package provider 'DockerMsftProvider'.

Name                     Version          DynamicOptions
----                     -------          --------------
DockerMsftProvider       1.0.0.1          Update


PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose -force
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Download size: 0MB
VERBOSE: Free space on the drive: 34595.54MB
VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from
'DockerDefault'.".
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: Verifying Hash of the downloaded file.
 VERBOSE: Microsoft.PowerShell.Utility\Get-FileHash C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe returns
 null
 WARNING: Cannot verify the file SHA256. Deleting the file.
 VERBOSE: Hash verified!
 WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist
 Install-Package : Cannot find path
 'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip'   because it does not exist.
 At line:1 char:1
+ Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
+       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~     ~~~~~
   + CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package],  Exception
  + FullyQualifiedErrorId :   PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
eManagement.Cmdlets.InstallPackage

It look likes get-filehash issue is there

@jianyunt
Copy link
Contributor

@jianyunt jianyunt commented Dec 23, 2016

@KrishnaV-MSFT, @lzybkr, @bmanikm, any ideas why Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe returns null?

@gaya3chandran1
Copy link

@gaya3chandran1 gaya3chandran1 commented Jun 1, 2017

I too encounter the same error. Any solution?
PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider

The package(s) come(s) from a package source that is not marked as trusted.
Are you sure you want to install software from 'DockerDefault'?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): A
WARNING: A restart is required to start docker service. Please restart your machine.
WARNING: After the restart please start the docker service.
Install-Package : Cannot rename because item at 'C:\Program Files\dummyName' does not exist.
At line:1 char:1

  • Install-Package -Name docker -ProviderName DockerMsftProvider
  •   + CategoryInfo          : InvalidOperation: (Microsoft.Power....InstallPackage:InstallPackage) [Install-Package],
     Exception
      + FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.RenameItemCommand,Microsoft.PowerShell.Pa
     ckageManagement.Cmdlets.InstallPackage
    
@InteXX
Copy link

@InteXX InteXX commented Jun 24, 2017

The solution, in my case at least, was to modify the Get-HttpResponse() function.

Beginning at line 1133, change this:

1133    $httpClient = New-Object System.Net.Http.HttpClient
1134    $response = $httpclient.GetAsync($Uri)

to this:

1133    $message = New-Object System.Net.Http.HttpRequestMessage
1134    $message.Method = [System.Net.Http.HttpMethod]::Head
1135    $message.RequestUri = $Uri
1136    $httpClient = New-Object System.Net.Http.HttpClient
1137    $response = $httpclient.SendAsync($message)

When the request sends the GET verb, the entire content is downloaded. On low-bandwidth connections such as mine, this results in a timeout. The Response.Result.Content property is null, which causes the subsequent package download to fail.

On the other hand, when we send the HEAD verb only the headers are returned. Since we're only looking for the Content-Length header in order to get the file size, this is all we need.

HTH

@laoshancun
Copy link

@laoshancun laoshancun commented Jun 6, 2018

any updates? still got Docker-17-06-2-ee-13.zip does not exist on windows server 1709
update on 2018-06-11:
it works fine on windows server 1803.

@pablodav
Copy link

@pablodav pablodav commented Jun 13, 2018

Got same issue on Windows server 1803, still trying to find a workaround.

@pablodav
Copy link

@pablodav pablodav commented Jun 13, 2018

What I have confirmed is that the hash is not verified because the file is not at destination, but couldn't find the line that downloads the docker...zip file as it looks like it's not the same as the function that downloads the .json file.

Doing:


$downloadURL = 'https://dockermsft.blob.core.windows.net/dockercontainer/docker-17-06-2-ee-13.zip'
$destination = 'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-17-06-2-ee-13.zip'
Invoke-WebRequest -Uri $downloadURL -OutFile $destination
Install-Package Docker -ProviderName DockerMsftProvider -RequiredVersion $RequiredVersion -Verbose

It will work only without -Force, because -Force flag also removes the file before downloading and then it doesn't downloads there.

File that I have been checking for debugging:

C:\Program Files\WindowsPowerShell\Modules\DockerMsftProvider\1.0.0.4\DockerMsftProvider.psm1

I have never seen the package in the temp folder $destination, looks like some bug in the script doesn't download it or downloads it to some other destination.

@vinicius91
Copy link

@vinicius91 vinicius91 commented Jun 28, 2018

First of all I would like to say thank you for all you guys for the info provided in this Issue.
I had the same problems installing Docker on a Windows Server 2016, but unfortunately @pablodav script didn't work for me. I created another one based on the solution that was giben by @danyhoron .

Here is it:
https://github.com/vinicius91/docker-windows-containers/blob/master/PowerShell-Scripts/install_docker_windows_server.ps1

Cheers!!!

@MagicJohnJang
Copy link

@MagicJohnJang MagicJohnJang commented Dec 21, 2018

Fix the script:

  1. New temp path.
  2. Temp path shows only after failed installation.
  3. Passing parameters.

The script tested on WindowsServer2016core-hyper 14393.0.161119-1705.RS1

work script: https://gist.github.com/MagicJohnJang/46655c4b3228ca2f636e38fc7c84eb11

@PlagueHO
Copy link

@PlagueHO PlagueHO commented Jan 11, 2019

This still occurs on a fresh install of Windows Server 2019:
image

Yes, the methods above do work around the problem. We know this is some issue with the binary file transfer and is resolved by using BITS to do the transfer.

So why is this still a problem after 3 years and why hasn't it been fixed yet?

@sgf
Copy link

@sgf sgf commented Jan 26, 2019

i have the same problem.
Cannot verify the file SHA256. Deleting the file.....

pull the 18.03 is ok
just on pull the 18.09 version.

@AdamWyzgol
Copy link

@AdamWyzgol AdamWyzgol commented Apr 8, 2019

to fix it just start BITS service

@rcaunt
Copy link

@rcaunt rcaunt commented Jul 22, 2019

Similar issue with DockerProvider for 19.03.0-rc2 except in this case the hash doesn't match:

MicrosoftDocs/Virtualization-Documentation#1158

@StefanScherer
Copy link
Contributor

@StefanScherer StefanScherer commented Jul 24, 2019

So, the general fix seems to update the package provider to the latest version available.
This can be done with this command:

Find-PackageProvider DockerMsftProvider | Install-PackageProvider

Then try to install a new Docker package.

@hades200082
Copy link

@hades200082 hades200082 commented Jul 27, 2019

Same issue here - none of the solutions provided above had worked for me.

Windows Server 2019.

@peterwishart
Copy link

@peterwishart peterwishart commented Jul 31, 2019

@hades200082 did you try the suggestion from @AdamWyzgol too? That just fixed this error for me on a fresh Server 2019 install.

@rukshandangalla
Copy link

@rukshandangalla rukshandangalla commented Aug 8, 2019

I faced the same issue and this is how I got it fixed.

  1. Navigate to C:\Users\<USERNAME>\AppData\Local\Temp\DockerMsftProvider

  2. Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-19-03-1.zip -Destination docker-19-03-1.zip

  3. Install-Package -Name Docker -ProviderName DockerMsftProvider -Verbose

Here the main point to look at DockerDefault_DockerSearchIndex.json file in the above mentioned directory. In there you can see SHA256 for docker distrubutions. Check that against the file you get (you can check the SHA by Get-FileHash -Path docker-18-09-0.zip -Algorithm SHA256). Then you can download the correct file and proceed.

@hades200082
Copy link

@hades200082 hades200082 commented Aug 8, 2019

I did manage to get around this issue eventually using the details above but I've given up with Docker on Windows for the time being because it just doesn't seem reliable and stable enough.

Too many things just don't work (try running a container as a DNS server - can't bind port 53 at all on windows)

I've spun up a Linux VM to run all things docker for now. Everything just works there.

@giggio
Copy link

@giggio giggio commented Aug 23, 2019

These fixes work every now and then. I am on a new installation of Server 2019 e I can't them to work.
We shouldn't have to be looking for workarounds. This is supposed to be an enterprise product, supported by Microsoft, but we can't get it to work.
@cwilhit I see you are core maintainer of this project. Would you please help us with this problem? This is a deal breaker to Docker usage on Windows.

@cwilhit
Copy link
Collaborator

@cwilhit cwilhit commented Aug 23, 2019

I haven't been able to repro this bug. An install on a fresh Windows Server 2019 and Windows Server, version 1903 just worked, so it's made it hard to track down what exactly is going wrong.

Reading through the thread, it appears the SHA errors all come back to the issue that Invoke-WebRequest is not completing it's download of the file. So, for folks poor network connectivity, the timeout occurs and the SHA fails. Thus. the suggestions by some folks to use BITStransfer as a work-around because I don't think this has a built-in timeout. That's the theory, anyways.

What would be most helpful to me is: for folks who are hitting this issue:

  1. does using the BITStransfer as described by this thread response successfully complete and download the file?
  2. If you try a standalone invocation of the download in PowerShell, does it succeed? For example: Invoke-WebRequest -Uri https://dockermsft.blob.core.windows.net/dockercontainer/docker-19-03-1.zip -Outfile "docker.zip"?

I am wondering if increasing Invoke-WebRequest's timeout limit is a viable path in that case.

@hades200082
Copy link

@hades200082 hades200082 commented Aug 23, 2019

@cwilhit Yes to both questions.

@InteXX
Copy link

@InteXX InteXX commented Aug 23, 2019

@cwilhit

Are you able to test on a deliberately-slowed network? Here's one way to create such an environment:

https://www.codeproject.com/Articles/18243/Bandwidth-throttling

You might consider using HEAD instead of GET in any case, as GET will always produce a lot of unnecessary overhead.

@StefanScherer
Copy link
Contributor

@StefanScherer StefanScherer commented Aug 24, 2019

Thanks for the hint @InteXX
Indeed, I can reproduce the issue in my VMware Fusion windows-docker-machine "2019-box" and a throttled network.

install-docker-network-192kb-5percent-drop-150ms-latency

@InteXX
Copy link

@InteXX InteXX commented Aug 24, 2019

@StefanScherer

FYI here's the detail again for the fix:

#15 (comment)

@StefanScherer
Copy link
Contributor

@StefanScherer StefanScherer commented Aug 24, 2019

I commented out the remove-item and added more write-verbose commands to see what happens.

PS C:\Users\vagrant> Install-Package Docker -ProviderName DockerMsftProvider -Force -Verbose
VERBOSE: Importing package provider 'DockerMsftProvider'.
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Cannot download link 'https://go.microsoft.com/fwlink/?LinkID=627338&clcid=0x409', retrying for '2' more
times.
VERBOSE: Download size: 0.02MB
VERBOSE: Free space on the drive: 54446.95MB
VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
C:\Users\vagrant\AppData\Local\Temp\1\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 39 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '19.03.1' from
'DockerDefault'.".
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: File C:\Users\vagrant\AppData\Local\Temp\1\DockerMsftProvider\Docker-19-03-1.zip does not exist.
VERBOSE: Verifying Hash of the downloaded file.
VERBOSE: downloaded file:
VERBOSE: SHA256 should be B2015B87D411D77C08FA71974867454F84B0B1B5EDD99E8F091F71B3E0B2770D
VERBOSE: SHA256 actually is
WARNING: Cannot verify the file SHA256. Deleting the file.

So, Invoke-WebRequest really timed out and didn't save any file and that's why the hashes don't match.

This was my changed code in C:\Program Files\WindowsPowerShell\Modules\DockerMsftProvider\1.0.0.7\DockerMsftProvider.psm1

function VerifyHashCheck
{
    param
	(
		[parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [System.String]        
    	$Destination, 
		
		[parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [System.String]        
    	$hash
	)

    if (!(Test-Path $Destination)) {
      Write-Verbose "File $Destination does not exist."
    }

    Write-Verbose "Verifying Hash of the downloaded file."
    Write-Verbose "downloaded file: $(dir $Destination)"

    $fileHash = Get-FileHash -Path $Destination `
                                -Algorithm SHA256
    
    if($fileHash.Psobject.properties.name -Contains "Hash")
    {
        $fileSha256 = $fileHash.Hash
    }
    else
    {
        Write-Verbose "Hash for the original file not available."
        return $false
    }

    Write-Verbose ("SHA256 should be $hash")
    Write-Verbose ("SHA256 actually is $fileSha256")
    return ($hash -ieq $fileSha256)
}

ping @cwilhit I don't know if we can fix this using Invoke-WebRequest, the -TimeoutSecs doesn't help.

I tried curl.exe, but that is only available in Windows Server 2019.

PS C:\Users\vagrant> curl.exe -o docker-19-03-1.zip https://dockermsft.blob.core.windows.net/dockercontainer/docker-19-03-1.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  124M    0  511k    0     0   2103      0 17:11:49  0:04:09 17:07:40  2766
@giggio
Copy link

@giggio giggio commented Aug 26, 2019

This is awesome! When should we expect to have it released?

@cwilhit
Copy link
Collaborator

@cwilhit cwilhit commented Aug 26, 2019

The update is out now

@giggio
Copy link

@giggio giggio commented Dec 19, 2019

I just had to do an install again, and it worked perfectly. It failed, then I updated the provider, tried again, and it worked. Tks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

You can’t perform that action at this time.