Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azure-arm WinRM Time out #31188

Closed
Anbukugan opened this issue May 14, 2019 · 37 comments
Closed

azure-arm WinRM Time out #31188

Anbukugan opened this issue May 14, 2019 · 37 comments

Comments

@Anbukugan
Copy link

Anbukugan commented May 14, 2019

Packer azure-arm WinRM Issue

I have included my JSON file below

{
"builders": [{
"type": "azure-arm",

"client_id": "XXXXXX",
"client_secret": "XXXXXX",
"tenant_id": "XXXXXX",
"subscription_id": "XXXXXX",

"managed_image_resource_group_name": "test",
"managed_image_name": "myPackerImage",

"os_type": "Windows",
"image_publisher": "MicrosoftWindowsServer",
"image_offer": "WindowsServer",
"image_sku": "2016-Datacenter",

"communicator": "winrm",
"winrm_use_ssl": true,
"winrm_insecure": true,
"winrm_timeout": "3m",
"winrm_password": "Welcome@123456",
"winrm_username": "packer",

"azure_tags": {
    "dept": "Engineering",
    "task": "Image deployment"
},
"build_resource_group_name" : "test",
"virtual_network_name": "test",
"virtual_network_resource_group_name" : "test",
"virtual_network_subnet_name" : "test",
"vm_size": "Standard_DS2_v2"
}],
"provisioners": [{
"type": "powershell",
"inline": [
  "Add-WindowsFeature Web-Server",
  "& $env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit",
  "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10  } else { break } }"
]
 }]
}

I have used the existing resource group, VNET and subnet. But it times out while connecting to the VM. It shows the following error
azure-arm output will be in this color.

==> azure-arm: Running builder ...
==> azure-arm: Getting tokens using client secret
azure-arm: Creating Azure Resource Manager (ARM) client ...
==> azure-arm: Using existing resource group ...
==> azure-arm: -> ResourceGroupName : 'test'
==> azure-arm: -> Location : 'uksouth'
==> azure-arm: Validating deployment template ...
==> azure-arm: -> ResourceGroupName : 'test'
==> azure-arm: -> DeploymentName : 'pkrdpn19ib1zopk'
==> azure-arm: Deploying deployment template ...
==> azure-arm: -> ResourceGroupName : 'test'
==> azure-arm: -> DeploymentName : 'kvpkrdpn19ib1zopk'
==> azure-arm: Getting the certificate's URL ...
==> azure-arm: -> Key Vault Name : 'pkrkvn19ib1zopk'
==> azure-arm: -> Key Vault Secret Name : 'packerKeyVaultSecret'
==> azure-arm: -> Certificate URL : 'https://pkrkvn19ib1zopk.vault.azure.net/secrets/packerKeyVaultSecret/37221457a5a3459fa0da7f3745e1c9ee'
==> azure-arm: Setting the certificate's URL ...
==> azure-arm: Validating deployment template ...
==> azure-arm: -> ResourceGroupName : 'test'
==> azure-arm: -> DeploymentName : 'pkrdpn19ib1zopk'
==> azure-arm: Deploying deployment template ...
==> azure-arm: -> ResourceGroupName : 'test'
==> azure-arm: -> DeploymentName : 'pkrdpn19ib1zopk'
==> azure-arm: Getting the VM's IP address ...
==> azure-arm: -> ResourceGroupName : 'test'
==> azure-arm: -> PublicIPAddressName : 'pkripn19ib1zopk'
==> azure-arm: -> NicName : 'pkrnin19ib1zopk'
==> azure-arm: -> Network Connection : 'PrivateEndpoint'
==> azure-arm: -> IP Address : '10.1.0.5'
==> azure-arm: Waiting for WinRM to become available...
==> azure-arm: Timeout waiting for WinRM.
==> azure-arm:
==> azure-arm: The resource group was not created by Packer, deleting individual resources ...
==> azure-arm: -> Deployment: pkrdpn19ib1zopk
==> azure-arm: -> Microsoft.Compute/virtualMachines : 'pkrvmn19ib1zopk'
==> azure-arm: -> Microsoft.Network/networkInterfaces : 'pkrnin19ib1zopk'
==> azure-arm: -> Microsoft.Compute/disks : '/subscriptions/2b87c584-986d-4a3c-9e56-012df416db72/resourceGroups/test/providers/Microsoft.Compute/disks/pkrosn19ib1zopk'
==> azure-arm:
==> azure-arm: The resource group was not created by Packer, deleting individual resources ...
==> azure-arm: Could not retrieve OS Image details
==> azure-arm: -> Deployment: kvpkrdpn19ib1zopk
==> azure-arm: -> Microsoft.KeyVault/vaults/secrets : 'pkrkvn19ib1zopk/packerKeyVaultSecret'
==> azure-arm: -> Microsoft.KeyVault/vaults : 'pkrkvn19ib1zopk'
==> azure-arm: -> : ''
==> azure-arm: Error deleting resource. Please delete manually.
==> azure-arm:
==> azure-arm: Name:
==> azure-arm: Error: Unable to parse path of image
==> azure-arm:
==> azure-arm: The resource group was not created by Packer, not deleting ...
Build 'azure-arm' errored: Timeout waiting for WinRM.

==> Some builds didn't complete successfully and had errors:
--> azure-arm: Timeout waiting for WinRM.

==> Builds finished but no artifacts were created.


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

@mimckitt
Copy link
Contributor

@Anbukugan please provide me with the link to the document you are following so I can better assist.

@Anbukugan
Copy link
Author

@MicahMcKittrick-MSFT I followed the guide form these websites below
https://packer.io/docs/builders/azure.html

https://docs.microsoft.com/en-us/azure/virtual-machines/windows/build-image-with-packer

@mimckitt
Copy link
Contributor

@cynthn Looks like I am seeing issues as well running through the document as is

PS C:\packer> ./packer build windows.json
azure-arm output will be in this color.

==> azure-arm: Running builder ...
==> azure-arm: Getting tokens using client secret
    azure-arm: Creating Azure Resource Manager (ARM) client ...
==> azure-arm: WARNING: Zone resiliency may not be supported in East US, checkout the docs at https://docs.microsoft.com/en-us/azure/availability-zones/
==> azure-arm: Creating resource group ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-sm3hz2jhln'
==> azure-arm:  -> Location          : 'East US'
==> azure-arm:  -> Tags              :
==> azure-arm:  ->> dept : Engineering
==> azure-arm:  ->> task : Image deployment
==> azure-arm: Validating deployment template ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-sm3hz2jhln'
==> azure-arm:  -> DeploymentName    : 'pkrdpsm3hz2jhln'
==> azure-arm: Deploying deployment template ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-sm3hz2jhln'
==> azure-arm:  -> DeploymentName    : 'kvpkrdpsm3hz2jhln'
==> azure-arm: Getting the certificate's URL ...
==> azure-arm:  -> Key Vault Name        : 'pkrkvsm3hz2jhln'
==> azure-arm:  -> Key Vault Secret Name : 'packerKeyVaultSecret'
==> azure-arm:  -> Certificate URL       : 'https://pkrkvsm3hz2jhln.vault.azure.net/secrets/packerKeyVaultSecret/c98583cbc60449da82587df60d9e6f4c'
==> azure-arm: Setting the certificate's URL ...
==> azure-arm: Validating deployment template ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-sm3hz2jhln'
==> azure-arm:  -> DeploymentName    : 'pkrdpsm3hz2jhln'
==> azure-arm: Deploying deployment template ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-sm3hz2jhln'
==> azure-arm:  -> DeploymentName    : 'pkrdpsm3hz2jhln'
==> azure-arm:
==> azure-arm: Future#WaitForCompletion: context has been cancelled: StatusCode=200 -- Original Error: context deadline exceeded
==> azure-arm:
==> azure-arm: Cleanup requested, deleting resource group ...
==> azure-arm: Resource group has been deleted.
Build 'azure-arm' errored: Future#WaitForCompletion: context has been cancelled: StatusCode=200 -- Original Error: context deadline exceeded

==> Some builds didn't complete successfully and had errors:
--> azure-arm: Future#WaitForCompletion: context has been cancelled: StatusCode=200 -- Original Error: context deadline exceeded

==> Builds finished but no artifacts were created.

@TomArcherMsft are you still/ also working on packer stuff? Any suggestions?

@mimckitt
Copy link
Contributor

mimckitt commented May 17, 2019

@Anbukugan can you try increasing the WinRM timeout to 10m?

"winrm_timeout": "10m",

This fixed the issue for me.

@mimckitt
Copy link
Contributor

@Anbukugan I updated the doc and increased the default timeout period from 3 minutes to 5 which seems to have fixed the error in my tests.

I will close this out but if you are still having issues after increasing the timeout period just let me know.

@sanchetanparmar
Copy link

i am using same code with "winrm_timeout": "20m", but still getting same error(azure-arm: Waiting for WinRM to become available...) with windows2016. its working with windows2019.

@AkosBatorfi
Copy link

I have the same issue, with the same configuration. On the hashicorp site I'm seeing others have the same issue, and indeed if I change from "2016-datacenter" to "2019-datacenter" the build runs just fine. This is happening for me both in Azure Devops with hosted agents as on my own laptop.

@harsh-vm
Copy link
Contributor

I had the same time-out issue. I got this working by increasing the WinRM time-out from 5 mins to 10 mins. But, I continued to use image_sku -'2016-datacenter', didn't change this one!

@sanchetanparmar
Copy link

@harsh-vm when did you try with latest 2016-Datacenter ? currently in 2016-datacenter WinRM disabled so how would you enable it ?

@harsh-vm
Copy link
Contributor

harsh-vm commented Dec 19, 2019

@sanchetanparmar I am not sure what's the source of your comment of WinRm being disabled in 2016-datacenter as I have used it yesterday and it still works as expected. For reference, this is what I have used under my 'builders' section of packer template:

   `"os_type": "Windows",

   "image_publisher": "MicrosoftWindowsServer",

   "image_offer": "WindowsServer",

   "image_sku": "{{user `image_sku`}}",

   "communicator": "winrm",

   "winrm_use_ssl": "true",

   "winrm_insecure": "true",

   "winrm_timeout": "10m",

   "winrm_username": "packer",`

where, image_sku is 2016-datacenter. Hope this helps..

@sanchetanparmar
Copy link

sanchetanparmar commented Dec 19, 2019

@harsh-vm Created a VM with 2016 and checked WinRM status which is disabled. Here my packer template

Packer - 1.4.5 and 1.5.0

{
  "builders": [{
    "type": "azure-arm",

    "subscription_id": "XXXXXX",
    "client_id":  "XXXXXX,
    "client_secret":    "XXXXXX",
    "tenant_id":  "XXXXXXX",

    "managed_image_resource_group_name": "myResourceGroup",
    "build_resource_group_name": "myResourceGroup",
    "managed_image_name": "PackerImagewin2016-DC",
    "os_type": "Windows",
    "image_publisher": "MicrosoftWindowsServer",
    "image_offer": "WindowsServer",
    "image_sku": "2016-Datacenter",

    "communicator": "winrm",
    "winrm_use_ssl": true,
    "winrm_insecure": true,
    "winrm_timeout": "10m",
    "winrm_username": "packer",

    "vm_size": "Standard_DS2_v2"
  }],
  "provisioners": [{
    "type": "powershell",
    "inline": [
      "Add-WindowsFeature Web-Server",
      "& $env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit",
      "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10  } else { break } }"
    ]
  }]
}

just tried above code working for 2012-R2-Datacenter and 2016-Datacenter-Server-Core but not for 2016-Datacenter

2016-Datacenter
==> azure-arm: Waiting for WinRM to become available...

==> azure-arm: Timeout waiting for WinRM.

@Mcphejoh
Copy link

Thank you for posting this I was updating powershell and the winrm is timing out - when I attempt to connect using MAC Remote Desktop I get connection refused. going to revert to 2012 for now just to get some Powershell testing done.

@Dilergore
Copy link

@harsh-vm Created a VM with 2016 and checked WinRM status which is disabled. Here my packer template

Packer - 1.4.5 and 1.5.0

{
  "builders": [{
    "type": "azure-arm",

    "subscription_id": "XXXXXX",
    "client_id":  "XXXXXX,
    "client_secret":    "XXXXXX",
    "tenant_id":  "XXXXXXX",

    "managed_image_resource_group_name": "myResourceGroup",
    "build_resource_group_name": "myResourceGroup",
    "managed_image_name": "PackerImagewin2016-DC",
    "os_type": "Windows",
    "image_publisher": "MicrosoftWindowsServer",
    "image_offer": "WindowsServer",
    "image_sku": "2016-Datacenter",

    "communicator": "winrm",
    "winrm_use_ssl": true,
    "winrm_insecure": true,
    "winrm_timeout": "10m",
    "winrm_username": "packer",

    "vm_size": "Standard_DS2_v2"
  }],
  "provisioners": [{
    "type": "powershell",
    "inline": [
      "Add-WindowsFeature Web-Server",
      "& $env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit",
      "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10  } else { break } }"
    ]
  }]
}

just tried above code working for 2012-R2-Datacenter and 2016-Datacenter-Server-Core but not for 2016-Datacenter

2016-Datacenter
==> azure-arm: Waiting for WinRM to become available...

==> azure-arm: Timeout waiting for WinRM.

I am experiencing the exact same issue. It was working before 17th of Nov (that was the last Sunday my pipeline was able to generate an image from the Marketplace 2016-Datacenter)

@harsh-vm
Copy link
Contributor

harsh-vm commented Jan 9, 2020

@harsh-vm Created a VM with 2016 and checked WinRM status which is disabled. Here my packer template

Packer - 1.4.5 and 1.5.0

{
  "builders": [{
    "type": "azure-arm",

    "subscription_id": "XXXXXX",
    "client_id":  "XXXXXX,
    "client_secret":    "XXXXXX",
    "tenant_id":  "XXXXXXX",

    "managed_image_resource_group_name": "myResourceGroup",
    "build_resource_group_name": "myResourceGroup",
    "managed_image_name": "PackerImagewin2016-DC",
    "os_type": "Windows",
    "image_publisher": "MicrosoftWindowsServer",
    "image_offer": "WindowsServer",
    "image_sku": "2016-Datacenter",

    "communicator": "winrm",
    "winrm_use_ssl": true,
    "winrm_insecure": true,
    "winrm_timeout": "10m",
    "winrm_username": "packer",

    "vm_size": "Standard_DS2_v2"
  }],
  "provisioners": [{
    "type": "powershell",
    "inline": [
      "Add-WindowsFeature Web-Server",
      "& $env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit",
      "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10  } else { break } }"
    ]
  }]
}

just tried above code working for 2012-R2-Datacenter and 2016-Datacenter-Server-Core but not for 2016-Datacenter

2016-Datacenter
==> azure-arm: Waiting for WinRM to become available...

==> azure-arm: Timeout waiting for WinRM.

@sanchetanparmar and @Dilergore,

This seems to be really an issue for most people. I wonder how it worked for me!

I had the image built just once as mentioned earlier by increasing the time out and the image was created with no further issues. What I hadn’t tried was creating VM with that image! Probably there might be an issue even there. I have currently moved on to 2019 core server from 2016 as per my work requirements.

But, will try to build image with 2016 data centre sku and will feed you guys if it works! If yes, will share the template I tried if not, Microsoft might have to take a look at it!

@Dilergore
Copy link

Dilergore commented Jan 10, 2020

@harsh-vm My Packer pipeline looks like, that I use the Marketplace "2016-Datacenter" to create a Bronze image. Then I push it to a Shared Image Gallery, then I use it again in Packer during a second build (second pipeline) to create a Golden Image which I trigger right after the Bronze Pipeline.

If I use the Bronze Image from 17th of Nov, that builds okay as a Golden Image. I think there is a new 2016-Datacenter image on the Marketplace since 17th of Nov and in that there was a change not to allow WinRM by default. I tried to look into the versions but it is really hard to determine which is the latest and also hard to determine when they released it (at least for some):
image

I would be grateful if you would have the time to test it again with the latest 2016-Datacenter Marketplace image, as I am currently very busy and I won't have time to pick this up in the next two weeks.
"os_type": "Windows", "image_publisher": "MicrosoftWindowsServer", "image_offer": "WindowsServer", "image_sku": "2016-Datacenter",

p.s.: I am still using Packer 1.4.5. If you can confirm that it is not working for you either I can fast track it with Microsoft.

@Mcphejoh
Copy link

Greetings - I have been using the dame Packer code and confirmed today the winrm timeout is still persistent - have tried with winrm ssl set to true and false user and password and with user name and password errors with connection reset by peer, the other ssl method just winrm times cancelling build. windows 2012 works but the power shell levels are ot compatible with code scripts.
Any help is appreciated. Same code as posted above and in packer samples - again worked before and now its not.

@Dilergore
Copy link

@Mcphejoh Can you check what is the difference between the WinRM / Windows Firewall (default) settings between 2012 and 2016 (or 2019 and 2016)?

@akingscote
Copy link

bump - so glad i found this thread. I was stuck on this for ages. I tried 2016-datacenter with 10m timeout and it failed with winRM error. Switched to 2012-R2-Datacenter with 3m timeout and it worked. Must have wasted 3 hours trying with 2016-datacenter.

This really needs to get fixed on the documentation, im new to packer and i thought for a long time i had done something wrong. At least make the default server 2012 or 2019 until the issue gets addressed.

@akingscote
Copy link

@MicahMcKittrick-MSFT can you re-open this?

@Dilergore
Copy link

@akingscote This is definitely not a documentation issue. This should work by default. We need to find the root cause, then engage Microsoft to fix (probably) the Marketplace Image for 2016-Datacenter.

@akingscote
Copy link

@Dilergore sure thing, but as someone who is trying to learn this stuff, its incredibly frustrating losing hours that on something that has a workaround. This issue has been open since May, so i doubt its getting fixed any time soon.

My first reference is this documentation. I understand it should work, but the issue is that it dosent work.

@Dilergore
Copy link

Dilergore commented Jan 22, 2020

@Dilergore sure thing, but as someone who is trying to learn this stuff, its incredibly frustrating losing hours that on something that has a workaround. This issue has been open since May, so i doubt its getting fixed any time soon.

My first reference is this documentation. I understand it should work, but the issue is that it dosent work.

I don't know anything about why this issue was created. I think back when this was created the problem was with the timeout. When the users started to increase it, it started working. There can be many reasons to that, one could be I can think of is that using a small VM may increase the configuration time for WinRM... If you check the config on the beginning of this issue, it is mentioned that DS2v2 was the VM used. That is 2 cores and 7 Gigs of memory. That combined with Standard HDD... I can imagine that booting up and configuring WinRM takes longer than the default timeout.

Anyway I can confirm since early Q3 it was working for me and it stopped working mid-November.

@SimonDoy
Copy link

I have been fighting this issue for a couple of days both with Windows 2016 and Windows 2019 DataCentre images.

I read some Microsoft documentation ([https://support.microsoft.com/en-gb/help/4494462/events-not-forwarded-if-the-collector-runs-windows-server](Microsoft support article)) that stated the winrm process runs differently on a Windows 2019 machine when the machine has less than 3.5GB.

So I thought I would change vm_size: "Standard_B1ms" which has 2GB ram.
This time Packer was able to successfully connect to WinRM.

@Dilergore
Copy link

@SimonDoy using a small VM definitely can be a problem.

I am using the following fairly big and fast VMs for my builds:

  • D4s_v3 (4c, 15g)
  • Premium SSD

...And I have the timeout set to 20 mins currently. For both 2016 and 2019 I am using the same specs. 2019 is working, 2016 is not. This problem is unique to the 2016 Marketplace Image / WinRM process.

@Adam-Goodwin
Copy link

Been running into the same problem, switching from Windows 2016 to 2019 solved the issue

@Dilergore
Copy link

Dilergore commented Jan 28, 2020

Hello,

Use this:
"image_publisher": "MicrosoftWindowsServer", "image_offer": "WindowsServer", "image_sku": "2016-Datacenter", "image_version": "14393.3326.1911120150",

With that image version WinRM connects. This is from 12th of November.

Update1:
It stopped working with 14393.3384.1912042333 which was released 10th of Dec. I am opening a new issue for this. Will link it here. Also planning to open a ticket for MS. Stay tuned!

Update2:
hashicorp/packer#8658

@Dilergore
Copy link

I cannot reproduce the issue anymore, so I closed the issue in the hashicorp queue along with the MS case.

I tested today multiple times and with all of the images since November and even with the latest one it is working for me. Don't know what changed.

@AliAllomani
Copy link

I still can reproduce the issue,

used image

   "image_publisher": "MicrosoftWindowsServer",
    "image_offer": "WindowsServer",
    "image_sku": "2016-Datacenter"

From initial troubleshooting it looks to me a certificate issue, trying to run winrm quickconfig on the machine during azure-arm: Waiting for WinRM to become available... resulting

WinRM service is already running on this machine.
WSManFault
    Message
        ProviderFault
            WSManFault
                Message = Cannot create a WinRM listener on HTTPS because this machine does not have an appropriate certificate. To be used for SSL, a certificate must have a CN matching the hostname, be appropriate for Server Authentication, and not be expired, revoked, or self-signed. 

Error number:  -2144108267 0x80338115
Cannot create a WinRM listener on HTTPS because this machine does not have an appropriate certificate. To be used for SSL, a certificate must have a CN matching the hostname, be appropriate for Server Authentication, and not be expired, revoked, or self-signed. 

And when trying to connect using openssl to retrieve the certificate i'm getting errno=54

openssl s_client -connect 13.95.122.54:5986 -showcerts
CONNECTED(00000003)
write:errno=54
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 307 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1580229460
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

Trying to re-generate self-signed certificate and reconfigure WinRM causing packer to immediately respond to the connection

$Cert = New-SelfSignedCertificate -CertstoreLocation Cert:\LocalMachine\My -DnsName "$env:COMPUTERNAME"
Remove-Item -Path WSMan:\Localhost\listener\listener* -Recurse
New-Item -Path WSMan:\LocalHost\Listener -Transport HTTPS -Address * -CertificateThumbPrint $Cert.Thumbprint -Force
Stop-Service winrm
Start-Service winrm

and from openssl showcerts i'm getting a correct answer

 openssl s_client -connect 13.95.122.54:5986 -showcerts
CONNECTED(00000003)
depth=0 CN = pkrvm39jkvjspuk
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = pkrvm39jkvjspuk
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/CN=pkrvm39jkvjspuk
   i:/CN=pkrvm39jkvjspuk
-----BEGIN CERTIFICATE-----
MIIDKjCCAhKgAwIBAgIQbI6Ll/YdLKZFm3XIDuCVEzANBgkqhkiG9w0BAQsFADAa
MRgwFgYDVQQDDA9wa3J2bTM5amt2anNwdWswHhcNMjAwMTI4MTYzNDI4WhcNMjEw
MTI4MTY1NDI4WjAaMRgwFgYDVQQDDA9wa3J2bTM5amt2anNwdWswggEiMA0GCSqG
SIb3DQEBAQUAA4IBDwAwggEKAoIBAQDTaBPCr8ImXt+wyDEcNVK3lW5HOme7X8h0
gl+ZTAmwhlzyZwWI1S5fW0Gfc+VQtwmscZs7in1/Rg0EBnhCHKiXYdJdWgiNQjp8
hxNHQlPzFMxBNHJCncs3cUjl8TBvWFVof+mNmv20IcoDfhkBXo8PBMC1M08krfGd
KXxvJ/Km3dfGvY3HKyMAdwJK/r4rENnTMIr5KgOv2cL4usTNS0o4nQSDVbL8rXdN
0Pfwui0ItGiZ7auul/tioQAmKpcxle7y16b/XnX1olQp59T7WklKcfS4Rt+XloAM
dyam22dhXaPQ9/03MBEqguO/SXDV2m+7RFLPRzHDPWwrQjE6eClDAgMBAAGjbDBq
MA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwEw
GgYDVR0RBBMwEYIPcGtydm0zOWprdmpzcHVrMB0GA1UdDgQWBBQYK0o8mxc3uUyn
9WAvpOzINrvkyzANBgkqhkiG9w0BAQsFAAOCAQEALIRGvoQONxX0RzdyOEX15dJm
tMChjVgU9y176UK03NcuNqfQqJXhnibZQO/+ApXT4C1YKUzZcmqkJpPkt2ufYmC1
sFLp3tGZ35zfjtU8Mm6xEHdQv4LGQzpCycVqlvFGrdWCMCB4EWZb0z7oqp+nsz2P
14HFaiPsHnfpJEMUF+jrMQkGb9bzMHTT4Y0q5TStVdc9q1cu3pWLnzJ6gaBlz0Iz
DG03HtTmwppmDLSE1RZYJBQ6UsgD/L/jbR2c08ko4t1uSMwRcANv5sGZ6TukyK95
JVnYbFrZWzcqWfE1uynTEdeb+l/aospY9g/Fjt4WKI0U0xnGuczsbx1KoO0ELg==
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=pkrvm39jkvjspuk
issuer=/CN=pkrvm39jkvjspuk
---
No client certificate CA names sent
Peer signing digest: SHA256
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1298 bytes and written 433 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5E200000884A7231C92707E15CD2222B4BE94DD50A3B61E7B8763B3BC0A2F615
    Session-ID-ctx:
    Master-Key: 6CF4DA86AEBEB597F72DB9DC9E8C8B59D8B240C7FE6F8491B14314E86529A338F07E1B2C5BEB300C48DE4D490978D5D5
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1580229891
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
---

I see that packer is using azure osProfile.windowsConfiguration.winRM value in the template to configure winRM on the VM,

So here i would assume that either there is an issue with creating the certificate from packer side before uploading it to azure vault, or and issue with azure that prevents the VM from configuring winRM correctly using the values from the template, this may needs more troubleshooting.

 "osProfile": {
                    "computerName": "[parameters('virtualMachines_pkrvm2nb5asnu2s_name')]",
                    "adminUsername": "packer",
                    "windowsConfiguration": {
                        "provisionVMAgent": true,
                        "enableAutomaticUpdates": true,
                        "winRM": {
                            "listeners": [
                                {
                                    "protocol": "https",
                                    "certificateUrl": "https://pkrkv2nb5asnu2s.vault.azure.net/secrets/packerKeyVaultSecret/05113faa18ee40a2b5465910b2f3dda1"
                                }
                            ]
                        }
                    },
                    "secrets": [
                        {
                            "sourceVault": {
                                "id": "[parameters('vaults_pkrkv2nb5asnu2s_externalid')]"
                            },
                            "vaultCertificates": [
                                {
                                    "certificateUrl": "https://pkrkv2nb5asnu2s.vault.azure.net/secrets/packerKeyVaultSecret/05113faa18ee40a2b5465910b2f3dda1",
                                    "certificateStore": "My"
                                }
                            ]
                        }
                    ]
                },

@Dilergore
Copy link

@AliAllomani reopened the other issue, let's continue there.

@nrajeshgoud
Copy link

Same issue for me for "image_sku": "2016-Datacenter" from few days, but somehow it worked with couple to retry's. my "winrm_timeout": "10m"

@danielsollondon
Copy link
Contributor

Hi All, I'm a Program Manager for the Azure VM image builder (which uses Packer under the hood), we have seen this too, and we have engaged the Windows PG to investigate. Initially there was a low memory condition which could cause problems, when using Standard D1_v2, this was due to Windows Update, and has been mitigated. However, there is still an issue, Windows PG is investigating, and I will report back when I hear. In the meantime one really kind member reached out with this workaround: danielsollondon/azvmimagebuilder#14 (comment)

@danielsollondon
Copy link
Contributor

danielsollondon commented Feb 9, 2020

Quick update, I spoke to the Windows team, they have identified an issue with the Windows Server 2016 image (November onwards), that impacts the time to initiate a WinRM connection with Packer, they are still working on this, and will update again mid next week. In the meantime please try increasing the Packer timeout to 30mins, and try a larger VM size.

@shurick81
Copy link

I am using "image_sku": "2019-Datacenter-smalldisk" and "image_sku": "2019-Datacenter-smalldisk" in combination with "winrm_timeout": "30m", still getting Timeout waiting for WinRM. in 95% of build attempts. Is there some open issue related to this problem that we can track? This issue seems old and closed.

@Dilergore
Copy link

@danielsollondon
Copy link
Contributor

Update from the Windows team: Windows Server 2016 images since November 2019 can have a post first boot performance issue related to an OS code integrity operation. This issue is more pronounced on small Azure VM sizes (with lower throughput and IO) rendering the VM not immediately usable after first boot. The performance issue is mitigated in February 2020 images and forward. Please use the latest February Windows Server 2016 image once it is available from the Marketplace (ETA 2/17).

@holtwilkins
Copy link

holtwilkins commented Feb 26, 2020

Hey @danielsollondon , I'm also seeing this issue pretty regularly with Win10 images, specifically I'm using offer Windows-10 and sku 19h2-evd. Any thoughts on when the issue will be fixed for this, or should I be using a different sku? I'm building for Windows Virtual Desktop, hence why I'm using the evd sku, but not sure if it matters?

EDIT: I found hashicorp/packer#8658, I'll continue the discussion over there instead if the switch from Standard_DS3_v2 to Standard_D3_v2 doesn't fix my win10 builds.

@Marcus-James-Adams
Copy link

Still occurring with a Standard_D4s_v3 on latest 2016 image
It's intermittent for me it's worse from mid-morning onwards - could external load factors also be playing into it @danielsollondon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests