Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New-NavContainer Stuck at Loading Symbols for TestLibraries image BC 14.10 #859

Closed
mmenninghaus opened this issue Feb 19, 2020 · 12 comments
Closed

Comments

@mmenninghaus
Copy link

@mmenninghaus mmenninghaus commented Feb 19, 2020

I am executing the following command:

New-NavContainer -accept_eula -containerName test -imageName 'mcr.microsoft.com/businesscentral/onprem:14.10.40471.0-de'  -doNotExportObjectsToText -TimeZoneId (Get-TimeZone).Id -shortcuts "Desktop" -alwaysPull -updateHosts -useBestContainerOS -accept_outdated -includeCSide -enableSymbolLoading -includeTestToolkit -includeTestLibrariesOnly

It runs forever (>1h whereas container creation usually only takes 5min) with the following output:

NavContainerHelper is version 0.6.4.25
NavContainerHelper is running as administrator
Host is Microsoft Windows 10 Enterprise - ltsc2019
Docker Client Version is 19.03.5
Docker Server Version is 19.03.5
Pulling image mcr.microsoft.com/businesscentral/onprem:14.10.40471.0-de-ltsc2019
14.10.40471.0-de-ltsc2019: Pulling from businesscentral/onprem
Digest: sha256:b248e37d07443a19b04e43b2dcf5757de817bb68d8ca31faae89dfb4db3e30b1
Using image mcr.microsoft.com/businesscentral/onprem:14.10.40471.0-de-ltsc2019
Creating Container test
Version: 14.10.40471.0-de
Style: onprem
Platform: 14.0.40464.0
Generic Tag: 0.0.9.98
Container OS Version: 10.0.17763.973 (ltsc2019)
Host OS Version: 10.0.17763.973 (ltsc2019)
Using locale de-DE
Using process isolation
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Files in C:\ProgramData\NavContainerHelper\Extensions\test\my:
- AdditionalOutput.ps1
- AdditionalSetup.ps1
- MainLoop.ps1
- SetupVariables.ps1
- updatehosts.ps1
Creating container test from image mcr.microsoft.com/businesscentral/onprem:14.10.40471.0-de-ltsc2019
e13b0fb69ee978688a8c8df35f63cd07f65cfdfbab939943a3caa9213f57f1ba
Waiting for container test to be ready

WARNING: DNS resolution not working from within the container.
Using Windows Authentication
Starting Local SQL Server
Starting Internet Information Server
Modifying Service Tier Config File with Instance Specific Settings
Starting Service Tier
Registering event sources
Creating DotNetCore Web Server Instance
Creating http download site
Creating Windows user mmenninghaus
Setting SA Password and enabling SA
Creating SUPER user
Container IP Address: 172.31.100.84
Container Hostname  : test
Container Dns Name  : test
Web Client          : http://test/NAV/
Dev. Server         : http://test
Dev. ServerInstance : NAV
Setting test to 172.31.100.84 in host hosts file

Files:
http://test:8080/al-3.0.193882.vsix

Initialization took 130 seconds
Ready for connections!
Set TimeZone in Container to W. Europe Standard Time
Reading CustomSettings.config from test
Creating Desktop Shortcuts for test
Importing Objects from C:\TestToolKit\CALTestLibraries.DE.fob (container path)
Generating Symbols while importing

All of my colleagues are experiencing the same issue.
It does not get stuck if I do not set the Symbol Loading flag. The 14.10 containers get also stuck when setting up without Symbol Loading or TestLibraries but trying to export C/AL files.

Thank you for any help on this issue!

@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 21, 2020

is there anything in the event log? You can use Get-NavContainerEventLog to fetch the event log (remember to delete it again from the event viewer)

@mmenninghaus

This comment has been minimized.

Copy link
Author

@mmenninghaus mmenninghaus commented Feb 21, 2020

See below, nothing that I am aware of.

{
    "container.logs":  [
                           "Initializing...",
                           "Setting host.docker.internal to 172.16.11.95 in container hosts file (copy from host hosts file)",
                           "Setting gateway.docker.internal to 172.16.11.95 in container hosts file (copy from host hosts file)",
                           "Setting host.docker.internal to 192.168.178.190 in container hosts file (copy from host hosts file)",
                           "Setting gateway.docker.internal to 192.168.178.190 in container hosts file (copy from host hosts file)",
                           "Setting host.docker.internal to 172.22.72.63 in container hosts file (copy from host hosts file)",
                           "Setting gateway.docker.internal to 172.22.72.63 in container hosts file (copy from host hosts file)",
                           "Setting host.docker.internal to 172.17.34.143 in container hosts file (copy from host hosts file)",
                           "Setting gateway.docker.internal to 172.17.34.143 in container hosts file (copy from host hosts file)",
                           "Setting host.docker.internal to 172.17.35.41 in container hosts file (copy from host hosts file)",
                           "Setting gateway.docker.internal to 172.17.35.41 in container hosts file (copy from host hosts file)",
                           "Setting kubernetes.docker.internal to 127.0.0.1 in container hosts file (copy from host hosts file)",
                           "Setting host.containerhelper.internal to 172.30.240.1 in container hosts file",
                           "Starting Container",
                           "Hostname is test",
                           "PublicDnsName is test",
                           "Using Windows Authentication",
                           "Starting Local SQL Server",
                           "Starting Internet Information Server",
                           "Modifying Service Tier Config File with Instance Specific Settings",
                           "Starting Service Tier",
                           "Registering event sources",
                           "Creating DotNetCore Web Server Instance",
                           "Creating http download site",
                           "Creating Windows user mmenninghaus",
                           "Setting SA Password and enabling SA",
                           "Creating SUPER user",
                           "Container IP Address: 172.30.250.227",
                           "Container Hostname  : test",
                           "Container Dns Name  : test",
                           "Web Client          : http://test/NAV/",
                           "Dev. Server         : http://test",
                           "Dev. ServerInstance : NAV",
                           "Setting test to 172.30.250.227 in host hosts file",
                           "",
                           "Files:",
                           "http://test:8080/al-3.0.193882.vsix",
                           "",
                           "Initialization took 128 seconds",
                           "Ready for connections!"
                       ],
    "docker.info":  [
                        "Client:",
                        " Debug Mode: false",
                        "",
                        "Server:",
                        " Containers: 4",
                        "  Running: 2",
                        "  Paused: 0",
                        "  Stopped: 2",
                        " Images: 42",
                        " Server Version: 19.03.5",
                        " Storage Driver: windowsfilter",
                        "  Windows: ",
                        " Logging Driver: json-file",
                        " Plugins:",
                        "  Volume: local",
                        "  Network: ics internal l2bridge l2tunnel nat null overlay private transparent",
                        "  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog",
                        " Swarm: inactive",
                        " Default Isolation: hyperv",
                        " Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)",
                        " Operating System: Windows 10 Enterprise Version 1809 (OS Build 17763.973)",
                        " OSType: windows",
                        " Architecture: x86_64",
                        " CPUs: 8",
                        " Total Memory: 15.85GiB",
                        " Name: SNC-NB-702",
                        " ID: AQSF:V6UL:M3CY:BXIX:NVN4:ZGVX:VB4H:RSEC:HP2D:6LJN:KMSN:E667",
                        " Docker Root Dir: C:\\ProgramData\\docker",
                        " Debug Mode: true",
                        "  File Descriptors: -1",
                        "  Goroutines: 45",
                        "  System Time: 2020-02-21T11:58:15.1325028+01:00",
                        "  EventsListeners: 1",
                        " Registry: https://index.docker.io/v1/",
                        " Labels:",
                        " Experimental: false",
                        " Insecure Registries:",
                        "  127.0.0.0/8",
                        " Live Restore Enabled: false",
                        " Product License: Community Engine",
                        ""
                    ],
    "container.labels":  {
                             "country":  "de",
                             "created":  "202002101431",
                             "cu":  "cu9",
                             "eula":  "https://go.microsoft.com/fwlink/?linkid=861843",
                             "legal":  "http://go.microsoft.com/fwlink/?LinkId=837447",
                             "maintainer":  "Dynamics SMB",
                             "nav":  "",
                             "osversion":  "10.0.17763.973",
                             "platform":  "14.0.40464.0",
                             "tag":  "0.0.9.98",
                             "version":  "14.10.40471.0"
                         },
    "container.ping":  [
                           "",
                           "Ping wird ausgeführt für test [172.30.250.227] mit 32 Bytes Daten:",
                           "Antwort von 172.30.250.227: Bytes=32 Zeit\u003c1ms TTL=128",
                           "Antwort von 172.30.250.227: Bytes=32 Zeit\u003c1ms TTL=128",
                           "Antwort von 172.30.250.227: Bytes=32 Zeit\u003c1ms TTL=128",
                           "Antwort von 172.30.250.227: Bytes=32 Zeit\u003c1ms TTL=128",
                           "",
                           "Ping-Statistik für 172.30.250.227:",
                           "    Pakete: Gesendet = 4, Empfangen = 4, Verloren = 0",
                           "    (0% Verlust),",
                           "Ca. Zeitangaben in Millisek.:",
                           "    Minimum = 0ms, Maximum = 0ms, Mittelwert = 0ms"
                       ],
    "container.env":  [
                          "username=mmenninghaus",
                          "ExitOnError=N",
                          "useSSL=N",
                          "accept_eula=Y",
                          "securePassword=\u003cspecified\u003e",
                          "passwordKeyFile=c:\\run\\my\\aes.key",
                          "removePasswordKeyFile=Y",
                          "accept_outdated=Y",
                          "licenseFile=\u003cspecified\u003e",
                          "auth=Windows",
                          "locale=de-DE",
                          "databaseServer=",
                          "databaseInstance=",
                          "enableApiServices=Y",
                          "enableSymbolLoading=Y",
                          "COMPLUS_NGenProtectedProcess_FeatureEnabled=0",
                          "DatabaseName=Cronusde",
                          "NAVDVDURL=\u003cspecified\u003e",
                          "VSIXURL=\u003cspecified\u003e",
                          "IsBcSandbox=N",
                          "CountryCode=de",
                          "COUNTRYURL=\u003cspecified\u003e"
                      ]
}
@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 21, 2020

Sorry, but that isn't the event log

@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 21, 2020

I can repro this on my machine.

@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 22, 2020

Something must have changed, if I do this:

if (-not $windowscredential) {
    $windowscredential = Get-Credential -UserName $env:USERNAME -Message "Enter your Windows Password"
}
New-NavContainer `
    -accept_eula `
    -accept_outdated `
    -containerName test `
    -imageName 'mcr.microsoft.com/businesscentral/onprem:1904-cu8-de-ltsc2019' `
    -doNotExportObjectsToText `
    -shortcuts "Desktop" `
    -updateHosts `
    -includeCSide `
    -enableSymbolLoading `
    -includeTestToolkit `
    -includeTestLibrariesOnly `
    -Credential $windowscredential

It works fine.
If I replace cu8 with cu9 - it hangs.

@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 22, 2020

The bug is in Microsoft.Dynamics.Nav.Ide.psm1 in 14.10, and it actually causes Import-NavApplicationObject, Export-NavApplicationObject, Compile-NavApplicationObject and more standard cmdlets to fail.

Reason is a new parameter added to RunNavIdeCommand, which doesn't get a value and subsequently all functions calling RunNavIdeCommand will fail/hang.

I have filed a bug to get this fixed for 14.11. In the meantime, I have uploaded a patch for Microsoft.Dynamics.Nav.Ide.psm1 here:

https://bcdocker.blob.core.windows.net/public/14.10.40471.0-Patch-Microsoft.Dynamics.Nav.Ide.psm1

And I have added code to New-NavContainer, which downloads and patches the container if 14.10 is being instantiated, you should see this when you have the right version:
image
Will ship in the next NavContainerHelper version 0.6.4.28 (today) and with that all pipelines etc. should work as before...

@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 22, 2020

This commit fixes the issue: ff02d65

@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 22, 2020

Shipped in 0.6.4.28

@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 22, 2020

@freddydk freddydk closed this Feb 22, 2020
@mboon75

This comment has been minimized.

Copy link

@mboon75 mboon75 commented Feb 24, 2020

We had the same sort of behavior with Import-ObjectsToNavContainer.ps1 since last thursday. We are not using CU9 yet, but CU5 for reasons. After long investigation we have noticed that on the host machines the Windows Updates had run in the night of wednesday and thursday.
Result was that Import-ObjectsToNavContainer was just hanging.
After reverting Security Update KB4532691 the issue was fixed. Where can we address this ?

@freddydk

This comment has been minimized.

Copy link
Contributor

@freddydk freddydk commented Feb 24, 2020

@mboon75 - you are running into the other problem we had over the last two weeks: https://freddysblog.com/2020/02/14/hyperv-isolation-to-the-rescue/ - where the February update caused things to fail.
There is no better solution to this than what's mentioned in the blog post above.

@mboon75

This comment has been minimized.

Copy link

@mboon75 mboon75 commented Feb 24, 2020

ah, sorry @freddydk ! I should read more of your blogs before asking. Thanks a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.