Slow module loading

[burnbay]

Hi.

I'm experiencing really slow loading times on the dbamodule. Looks like it's the same pattern on different machines. From what I can see it's the dot sourcing of all the function files that's taking time.

Read about the same issue here: http://superuser.com/questions/1170619/is-dot-sourcing-slower-than-just-reading-file-content

So I replaced all function loading with Invoke-Expression and that made a significant performance boost.

Before:

After:

It's not a module bug I guess, but more a Powershell runtime problem. Does anybody experience the same problems?

System Details

• Operating system name and version:
  Windows Server 2012 R2 Standard

• Output from $PSVersionTable:
  PSVersion 5.0.10586.117
  PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
  BuildVersion 10.0.10586.117
  CLRVersion 4.0.30319.42000
  WSManStackVersion 3.0
  PSRemotingProtocolVersion 2.3
  SerializationVersion 1.1.0.1

• Output of dbatools version:
  0.8.900

• SQL Server version for source/target
  SQL Server 2016 SP1

[potatoqualitee]

Hey @burnbay - we had the same issues and fixed it in 8.0.902 -- please update and let me know if you continue to have issues.

[burnbay]

Hi.

On version 8.0.902 I'm just getting "Install-Package : Illegal characters in path" Version 8.0.900 works fine on installation.

[potatoqualitee]

Dang, can you try

Uninstall-Module dbatools -AllVersions
Install-Module dbatools -Force

See if you continue to get encoding issues? And if so, try this (temporarily)

Invoke-Expression (Invoke-WebRequest -UseBasicParsing https://dbatools.io/in)

Let me know if any of those work. I can install dbatools no problem on both Win10 and Server2016 so I'm trying to find the culprit. If you continue to have issues,c an you please paste the whole error?

Thanks!

[burnbay]

Removed all verssions and then ran Install-Module dbatools -Force:

Invoke-Expression (Invoke-WebRequest -UseBasicParsing https://dbatools.io/in) worked!

[burnbay]

Here's the verbose output:

[potatoqualitee]

WTF agghh. k, thanks. Will try to figure out what is up and fix that. My apologies.

How's the new loading time?

[burnbay]

Looks good! :)

[potatoqualitee]

Breaks my heart to see 10 seconds -- on a regular machine, that should be subsecond (v 0.8.900 was 3 secs).

@PowerDBAKlaas looked into this too and found it's a GPO setting. Klaas, did you ever find which specific setting increased the time?

[burnbay]

Well ... compared to 12 minutes it's blazing fast :)

[potatoqualitee]

LOL!! Oh man, I thought that was 12 seconds to 10 seconds! Torturous!

FWIW, the offending line was in an older copy of Invoke-Sqlcmd2, so your system really doesn't like Get-Command -ListImported

[burnbay]

Would be really interesting to know what GPO setting that was @PowerDBAKlaas? Our IT department love GPO's a bit too much IMO

[PowerDBAKlaas]

Can you show what:
Get-ExecutionPolicy -List
returns, please?

[burnbay]

No restrictinos on this dev server:

[gbargsley]

I used to have slow loading. But the latest release is super fast for me.

[PowerDBAKlaas]

@burnbay that's different from what I had, but it can still be a problem:
the Policies should be "Undefined".
When it's set through a GPO, additional checking is done.

Instead of the executionpolicy, a registry key should be set in the GPO.

Tobias wrote this blog post after our conversation on my module loading times (63 sec):
http://www.powertheshell.com/understanding-execution-policy/

[potatoqualitee]

Thanks so much for the link @PowerDBAKlaas! Tweeted it.

[potatoqualitee]

@gbargsley - yeah, seems I introduced the slowness in middle of December 👎

[potatoqualitee]

@burnbay - please try to install the latest version (0.8.903) from the gallery, I updated a bunch of encoding.

[burnbay]

Sorry to say ....

[potatoqualitee]

dangit. what language is your windows? it's win10 right?

[FriedrichWeinmann]

Heya, we're having some trouble reproducing the issue.
Would you mind providing:

• Operating System, Edition and build type (e.g.: Windows 10 Professional x64)
• An export of your $host variable: $host | Export-Clixml host.xml
• An export of the error record that is thrown: $error[0] | Export-clixml error.xml
  (The error export only if the failed install was the last command you used, so it's the right error. Otherwise you may need to find the correct error to export first)

[potatoqualitee]

Also can you Save-Module -Name dbatools -Path C:\temp

then

Import-Module C:\temp\dbatools\0.8.903\dbatools.psd1

See what happens?

[burnbay]

@ctrlbold Works like a charm with Save-Module, then Import-Module.

@FriedrichWeinmann System:
Windows 2012 Server R2 Standard SP1 x64

info.zip

[FriedrichWeinmann]

Hi, thanks for the data. I went through it, was a bit at a loss and set up an identical machine:
Windows Server 2012 R2 SP1 x64

• Language & Locale: Norwegian (Bokmål)
• PowerShell UI language: en-US
• Management Framework 5 RTM (5.0.10586.117)
  It installed the module flawlessly on import.

This - combined with your own example of saving and importing the module - strongly implies that something on your machine is broken, either PowerShellGet, PackageManagement, PowerShell or .NET in general.

Recommended steps to solve this:

• Install WMF 5.1. It replaces WMF 5.0 components and it is plain better all around, making this a good update. (Meaning you should do it, issue or no issue)
• Reinstall the package management module from a healthy system of as close to equal patchlevel as you can manage
• Reinstall PowerShellGet
• Try a .NET repair

The first is generally safe. For 2 & 3, copying the existing module directory as backup is sufficient but recommended. Before doing the .NET repair I highly recommend making sure you've got a backup of the server. It's usually safe to use, but 'usually' and 50 cent will get you a coke.

[burnbay]

Wow! What an effort. Thank's so much. I suspect something is broken too. Never had any problems before on this particular machine, but I don't control software installation/patching. Could be that something's been done. Anyway, will look into it tomorrow.

[FriedrichWeinmann]

Hey, no worries :)
I still have a W2K12R2 template on my test ESX and had my test environment shutdown due to maintenance today, so I had the capacity available to do a quick clone and language-pack transform.

[burnbay]

Still, installing version 0.8.900 works great on the same machine. Tried on a different server now and the same error. W2K12R2 with CurrentCulture nb-NO and CurrentUICulture en-US, same as the first server. Installing version 0.8.900 works on both.

[burnbay]

Tried on a W7EntSp1 with no luck:

$host:

Will try upgrading to WMF 5.1 on the W7 machine. To be continued ....

[burnbay]

Success:

I'll test the same on the first W2K12R2 server, but cannot restart immediately. Will update as soon as possible.

[potatoqualitee]

Interesting! The thing that drives me crazy is that it used to work. Can you do one more thing the next time you come across that error? Paste the results of $error[0] | select *

…

On Tue, Feb 21, 2017 at 9:36 AM, burnbay ***@***.***> wrote: Tried on a W7EntSp1 with no luck: [image: image] <https://cloud.githubusercontent.com/assets/2721929/23156863/f06df1ca-f818-11e6-9401-2248d09c90b8.png> $host: [image: image] <https://cloud.githubusercontent.com/assets/2721929/23156875/fded792e-f818-11e6-8716-cb9b92a74e9b.png> Will try upgrading to WMF 5.1 on the W7 machine. To be continued .... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#779 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH5QES2m5Pvi2E913X5I9qlj-mtYYhLrks5reqIegaJpZM4MGJIv> .

[potatoqualitee]

Nevermind, @sanderstad ran it with that and it didn't provide adequate info :/

[burnbay]

Well @FriedrichWeinmann and @ctrlbold ..... worked out just fine with WMF 5.1 on W2K12R2 too:

A bit bugged by the error on WMF 5.0. I guess a lot of people run that version.

Thanks for all your help!

[FriedrichWeinmann]

Damn, and I figured out what buggered my test: One of our trainees had patched my templates and forgotten to mention it to me * grml *
I'll see whether I can troubleshoot the issue - after all, as you said, many people still using that.
Thanks for reporting this and helping is pinpoint it.

[ConstantineK]

Also, for anyone who is having an issue with the Illegal characters in path please post the output of nuget help and report their nuget version?

I am seeing reinstalling fixing issues and a few other projects on gh reporting specific nuget version issues, so this would be very helpful to isolate the issue.

Thanks!

[potatoqualitee]

I got another report via email D: I'll ask him to run nuget help, thanks @ConstantineK

[ConstantineK]

Talked to @sanderstad, looks like the location of the nuget executable might be local to the issue, going to find the exact location and report back which nuget version you want to get :)

[potatoqualitee]

Yessssss!

[potatoqualitee]

hey @burnbay, can you run this

Get-HotFix | ? installedon -gt 2/15/2013

Let us know and we'll compare with @sanderstad's

[sanderstad]

Here is my "little" list
installedupdates_sstad.txt

[burnbay]

Here's my list
installedupdates.txt

[potatoqualitee]

Thank you both!

[potatoqualitee]

agggh, it doesnt look like a recent update caused the problem 👎

[potatoqualitee]

I am so pleased to say that both the speed and the illegal characters issue has been resolved by @ConstantineK and @FriedrichWeinmann in v0.8.938

ck found out after recompiling some DLL's that loading the module faster with the . caused illegal chars.

This commit shows our ultimate solution by @FriedrichWeinmann

f6dcc2d

Module now loads on my machine at 1.5 secs avg.

[burnbay]

Great news!
👍🏻👍🏻