New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Using module" statement does not reload module after changes are made #7654

Open
deadlydog opened this Issue Aug 28, 2018 · 13 comments

Comments

Projects
None yet
7 participants
@deadlydog

deadlydog commented Aug 28, 2018

Problem Description

If you are using the using module statement, it will load the module without any issues. However, if you make a change to the module and run the using module statement again without restarting your PowerShell session, it will not load the new modified module into memory, and instead will continue to use the old existing module that was loaded. This makes doing development with using module statements very tedious as you must continually reload your PowerShell session whenever you make changes to the module.

Ideally we would just use Import-Module -Name [module name] -Force to avoid this issue, but that does not work for importing classes, so the using module statement must be used.

Update: This problem also exists in Windows PowerShell (not Core). Go here to upvote that it gets fixed there too.

Steps to reproduce

Create the following module and script files:

In TestModule.psm1:

class TestClass 
{
	[string] $ClassPropertyText = "Initial text"
}

function Get-ModuleFunctionText()
{
	return "Initial text"
}

Export-ModuleMember -Function Get-ModuleFunctionText

In Test.ps1:

using module '.\TestModule.psm1'

$instance = [TestClass]::new()
$instance.ClassPropertyText

Get-ModuleFunctionText

Run Test.ps1 and it will output:

Initial text
Initial text

Then modify TestModule.psm1 to:

class TestClass 
{
	[string] $ClassPropertyText = "Updated text"
}

function Get-ModuleFunctionText()
{
	return "Updated text"
}

Export-ModuleMember -Function Get-ModuleFunctionText

Run Test.ps1 again and it will still output:

Initial text
Initial text

Expected behavior

When running Test.ps1 the 2nd time, it should output:

Updated text
Updated text

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      5.1.17134.228
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17134.228
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Additional Info

The problem can be somewhat reduced by using both the using module statement and Import-Module -Force statement, like this:

using module '.\TestModule.psm1'
Import-Module -Name '.\TestModule.psm1' -Force

$instance = [TestClass]::new()
$instance.ClassPropertyText

Get-ModuleFunctionText

This code would result in the following output:

Initial text
Updated text

So here the module code was updated, but the class code was not.

Proposed Solution

To fix this issue I propose adding a -Force flag to the using module statement that forces the module to be reloaded into memory; the same way that the -Force flag works for the Import-Module cmdlet. This will allow developing code that uses the class construct to work much smoother.

@MaximoTrinidad

This comment has been minimized.

Show comment
Hide comment
@MaximoTrinidad

MaximoTrinidad Aug 29, 2018

Please remember, this Github repository is only for PowerShell Core and not Windows PowerShell. If you have a Window PowerShell, submit the issue at the Uservoice forum: https://windowsserver.uservoice.com/forums/301869-powershell

As stated in the PowerShell Core landing page in the section "Windows PowerShell vs PowerShell Core": https://github.com/PowerShell/PowerShell#windows-powershell-vs-powershell-core

This way the issue will reach the correct support team .

Now, if you can use PowerShell Core to reproduce the issue you're experiencing, then provide which version of PowerShell Core: GA or RC1.

Any enhancements will be done on PowerShell Core only.

Thanks

MaximoTrinidad commented Aug 29, 2018

Please remember, this Github repository is only for PowerShell Core and not Windows PowerShell. If you have a Window PowerShell, submit the issue at the Uservoice forum: https://windowsserver.uservoice.com/forums/301869-powershell

As stated in the PowerShell Core landing page in the section "Windows PowerShell vs PowerShell Core": https://github.com/PowerShell/PowerShell#windows-powershell-vs-powershell-core

This way the issue will reach the correct support team .

Now, if you can use PowerShell Core to reproduce the issue you're experiencing, then provide which version of PowerShell Core: GA or RC1.

Any enhancements will be done on PowerShell Core only.

Thanks

@deadlydog

This comment has been minimized.

Show comment
Hide comment
@deadlydog

deadlydog Aug 29, 2018

Thanks @MaximoTrinidad I didn't realize that this repo was only for PowerShell Core. I've now submitted this issue in the Windows PowerShell UserVoice as well here.

I was able to reproduce the problem on PowerShell Core as well. Here's the version info for PowerShell Core:

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.0.4
PSEdition                      Core
GitCommitId                    v6.0.4
OS                             Microsoft Windows 10.0.17134
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

deadlydog commented Aug 29, 2018

Thanks @MaximoTrinidad I didn't realize that this repo was only for PowerShell Core. I've now submitted this issue in the Windows PowerShell UserVoice as well here.

I was able to reproduce the problem on PowerShell Core as well. Here's the version info for PowerShell Core:

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.0.4
PSEdition                      Core
GitCommitId                    v6.0.4
OS                             Microsoft Windows 10.0.17134
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0
@MaximoTrinidad

This comment has been minimized.

Show comment
Hide comment
@MaximoTrinidad

MaximoTrinidad Aug 29, 2018

@deadlydog

Thanks a lot!! Yes this repo is only for PowerShell Core but your contribution help identifying issues that have been carried from the Windows PowerShell product code is valuable.

Please keep submitting anything you find.

Greatly Appreciated!

MaximoTrinidad commented Aug 29, 2018

@deadlydog

Thanks a lot!! Yes this repo is only for PowerShell Core but your contribution help identifying issues that have been carried from the Windows PowerShell product code is valuable.

Please keep submitting anything you find.

Greatly Appreciated!

@MaximoTrinidad

This comment has been minimized.

Show comment
Hide comment
@MaximoTrinidad

MaximoTrinidad Aug 29, 2018

@deadlydog,

By the way!! Cool finding.

Yes!! I can reproduce the issue on PowerShell Core version 6.1.0-rc1.

PS [20] > $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.1.0-rc.1
PSEdition                      Core
GitCommitId                    6.1.0-rc.1
OS                             Microsoft Windows 10.0.17746
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

MaximoTrinidad commented Aug 29, 2018

@deadlydog,

By the way!! Cool finding.

Yes!! I can reproduce the issue on PowerShell Core version 6.1.0-rc1.

PS [20] > $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.1.0-rc.1
PSEdition                      Core
GitCommitId                    6.1.0-rc.1
OS                             Microsoft Windows 10.0.17746
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0
@BrucePay

This comment has been minimized.

Show comment
Hide comment
@BrucePay

BrucePay Aug 29, 2018

Member

Since this is a dev-mode problem, I would suggest having a dev-mode flag for PowerShell such that modules are always reloaded when referenced. The problem with -force on using is that it's fragile. Miss one location and you can't figure out what's going on. Running in this mode is likely to be slow (or slower) but reloading is guaranteed. (Note: this is the solution that the DSC team settled on years ago.)

Member

BrucePay commented Aug 29, 2018

Since this is a dev-mode problem, I would suggest having a dev-mode flag for PowerShell such that modules are always reloaded when referenced. The problem with -force on using is that it's fragile. Miss one location and you can't figure out what's going on. Running in this mode is likely to be slow (or slower) but reloading is guaranteed. (Note: this is the solution that the DSC team settled on years ago.)

@datenschieber

This comment has been minimized.

Show comment
Hide comment
@datenschieber

datenschieber Aug 29, 2018

Is it already covered in #2505?

datenschieber commented Aug 29, 2018

Is it already covered in #2505?

@deadlydog

This comment has been minimized.

Show comment
Hide comment
@deadlydog

deadlydog Aug 29, 2018

@datenschieber I saw #2505 as well, but it is a different issue.

I didn't read the entire thread, but I believe #2505 is about how using Import-Module -Force does not also reload submodules referenced by the specified module, so they are considering adding a -Recurse parameter as a way to to ensure that submodules get reloaded as well.

This issue is that using module does not reload the specified module. It would be a good idea though when this is implemented to have it automatically reload submodules as well, or else implement both -Force and -Recurse switches from the start so that you don't end up with a new issue similar to #2505, but for using module instead of Import-Module.

deadlydog commented Aug 29, 2018

@datenschieber I saw #2505 as well, but it is a different issue.

I didn't read the entire thread, but I believe #2505 is about how using Import-Module -Force does not also reload submodules referenced by the specified module, so they are considering adding a -Recurse parameter as a way to to ensure that submodules get reloaded as well.

This issue is that using module does not reload the specified module. It would be a good idea though when this is implemented to have it automatically reload submodules as well, or else implement both -Force and -Recurse switches from the start so that you don't end up with a new issue similar to #2505, but for using module instead of Import-Module.

@BrucePay

This comment has been minimized.

Show comment
Hide comment
@BrucePay

BrucePay Aug 30, 2018

Member

@deadlydog Having to change your code and add -Force -Recurse everywhere during development then removing it for production sounds unappealing. Having a universal DevMode flag seems more manageable.

Member

BrucePay commented Aug 30, 2018

@deadlydog Having to change your code and add -Force -Recurse everywhere during development then removing it for production sounds unappealing. Having a universal DevMode flag seems more manageable.

@deadlydog

This comment has been minimized.

Show comment
Hide comment
@deadlydog

deadlydog Aug 31, 2018

@BrucePay I suggested -Force (and potentially also -Recurse) for consistency with Import-Module and many other native cmdlets.

Perhaps a combination of both approaches would be best. Add in the -Force (and -Recurse if that gets added to Import-Module) so they can be used natively/explicitly, but also add in a dev-mode flag that would implicitly provide those parameters even if you don't provide them. The reason I mention having both is I imagine there will be other changes people would expect dev-mode to toggle on, such as providing -Force and -Recurse whenever Import-Module is called, and I'm sure there would be some people upset if certain things could only be enabled by having the dev-mode toggle turned on; e.g. there might be a reason why people want to use using module ... -Force in production code without also turning on all of the other things that dev-mode might turn on.

Having a dev-mode toggle that simply sets some defaults like that would be nice, but I think the scope of that request is beyond this issue. Plus, if all of the options are available to set natively/explicitly, then we wouldn't need to rely on the PowerShell team to provide the dev-mode toggle. Anybody would be able to write their own "dev-mode" module that could be imported and simply hides the native cmdlets and exposes theirs instead, where there's simply delegate the work to the native cmdlets, but with additional parameters provided (e.g. the decorator pattern).

For those reasons, I would still propose simply adding -Force (and -Recurse if needed) to using module. Dev-mode could be it's own new feature request, since it would apply to many more cmdlets than just using module.

deadlydog commented Aug 31, 2018

@BrucePay I suggested -Force (and potentially also -Recurse) for consistency with Import-Module and many other native cmdlets.

Perhaps a combination of both approaches would be best. Add in the -Force (and -Recurse if that gets added to Import-Module) so they can be used natively/explicitly, but also add in a dev-mode flag that would implicitly provide those parameters even if you don't provide them. The reason I mention having both is I imagine there will be other changes people would expect dev-mode to toggle on, such as providing -Force and -Recurse whenever Import-Module is called, and I'm sure there would be some people upset if certain things could only be enabled by having the dev-mode toggle turned on; e.g. there might be a reason why people want to use using module ... -Force in production code without also turning on all of the other things that dev-mode might turn on.

Having a dev-mode toggle that simply sets some defaults like that would be nice, but I think the scope of that request is beyond this issue. Plus, if all of the options are available to set natively/explicitly, then we wouldn't need to rely on the PowerShell team to provide the dev-mode toggle. Anybody would be able to write their own "dev-mode" module that could be imported and simply hides the native cmdlets and exposes theirs instead, where there's simply delegate the work to the native cmdlets, but with additional parameters provided (e.g. the decorator pattern).

For those reasons, I would still propose simply adding -Force (and -Recurse if needed) to using module. Dev-mode could be it's own new feature request, since it would apply to many more cmdlets than just using module.

@BrucePay

This comment has been minimized.

Show comment
Hide comment
@BrucePay

BrucePay Aug 31, 2018

Member

@deadlydog Some context: using module and Import-Module are two very different things. The using statement is part of the PowerShell language, just like if, while etc. Characteristically keywords do not take parameters (with a few exceptions e.g. switch -regex (which was a mistake - sorry - it seemed like a good idea at the time)). Import-Module, on the other hand, is "just" a cmdlet and so, like all cmdlets, it takes parameters. Behaviorally, keywords are processed once at parse time. This is why you have to use using when importing a class definition that you are deriving from. Cmdlets are processed at runtime and are evaluated each time they are encountered during script execution. So you can, for example, put Import-Module in a loop like:

foreach ($m in 'module1", "module2", "module3") { Import-Module $m }

whereas using statements have to be the first non-comment lines in a compile unit (script) and will be executed only once when the compile unit is parsed. This is important to why we don't want parameters on keywords. In PowerShell, parameters take arguments that are evaluated at runtime e.g. Import-Module -force:$foo bar. Keywords are processed at parse time and so can only take constant arguments (usually no arguments at all). For example switch takes -regex so you can write

switch -regex ('aaa') { a+ {'Eh!'} b+ {"Bee!"}}

but you can't write

switch -regex:$foo ('aaa') { a+ {'Eh!'} b+ {"Bee!"}}

because switch is processed entirely at parse time.

So in summary, using namespace is a keyword and keywords, as a rule, don't take parameters. And even if they had parameters the behavior would be inconsistent with cmdlets.

Another point to consider is that much of the time, people don't explicitly import modules, they let the autoloader do it. How do you pass a switch to the autoloader? Dev-mode fixes that.

Having a dev-mode toggle that simply sets some defaults like that would be nice, but I think the scope of that request is beyond this issue. Plus, if all of the options are available to set natively/explicitly, then we wouldn't need to rely on the PowerShell team to provide the dev-mode toggle.

Since, per the above, we are talking about making changes to the PowerShell language, either way, dev-mode or -force require changes to be made to the core engine, typically (but no longer exclusively) the domain of the PowerShell team. (Also, don't confuse dev-mode with strict-mode. The types of behaviours defined by strict mode are definitely something you'd want to leave on in production. On the other hand, dev-mode - forcing modules to be reloaded every time - will be a significant performance hit that you would definitely not want in production.)

Finally, you can use $PSDefaultParameterValues to make module imports -Force by default, so sort of a dev-mode switch:

$PSDefaultParameterValues."Import-Module:Force" = $true
Member

BrucePay commented Aug 31, 2018

@deadlydog Some context: using module and Import-Module are two very different things. The using statement is part of the PowerShell language, just like if, while etc. Characteristically keywords do not take parameters (with a few exceptions e.g. switch -regex (which was a mistake - sorry - it seemed like a good idea at the time)). Import-Module, on the other hand, is "just" a cmdlet and so, like all cmdlets, it takes parameters. Behaviorally, keywords are processed once at parse time. This is why you have to use using when importing a class definition that you are deriving from. Cmdlets are processed at runtime and are evaluated each time they are encountered during script execution. So you can, for example, put Import-Module in a loop like:

foreach ($m in 'module1", "module2", "module3") { Import-Module $m }

whereas using statements have to be the first non-comment lines in a compile unit (script) and will be executed only once when the compile unit is parsed. This is important to why we don't want parameters on keywords. In PowerShell, parameters take arguments that are evaluated at runtime e.g. Import-Module -force:$foo bar. Keywords are processed at parse time and so can only take constant arguments (usually no arguments at all). For example switch takes -regex so you can write

switch -regex ('aaa') { a+ {'Eh!'} b+ {"Bee!"}}

but you can't write

switch -regex:$foo ('aaa') { a+ {'Eh!'} b+ {"Bee!"}}

because switch is processed entirely at parse time.

So in summary, using namespace is a keyword and keywords, as a rule, don't take parameters. And even if they had parameters the behavior would be inconsistent with cmdlets.

Another point to consider is that much of the time, people don't explicitly import modules, they let the autoloader do it. How do you pass a switch to the autoloader? Dev-mode fixes that.

Having a dev-mode toggle that simply sets some defaults like that would be nice, but I think the scope of that request is beyond this issue. Plus, if all of the options are available to set natively/explicitly, then we wouldn't need to rely on the PowerShell team to provide the dev-mode toggle.

Since, per the above, we are talking about making changes to the PowerShell language, either way, dev-mode or -force require changes to be made to the core engine, typically (but no longer exclusively) the domain of the PowerShell team. (Also, don't confuse dev-mode with strict-mode. The types of behaviours defined by strict mode are definitely something you'd want to leave on in production. On the other hand, dev-mode - forcing modules to be reloaded every time - will be a significant performance hit that you would definitely not want in production.)

Finally, you can use $PSDefaultParameterValues to make module imports -Force by default, so sort of a dev-mode switch:

$PSDefaultParameterValues."Import-Module:Force" = $true
@deadlydog

This comment has been minimized.

Show comment
Hide comment
@deadlydog

deadlydog Sep 2, 2018

Thanks for the explanation and clarification @BrucePay 👍 I hadn't considered that using was a language keyword while Import-Module was a cmdlet, and as such the two adhere to different rules/conventions. Given that and what you mentioned above, the dev-mode flag makes more sense than adding -Force parameters.

So I guess I see 3 potential ways to go here:

  1. Introduce a new dev-mode flag as previously described.
  2. Simply change the existing using statement to automatically reload modules; I think that would be backward compatible, but if it would incur a performance hit at run-time that's probably not the way to go.
  3. Introduce a new language keyword (e.g. using reloaded module or some such thing) that removes the module before loading it.

Of those options, if it's possible and doesn't introduce any backward compatibility issues or performance costs then I think 2. would be the way to go. Otherwise, the dev-mode flag may be the best 🙂

deadlydog commented Sep 2, 2018

Thanks for the explanation and clarification @BrucePay 👍 I hadn't considered that using was a language keyword while Import-Module was a cmdlet, and as such the two adhere to different rules/conventions. Given that and what you mentioned above, the dev-mode flag makes more sense than adding -Force parameters.

So I guess I see 3 potential ways to go here:

  1. Introduce a new dev-mode flag as previously described.
  2. Simply change the existing using statement to automatically reload modules; I think that would be backward compatible, but if it would incur a performance hit at run-time that's probably not the way to go.
  3. Introduce a new language keyword (e.g. using reloaded module or some such thing) that removes the module before loading it.

Of those options, if it's possible and doesn't introduce any backward compatibility issues or performance costs then I think 2. would be the way to go. Otherwise, the dev-mode flag may be the best 🙂

@LosManos

This comment has been minimized.

Show comment
Hide comment
@LosManos

LosManos Sep 25, 2018

Can we solve the problem by having Import-Module import class and enum?

TL:DR

If using cannot be extended to force loading of a module, then we should be able to have Import-Module also import enums and classes. At the time of writing I am forced to use using as my psm1-file uses enums and classes but like wise forced to use Import-Module .. -Force since I am developing.
Practically it means I have to do every development iteration run twice, the first to fail and load the new module and the second for the real run.

FWIW I have a Stackoverflow question for answers outside of this forum.

LosManos commented Sep 25, 2018

Can we solve the problem by having Import-Module import class and enum?

TL:DR

If using cannot be extended to force loading of a module, then we should be able to have Import-Module also import enums and classes. At the time of writing I am forced to use using as my psm1-file uses enums and classes but like wise forced to use Import-Module .. -Force since I am developing.
Practically it means I have to do every development iteration run twice, the first to fail and load the new module and the second for the real run.

FWIW I have a Stackoverflow question for answers outside of this forum.

@vexx32

This comment has been minimized.

Show comment
Hide comment
@vexx32

vexx32 Sep 25, 2018

I would also prefer that to be the case. The fact that it doesn't import classes into the current scope makes it very difficult to, for example, discover what is valid input for cmdlets with a custom typed parameter.

Perhaps one of the only ways might be & (GMO 'module') {$host.EnterNestedPrompt()}

The difficulty is, however that reloading classes seems to be an order of magnitude more difficult to accomplish. :/

vexx32 commented Sep 25, 2018

I would also prefer that to be the case. The fact that it doesn't import classes into the current scope makes it very difficult to, for example, discover what is valid input for cmdlets with a custom typed parameter.

Perhaps one of the only ways might be & (GMO 'module') {$host.EnterNestedPrompt()}

The difficulty is, however that reloading classes seems to be an order of magnitude more difficult to accomplish. :/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment