Fix Various PowerShell Class Issues

[rjmholt]

Meta-issue to track work on improving the experience of PowerShell class usage as much as possible.

Note on By-Design Behaviours

PowerShell classes currently are a bit finnicky. Some of this is a necessary pain because of the design constraints behind classes.

Classes are currently implemented as compiling to .NET IL so that we can take advantage of the .NET object model. Not doing this would mean us reinventing (and having to maintain) the wheel on essentially all object-oriented features available in .NET (or anywhere else) and paying a high runtime overhead on shimming compatibility with the .NET object model (e.g. faking inheritance from .NET classes).

Instead, we compile PowerShell classes to dynamic assemblies and bake in calls to PowerShell in the generated IL. To do this at runtime would mean either breaking dynamic- and module-scoped behaviours in classes, or emitting a new dynamic assembly every time we hit a class definition (and since PowerShell's highest compilation unit is the scriptblock, and scriptblocks are permitted in for-loops, the performance penalty here could be extreme). (I'm still a bit hazy on the details about the mechanisms here, especially in terms of why caching is made impossible by module scope or in comparison to Add-Type, but @daxian-dbw or @lzybkr will be able to add to/correct me).

The need to compile classes at parse-time means the types required to define a class (in IL) must also be known at parse-time. The module-scoping issue means that a using module statement is required to import classes from modules (@daxian-dbw might like to add information here about the specifics governing this need), or by exporting class usage in PowerShell functions (classes having a sort of module-private behaviour).

Issues to Improve

There are, however, a few improvements possible to PowerShell classes that might not run up against the design constraints given above. Below is a list of open issues for classes in PowerShell.

Classes in modules

• PowerShell classes behaviour inconsistent #2449 Exporting classes in PS modules consistently.
• Reloading module does not reload submodules. #2505 Classes in nested modules are not updated by Import-Module -Force.
• Using module statement doesn't work with ScriptsToProcess #2962 Using ScriptsToProcess with using module stops classes from being imported
• Using module behavior on nested modules is inconsistent #2964 using module imports nested module classes in a script but not interactively.
• using module test doesn't load the powershell class defined in 'test' if 'test.psd1' has 'FunctionToExport' or 'CmdletToExport' specified #4112 using module doesn't load classes when FunctionToExport or CmdletToExport are specified in the .psd1
• If a module 'test' has multiple nested .psm1 modules that define powershell classes, then using module test can only load the classes defined in test.psm1 but not other .psm1 files #4114 using module does not find classes in nested modules.
• PowerShell class methods cannot invoke non-exported functions. #4713 PS classes cannot invoke non-exported functions from their module (possibly a New-Module problem).
• Make PowerShell Classes Exportable Module Members #6293 Making PS classes exportable module members.

Type errors

• Classes inheriting from [System.Attribute] cannot be defined and used in the same PowerShell module #1762 Custom attributes can't be defined and used in the same PS module
• Using external types / classes in Custom PowerShell Classes (v5) results in Parsing Error #2074 Types not available at parse-time used in PS class bodies cause parse errors. @daxian-dbw noted that we need some types at parse-time (such as method return type) to generate the IL, but type resolution inside method/constructor scriptblocks may not be needed and be more appropriate at runtime. Noted as by design.
• Defining a PowerShell class in a script causes a parser error if the class references external types that aren't currently loaded #3641 Type availability at parse-time. Need to implement the metadata reader (Use the MetadataLoadContext to restore reflection-only type discovery for mechanisms like using assembly #6653).
• See also: PSScriptAnalyzer #850

Other bugs

• Move classes error messages from runtime to compiletime/parsetime #2224 Some class errors occur at runtime when they can be detected at parse-time. (We may want to loosen the error-checking behaviour here if it makes sense to do so).
• New-Object doesn't work with classes in nested modules #2963 New-Object does not work for PS classes as it does for .NET classes.
• Error trying to use XmlSerializer with PS5-styled classes #5337 PS Classes in collectible assemblies cause XML serialisation problems.
• Impossible to call overridden protected internal method of base class. #7622 protected internal override doesn't work (program diverges on method call).
• Class : Methods and properties can't have the same name #8235 Properties and methods with the same name are permitted in a declaration, but attempting to call the method results in an error that no such method exists.

New feature requests

• Support interface declaration using classes #2223 Interface declarations
• Parameters to methods in classes should be able to take [VALIDATExxx] attributes. #2485 Validation attributes for PS classes
• Enable native interop with static extern class methods #2876 static extern methods and struct definition in PowerShell
• Add use static <class> usage to allow static member access in PowerShell #5099 Static class member imports
• Improvements for classes #6418 Other proposed class extensions.
• enum: Underlying type #8028 Support enum types other than Int32

A couple of other resources on these issues:

• @devblackops' talk on issues with PS classes: video, code.
• The SAPIEN page on the using statement: here.

[sancarn]

Have override methods been suggested?

For example I have had to implement GUIs while overriding the WndProc method before. So a way to override methods would be a great addition to Powershell classes.

Edit:

I noticed #6418 suggested overriding property setters and getters. I suppose that is similar to this.

[christru]

Is there any community projects that work around these issues for example? I cannot for the life of me get a decent project implemented in PS w/ classes broken out into a sensible manner without running into one of the above issues..

[devblackops]

@christru You can take a look at my PoshBot module here: https://github.com/poshbotio/PoshBot

That module is almost entirely PowerShell class based.

[christru]

@devblackops as I’m litterally watching your YouTube video talk on classes. Thanks buddy!

[devblackops]

@christru Cool. I did the same talk at the PowerShell Summit. It is a more polished version. https://www.youtube.com/watch?v=i1DpPU_xxBc&list=PLfeA8kIs7CocGXuezOoYtLRdnK9S_Mq3e

[IISResetMe]

@rjmholt I believe #8302 (classes don't produce valid interface property methods) should be filed under "Other bugs"

[rjmholt]

@IISResetMe currently classes don't allow overrides. Wouldn't the virtual addition be part of the interface inheritance?

[IISResetMe]

@rjmholt well, we already mark all generated methods virtual, allowing class definitions to inherit existing interfaces:

class PleaseDisposeOfMe : IDisposable
{
  [void]Dispose(){ <# free unmanaged resources #> }
}

but since we never mark the underlying get/set methods of properties virtual (looks like an oversight), you can't actually implement an interface with a property:

class MyPrincipal : System.Security.Principal.IPrincipal
{
  [System.Security.Principal.IPrincipal]$Identity
  [bool]IsInRole()
}

At line:2 char:1
+ class MyPrincipal : System.Security.Principal.IPrincipal
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error during creation of type "MyPrincipal". Error message:
Method 'get_Identity' in type '<404e3736>.MyPrincipal' from assembly '⧹powershell, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : TypeCreationError

[KirkMunro]

Forgive some ignorance on my part, but looking at the number of issues and the complications that exist in classes in PowerShell today, I can't help but feel this is a design problem rather than a collection of issues/bugs to be addressed.

Wouldn't it make more sense and be much easier if:

• PowerShell had a new module type specifically for class definitions, called something like class module that could only include class definitions, with a psx1 file extension; nothing else would be allowed in these files (no functions, variables, command invocations, etc.)
• Import-Module automatically recognized psx1 files defined in RootModule, or class modules included in NestedModules, or RequiredModules fields in a module manifest, as well as psx1 modules that don't have a manifest at all, and loaded the types defined in those psx1 files accordingly (RequiredModules first, then NestedModules, then RootModule, so that you can derive from other types in other modules)
• PowerShell automatically identified classes derived from PSCmdlet and added them to the current session without the extra work that is being done here.
• PowerShell automatically exported classes that are associated with cmdlets that are exported from a module.
• PowerShell supported ClassesToExport in module manifests to identify types in modules so that implicit loading could still work, even with classes.
• Export-ModuleMember had a -Class parameter to explicitly identify which classes you want to export

One of the drivers behind this approach is that it would do away with the need for using for modules containing classes, and supports #requires, NestedModules, RequiredModules, which makes installing modules from the PowerShell Gallery work as expected, and everything else that people already do with Import-Module today. Having both using and Import-Module is confusing and already requires a lot of re-thinking when it comes to how you set up modules to properly load classes. It also facilitates defining an equivalent of binary modules from within PowerShell itself.

There are more thoughts behind this, but I've shared enough to see what others think for now.

[KirkMunro]

On second thought, I'd prefer not having to deal with Export-ModuleMember or ClassesToExport as suggested in those last two bullets at all, having psx1 files be a PowerShell-ish equivalent of a type library, with types automatically exported just as they are with dlls (ideally with something resembling public/private to control visibility). With that approach, the last three bullets wouldn't be necessary -- classes would just be loaded and available once a module was imported.