Invoke-DscResource bypassing LCM for PS7

[gaelcolas]

This is an RFC to add support for an Invoke-DscResource in PowerShell 7.
Since Get-DscResource and Configuration have been fixed, this RFC is only to add something similar in syntax than the WMF 5.1's Invoke-DscResource, but not using the LCM or CIM, and only consuming PowerShell resource directly.

This RFC mainly specify an MVP (Minimum Viable Product), but feel free to comment or add in the out of scope section, what should come next.

[dmilov]

In PowerShell 5.1 resources property types are limited to the CIM type system because of the LCM. Since the new Invoke-DSCResource will call PowerShell, is it expected all PowrShell types to be supported for resource properties?

For instance Hashtable wasn’t supported and the resource have to do conversion to CIM type to present Hashtable user input.

What would be supported types for properties?

[SimeonGerginov]

Following the previous comment about the properties I was wondering if passing one DSC Resource as property to another would be available with Invoke-DscResource?

[TravisEz13]

@SimeonGerginov Invoke-DscResource would invoke individual resources. So you could get one resource, and the pass the result to another, but you would arrange passing the data.

[TravisEz13]

@dmilov Invoke-DscResource will return the exact object that the resource returns for the get method. for test, true or false would be allowed. No, output is expected for the set method and the output will be logged to the verbose stream.

[jpogran]

@TravisEz13 is it correct that Invoke-DscResource will not return anything for set? How are things that use that cmdlet to know whether the cmdlet succeeded, or if state was changed? How will errors be handled?

The reason I ask is that Puppet uses Invoke-DscResource as an integration point, and we inspect the return of Test and Set to check state and inspect whether state was changed. If we lose that then we lose the integration point.

[TravisEz13]

@jpogran the PowerShell convention for Invoke is to Write an error for failure. This is what the LCM would expect for the failure Set method and therefore so would Invoke-DscResource. It would be possible of all other methods, but Test should return $true or $false if it can.

[jpogran]

@TravisEz13 apologies, my reply was slightly confusing to read before, so I'm trying again. I'm not talking about implementing DSC Resources, I'm talking about a vendor using Invoke-DscResource.

For example, if you run the following on v5.1, you get back an indication if the state has drifted on the target machine:

PS C:\Users\james> $invokeParams = @{
>>   Name          = 'User'
>>   Modulename    = 'PSDesiredStateConfiguration'
>>   Method        = 'test'
>>   Property      = @{
>>     username = 'foobar'
>>     disabled = $true
>>   }
>> }
PS C:\Users\james> Invoke-DscResource @invokeParams

InDesiredState
--------------
False

Puppet uses the value of InDesiredState to determine if it needs to run Set. If Test returns the results of Get, is there still some value returned that tells the caller it is not in desired state? Or is the caller now responsible for determining if there is drift?

[TravisEz13]

I can update that. Gale, the LCM, I'm sure just returns this based on if Set succeeds or fails. Is this the behavior we want?

[TravisEz13]

@jpogran looking at the test method in windows PowerShel, it returns the same thing as the set method. I'll update that as well, but both are CIM object. I recommend we use a new class with the same properties.

[gaelcolas]

@dmilov The Invoke-DscResource for PS7+ would not go through CIM, so it would support other types for the parameters of the Get/Set/Test functions, but it's worth noting that doing so would mean the resource would not be backward compatible with DSC in WMF5.1, and by way of things all solutions leveraging 5.1 and not yet 7+ (DSC in 5.1, Chef, Puppet & probably Ansible).
I'd still recommend against it for someone who creates resources that targets many customers and different solutions, such as in the DSC Resource Kit.

Eventually the bigger questions is what type can be easily serialized & deserialized, even if we don't use CIM, so that any tool/solution can integrate with it. But we're not there yet.

[gaelcolas]

@jpogran looking at the test method in windows PowerShell, it returns the same thing as the set method. I'll update that as well, but both are CIM object. I recommend we use a new class with the same properties.

Yes for a new class with same property (not CIM).

The TEST returns:

InDesiredState
--------------
True

The SET returns:

RebootRequired
--------------
False

I'll provide more details and update the RFC, but the returned object will expose the same properties, from different classes (not CIM).

[jborean93]

Just thought I would comment on this, while Ansible does not support PowerShell Core right now we do currently have a Ansible win_dsc module that uses Invoke-DscResource. We currently rely on WMI/CIM to list and validate the available input options for a resource so removing WMI would break that ability. That is fine as I understand this is trying to be cross platform but having a replacement way to scan the DSC resource parameters, their types, and whether they are required would be great. You can technically do this with Get-DscResource but it falls flat when you get to properties that are a CimInstance/nested type. Using xWebsite in xWebAdministration as an example

PS C:\Users\vagrant-domain> (Get-DscResource -Name xWebsite).Properties

Name                     PropertyType                         IsMandatory Values
----                     ------------                         ----------- ------
Name                     [string]                                    True {}
ApplicationPool          [string]                                   False {}
ApplicationType          [string]                                   False {}
AuthenticationInfo       [MSFT_xWebAuthenticationInformation]       False {}
BindingInfo              [MSFT_xWebBindingInformation[]]            False {}
DefaultPage              [string[]]                                 False {}
DependsOn                [string[]]                                 False {}
EnabledProtocols         [string]                                   False {}
Ensure                   [string]                                   False {Absent, Present}
LogCustomFields          [MSFT_xLogCustomFieldInformation[]]        False {}
LogFlags                 [string[]]                                 False {BytesRecv, BytesSent, ClientIP, ComputerN...
LogFormat                [string]                                   False {IIS, NCSA, W3C}
LoglocalTimeRollover     [bool]                                     False {}
LogPath                  [string]                                   False {}
LogPeriod                [string]                                   False {Daily, Hourly, MaxSize, Monthly...}
LogTargetW3C             [string]                                   False {ETW, File, File,ETW}
LogTruncateSize          [string]                                   False {}
PhysicalPath             [string]                                   False {}
PreloadEnabled           [bool]                                     False {}
PsDscRunAsCredential     [PSCredential]                             False {}
ServiceAutoStartEnabled  [bool]                                     False {}
ServiceAutoStartProvider [string]                                   False {}
SiteId                   [UInt32]                                   False {}
State                    [string]                                   False {Started, Stopped}

We can see all the applicable properties, their types, whether it is mandatory, and available choices for that property which is great. What we can't do without WMI is see those same details for the AuthenticationInfo, BindingInfo, or LogCustomFields properties. These details are stored in the MOF https://github.com/PowerShell/xWebAdministration/blob/dev/DSCResources/MSFT_xWebsite/MSFT_xWebsite.schema.mof so it's not just an arbitrary dictionary. Using WMI this is possible with (Get-CimClass -ClassName "MSFT_xWebAuthenticationInformation" -Namespace root\Microsoft\Windows\DesiredStateConfiguration).CimClassProperties:

PS C:\Windows\system32> (Get-CimClass -ClassName "MSFT_xWebAuthenticationInformation" -Namespace root\Microsoft\Windows\DesiredStateConfiguration).CimClassProperties


Name               : Anonymous
Value              :
CimType            : Boolean
Flags              : Property, NullValue
Qualifiers         : {write}
ReferenceClassName :

Name               : Basic
Value              :
CimType            : Boolean
Flags              : Property, NullValue
Qualifiers         : {write}
ReferenceClassName :

Name               : Digest
Value              :
CimType            : Boolean
Flags              : Property, NullValue
Qualifiers         : {write}
ReferenceClassName :

Name               : Windows
Value              :
CimType            : Boolean
Flags              : Property, NullValue
Qualifiers         : {write}
ReferenceClassName :

Once again we have access to the same information it's just in a different form. Implementing the ability to get sub type details in Get-DscResource or some other cmdlet would solve that issue for us.

[TravisEz13]

My Invoke-DscResource is using Get-DscResource to find the resources. I'm making fixes to make it work on Linux and macOS. The first example is working.

Example from my macOS dev machine:

❯❯❯ (Get-DscResource -Name Script -Module PSDscResources).Properties                                                                                                                                                                                                                                              
Name                 PropertyType   IsMandatory Values                                                                                                   
----                 ------------   ----------- ------                                                                                                   
GetScript            [string]              True {}                                                                                                       
SetScript            [string]              True {}                                                                                                       
TestScript           [string]              True {}                                                                                                       
Credential           [PSCredential]       False {}                                                                                                       
DependsOn            [string[]]           False {}                                                                                                       
PsDscRunAsCredential [PSCredential]       False {}

[jborean93]

@TravisEz13 yep that's how it works right now in v5 but I'm talking about nested types and getting those properties. Maybe considering WMI will be removed then that won't be a thing anymore?

[TravisEz13]

On windows, Cim cmdlets will still be there. Therefore, you can examine the types. On non-windows... there probably still is a way. The difference is we won't be using CIM to deliver the objects to you.

[TravisEz13]

@jborean93 Nope... no way, because the nested types are not working:

I'll open an issue on DSC. cc @mgreenegit

[jpogran]

To add to what @jborean93 stated, Puppet uses that information also in one of the modules it maintains to build parameter validation logic. That PowerShell didn't give that information in pwsh6, meant we used a mof parser in ruby to build our logic.

[gaelcolas]

Quick update that could be of interest for @jpogran & @jborean93: I think there's a way to find the type information without using Get-CimClass.

I got it to work with MOF based resource, exploring for class based ones atm.
Will talk to @TravisEz13 about what we can do with that.

[Microsoft.PowerShell.DesiredStateConfiguration.Internal.DscClassCache]::ClearCache()
[Microsoft.PowerShell.DesiredStateConfiguration.Internal.DscClassCache]::Initialize()


$module = Get-Module -Name mod1 -ListAvailable -SkipEditionCheck
$resourceName = 'RsrcWithSubType'
$schemaFilePath = $null
$functionsToDefine = [System.Collections.Generic.Dictionary[string,ScriptBlock]]::new(([System.StringComparer]::OrdinalIgnoreCase))


$MyErrors = [System.Collections.ObjectModel.Collection[System.Exception]]::new()

[Microsoft.PowerShell.DesiredStateConfiguration.Internal.DscClassCache]::ImportCimKeywordsFromModule($module, $ResourceName, [ref] $SchemaFilePath, $functionsToDefine, $MyErrors)
[System.Management.Automation.Language.DynamicKeyword]::GetKeyword($ResourceName).Properties
# Key                  Value
# ---                  -----
# DependsOn            System.Management.Automation.Language.DynamicKeywordProperty
# PsDscRunAsCredential System.Management.Automation.Language.DynamicKeywordProperty
# Ensure               System.Management.Automation.Language.DynamicKeywordProperty
# Name                 System.Management.Automation.Language.DynamicKeywordProperty
# SubTypeProperty      System.Management.Automation.Language.DynamicKeywordProperty

[System.Management.Automation.Language.DynamicKeyword]::GetKeyword('SubType').Properties

# Key       Value
# ---       -----
# Property1 System.Management.Automation.Language.DynamicKeywordProperty
# Property2 System.Management.Automation.Language.DynamicKeywordProperty
# Property3 System.Management.Automation.Language.DynamicKeywordProperty

This works (on Windows) both for PS 5.1 & PS7.preview3 (minus the -SkipEditionCheck for 5.1, that's my dummy module issue).

[edit]
And it's available in 6.2 as well.

[edit 2020-04-19]

Adding the equivalent with Class based resources:

[Microsoft.PowerShell.DesiredStateConfiguration.Internal.DscClassCache]::ClearCache()
[Microsoft.PowerShell.DesiredStateConfiguration.Internal.DscClassCache]::Initialize()
$module = Get-Module -Name mod1 -ListAvailable -SkipEditionCheck
$functionsToDefine = [System.Collections.Generic.Dictionary[string,ScriptBlock]]::new(([System.StringComparer]::OrdinalIgnoreCase))
$className = [string[]]'ClassRsrcWithSubType'
$resources = [Microsoft.PowerShell.DesiredStateConfiguration.Internal.DscClassCache]::ImportClassResourcesFromModule($Module, $className, $functionsToDefine)
[System.Management.Automation.Language.DynamicKeyword]::GetKeyword($resources[0]).Properties

# Key       Value
# ---       -----
# Property1 System.Management.Automation.Language.DynamicKeywordProperty

[System.Management.Automation.Language.DynamicKeyword]::GetKeyword($resources[1]).Properties

# Key                  Value
# ---                  -----
# DependsOn            System.Management.Automation.Language.DynamicKeywordProperty
# PsDscRunAsCredential System.Management.Automation.Language.DynamicKeywordProperty
# Ensure               System.Management.Automation.Language.DynamicKeywordProperty
# Name                 System.Management.Automation.Language.DynamicKeywordProperty
# SubTypeProperty      System.Management.Automation.Language.DynamicKeywordProperty

[JamesWTruher]

we should be using ThrowTerminatingError

[TravisEz13]

The module is written in PowerShell...

[JamesWTruher]

a mutex should enable system wide locking if you go down this path.

[gaelcolas]

I think system wide locking is not desired.
Allowing concurrent execution is good, just wanted to warn about the implication to the end user.

[JamesWTruher]

should this be strongly typed (rather than a pscustomobject with NoteProperties?

[TravisEz13]

It is

[JamesWTruher]

as noted in Motivation is this work sufficient to unblock those that can't move to PS7?