The PSObject implementation does not handle classes derived from DynamicObject properly

[jazzdelightsme]

This repro’s on 5.1 and 6. The problem is that if I have a DynamicObject that has a dynamic property “Foo”, things like $obj.Foo work, but Select-Object -ExpandProperty Foo do not.

Steps to reproduce

Add-Type @"
using System;
using System.Collections.Generic;
using System.Dynamic;

public class MyDynamicObject : DynamicObject
{
    public override IEnumerable<string> GetDynamicMemberNames()
    {
        return new string[] { "Foo" };
    }

    public override bool TryGetMember( GetMemberBinder binder, out object result )
    {
        result = null;
        bool succeeded = false;

        if( binder.Name == "Foo" )
        {
            result = 123;
            succeeded = true;
        }

        return succeeded;
    }
}
"@

$myObj = [MyDynamicObject]::new()

$myObj.Foo                                  # <-- works
$myObj | %{ $_.Foo }                        # <-- works
$myObj | Select-Object -ExpandProperty Foo  # <-- DOES NOT WORK

Expected behavior

The last three lines should all work (yield 123).

Actual behavior

The Select-Object statement throws. ("Select-Object : Property "Foo" cannot be found.")

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.1.0-preview.1
PSEdition                      Core
GitCommitId                    v6.1.0-preview.1
OS                             Microsoft Windows 10.0.16299
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[BrucePay]

This is not specific to select. For example:

PS[1] (459) > $myObj | foreach { $_.Foo }
123
PS[1] (460) > $myObj | foreach Foo
PS[1] (461) >

It looks like we aren't handling DynamicObject in the PSObject wrapper. Foo doesn't show up as a property:

PS[1] (467) > $myObj.PSObject.Properties
PS[1] (468) >

[BrucePay]

The original title was a bit confusing given that PowerShell, from V3 on, uses the DLR extensively. I've updated it to more accurately reflect what I think the problem is. @jazzdelightsme - are you ok with the updated title?

[jazzdelightsme]

Sure. Thank you!

[jazzdelightsme]

PSDynamicMember appears to be a mostly-useless stub: the Value getter/setters just throw, and it doesn't even hold a reference to the owning object. It seems like the solution will involve teaching this thing how to get a value... first I suppose we need to be able to determine if the "Foo" member even is a property or not (maybe it's a method). Then, if it's a property, we have to figure out if it is settable and/or gettable. @lzybkr / @BrucePay, is that possible with some DLR-fu?

Looking at the DLR stuff is ... intimidating. It's hard to tell where to grab on first. Any hints?

[lzybkr]

PSDynamicMember exists just for Get-Member - and the results of Get-Member don't reference an owning object.

I believe this is the code that is supposed to cover your scenario. It was likely tested (I don't typically write code like that without testing) so I'm not sure why it doesn't work - but I'd start there.

[jazzdelightsme]

Thanks @lzybkr!

I believe that code does in fact work. That's why things like $myObj.Foo work: PSObject dynamically binds the member access, and all is well.

But for Select-Object -ExpandProperty, on the other hand... that code does not come into play.

When using wildcards (like $myObj | select -exp *Foo*), DotNetAdapter.AddAllDynamicMembers correctly finds the "Foo" member, and adds a PSDynamicMember to the collection, so the Match method, when it calls GetIntegratedMembers, has the "Foo" member in the list.

However, the Match method is instructed to only return members of type Properties or PropertySet (as specified by MshExpression.ResolveNames(PSObject, bool)), but the "Foo" member is a Dynamic, so it gets filtered out.

For the non-wildcard case (like $myObj | select -exp Foo), ResolveNames just looks for target.Members[_stringValue] (where _stringValue is "Foo"), and dynamic members don't seem to show up in that PSMemberInfoIntegratingCollection.

If dynamic members were to be exposed to ResolveNames (via Members.Match or Members[string indexer], though... perhaps things could work. But wouldn't PSDynamicMember would have to built up a bit, for that to work? And I'm not sure if PSDynamicMember can be taught to do what it needs to do.

If that's not the right approach... then what will
MshExpression.GetValue do? (if the dynamic member does not show up in Properties?)

[lzybkr]

Ah, of course. That's the problem then. The PSObject pseudo-reflection like api can't work for dynamic objects, at least not in general.

The GetDynamicMemberNames api that a dynamic object implements is not a contract. For example, a dynamic object could synthesize a property just by trying to access it - essentially giving the object infinite members. The primary use case for the api is a debugger, or in PowerShell's case, Get-Member - in other words - for a somewhat improved UI, but nothing more.

I believe the right fix is to use a dynamic site in the same way that PowerShell does for the $obj.member syntax.

This problem affects Select-Object, ForEach-Object, Sort-Object, Measure-Object, Where-Object, Format-List, Format-Table and maybe more.

It would seem this scenario is a corner case - it has never worked and this is the first report I've seen. That said, fixing it has a nice benefit - performance should improve even when not using dynamic objects.

A fix does need to be careful though - some of those commands accept wildcards for property names. It might be best to keep the existing code for any use of wildcards.

[jazzdelightsme]

Okay, it may be beyond my ken, but I'll buy you lunch, @lzybkr, if you are willing to talk me through what you think the "right" change would look like...

[lzybkr]

No need to buy me lunch, but I'm happy to chat.

To get you started - you can find similar code here.

The difference is - you'll use PSGetMemberBinder.Get("PropertyName"). It really does just boil down to a couple of lines of code - creating the call site with the right binder, then invoking the delegate for that site.

[jazzdelightsme]

I came up with an PoC (see here) that seems to work well, but it only addresses Select-Object--not other things like Where-Object.

Next I'll try pushing the solution down a bit further, into the .Members collection itself.