Support customizing COM behaviors

[bclothier]

One major issue I have had with VBx was that Microsoft simply tried too hard to make it hard to customize the behavior of the COM objects. As a consequence, the VBx is pretty much limited to Automation specifications, which is a pretty narrow subset of COM ecosystem. In fact, .NET COM interop is considerably more powerful compared to what can be achieved in VBx and that is in spite of the interop layer that is required to work between COM and .NET. In the thread about inheritance, I commented:

One thing that is not easily done inside VB*, however, is the ability to override the behavior of IUnknown::AddRef, IUnknown::Release and IUnknown::QueryInterface. In .NET, you can override the IUnknown::QueryInterface by implementing ICustomQueryInterface interface. Similarly, you can make a .NET object extendable by implementing IReflect (which is compatible with IDispatch-bound calls). The benefit of using ICustomQueryInterface and IReflect is that you get the benefit of type safety that you otherwise wouldn't have if you were implementing the IUnknown directly.

Originally posted by @bclothier in https://github.com/WaynePhillipsEA/twinbasic/issues/73#issuecomment-826012467

I'm splitting this out to its own issue so that it can be considered as I think that is something that doesn't require inheritance but would go a long way toward making tB more powerful and capable of working with non-Automation COM objects. The interface approach that .NET uses for customizing the COM behavior seems to be a good way of enabling higher degree of control without having to wade into the level of C plumbing.

References:

ICustomQueryInterface
IReflect
ICustomMarshaler

[Kr00l]

When tB supports #9 (unsigned datatypes) then we have full control of stdole.IUnknown. :)

[bclothier]

Yes but if tB generates code for handling the IUnknown, that would cause problems. We'd need a way of ensuring that no codegen are made for classes that implements the IUnknown or IDispatch themselves. ICustomQueryInterface is a cheap way of extending the class's behavior without taking on the burden of the reference counting that's handled by the codegen from the tB's compiler.

[Kr00l]

Sure. I meant it's then easier to "interact", not to replace the "implementation".
For example for twinbasic/twinbasic#433 I use a custom IUnknownUnrestricted which returns LONG instead of ULONG.
Also you can QueryInterface, like in below example where you can test for an stringified iid.

Public Function VTableInterfaceSupported(ByVal This As OLEGuids.IUnknownUnrestricted, ByVal IIDString As String) As Boolean
Debug.Assert Not (This Is Nothing)
Dim HResult As Long, IID As OLEGuids.OLECLSID, ObjectPointer As Long
CLSIDFromString StrPtr(IIDString), IID
HResult = This.QueryInterface(VarPtr(IID), ObjectPointer)
If ObjectPointer <> 0 Then
    Dim IUnk As OLEGuids.IUnknownUnrestricted
    CopyMemory IUnk, ObjectPointer, 4
    IUnk.Release
    CopyMemory IUnk, 0&, 4
End If
VTableInterfaceSupported = CBool(HResult = S_OK)
End Function

Though one obstacle would remain. stdole.IUnknown::QueryInterface returns HRESULT, thus a sub in tB.
The IUnknownRestricted just returns LONG instead of HRESULT.
Maybe that can be addressed in tB as well? I remember @WaynePhillipsEA mentioned this already once.

[Kr00l]

Btw, IUnknown and IDispatch cannot be overriden or "reimplemented" in classes.
So, having a in-built ICustomQueryInterface would be the only easy way w/o hacking the vtable and lightweight COM.

Even by custom interfaces as of twinbasic/twinbasic#388 are not allowed to match VBx behavior.

[bclothier]

The procedure that returns a HRESULT should be decorated with to-be-implemented PreserveSig attribute which was mentioned in the twinbasic/twinbasic#25 as to allow tB to define as a function returning a long, rather than a sub.

I have yet to have a need to implement the AddRef and Release -- extending the QueryInterface and the IDispatch::Invoke is usually good enough for me.

[Kr00l]

How/where could we define (once implemented) a [PreserveSig] attribute on stdole.IUnknown::QueryInterface ?
There is no "declare" or so.. (e.g. like a DllImportAttribute)

[bclothier]

The redefinition of an existing interface would look something like this:

[ 
  ComImport,
  Guid("<guid of the IUnkown>"
]
Public Interface IUnknown
  [ PreserveSig ]
  Function AddRef() As Long
  [ PreserveSig ]
  Function Release() As Long
  [ PreserveSig ]
  Function QueryInterface(ByRef refiid As GUID, ByRef ppvoid As Object)
End Interface

1. ComImport indicates that this is an interface imported from some other place, rather than a new definition.
2. Guid can match the IUnknown's GUID as to replace the original definition
3. PreserveSig allows the functions to return HRESULT directly rather than implicitly via the codegen.

With the redefined interfaces, you can then cast an object into MyLibrary.IUnknown and call the methods directly on it. You could Implements it on an class and allow it to implement the methods instead.

As you can see, this wouldn't require you to write a IDL file; you'd be able to do it all in the tB source. .NET COM interop is pretty powerful because of capabilities like those.

[mburns08109]

Ben,
I'm kind-of following this, but I'm a bit lost on the details.
Could you give a few "Whys" and/or examples on some of your points?

Also, was there anything like this sort of "extending the COM interface" thinking that led MS into the infamous "Great ADO library f*ck-up"?

[bclothier]

As I mentioned, Automation is a pretty narrow subset of COM and that limits the potential of VBx a lot. There are far more COM stuff out there that aren't Automation-compatible. To provide an example, a large majority of them implements only the IUnknown. To use those, you must supply the interface definition yourself. You can write your own type library but that requires that you install the MIDL compiler, which is a part of C++ toolchain and it's pretty huge. Alternatively, you can just define the interface in the .NET language and start using it.

Another example of non-Automation-compatible interface would be those that exposes a C-style array in the definition. That's a big No-No in VBlandia - they must be all SAFEARRAY! Structs with pinned members with variable length is another example.

Then sometimes there are private/undocumented COM interfaces that is known and you want to use it instead.

Finally, sometime, it's just more convenient to redefine the interface with compatible data types for ease of use within your code.

For some examples, take look at Rubberduck's TypeLib API which was helpfully developed by Wayne.

This class is particularly a good example where we can work around buggy implementations and make it work for our purposes. Take particular note of this line:

internal sealed class TypeInfoWrapper : TypeInfoInternalSelfMarshalForwarderBase, ITypeInfoWrapper

If you follow the definition of the TypeInfoInternalSelfMarshalForwarderBase, you get to the ITypeInfoInternal which is a redefinition of the ITypeInfo library. Look at the definitions where we basically substitute various parameter types as to get more control and avoid bugs arising from bad implementations.

Could it have had helped with ADO fiasco? Maybe, but it's only to work for your own codebase, not for the other guy's codebase using the provided ADO type library.