Skip to content

Stateless CustomMarshaller that operates both on object and struct doesn't work #116790

@m-celikba

Description

@m-celikba

Description

Consider this marshaller:

[CustomMarshaller(typeof(object), MarshalMode.Default, typeof(MyStructMarshaller))]
//[CustomMarshaller(typeof(MyStruct), MarshalMode.Default, typeof(MyStructMarshaller))] COMMENT 1
public static class MyStructMarshaller
{
    public static MyStruct ConvertToUnmanaged(object? managed)
    {
        ...
    }

    public static object? ConvertToManaged(MyStruct unmanaged)
    {
        ...
    }

    public static void Free(MyStruct unmanaged)
    {
        ...
    }
    
    //public static MyStruct ConvertToUnmanaged(MyStruct managed) => ConvertToUnmanaged((object?)managed); COMMENT 2

    //public static MyStruct ConvertToManaged(MyStruct managed)  { ... }; COMMENT 3
}

this works however for the following scenario
COM method of an interface marked with GeneratedComInterface:
void CreateMyStruct(out MyStruct myStruct);

I'm getting
error SYSLIB1051: The type 'XXX.MyStruct' is not supported by source-generated COM. The generated source will not handle marshalling of parameter 'myStruct'

If I uncomment COMMENT 1 and COMMENT 2 lines above, the compile errors are gone however the generated code for the COM interface doesn't contain code for calling the ConvertToManaged() method for this other COM method:
MyStruct GetMyStruct();

I've tracked down the issue to dotnet\runtime\src\libraries\System.Runtime.InteropServices\gen\Microsoft.Interop.SourceGeneration\MarshallerShape.cs:ConvertToManaged() which expects to find a method ConvertToManaged() whose return type is of type MyStruct.

This is not possible: we cannot uncomment COMMENT 3 line above as functions with same signature but different returned type are not allowed.

PS: MyStruct is marked with [NativeMarshalling(typeof(MyStructMarshaller))]

Reproduction Steps

see description

Expected behavior

I'm not sure if this is a supported scenario. It seems to be since I get no analyzer complains.
Maybe MarshallerShape.cs:ConvertToManaged() needs to consider function returning object as positives.

Actual behavior

the generated COM code is missing calls to ConvertToManaged()

Regression?

No response

Known Workarounds

No response

Configuration

.NET 9

Other information

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions