Reflection free mode: using sqlclient - possible options for obfuscation

[LLT21]

@MichalStrehovsky : now that it is possible with .NET 8 to run both the kestrel server and npgsql in reflection free mode (see https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/CSharp/appmpower) I am trying to close my last open item, running the sqlclient in:

1. either reflection free mode: the first blocking factor here seems to be the missing option for flags something like

    <RuntimeHostConfigurationOption Include="Switch.System.Reflection.Disabled.DoNotThrowForAttributeFlags" Value="true" />

Unhandled Exception: MT4340638104: TypeInitialization_Type_NoTypeAvailable
 ---> MT4340635080: Reflection_Disabled
   at Internal.Reflection.RuntimeTypeInfo.GetAttributeFlagsImpl() + 0x34

This has been reported in other issues and I have seen that was omitted for a better way of dealing with reflection; would still be nice however to have this one

2. or a way to exclude the System.Data.SqlClient from the reflection free mode, so having a custom library in a reflection free mode using System.Data.SqlClient with reflection. You earlier pointed me to the following solution, but I wonder whether it is still working (and if so how I can test that the custom library is reflection free):

namespace System.Runtime.CompilerServices
{
    [AttributeUsage(AttributeTargets.All)]
    internal class __BlockAllReflectionAttribute : Attribute { }
}

The goal here is purily obfuscation, where I don't mind having open source libraries un-obfuscated, but like to have some of my libraries obfuscated (reflection free).

First step I would like to get working is:

            using var sqlConnection = new SqlConnection("...");

            sqlConnection.Open();
            Console.WriteLine("state:  " + sqlConnection.State);
            sqlConnection.Close();

[ghost]

Tagging subscribers to this area: @dotnet/area-system-reflection
See info in area-owners.md if you want to be subscribed.

Issue Details

---

@MichalStrehovsky : now that it is possible with .NET 8 to run both the kestrel server and npgsql in reflection free mode (see https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/CSharp/appmpower) I am trying to close my last open item, running the sqlclient in:

1. either reflection free mode: the first blocking factor here seems to be the missing option for flags something like

    <RuntimeHostConfigurationOption Include="Switch.System.Reflection.Disabled.DoNotThrowForAttributeFlags" Value="true" />

Unhandled Exception: MT4340638104: TypeInitialization_Type_NoTypeAvailable
 ---> MT4340635080: Reflection_Disabled
   at Internal.Reflection.RuntimeTypeInfo.GetAttributeFlagsImpl() + 0x34

This has been reported in other issues and I have seen that was omitted for a better way of dealing with reflection; would still be nice however to have this one

2. or a way to exclude the System.Data.SqlClient from the reflection free mode, so having a custom library in a reflection free mode using System.Data.SqlClient with reflection. You earlier pointed me to the following solution, but I wonder whether it is still working (and if so how I can test that the custom library is reflection free):

namespace System.Runtime.CompilerServices
{
[AttributeUsage(AttributeTargets.All)]
internal class __BlockAllReflectionAttribute : Attribute { }
}

The goal here is purily obfuscation, where I don't mind having open source libraries un-obfuscated, but like to have some of my libraries obfuscated (reflection free).

Author:	LLT21
Assignees:	-
Labels:	area-System.Reflection, untriaged
Milestone:	-

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

@MichalStrehovsky : now that it is possible with .NET 8 to run both the kestrel server and npgsql in reflection free mode (see https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/CSharp/appmpower) I am trying to close my last open item, running the sqlclient in:

1. either reflection free mode: the first blocking factor here seems to be the missing option for flags something like

    <RuntimeHostConfigurationOption Include="Switch.System.Reflection.Disabled.DoNotThrowForAttributeFlags" Value="true" />

Unhandled Exception: MT4340638104: TypeInitialization_Type_NoTypeAvailable
 ---> MT4340635080: Reflection_Disabled
   at Internal.Reflection.RuntimeTypeInfo.GetAttributeFlagsImpl() + 0x34

This has been reported in other issues and I have seen that was omitted for a better way of dealing with reflection; would still be nice however to have this one

2. or a way to exclude the System.Data.SqlClient from the reflection free mode, so having a custom library in a reflection free mode using System.Data.SqlClient with reflection. You earlier pointed me to the following solution, but I wonder whether it is still working (and if so how I can test that the custom library is reflection free):

namespace System.Runtime.CompilerServices
{
    [AttributeUsage(AttributeTargets.All)]
    internal class __BlockAllReflectionAttribute : Attribute { }
}

The goal here is purily obfuscation, where I don't mind having open source libraries un-obfuscated, but like to have some of my libraries obfuscated (reflection free).

Author:	LLT21
Assignees:	-
Labels:	untriaged, area-NativeAOT-coreclr
Milestone:	-

[MichalStrehovsky]

This has been reported in other issues and I have seen that was omitted for a better way of dealing with reflection; would still be nice however to have this one

What is the stack that leads to type attributes being read? We don't really have a good way to produce the answer for this. We at most know if something is a class or interface, but depending on what SqlClient wants to get from this, the rest will likely be a wrong answer.

You earlier pointed me to the following solution, but I wonder whether it is still working

Support for this attribute was deleted from the product.

The goal here is purily obfuscation

At this point, if you really don't use reflection, adding <StackTraceSupport>false</StackTraceSupport> will get you very close to the no reflection mode, except things will still work...

[LLT21]

The stack is the following:

Unhandled Exception: MT4335870616: TypeInitialization_Type_NoTypeAvailable
 ---> MT4335867592: Reflection_Disabled
   at Internal.Reflection.RuntimeTypeInfo.GetAttributeFlagsImpl() + 0x34
   at Internal.Reflection.Extensions.NonPortable.CustomAttributeSearcher`1.GetMatchingCustomAttributes(E, Type, Boolean, Boolean) + 0xa0
   at System.Reflection.CustomAttributeExtensions.GetCustomAttribute[T](Assembly) + 0x50
   at System.Resources.ManifestBasedResourceGroveler.GetNeutralResourcesLanguage(Assembly, UltimateResourceFallbackLocation&) + 0x20
   at System.Resources.ResourceManager.CommonAssemblyInit() + 0xa0
   at System.Resources.ResourceManager..ctor(Type) + 0xb4
   at System.SR.get_ResourceManager() + 0x48
   at System.SR.GetResourceString(String, String) + 0x38
   at System.Data.Common.ADP..cctor() + 0x11c
   at System.Runtime.CompilerServices.ClassConstructorRunner.EnsureClassConstructorRun(StaticClassConstructionContext*) + 0xbc
   Exception_EndOfInnerExceptionStack
   at System.Runtime.CompilerServices.ClassConstructorRunner.EnsureClassConstructorRun(StaticClassConstructionContext*) + 0xb0
   at System.Runtime.CompilerServices.ClassConstructorRunner.CheckStaticClassConstructionReturnGCStaticBase(StaticClassConstructionContext*, Object) + 0x14
   at System.Data.Common.ADP.IsCatchableExceptionType(Exception) + 0xa0
   at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection, DbConnectionOptions, DbConnectionInternal) + 0x204
   at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection, DbConnectionOptions, DbConnectionInternal) + 0x74
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection, UInt32, Boolean, Boolean, DbConnectionOptions, DbConnectionInternal&) + 0x304
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection, TaskCompletionSource`1, DbConnectionOptions, DbConnectionInternal&) + 0x8c
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection, TaskCompletionSource`1, DbConnectionOptions, DbConnectionInternal, DbConnectionInternal&) + 0x160
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection, DbConnectionFactory, TaskCompletionSource`1, DbConnectionOptions) + 0x120
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1) + 0x114
   at System.Data.SqlClient.SqlConnection.Open() + 0xe4
   at test3.Program.Main(String[]) + 0x50

When I return nothing I can go a step further, but then similar errors come back; so not sure indeed whether it will be sufficient.

Maybe a more important question: all internal/private classes, methods, properties -> are these included in the reflection data or only the public ones ? Any hint on how this might improve obfuscation ?

[teo-tsirpanis]

Without being 100% sure, I think that if your library's APIs are not accessed via reflection, Native AOT will trim reflection metadata for them, even if reflection-free mode is not enabled. My understanding is that the reflection-free mode is not as valuable as in the past, and there were even talks to remove it.

[MichalStrehovsky]

When I return nothing I can go a step further, but then similar errors come back; so not sure indeed whether it will be sufficient.

It is unlikely ResourceManager could be made to work by shimming reflection APIs. In the end it needs to look at resources.

Maybe a more important question: all internal/private classes, methods, properties -> are these included in the reflection data or only the public ones ? Any hint on how this might improve obfuscation ?

The compiler by default keeps names of types. Names of methods are included if stack trace data is generated (the suggested <StackTraceSupport>false</StackTraceSupport> disables that). Other than that, the compiler only generates reflection metadata for things that were visible targets of reflection. Only a very small portion of the program will be generated like that.

[Wraith2]

You should strongly consider upgrading from the maintainance only System.Data.SqlClient to the modern replacement which is Microsoft.Data.SqlClient where work is being done to make things aot friendlier e.g. dotnet/SqlClient#2226

[LLT21]

The compiler by default keeps names of types. Names of methods are included if stack trace data is generated (the suggested <StackTraceSupport>false</StackTraceSupport> disables that). Other than that, the compiler only generates reflection metadata for things that were visible targets of reflection. Only a very small portion of the program will be generated like that.

@MichalStrehovsky Would it be possible to list a number of namespaces in the csproj file that do not generate reflection data at all (even no names of types) ? This would allow to remove full disabling of reflection for obfuscation reasons. As I understand in such scenario only the reflection stack would be included and reflection info for referenced namespaces like System... and Microsoft... which is fine. This is a potential enhancement request for a future version.

[MichalStrehovsky]

The compiler by default keeps names of types. Names of methods are included if stack trace data is generated (the suggested <StackTraceSupport>false</StackTraceSupport> disables that). Other than that, the compiler only generates reflection metadata for things that were visible targets of reflection. Only a very small portion of the program will be generated like that.

@MichalStrehovsky Would it be possible to list a number of namespaces in the csproj file that do not generate reflection data at all (even no names of types) ? This would allow to remove full disabling of reflection for obfuscation reasons. As I understand in such scenario only the reflection stack would be included and reflection info for referenced namespaces like System... and Microsoft... which is fine. This is a potential enhancement request for a future version.

That would have the same problem as #88748 (comment) - there is no way to audit whether it's safe to do this. By definition we don't have information about whether reflection is used (or isn't used) with a certain type. If we had this information, we could optimize it out transparently. This would inject random failure modes, same way as IlcDisableReflection does. It's the main reason why we'll never productize it.

[LLT21]

You should strongly consider upgrading from the maintainance only System.Data.SqlClient to the modern replacement which is Microsoft.Data.SqlClient where work is being done to make things aot friendlier e.g. dotnet/SqlClient#2226

@Wraith2 : based on your feedback, I made a fork of SqlClient and was able with the changes in:
dotnet/SqlClient@5.1-servicing...LLT21:SqlClient:5.1-servicing
to make the full disable reflection work for the following basic, but for me probably sufficient, statements:

            using var sqlConnection = new SqlConnection("server=xxx.database.windows.net;user id=xxx;password=xxx;Initial Catalog=xxx;Pooling=true;Connection Timeout=120;");

            sqlConnection.Open();

            using var sqlCommand = new SqlCommand("SELECT * FROM Country; SELECT * FROM Language", sqlConnection);

            var dataReader = await sqlCommand.ExecuteReaderAsync();
            int i = 0; 

            do
            {
                while (await dataReader.ReadAsync())
                {
                    if (i == 0) Console.WriteLine(dataReader["Code"] + " " + dataReader["Name"]);
                    else Console.WriteLine(dataReader["Language"] + " " + dataReader["Name"]);
                }

                i++; 
            } while (await dataReader.NextResultAsync());

            dataReader.Close();            

            using var sqlCommand2 = new SqlCommand("UPDATE Country SET Name = 'Belgique' WHERE Code = 'BE'", sqlConnection);

            sqlCommand2.ExecuteNonQuery();

            sqlConnection.Close();

I have done this on the 5.1-servicing branch, because the SqlClient main branch has some debugging issues on my Mac M1 while connecting to an Azure database: it tries to connect via the ip address which is not allowed. Besides of this, I am impressed by the few changes necessary to support this flow without reflection.

[LLT21]

The compiler by default keeps names of types. Names of methods are included if stack trace data is generated (the suggested <StackTraceSupport>false</StackTraceSupport> disables that). Other than that, the compiler only generates reflection metadata for things that were visible targets of reflection. Only a very small portion of the program will be generated like that.

@MichalStrehovsky Would it be possible to list a number of namespaces in the csproj file that do not generate reflection data at all (even no names of types) ? This would allow to remove full disabling of reflection for obfuscation reasons. As I understand in such scenario only the reflection stack would be included and reflection info for referenced namespaces like System... and Microsoft... which is fine. This is a potential enhancement request for a future version.

That would have the same problem as #88748 (comment) - there is no way to audit whether it's safe to do this. By definition we don't have information about whether reflection is used (or isn't used) with a certain type. If we had this information, we could optimize it out transparently. This would inject random failure modes, same way as IlcDisableReflection does. It's the main reason why we'll never productize it.

@MichalStrehovsky I will describe the use case better in a separate issue (as this one is closed), which then might or might not be linked to issue #96511. Thanks for the info provided on this issue, it helps me to better explain the requirement in the next one.

[charlesroddie]

The compiler by default keeps names of types. Names of methods are included if stack trace data is generated (the suggested <StackTraceSupport>false</StackTraceSupport> disables that). Other than that, the compiler only generates reflection metadata for things that were visible targets of reflection. Only a very small portion of the program will be generated like that.

This is interesting. I believe the prior art for UWP and Mono AOT IOS apps is to have stack traces without symbols , have symbols uploaded and then have traces resymbolicated when displaying to developers in app stores or appcenter etc..

It sounds like this is not yet possible with NativeAOT but that the <StackTraceSupport>false</StackTraceSupport> feature a step in this direction. Is this right @MichalStrehovsky ?

[MichalStrehovsky]

Stack traces work pretty much the same how they worked in uwp. If stack trace support is disabled, stack traces only have addresses one can symbolicate with the symbols file. I would think stack traces on Mono ios just work all the time since Mono requires the underlying il metadata to be present.