.NET Framework 4.6.2 Long Paths: Illegal Characters in path exception

[zhawkins-viasat]

Environment

• Pythonnet version: 2.5.2
• Python version: 3.8.10
• Operating System: Windows 10
• .NET Runtime: 4.6.2

Details

• .NET Framework 4.6.2 supports long paths by default. When calling an assembly that a targets .NET Framework 4.6.2, long path support is disabled and causes an Illegal characters in path exception when using a FileStream constructor:

System.ArgumentException: Illegal characters in path.
   at System.Security.Permissions.FileIOPermission.EmulateFileIOPermissionChecks(String fullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)

Workaround:
For now, the workaround is to create a python.exe.config application config file with the following:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <startup>
    <supportedRuntime version=".NETFramework,Version=v4.6.2"/>
  </startup>
  <runtime>
    <AppContextSwitchOverrides value="Switch.System.IO.UseLegacyPathHandling=false;Switch.System.IO.BlockLongPaths=false" />
  </runtime> 
</configuration>

This, however, creates a global solution which is not desired. An alternate workaround is to explicitly update the AppContext variables in the Python script before any code is executed (credit to https://stackoverflow.com/questions/53193280/long-path-workaround-not-working-on-some-installs):

type = Type.GetType("System.AppContext")
if type:
    AppContext.SetSwitch("Switch.System.IO.UseLegacyPathHandling", False)
    AppContext.SetSwitch("Switch.System.IO.BlockLongPaths", False)

    switchType = Type.GetType("System.AppContextSwitches")
    if switchType:
        # We also have to reach into System.AppContextSwitches and manually update the cached private versions of these properties (don't ask me why):

        legacyField = switchType.GetField("_useLegacyPathHandling", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic)
        if legacyField:
            legacyField.SetValue(None, -1) # <- caching uses 0 to indicate no value, -1 for false, 1 for true.

        blockingField = switchType.GetField("_blockLongPaths", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic)
        if blockingField:
            blockingField.SetValue(None, -1) # <- caching uses 0 to indicate no value, -1 for false, 1 for true.

Again, not an ideal solution because this uses reflection to set values of private variables by name.

Ideally, the CLR loader would load the proper AppContext defaults based on the version of target framework like is shown here:

https://referencesource.microsoft.com/#mscorlib/system/AppContext/AppContextDefaultValues.Defaults.cs,60

[filmor]

clr-loader supports specifying a config file for .NET Framework, that would probably be the way to go. Either initialise like this:

from pythonnet import set_runtime
set_runtime("netfx", config_file=<path-to-config>)

import clr

or via the environment, setting PYTHONNET_NETFX_CONFIG_FILE=....

[zhawkins-viasat]

clr-loader supports specifying a config file for .NET Framework, that would probably be the way to go. Either initialise like this:

from pythonnet import set_runtime
set_runtime("netfx", config_file=<path-to-config>)

import clr

or via the environment, setting PYTHONNET_NETFX_CONFIG_FILE=....

A config file should NOT be necessary in order to utilize default features of the framework.

[filmor]

You don't need to shout. Have a look at https://github.com/pythonnet/clr-loader/tree/master/netfx_loader. We do not set anything there, so I am assuming we are using all the default values. It's not like we are explicitly overriding any configuration to make it worse.

[zhawkins-viasat]

You don't need to shout. Have a look at https://github.com/pythonnet/clr-loader/tree/master/netfx_loader. We do not set anything there, so I am assuming we are using all the default values. It's not like we are explicitly overriding any configuration to make it worse.

Do you not believe that users have a reasonable expectation that if they compile a DLL using .NET Framework 4.6.2, that the default features of 4.6.2 would be available when using that DLL with PythonNet?

You might not be setting anything in your code and using default values, but that doesn't mean you're doing it "right" either.

[lostmsu]

.NET configuration is per-executable, not per-DLL. You have a script that is a Python entry point, which loads the DLL. If the script were .NET executable, you'd create a .config for it. But because it is not, you have to use some other way, because as you mentioned earlier the config on Python executable would affect all scripts. So logically setting the config in the script is the right way to do it, and since you can do it explicitly, that seems fine by me.

Not sure if hosting process can affect AppDomain.CurrentDomain.SetupInformation in the way that you want it to. Contributions are welcome.

[zhawkins-viasat]

If the script were .NET executable, you'd create a .config for it.

When I create a .NET executable leveraging my library (DLL), I do not get exceptions so no config file is necessary. Exceptions were only found when using PythonNet because of what I have outlined. The scenario is as such:

• Create a .NET executable using the library - no exceptions, default features of the target framework are honored
• Use PythonNet to call the library - immediate exceptions and code cannot run because of errors never seen before. Errors resulted because the target framework's default settings were not honored

Actually figuring out what was wrong required stepping into the .NET Framework source code to find that there was a context switch that was not setup with the expected default value for the framework in which the dll was targeted to. The Python interpreter executable is written in C so it has no notion of the .NET framework. Since PythonNet "adds" the ability for the C-based interpreter executable to integrate with the .NET CLR, I would say that it is PythonNet's job to properly initialize the CLR based on the specifications of the assembly being loaded.

[lostmsu]

When I create a .NET executable

That's because your executable has target .NET version set, so it gets implicit config. To do that you set TargetFramework in .csproj. If you tried to load the DLL from an executable with older TargetFramework, it would fail similarly. The TargetFramework of your DLL does not matter at all.

Cause there's no executable built from a .csproj when you launch .NET, only the Python script, logically, the Python script is the only place your setting could go.

[zhawkins-viasat]

That's because your executable has target .NET version set, so it gets implicit config. To do that you set TargetFramework in .csproj. If you tried to load the DLL from an executable with older TargetFramework, it would fail similarly. The TargetFramework of your DLL does not matter at all.

Cause there's no executable built from a .csproj when you launch .NET, only the Python script, logically, the Python script is the only place your setting could go.

Thank you for proving my point. I keep saying it and I don't know how to make it any clearer to you - PythonNet should honor the target framework of the assembly it is loading. In your words - it should utilize an implicit config based on the target version. That is the logical thing to do.

[lostmsu]

Lol, your point has no proof here. I just told you that the script should set the target framework, because the script is the entry point, and that's how it works in .NET. Additionally, see Python's motto.

[vadimkatsman]

Target framework is not the only reason we need application config. If I call the domain library that is pretty much only thin client away from the complete application, my domain objects will very much read configuration entries. If - inside the downstream of the assembly we load - we happen to use EF, WCF, WWF, logging etc. you cannot get away from not having the config file. And those config files cannot be global per machine - tons of projects will require tons of different configuration setups.

[lostmsu]

The way to specify config was listed in the second comment before the issue was closed.

[vadimkatsman]

I posted my comment to contribute to the debate to emphasize wider importance of this feature to the benefit of other readers- who like me just a day ago - would be asking the question of config in general. Plus, the referred functionality either never worked or was broken in later releases - I posted 2 related issues / bugs (one i could temporarily fix locally but another one is a true showstopper). Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Victor ***@***.***> Sent: Wednesday, March 1, 2023 12:00 AM To: pythonnet/pythonnet ***@***.***> Cc: vadimkatsman ***@***.***>; Comment ***@***.***> Subject: Re: [pythonnet/pythonnet] .NET Framework 4.6.2 Long Paths: Illegal Characters in path exception (Issue #1754) The way to specify config was listed in the second comment before the issue was closed. — Reply to this email directly, view it on GitHub<#1754 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANSW6JBEVDT7DAKHMBWMUJLWZ3RDHANCNFSM5SXEK6GA>. You are receiving this because you commented.Message ID: ***@***.***>

[zhawkins-viasat]

The config file is important but is not what was being asked here. Unfortunately, the answers from the contributors devolved and started to be insulting so the discussion could not be continued in a respectful and logical manner. Configs should absolutely be honored and using them is still relevant. However, in absence of a config file, the defaults for the framework version under which the dll was compiled should be honored. It shouldn't require a specific config file in order to enable settings that were already implicit in the version of the framework that was being used.