Windows 10 Long Filename Support

[apparentlymart]

Windows 10 has a mechanism to support file paths longer than MAX_PATH, but for backward compatibility it is disabled by default and must be opted in explicitly in an application manifest.

<application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="https://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
</application>

There is some discussion about whether Go programs should automatically opt in to this (and, it seems, the possibility that they already do) in golang/go#17835.

However, we may be able to force it ourselves (since Go apparently uses all of the appropriate long-path-compatible APIs internally anyway) by attaching our own manifest at link time, perhaps using the rsrc tool. This would then avoid errors on Windows systems when users have deeply-nested module hierarchies or are maintaining their configurations in deep subdirectories.

When a long path error is hit, Terraform will usually return an error message including the standard string representation of the underlying Windows API error, which is "The filename or extension is too long."

This issue represents investigating this issue further and making a decision about what to do here. If you have run into filename-length related problems on Windows 10 with long path support enabled, please add a 👍 reaction to this comment (don't post a new comment) so we can gauge the impact that the current situation has in real-world use.

[teamterraform]

This issue hasn't been as prolific as we initially feared, and so it never rose to the top of the priority list. Windows has been getting gradually more friendly to applications expecting a Unix-like environment, and so possibly the situation has also changed such that this problem is harder to hit than it was at the time we first became aware of it.

Given that the problem is uncommon -- uncommon enough that the corresponding Go issue has also been silent for quite some time -- we're going to close this out for now as part of some general issue gardening.

If we find that this becomes a significant problem again (affecting many Windows users, and with no reasonable workaround) then we'll open a new issue to discuss a more specific plan of action.

[jakauppila]

@teamterraform This is still very much an issue. There likely hasn't been much noise because this issue was quite concise both on the the problem as well as the proposed solution, so there wasn't much more to add to the conversation.

I think it's fine if it's not high on the list of priorities, but can it be re-opened as it is still an open issue for all Windows users?

[teamterraform]

Understood, @jakauppila! Thanks for letting us know.

For the moment we're going to leave this closed because any work on this ought to be on golang/go#17835 unless we hear official word that the Go team considers it to be out of scope. While in principle we could implement something custom to insert a manifest resource into the binary after it is built, our current build process does not have any room for per-platform customizations and rather assumes that the Go compiler knows how to build working binaries for every platform.

Were this it rise to the top of the priority list we'd prefer to address it by making a contribution to Go rather than adding platform-specific complexity to our local build process. We can't prioritize this right now either way, so it's kinda a moot point.

In the meantime, it might work to put something like the XML example in the original comment here in a file named terraform.exe.manifest alongside the installed terraform.exe. We haven't tested this, but documentation from Microsoft seems to suggest that this should be equivalent to having the manifest embedded inside the executable, and if so that at least provides a way to do it independently of what the Go compiler produces, without post-processing the actual .exe file.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.