Start-Process fails with EnvironmentBlockTooLong error

[hbuckle]

It seems that Start-Process has a hardcoded limit for the environment block, which I am hitting when running in Azure Pipelines (which sets lots of environment variables)

According to the documentation this limit only existed for Windows XP, is there any reason for it to remain?

Steps to reproduce

Start-Process -FilePath "myprocess.exe"

Expected behavior

Process is executed

Actual behavior

2020-11-17T14:21:34.1311704Z Start-Process : EnvironmentBlockTooLong
2020-11-17T14:21:34.1313656Z At C:\Program Files\PowerShell\Modules\PSDeployTools\1.1.55\Functions\Invoke-Process.ps1:44 char:16
2020-11-17T14:21:34.1315142Z +     $process = Start-Process @processParams
2020-11-17T14:21:34.1316243Z +                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-11-17T14:21:34.1317681Z + CategoryInfo          : NotSpecified: (:) [Start-Process], InvalidOperationException
2020-11-17T14:21:34.1319184Z + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand

[iSazonov]

@hbuckle Do you want to pull PR?

[hbuckle]

Absolutely, if it's fine to remove the limit I'll create a PR

[iSazonov]

I believe we can remove the limit because Windows XP is out of support.

Since the issue is related to Azure Pipelines I'd ask MSFT team to include the fix in servicing release 7.1.1.

[jborean93]

IIRC there's a limit of 32767 (0xFFFF as a signed Int16) on environment blocks defined by the user https://docs.microsoft.com/en-us/windows/win32/procthread/environment-variables on Windows. Removing this limit could potentially cause undefined errors.

Environment Variables - Win32 apps

Every process has an environment block that contains a set of environment variables and their values. There are two types of environment variables: user environment variables (set for each user) and system environment variables (set for everyone).

[hbuckle]

@jborean93 - from that documentation

The maximum size of a user-defined environment variable is 32,767 characters. There is no technical limitation on the size of the environment block.

So the limit seems to be on the size of individual variables, not on the size of the block?

[jborean93]

That's true, It does also say

However, there are practical limits depending on the mechanism used to access the block. For example, a batch file cannot set a variable that is longer than the maximum command line length.

But it's definitely more on the side of it should work and it's up to the user of the environment block to handle it properly and not something PowerShell should limit.

[MattJeanes]

This issue appeared in PowerShell 7.1.0 as part of #10830

And appeared in Azure Pipelines for me today as the agent got updated to use PowerShell 7.1.0

Until the linked PR #14111 is in a workaround for some scenarios where you don't need to inherit environment variables from the process (or deployment pipeline in the case of Azure Pipelines) is to use -NoNewEnvironment which will as of 7.1.0 initialise a standard set of environment variables for the current user.

This fixed my issue trying to run an app inside of Azure Pipelines for now.

[hbuckle]

@MattJeanes - I don't think it was introduced in 7.1, I'm assuming it has been there since the days of Windows XP. I wonder if something changed in Azure Pipelines that meant they started creating more environment variables

[MattJeanes]

The behaviour of PowerShell was changed to call ConvertEnvVarsToByteArray which contains the EnvironmentBlockTooLong check in 7.1 when not using -UseNewEnvironment as well, as the PR prepared PowerShell for the ability to pass in custom environment variable blocks.

You can see that change here: https://github.com/PowerShell/PowerShell/pull/10830/files#diff-a87e81856755f9e2a0f6002cf90c4cc3d0a4d1e81394bee7ac61543f71138b71R2464

GitHub

Update UseNewEnvironment parameter behavior of Start-Process cmdlet on Windows by iSazonov · Pull Request #10830 · PowerShell/PowerShell

PR Summary The PR effects only Windows behavior.

With UseNewEnvironment parameter Start-Process run a child process with environment variables without modifications in current session (state befor...

[iSazonov]

Yes, this is a side effect from old code. The implementation was changed as first step to open a way for supporting new features (like adding new env vars on the fly by new parameter (I already forget approved name :-) ) and language feature like bash style LC=ALL grep...) but we stopped due to lack of resources :-( If you have an interest you can grab this work.

[MattJeanes]

I have run into a situation where I needed to pass in environment variables to a process, so to work around the issue I essentially temporarily move a bunch of environment variables to a variable, run the processes and then move them back.

Here are the scripts to do it, they remove all RELEASE_* environment variables but you can tweak for your own needs:

function Clear-EnvironmentVariablesTemporary {
    # TEMPORARY: https://github.com/PowerShell/PowerShell/issues/14110

    $global:ClearedEnvironmentVariables = (Get-Item "env:").ForEach({
        if ($_.Key.StartsWith("RELEASE_")) {
            return $_
        }
    })
    $global:ClearedEnvironmentVariables | ForEach-Object { Remove-Item (Join-Path "env:" $_.Key) }
    Write-Host "Cleared $($global:ClearedEnvironmentVariables.Count) environment variables"
}

function Restore-EnvironmentVariablesTemporary {
    # TEMPORARY: https://github.com/PowerShell/PowerShell/issues/14110

    if ($global:ClearedEnvironmentVariables) {
        $global:ClearedEnvironmentVariables | ForEach-Object { Set-Item (Join-Path "env:" $_.Key) -Value $_.Value }
    }
    else {
        Write-Warning "No cleared environment variables saved, cannot restore"
    }

    Write-Host "Restored $($global:ClearedEnvironmentVariables.Count) environment variables"
    $global:ClearedEnvironmentVariables = $null
}

[ghost]

🎉This issue was addressed in #14111, which has now been successfully released as v7.1.3.:tada:

Handy links:

• Release Notes

[ghost]

🎉This issue was addressed in #14111, which has now been successfully released as v7.2.0-preview.4.:tada:

Handy links:

• Release Notes