Performance problem: (implicitly) dot-sourced code is slowed down by variable lookups

[mklement0]

Summary of the performance problem

^{Note: This SO question inspired this suggestion.}

Updated based on feedback from @lzybkr below.

# REGULAR, direct invocation of an expression (a `foreach` statement 
# that loops 1 million (1e6) times, in this case) - implicitly DOT-SOURCED
(Measure-Command { $result = foreach ($n in 1..1e6) { $n } }).TotalSeconds

# Invocation in CHILD SCOPE, using & { ... }
# 10+ TIMES FASTER.
(Measure-Command { $result = & { foreach ($n in 1..1e6) { $n } } }).TotalSeconds

Sample timings (note the ratio) on macOS:

2.5900995  # DIRECT invocation (implicitly DOT-SOURCED)
0.2580877  # in CHILD SCOPE, via & { ... }

That is, direct invocation was about 10 times slower than invocation via &.

Possible implementation

See @lzybkr's comments below, which also explains the root cause.

[lzybkr]

That analysis is incorrect. The difference is dot sourcing.

Try this for comparison:

$cmdtest = {
    $stream = [System.IO.StreamWriter] $args[0]
    $i = 0
    while ($i -le 1000000) {
        $stream.WriteLine("This is the line number: $i")
        $i++
    }
    $stream.Close()
}

Measure-Command {
    & $cmdtest "$PSScriptRoot\t_i.txt"
} # takes ~1 sec

Measure-Command {
    . $cmdtest "$PSScriptRoot\t_d.txt"
} # takes ~4 sec

Measure-Command {
    Invoke-Command -ScriptBlock $cmdtest -Arg "$PSScriptRoot\t_icm.txt"
} # Takes ~1 second

The basic problem is doing a dictionary lookup for every variable reference when dot sourcing.

A fix is possible but a little tricky. You can cache the PSVariable that the lookup finds, but:

1. The cache entry should be a weak reference
2. If the weak reference is still alive, you still need to check if the variable was removed (e.g. with Remove-Variable).

The cache should be indexed by a small integer in the same way that regular (not dot sourced) variables are.

[mklement0]

Thanks, @lzybkr - I've updated the issue title and OP accordingly.