Start-RSJob error when Collection in argument list is modified before all jobs have started.

[kwslavens]

I ran into a issue in Start-RSJobs module.

Collection was modified; enumeration operation may not execute.
At E:\Scripts\MultiThread-RSJobs\PoshRSJob\Public\Start-RSJob.ps1:381 char:25
+ ,$ArgumentList | ForEach {
**+ ~~~~~~~~~~~~~~~~~~~~~~~~~~**
+ CategoryInfo : OperationStopped: (:) [], InvalidOperationException
+ FullyQualifiedErrorId : System.InvalidOperationException

When the number of files in $targetfiles is fairly small 350ish the issue doesn't happen. It seems to occur when a large number of jobs are started and some of the first jobs are running and modifying the $SyncArrayLogOutput before all the other jobs have started. Here is the code I'm working on below.

 $ErrorActionPreference = "Stop"
 $path = $PSScriptRoot
 # Import the RSJobs module.
 Import-Module $path\PoshRSJob\PoshRSJob.psm1
 #import AlphaFS for long path support.
 Import-Module -Name "E:\Scripts\Modules\AlphaFS.dll"
 $LogFile = $Path+"\Log.csv"
 $LogFileErrors = $Path+"\ErrorLog.csv"


 #region Functions
 #Function to provide a more robust Synchronized variable locking and unlocking. 
 function Get-AlphaFSChildItem {
    <#
    .SYNOPSIS
        Gets the files and folders in a file system drive and can handle Folder and File paths that are greater than 260 characters.
    .DESCRIPTION
        This command is similar to Get-ChildItem except it takes advantage of the AlphaFS(.NET) module in order to get around the 260 character path limit.
        ***************************************************************
        The AlphaFS.dll must be downloaded: http://alphafs.alphaleonis.com/index.html 
        Then imported: PS C:\> Import-Module -Name "%FilePath%/AlphaFS.dll"

        Information can be found: https://github.com/alphaleonis/AlphaFS/wiki/PowerShell
        ***************************************************************
    .PARAMETER Path
    .EXAMPLE
        Get-AlphaFSChildItem

        Description

        -----------

        This command gets the files and subdirectories in the current directory. If the current directory does not have
        child items, the command does not return any results.
    .EXAMPLE
        Get-AlphaFSChildItem -path C:\ps-test

        Description

        -----------

        This command gets the child items in the C:\ps-test directory
    .EXAMPLE
        Get-AlphaFSChildItem -path C:\ps-test -recurse

        Description

        -----------

        This command gets the system files and folders in the specified directory and its subdirectories.
    .EXAMPLE
        Get-AlphaFSChildItem -path C:\ps-test -SearchPattern "*.txt"

        Description

        -----------

        This command gets the system files that end in  "*.txt"


    .LINK
        http://alphafs.alphaleonis.com/index.html
        http://alphafs.alphaleonis.com/doc/2.0/index.html
        https://github.com/alphaleonis/AlphaFS/wiki/PowerShell

    #>
    param(
        [Parameter(
            ValueFromPipeline=$True,
            ValueFromPipelineByPropertyName=$True
        )
        ][string]$Path = (Get-Location).Path,
        [string]$SearchPattern = '*',
        [array]$Include,
        [array]$Exclude,
        [Switch]$Recurse
    )
    $Error.Clear()

    if(!(Get-Module -name AlphaFS -ErrorAction SilentlyContinue)){  
        Import-Module -name AlphaFS
    }
    if($Error){
        return
    }

    $items = @()
    $FileSystemEntryInfo = @()
    if($recurse){
        $SearchOption = 'AllDirectories'
    }
    else{
        $SearchOption = 'TopDirectoryOnly'
    }
#   Get all folders
    $array = @()
    Try {
        $array = [Alphaleonis.Win32.Filesystem.Directory]::EnumerateDirectories($Path,$SearchPattern,[System.IO.SearchOption]::$SearchOption)
        Foreach ($file in $array) { $items += $file } 
    }
    Catch [System.UnauthorizedAccessException] {
        # Report exception.
        Write-Error $_.Exception.Message
    }
#   Get all files
    Try {
        $array = [Alphaleonis.Win32.Filesystem.Directory]::EnumerateFiles($Path,$SearchPattern,[System.IO.SearchOption]::$SearchOption)
        Foreach ($file in $array) { $items += $file } 
    }
    Catch [System.UnauthorizedAccessException] {
        #   Report exception.
        Write-Error $_.Exception.Message
    }
    foreach($item in $items){
        $FileSystemEntryInfo += [Alphaleonis.Win32.Filesystem.File]::GetFileSystemEntryInfo($item)
    }
    if($Include){
        $IncludedItems = @()
        foreach($inc in $Include){
            $IncludedItems += $FileSystemEntryInfo | Where-Object {$_.FullPath -like "$inc"}
        }
        $FileSystemEntryInfo = $IncludedItems
    }   
    if($Exclude){
        foreach($Exc in $Exclude){
            $FileSystemEntryInfo = $FileSystemEntryInfo | Where-Object {$_.FullPath -notlike "$Exc"}
        }
    }
    return $FileSystemEntryInfo | Select @{N='FullName'; E={$_.FullPath}},@{N='LongFullName'; E={$_.LongFullPath}},* 
}
Function Write-Log {
    [cmdletbinding()]
    Param ([Parameter(ValueFromPipeline)]
          [string[]]$MessageList
    )
    process
    {
        foreach ($Message in $MessageList) {
            add-content $LogFile $Message
        }
    }
}
Function Write-ErrorLog {
    [cmdletbinding()]
    Param ([Parameter(ValueFromPipeline)]
          [string[]]$MessageList
    )
    process
    {
        foreach ($Message in $MessageList) {
            add-content $LogFileErrors $Message
        }
    }
}
function Get-AlphaFSHexString {
    param(
        [Parameter(Mandatory=$True,Position=1)]
        [string]$Path = '',
        [Parameter(Mandatory=$True,Position=2)]
        [int]$Bytes = ''
    )
    $Signatures = @(
        '0DE0AD0BE0EF040E70490880B202C01D027020010680D40C10B304D0C803A09F0DE0580C0C90310B0880C408D0AC0240F00C509401F03308C010',
        '0DE0AD0BE0EF040920CB03C0320B80DA0BD0C90FF0CC01F050EC0E0580E10D60C80D307704F0C20E90C20C803E0570940D908B053043065081034',
        '0DE0AD0BE0EF040DD0600450450360B106C0610CC0370790350B606707803607B02D0720670E60640330F80D70C90980710C7010CD0760FA0EE090',
        '0DE0AD0BE0EF0409F08309F0FA02D0AE0CD06A06D0840F20920270440E707D020820FF05A082040130CF0920C06301701C061027089080E908F',
        '0DE0AD0BE0EF040E707905808D0F009C0630E50650A30E00F00F10310707E08A0103103A0C50B40FD07003B08408202305B0F30DD0A703B0C509E',
        '0DE0AD0BE0EF040FB0710660B20620E60C6062053060B20800970120AE0E50902105604F0B40300D30A80890490A00130DB04F0650C80410960EA',
        '0DE0AD0BE0EF040FF0E00970F103E0B509105001C06409001B03309F0BD0D60D20540180F7083010560B0020AE010EB0FE05101E03501409F0B3',
        '0DE0AD0BE0EF040D50270B80002B07609D04F0310550960820C308D0CF0170C50290DA0C0BD0410B909B0EC08A0810FC0DE07F01308D0D80A80D8',
        '0DE0AD0BE0EF040DD0960C90CD0BA0BC0B102E03004D000A30F10700270D002F0770F10AD0FB0CF031084090A60A10DE0490EC0D10560AD0670EA',
        '0DE0AD0BE0EF040B50330990490120B80BB0B707D0620D0C80C30580E208A08002109F0A901A0580D0500BE05807D0BA0DF02208905D000980FA',
        '0DE0AD0BE0EF04080980F60FF0FC0160710906A01201508E0660640F609F0620350470301802908A05A02F0D50750A50C80C60F90B60E30E08C',
        '0DE0AD0BE0EF0409B09C09A04604501C04B0440F902E09E0C20D00EB049080180A405903809507701A05C09B0A00F50FD0EF0FD0A604E016048068'
    )
    if ($HeaderHexString) { Remove-Variable HeaderHexString }
    Try {
      $File= [Alphaleonis.Win32.Filesystem.File]::OpenRead($Path)
      for ($i = 0; $i -lt $Bytes; $i++)
        { 
            if ($byte) { Remove-Variable byte }
            $byte = $($File.ReadByte())
            # if ( ( '{0:X}' -f $byte ).Length -eq 1 ) {
                    $HeaderHexString += '0{0:X}' -f $byte
            #    }
             #   else {
             #       $HeaderHexString += '{0:X}' -f $byte
              #  }
        }
        $File.Close()
    }
    Catch {
        # Report exception.
    }
    $HeaderHexString 
}

Get-RSJob | Remove-RSJob -force
#endregion

#Remove any stale jobs
Get-RSJob | Remove-RSJob

$SyncArrayLogOutput = [System.Collections.ArrayList]::Synchronized(@{}) ; $SyncArrayLogOutput.Clear()

$ScriptBlock = {
    Param($SyncArrayLog)
    $TD = get-Date -f "yyyy:MM:dd:HH:mm:ss.fff"
    $ExecuteTime = $(Measure-Command {
        if ($Header) { Remove-Variable Header }
        $header = Get-AlphaFSHexString -path $_.LongFullName -Bytes 40

    }).Milliseconds
    [System.Threading.Monitor]::Enter($SyncArrayLog.SyncRoot)
        $SyncArrayLog.add("$td,$($_.LongFullName),$ExecuteTime")
    [System.Threading.Monitor]::Exit($SyncArrayLog.SyncRoot)
    return $header
}

#$TargetFiles = get-AlphaFSchilditem -Path $TargetPath -Recurse
$TargetFiles = get-AlphaFSchilditem -Path E:\EncryptedArchive\twhite -Recurse


$TargetFiles | Start-RSjob -ScriptBlock $ScriptBlock -ArgumentList $SyncArrayLogOutput -FunctionsToLoad Get-AlphaFSHexString -ModulesToImport "E:\Scripts\Modules\AlphaFS.dll"

do {
    sleep 1

    $failed =Get-RSJob -State Failed
    $failed | Write-ErrorLog
    $failed | Remove-RSJob -force
    Get-RSJob -State Completed | Remove-RSJob
    write-host "Checking"
    $temp = New-Object System.Collections.ArrayList($null)
    [System.Threading.Monitor]::Enter($SyncArrayLogOutput.SyncRoot)
        $temp.AddRange($SyncArrayLogOutput)
        $SyncArrayLogOutput.Clear()
    [System.Threading.Monitor]::Exit($SyncArrayLogOutput.SyncRoot)
    write-host "-----------------------------------------------------------------"  
    $temp | Write-Log
    $temp
}while ($($(Get-RSJob).length) -gt 0)

[proxb]

Thanks for providing the code to duplicate this issue. I'll take a look at it and see what I can find.

[proxb]

Just out of curiosity and I know this is many many months late, but I am curious if anything has changed with the recent updates that I have pushed out.

[Fxbouffant]

I do have this issue too with version 1.5.7.6.
It seems caused by [System.Collections.ArrayList]::Synchronized as it doesn't occur with a [hashtable]::Synchronized

[proxb]

Weird, did you use the same code as OP or did you use something different? If different, can you post it here?

[proxb]

@Fxbouffant @kwslavens Can you try again with the latest build and let me know if you still see issues?

[proxb]

@codykonior Just to make sure, the same error as this:

Collection was modified; enumeration operation may not execute.
At E:\Scripts\MultiThread-RSJobs\PoshRSJob\Public\Start-RSJob.ps1:381 char:25
+ ,$ArgumentList | ForEach {
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], InvalidOperationException
+ FullyQualifiedErrorId : System.InvalidOperationException

[MVKozlov]

lines from 241:

        If ($PSBoundParameters.ContainsKey('ArgumentList')) {
            If (@($ArgumentList).count -match '0|1') {
                $SingleArgument = $True
            } 
            Else {
                $SingleArgument = $False
            }
        }

-match is a bug. it also matches 10 20 and so on
there must be -le 1 !

may be this bug can affect on above error

btw, may be stuff like $ArgumentList -is [array] can be used instead ?