ConvertFrom-RobocopLog.ps1 is a powerful PowerShell tool designed to analyze log files generated by Robocopy, providing a clear and structured overview of file copy operations.
It converts logs into information-rich PowerShell objects, facilitating automation, reporting, and analysis of data transfers.
This script supports various Robocopy log formats with different log options, including those with full or partial file paths.
- Detailed extraction of header information (source, destination, options).
- Complete analysis of copied, modified, error, or ignored files.
- Processing of overall statistics (number of files, bytes, errors) with the possibility of calculating the total processing time.
- Flexible filtering options through parameters to include or exclude file and directory classes for parsing.
- Raw data extraction: With the
-RawParsingoption, users can retrieve unprocessed log lines for further customization or external analysis. - Verbose output: Enhanced progress tracking and diagnostic output with
Write-Verbose, available by specifying the-Verboseflag.
This script is ideal for system administrators looking to better understand or automate the management of Robocopy operations in complex environments, while maintaining an accurate view of performance and results.
To use ConvertFrom-RobocopLog.ps1, you must ensure that your environment meets the following prerequisites:
- Operating system: Windows with PowerShell (version 5.1 or later).
- Robocopy: The log file must be generated by a Robocopy command with the log generation option enabled (e.g.,
/LOG,/UNILOG). - Permissions: You must have the necessary permissions to read the specified log file.
- Full file paths: For analysis, Robocopy should be run with the
/fp(Full Path) option to log full file paths. - Show Class: For analysis, Robocopy should be run without the
/nc(Class not logged) option.
For French logs, use the /unilog option instead of /log.
To use ConvertFrom-RobocopLog.ps1, specify the Robocopy log file to analyze using the required -RoboLog parameter. The script will parse the content and return a structured PowerShell object containing the header information, processed files, and summary statistics.
-RoboLog: Path to the Robocopy log file to analyze (required).-LogLanguage: Log language (optional, possible values: "en-US" or "fr-FR", default "en-US").-IncludeFileClass: File classes to include in the parsing (optional, default "All").-ExcludeFileClass: File classes to exclude from the parsing (optional).-IncludeDirClass: Directory classes to include in the parsing (optional, default "All").-ExcludeDirClass: Directory classes to exclude from the parsing (optional).-RawParsing: Enables output of unprocessed log lines, bypassing structured parsing (optional).
You can copy/paste the function ConvertFrom-RobocopLog { ...} and either declare it in your script or create your own .psm1 module to integrate it without polluting your script.
The settings are completely identical.
Complete analysis of a log file in English (default):
$LOG = .\ConvertFrom-RobocopLog.ps1 -RoboLog "C:\Logs\RobocopyLog.txt"
Analysis of a log file in French with specific directory classes excluded:
$LOG = .\ConvertFrom-RobocopLog.ps1 -RoboLog "C:\Logs\RobocopyLog.txt" -LogLanguage fr-FR -ExcludeDirClass "NewDirs"
Analysis excluding all file and directory classes (only header and summary will be parsed):
$LOG = .\ConvertFrom-RobocopLog.ps1 -RoboLog "C:\Logs\RobocopyLog.txt" -ExcludeFileClass "All" -ExcludeDirClass "All"
Raw data extraction mode for unprocessed log lines:
$LOG = .\ConvertFrom-RobocopLog.ps1 -RoboLog "C:\Logs\RobocopyLog.txt" -RawParsing
HeaderInfo: Contains log header information.Start: Start date and time.Source: Transfer source.Destination: Destination of the transfer.Files: File criteria.Options: Options used with Robocopy
Exemple
$LOG.HeaderInfo.Start # 2024/09/25 08:14:10
$LOG.HeaderInfo.Source # C:\SourceFolder\
$LOG.HeaderInfo | Format-List
######
Start : 2024/09/25 08:14:10
Source : C:\SourceFolder\
Destination : C:\DestFolder\
FilesOptions : *.*
Options : *.* /V /X /TS /FP /S /E /COPYALL /PURGE /MIR /NP /R:1000000 /W:30
Files: Contains information about processed files according to their classNew,Same,Newer,Older,Modified,Lonely,TweakedFilePath: Full path of fileFileSize: With/byteor with default Robocopy unit (m, g, t, ..)TimeStamp
Failed: Contains the list of files with error.FilePath: Full path of fileErrorInfo: Code errorErroAction: Indicates what action was attempted (Copy, Delete, etc.)
Destination: Destination of the transfer.Files: File criteria.Options: Options used with Robocopy
Exemple
$LOG.Files.Extra
######
FilePath FileSize Timestamp
-------- -------- ---------
C:\SourceFolder\LargeFile_1.dat 1.0 g 25/09/2024 14:11:56
C:\SourceFolder\LargeFile_2.dat 1.0 g 25/09/2024 14:12:00
$LOG.Files.New | Out-GridView
$LOG.Files.Failed
######
FilePath ErrorInfo ErrorAction
-------- --------- -----------
C:\SourceFolder\LargeFile_10.dat ERROR 32 0x00000020 Copying File
C:\SourceFolder\LargeFile_11.dat ERROR 32 0x00000020 Copying File
C:\SourceFolder\LargeFile_12.dat ERROR 32 0x00000020 Copying File
Dirs: Contains information about processed Directory according to their classNew,Extra,LonelyDirpathItemCount: Count Inner Directory Object (-1 if Deleting)
Exemple
$log.Dirs.New[0].DirPath # "C:\SourceFolder\NewDirectory\"
SummaryInfo: Overall summary of operations.Dirs:Total,Copied,Skipped,Mismatch,Failed,Extras.
Files:Total,Copied,Skipped,Mismatch,Failed,Extras`.
Ended: End date and time.TotalTimeFromInnerLog: Total duration calculated from Header et Summary Data Date.TotalTimeFromFile: Total duration calculated from log file metadata
Exemple
$LOG.SummaryInfo
######
Dirs : @{Total=2; Copied=1; Skipped=1; Mismatch=0; Failed=0; Extras=1}
Files : @{Total=18; Copied=3; Skipped=15; Mismatch=0; Failed=0; Extras=2}
Ended : 2024/09/25 08:25:10
TotalTimeFromInnerLog : 00d:00h:00m:00s
TotalTimeFromFile : 00d:00h:53m:45s
$LOG.SummaryInfo.Files.Skipped # 15
| Language | Comments |
|---|---|
| English | By Default, can use with /log option from robocopy command. |
| French | By Default, can use with /unilog option from robocopy command. Needed to handle accents |
Just add Your Language CultureMap From the variable $CultureMap = @{...}.
Then remember to add your language option to the function and/or script parameter.
[Parameter(Mandatory = $false)]
[ValidateSet("en-US", "fr-FR")]
[String]$LogLanguage = "en-US",
This section covers the options that Robocopy must use to generate its log in order to be analyzed by RobocopLog For more information about Robocopy options click on the link: Microsoft Learn : Robocopy
JOB Options : Not tested.
| Options | Supported | Microsoft Description |
|---|---|---|
| /xf | no | Excludes files that match the specified names or paths. Wildcard characters (* and ?) are supporte |
| /nc | no | Excludes directories that match the specified names and paths. |
| Log Options | Supported | Comments | Microsoft Description |
|---|---|---|---|
| /fp | yes | Mandatory : FullPath need to be generated for bein captured by parsing | Includes the full path names of the files in the output. |
| /nc | no | Forbidden : Class need to be generated during log because they are used for parsing | Specifies that file classes aren't to be logged. |
| Log Options | Supported | Recommended* | Microsoft Description |
|---|---|---|---|
| /log+ | no | - | Writes the status output to the log file (appends the output to the existing log file). |
| /unilog+ | no | - | Writes the status output to the log file (appends the output to the existing log file). |
| /unicode | no | - | Displays the status output as unicode text. |
| /v | yes | yes | Produces verbose output, and shows all skipped files. |
| /ts | yes | yes | Includes source file time stamps in the output. |
| /byte | yes | yes | Prints sizes as bytes. |
| /ns | yes | no | Specifies that file sizes aren't to be logged. |
| /np | yes | yes | Specifies that the progress of the copying operation (the number of files or directories copied so far) won't be displayed. |
| /ndl | yes | no | Specifies that directory names aren't to be logged. |
| /nfl | yes | no | Specifies that file names aren't to be logged. |
| /njh | yes | no | Specifies that there's no job header. |
| /njs | yes | no | Specifies that there's no job summary. |
| /x | yes | yes | Reports all extra files, not just the ones that are selected. |
* Note : Recommended for Optimal Analysis
File Exists In Exists In Source/Dest Source/Dest Source/Dest
Class Source Destination File Times File Sizes Attributes
=========== =========== ================ =============== ============= ============
Lonely Yes No n/a n/a n/a
Tweaked Yes Yes Equal Equal Different
Same Yes Yes Equal Equal Equal
Changed Yes Yes Equal Different n/a
Newer Yes Yes Source > Dest n/a n/a
Older Yes Yes Source < Dest n/a n/a
Extra No Yes n/a n/a n/a
Mismatched Yes (file) Yes (directory) n/a n/a n/a
Classe de Existe dans Existe dans Source/Dest Source/Dest Source/Dest
Fichier la Source la Destination Date fichier Taille fichier Attributs
=========== =========== ================ =============== ============= ============
Solitaire Oui Non n/a n/a n/a
Tweaked * Oui Oui Identique Identique Différent
Identique Oui Oui Equal Identique Identique
Modifié Oui Oui Equal Different n/a
Plus récent Oui Oui Source > Dest n/a n/a
Plus ancien Oui Oui Source < Dest n/a n/a
Supplément. Non Oui n/a n/a n/a
Discordance Yes (Fic) Yes (Rép) n/a n/a n/a
* Note : Tweaked is KeyWord used for french too.
- Version
1.0| 2024-09-26 - Version
1.01| 2024-09-27- Bug fix : When the $ParseType option was full, the end summary was not correctly parsed due to a bad condition
- Version
2.0| 2024-10-27- Issue #2 : New Method to browse log file. This script now integrate the log file into an array and browse it.
- Version
2.1| 2024-11-03- Issue #4 : Integration of parameters for selecting file and directory classes to return in the object.
- Removal of the ParseType option in favor of new options allowing you to exclude the recovery of file or directory classes to only manage the header and summary.
- Version
2.2| 2024-11-03- Issue #7 : Add RawParsing Option to return the raw data of the file classes without structuring them.
- Add Write-Verbose to follow the progress of the script When the -Verbose parameter is used.
- Update the help section to include the new RawParsing parameter and the new behavior of the script.