restic 0.18.0

Changelog for restic 0.18.0 (2025-03-27)

The following sections list the changes in restic 0.18.0 relevant to restic users. The changes are ordered by importance.

Summary

• Sec #5291: Mitigate attack on content-defined chunking algorithm
• Fix #1843: Correctly restore long filepaths' timestamp on old Windows
• Fix #2165: Ignore disappeared backup source files
• Fix #5153: Include root tree when searching using find --tree
• Fix #5169: Prevent Windows VSS event log 8194 warnings for backup with fs snapshot
• Fix #5212: Fix duplicate data handling in prune --max-unused
• Fix #5249: Fix creation of oversized index by repair index --read-all-packs
• Fix #5259: Fix rare crash in command output
• Chg #4938: Update dependencies and require Go 1.23 or newer
• Chg #5162: Promote feature flags
• Enh #1378: Add JSON support to check command
• Enh #2511: Support generating shell completions to stdout
• Enh #3697: Allow excluding online-only cloud files (e.g. OneDrive)
• Enh #4179: Add sort option to ls command
• Enh #4433: Change default sort order for find output
• Enh #4521: Add support for Microsoft Blob Storage access tiers
• Enh #4942: Add snapshot summary statistics to rewritten snapshots
• Enh #4948: Format exit errors as JSON when requested
• Enh #4983: Add SLSA provenance to GHCR container images
• Enh #5054: Enable compression for ZIP archives in dump command
• Enh #5081: Add retry mechanism for loading repository config
• Enh #5089: Allow including/excluding extended file attributes during restore
• Enh #5092: Show count of deleted files and directories during restore
• Enh #5109: Make small pack size configurable for prune
• Enh #5119: Add start and end timestamps to backup JSON output
• Enh #5131: Add DragonFlyBSD support
• Enh #5137: Make tag command print which snapshots were modified
• Enh #5141: Provide clear error message if AZURE_ACCOUNT_NAME is not set
• Enh #5173: Add experimental S3 cold storage support
• Enh #5174: Add xattr support for NetBSD 10+
• Enh #5251: Improve retry handling for flaky rclone backends
• Enh #52897: Make recover automatically rebuild index when needed

Details

• Security #5291: Mitigate attack on content-defined chunking algorithm

  Restic uses Rabin Fingerprints for its content-defined chunker. The algorithm relies on a secret polynomial to split files into chunks.

  As shown in the paper "Chunking Attacks on File Backup Services using Content-Defined Chunking" by Boris Alexeev, Colin Percival and Yan X Zhang, an attacker that can observe chunk sizes for a known file can derive the secret polynomial. Knowledge of the polynomial might in some cases allow an attacker to check whether certain large files are stored in a repository.

  A practical attack is nevertheless hard as restic merges multiple chunks into opaque pack files and by default processes multiple files in parallel. This likely prevents an attacker from matching pack files to the attacker-known file and thereby prevents the attack.

  Despite the low chances of a practical attack, restic now has added mitigation that randomizes how chunks are assembled into pack files. This prevents attackers from guessing which chunks are part of a pack file and thereby prevents learning the chunk sizes.

  #5291 #5295

• Bugfix #1843: Correctly restore long filepaths' timestamp on old Windows

  The restore command now correctly restores timestamps for files with paths longer than 256 characters on Windows versions prior to Windows 10 1607.

  #1843 #5061

• Bugfix #2165: Ignore disappeared backup source files

  The backup command now quietly skips files that are removed between directory listing and backup, instead of printing errors like:

  error: lstat /some/file/name: no such file or directory
  

  #2165 #3098 #5143 #5145

• Bugfix #5153: Include root tree when searching using find --tree

  The restic find --tree command did not find trees referenced by restic snapshot --json. It now correctly includes the root tree when searching.

  #5153

• Bugfix #5169: Prevent Windows VSS event log 8194 warnings for backup with fs snapshot

  When running backup with the --use-fs-snapshot option in Windows with admin rights, event logs like

  Volume Shadow Copy Service error: Unexpected error querying for the IVssWriterCallback interface.  hr = 0x80070005, Access is denied.
  . This is often caused by incorrect security settings in either the writer or requester process.
  
  Operation:
     Gathering Writer Data
  
  Context:
     Writer Class Id: {e8132975-6f93-4464-a53e-1050253ae220}
     Writer Name: System Writer
     Writer Instance ID: {54b151ac-d27d-4628-9cb0-2bc40959f50f}
  

  Are created several times even though the backup itself succeeds. This has now been fixed.

  #5169 #5170 https://forum.restic.net/t/windows-shadow-copy-snapshot-vss-unexpected-provider-error/3674/2

• Bugfix #5212: Fix duplicate data handling in prune --max-unused

  The prune --max-unused size command did not correctly account for duplicate data. If a repository contained a large amount of duplicate data, this could previously result in pruning too little data. This has now been fixed.

  #5212 https://forum.restic.net/t/restic-not-obeying-max-unused-parameter-on-prune/8879

• Bugfix #5249: Fix creation of oversized index by repair index --read-all-packs

  Since restic 0.17.0, the new index created by repair index --read-all-packs was written as a single large index. This significantly increased memory usage while loading the index.

  The index is now correctly split into multiple smaller indexes, and repair index now also automatically splits oversized indexes.

  #5249

• Bugfix #5259: Fix rare crash in command output

  Some commands could in rare cases crash when trying to print status messages and request retries at the same time, resulting in an error like the following:

  panic: runtime error: slice bounds out of range [468:156]
  [...]
  github.com/restic/restic/internal/ui/termstatus.(*lineWriter).Write(...)
  	/restic/internal/ui/termstatus/stdio_wrapper.go:36 +0x136
  

  This has now been fixed.

  #5259 #5300

• Change #4938: Update dependencies and require Go 1.23 or newer

  We have updated all dependencies. Restic now requires Go 1.23 or newer to build.

  This also disables support for TLS versions older than TLS 1.2. On Windows, restic now requires at least Windows 10 or Windows Server 2016. On macOS, restic now requires at least macOS 11 Big Sur.

  #4938

• Change #5162: Promote feature flags

  The deprecate-legacy-index, deprecate-s3-legacy-layout, explicit-s3-anonymous-auth and safe-forget-keep-tags features are now stable and can no longer be disabled. The corresponding feature flags will be removed in restic 0.19.0.

  #5162

• Enhancement #1378: Add JSON support to check command

  The check command now supports the --json option to output all statistics in JSON format.

  #1378 #5194

• Enhancement #2511: Support generating shell completions to stdout

  The generate command now supports using - as the filename with the --[shell]-completion option to ...

restic 0.17.3

Changelog for restic 0.17.3 (2024-11-08)

The following sections list the changes in restic 0.17.3 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4971: Fix unusable mount on macOS Sonoma
• Fix #5003: Fix metadata errors during backup of removable disks on Windows
• Fix #5101: Do not retry load/list operation if SFTP connection is broken
• Fix #5107: Fix metadata error on Windows for backups using VSS
• Enh #5096: Allow prune --dry-run without lock

Details

• Bugfix #4971: Fix unusable mount on macOS Sonoma

  On macOS Sonoma when using FUSE-T, it was not possible to access files in a mounted repository. This issue is now resolved.

  #4971 #5048

• Bugfix #5003: Fix metadata errors during backup of removable disks on Windows

  Since restic 0.17.0, backing up removable disks on Windows could report errors with retrieving metadata like shown below.

  error: incomplete metadata for d:\filename: get named security info failed with: Access is denied.
  

  This has now been fixed.

  #5003 #5123 https://forum.restic.net/t/backing-up-a-folder-from-a-veracrypt-volume-brings-up-errors-since-restic-v17-0/8444

• Bugfix #5101: Do not retry load/list operation if SFTP connection is broken

  When using restic with the SFTP backend, backend operations that load a file or list files were retried even if the SFTP connection was broken. This has now been fixed.

  #5101 https://forum.restic.net/t/restic-hanging-on-backup/8559

• Bugfix #5107: Fix metadata error on Windows for backups using VSS

  Since restic 0.17.2, when creating a backup on Windows using --use-fs-snapshot, restic would report an error like the following:

  error: incomplete metadata for C:\: get EA failed while opening file handle for path \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopyXX\, with: The process cannot access the file because it is being used by another process.
  

  This has now been fixed by correctly handling paths that refer to volume shadow copy snapshots.

  #5107 #5110 #5112

• Enhancement #5096: Allow prune --dry-run without lock

  The prune --dry-run --no-lock now allows performing a dry-run without locking the repository. Note that if the repository is modified concurrently, prune may return inaccurate statistics or errors.

  #5096

restic 0.17.2

Changelog for restic 0.17.2 (2024-10-27)

The following sections list the changes in restic 0.17.2 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4004: Support container-level SAS/SAT tokens for Azure backend
• Fix #5047: Resolve potential error during concurrent cache cleanup
• Fix #5050: Return error if tag fails to lock repository
• Fix #5057: Exclude irregular files from backups
• Fix #5063: Correctly backup extended metadata when using VSS on Windows

Details

• Bugfix #4004: Support container-level SAS/SAT tokens for Azure backend

  Restic previously expected SAS/SAT tokens to be generated at the account level, which prevented tokens created at the container level from being used to initialize a repository. This caused an error when attempting to initialize a repository with container-level tokens.

  Restic now supports both account-level and container-level SAS/SAT tokens for initializing a repository.

  #4004 #5093

• Bugfix #5047: Resolve potential error during concurrent cache cleanup

  When multiple restic processes ran concurrently, they could compete to remove obsolete snapshots from the local backend cache, sometimes leading to a "no such file or directory" error. Restic now suppresses this error to prevent issues during cache cleanup.

  #5047

• Bugfix #5050: Return error if tag fails to lock repository

  Since restic 0.17.0, the tag command did not return an error when it failed to open or lock the repository. This issue has now been fixed.

  #5050 #5056

• Bugfix #5057: Exclude irregular files from backups

  Since restic 0.17.1, files with the type irregular could mistakenly be included in snapshots, especially when backing up special file types on Windows that restic cannot process. This issue has now been fixed.

  Previously, this bug caused the check command to report errors like the following one:

    tree 12345678[...]: node "example.zip" with invalid type "irregular"
  

  To repair affected snapshots, upgrade to restic 0.17.2 and run:

  restic repair snapshots --forget
  

  This will remove the irregular files from the snapshots (creating a new snapshot ID for each of the affected snapshots).

  #5057 https://forum.restic.net/t/errors-found-by-check-1-invalid-type-irregular-2-ciphertext-verification-failed/8447/2

• Bugfix #5063: Correctly backup extended metadata when using VSS on Windows

  On Windows, when creating a backup with the --use-fs-snapshot option, restic read extended metadata from the original filesystem path instead of from the snapshot. This could result in errors if files were removed during the backup process.

  This issue has now been resolved.

  #5063 #5097 #5099

restic 0.17.1

Changelog for restic 0.17.1 (2024-09-05)

The following sections list the changes in restic 0.17.1 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #2004: Correctly handle volume names in backup command on Windows
• Fix #4945: Include missing backup error text with --json
• Fix #4953: Correctly handle long paths on older Windows versions
• Fix #4957: Fix delayed cancellation of certain commands
• Fix #4958: Don't ignore metadata-setting errors during restore
• Fix #4969: Correctly restore timestamp for files with resource forks on macOS
• Fix #4975: Prevent backup --stdin-from-command from panicking
• Fix #4980: Skip extended attribute processing on unsupported Windows volumes
• Fix #5004: Fix spurious "A Required Privilege Is Not Held by the Client" error
• Fix #5005: Fix rare failures to retry locking a repository
• Fix #5018: Improve HTTP/2 support for REST backend
• Chg #4953: Also back up files with incomplete metadata
• Enh #4795: Display progress bar for restore --verify
• Enh #4934: Automatically clear removed snapshots from cache
• Enh #4944: Print JSON-formatted errors during restore --json
• Enh #4959: Return exit code 12 for "bad password" errors
• Enh #4970: Make timeout for stuck requests customizable

Details

• Bugfix #2004: Correctly handle volume names in backup command on Windows

  On Windows, when the specified backup target only included the volume name without a trailing slash, for example, C:, then restoring the resulting snapshot would result in an error. Note that using C:\ as backup target worked correctly.

  Specifying volume names is now handled correctly. To restore snapshots created before this bugfix, use the : syntax. For example, to restore a snapshot with ID 12345678 that backed up C:, use the following command:

  restic restore 12345678:/C/C:./ --target output/folder
  

  #2004 #5028

• Bugfix #4945: Include missing backup error text with --json

  Previously, when running a backup with the --json option, restic failed to include the actual error message in the output, resulting in "error": {} being displayed.

  This has now been fixed, and restic now includes the error text in JSON output.

  #4945 #4946

• Bugfix #4953: Correctly handle long paths on older Windows versions

  On older Windows versions, like Windows Server 2012, restic 0.17.0 failed to back up files with long paths. This problem has now been resolved.

  #4953 #4954

• Bugfix #4957: Fix delayed cancellation of certain commands

  Since restic 0.17.0, some commands did not immediately respond to cancellation via Ctrl-C (SIGINT) and continued running for a short period. The most affected commands were diff,find, ls, stats and rewrite. This is now resolved.

  #4957 #4960

• Bugfix #4958: Don't ignore metadata-setting errors during restore

  Previously, restic used to ignore errors when setting timestamps, attributes, or file modes during a restore. It now reports those errors, except for permission related errors when running without root privileges.

  #4958

• Bugfix #4969: Correctly restore timestamp for files with resource forks on macOS

  On macOS, timestamps were not restored for files with resource forks. This has now been fixed.

  #4969 #5006

• Bugfix #4975: Prevent backup --stdin-from-command from panicking

  Restic would previously crash if --stdin-from-command was specified without providing a command. This issue has now been fixed.

  #4975 #4976

• Bugfix #4980: Skip extended attribute processing on unsupported Windows volumes

  With restic 0.17.0, backups of certain Windows paths, such as network drives, failed due to errors while fetching extended attributes.

  Restic now skips extended attribute processing for volumes where they are not supported.

  #4955 #4950 #4980 #4998

• Bugfix #5004: Fix spurious "A Required Privilege Is Not Held by the Client" error

  On Windows, creating a backup could sometimes trigger the following error:

  error: nodeFromFileInfo [...]: get named security info failed with: a required privilege is not held by the client.
  

  This has now been fixed.

  #5004 #5019

• Bugfix #5005: Fix rare failures to retry locking a repository

  Restic 0.17.0 could in rare cases fail to retry locking a repository if one of the lock files failed to load, resulting in the error:

  unable to create lock in backend: circuit breaker open for file <lock/1234567890>
  

  This issue has now been addressed. The error handling now properly retries the locking operation. In addition, restic waits a few seconds between locking retries to increase chances of successful locking.

  #5005 #5011 #5012

• Bugfix #5018: Improve HTTP/2 support for REST backend

  If rest-server tried to gracefully shut down an HTTP/2 connection still in use by the client, it could result in the following error:

  http2: Transport: cannot retry err [http2: Transport received Server's graceful shutdown GOAWAY] after Request.Body was written; define Request.GetBody to avoid this error
  

  This issue has now been resolved.

  #5018 https://forum.restic.net/t/receiving-http2-goaway-messages-with-windows-restic-v0-17-0/8367

• Change #4953: Also back up files with incomplete metadata

  If restic failed to read extended metadata for a file or folder during a backup, then the file or folder was not included in the resulting snapshot. Instead, a warning message was printed along with returning exit code 3 once the backup was finished.

  Now, restic also includes items for which the extended metadata could not be read in a snapshot. The warning message has been updated to:

  incomplete metadata for /path/to/file: <details about error>
  

  #4953 #4977

• Enhancement #4795: Display progress bar for restore --verify

  When the restore command is run with --verify, it now displays a progress bar while the verification step is running. The progress bar is not shown when the --json flag is specified.

  #4795 #4989

• Enhancement #4934: Automatically clear removed snapshots from cache

  Previously, restic only removed snapshots from the cache on the host where the forget command was executed. On other hosts that use the same repository, the old snapshots remained in the cache.

  Restic now automatically clears old snapshots from the local cache of the current host.

  #4934 #4981

• Enhancement #4944: Print JSON-formatted errors during restore --json

  Restic used to print any restore errors directly to the console as freeform text messages, even when using the --json option.

  Now, when --json is specified, restic prints them as JSON formatted messages.

  #4944 #4946

• Enhancement #4959: Return exit code 12 for "bad password" errors

  Restic now returns exit code 12 when it cannot open the repository due to an incorrect password.

  #4959

• Enhancement #4970: Make timeout for stuck requests customizable

  Restic monitors connections to the backend to detect stuck requests. If a request does not return any data within five minutes, restic assumes the request is stuck and retries it. However, for large repositories this timeout might be insufficient to collect a list of all files, causing the following error:

  `List(data) returned error,...

restic 0.17.0

Changelog for restic 0.17.0 (2024-07-26)

The following sections list the changes in restic 0.17.0 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #3600: Handle unreadable xattrs in folders above backup source
• Fix #4209: Fix slow SFTP upload performance
• Fix #4503: Correct hardlink handling in stats command
• Fix #4568: Prevent forget --keep-tags <invalid> from deleting all snapshots
• Fix #4615: Make find not sometimes ignore directories
• Fix #4656: Properly report ID of newly added keys
• Fix #4703: Shutdown cleanly when receiving SIGTERM
• Fix #4709: Correct --no-lock handling of ls and tag commands
• Fix #4760: Fix possible error on concurrent cache cleanup
• Fix #4850: Handle UTF-16 password files in key command correctly
• Fix #4902: Update snapshot summary on rewrite
• Chg #956: Return exit code 10 and 11 for non-existing and locked repository
• Chg #4540: Require at least ARMv6 for ARM binaries
• Chg #4602: Deprecate legacy index format and s3legacy repository layout
• Chg #4627: Redesign backend error handling to improve reliability
• Chg #4707: Disable S3 anonymous authentication by default
• Chg #4744: Include full key ID in JSON output of key list
• Enh #662: Optionally skip snapshot creation if nothing changed
• Enh #693: Include snapshot size in snapshots output
• Enh #805: Add bitrot detection to diff command
• Enh #828: Improve features of the repair packs command
• Enh #1786: Support repositories with empty password
• Enh #2348: Add --delete option to restore command
• Enh #3067: Add extended options to configure Windows Shadow Copy Service
• Enh #3406: Improve dump performance for large files
• Enh #3806: Optimize and make prune command resumable
• Enh #4006: (alpha) Store deviceID only for hardlinks
• Enh #4048: Add support for FUSE-T with mount on macOS
• Enh #4251: Support reading backup from a command's standard output
• Enh #4287: Support connection to rest-server using unix socket
• Enh #4354: Significantly reduce prune memory usage
• Enh #4437: Make check command create non-existent cache directory
• Enh #4472: Support AWS Assume Role for S3 backend
• Enh #4547: Add --json option to version command
• Enh #4549: Add --ncdu option to ls command
• Enh #4573: Support rewriting host and time metadata in snapshots
• Enh #4583: Ignore s3.storage-class archive tiers for metadata
• Enh #4590: Speed up mount command's error detection
• Enh #4601: Add support for feature flags
• Enh #4611: Back up more file metadata on Windows
• Enh #4664: Make ls use message_type field in JSON output
• Enh #4676: Make key command's actions separate sub-commands
• Enh #4678: Add --target option to the dump command
• Enh #4708: Back up and restore SecurityDescriptors on Windows
• Enh #4733: Allow specifying --host via environment variable
• Enh #4737: Include snapshot ID in reason field of forget JSON output
• Enh #4764: Support forgetting all snapshots
• Enh #4768: Allow specifying custom User-Agent for outgoing requests
• Enh #4781: Add restore options to read include/exclude patterns from files
• Enh #4807: Support Extended Attributes on Windows NTFS
• Enh #4817: Make overwrite behavior of restore customizable
• Enh #4839: Add dry-run support to restore command

Details

• Bugfix #3600: Handle unreadable xattrs in folders above backup source

  When backup sources are specified using absolute paths, backup also includes information about the parent folders of the backup sources in the snapshot.

  If the extended attributes for some of these folders could not be read due to missing permissions, this caused the backup to fail. This has now been fixed.

  #3600 #4668 https://forum.restic.net/t/parent-directories-above-the-snapshot-source-path-fatal-error-permission-denied/7216

• Bugfix #4209: Fix slow SFTP upload performance

  Since restic 0.12.1, the upload speed of the sftp backend to a remote server has regressed significantly. This has now been fixed.

  #4209 #4782

• Bugfix #4503: Correct hardlink handling in stats command

  If files on different devices had the same inode ID, the stats command did not correctly calculate the snapshot size. This has now been fixed.

  #4503 #4006 https://forum.restic.net/t/possible-bug-in-stats/6461/8

• Bugfix #4568: Prevent forget --keep-tags <invalid> from deleting all snapshots

  Running forget --keep-tags <invalid>, where <invalid> is a tag that does not exist in the repository, would remove all snapshots. This is especially problematic if the tag name contains a typo.

  The forget command now fails with an error if all snapshots in a snapshot group would be deleted. This prevents the above example from deleting all snapshots.

  It is possible to temporarily disable the new check by setting the environment variable RESTIC_FEATURES=safe-forget-keep-tags=false. Note that this feature flag will be removed in the next minor restic version.

  #4568 #4764

• Bugfix #4615: Make find not sometimes ignore directories

  In some cases, the find command ignored empty or moved directories. This has now been fixed.

  #4615

• Bugfix #4656: Properly report ID of newly added keys

  restic key add now reports the ID of the newly added key. This simplifies selecting a specific key using the --key-hint key option.

  #4656 #4657

• Bugfix #4703: Shutdown cleanly when receiving SIGTERM

  Previously, when restic received the SIGTERM signal it would terminate immediately, skipping cleanup and potentially causing issues like stale locks being left behind. This primarily effected containerized restic invocations that use SIGTERM, but could also be triggered via a simple killall restic.

  This has now been fixed, such that restic shuts down cleanly when receiving the SIGTERM signal.

  #4703

• Bugfix #4709: Correct --no-lock handling of ls and tag commands

  The ls command never locked the repository. This has now been fixed, with the old behavior still being supported using ls --no-lock. The latter invocation also works with older restic versions.

  The tag command erroneously accepted the --no-lock command. This command now always requires an exclusive lock.

  #4709

• Bugfix #4760: Fix possible error on concurrent cache cleanup

  If multiple restic processes concurrently cleaned up no longer existing files from the cache, this could cause some of the processes to fail with an no such file or directory error. This has now been fixed.

  #4760 #4761...

restic 0.16.5

Changelog for restic 0.16.5 (2024-07-01)

The following sections list the changes in restic 0.16.5 relevant to restic users. The changes are ordered by importance.

Summary

• Enh #4799: Add option to force use of Azure CLI credential
• Enh #4873: Update dependencies

Details

• Enhancement #4799: Add option to force use of Azure CLI credential

  A new environment variable AZURE_FORCE_CLI_CREDENTIAL=true allows forcing the use of Azure CLI credential, ignoring other credentials like managed identity.

  #4799

• Enhancement #4873: Update dependencies

  A few potentially vulnerable dependencies were updated.

  #4873 #4878

restic 0.16.4

This release works around and improves detection of a bug in the compression library used by restic. The resulting issue only happens when using restic 0.16.3 and the max compression level (the default auto and off compression levels are not affected), and when the source files being backed up have specific data in them to trigger the bug. If you use max compression, you can use restic check --read-data to make sure you're not affected.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.4 (2024-02-04)

The following sections list the changes in restic 0.16.4 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4677: Downgrade zstd library to fix rare data corruption at max. compression
• Enh #4529: Add extra verification of data integrity before upload

Details

• Bugfix #4677: Downgrade zstd library to fix rare data corruption at max. compression

  In restic 0.16.3, backups where the compression level was set to max (using --compression max) could in rare and very specific circumstances result in data corruption due to a bug in the library used for compressing data. Restic 0.16.1 and 0.16.2 were not affected.

  Restic now uses the previous version of the library used to compress data, the same version used by restic 0.16.2. Please note that the auto compression level (which restic uses by default) was never affected, and even if you used max compression, chances of being affected by this issue are small.

  To check a repository for any corruption, run restic check --read-data. This will download and verify the whole repository and can be used at any time to completely verify the integrity of a repository. If the check command detects anomalies, follow the suggested steps.

  #4677 #4679

• Enhancement #4529: Add extra verification of data integrity before upload

  Hardware issues, or a bug in restic or its dependencies, could previously cause corruption in the files restic created and stored in the repository. Detecting such corruption previously required explicitly running the check --read-data or check --read-data-subset commands.

  To further ensure data integrity, even in the case of hardware issues or software bugs, restic now performs additional verification of the files about to be uploaded to the repository.

  These extra checks will increase CPU usage during backups. They can therefore, if absolutely necessary, be disabled using the --no-extra-verify global option. Please note that this should be combined with more active checking using the previously mentioned check commands.

  #4529 #4681

restic 0.16.3

This release fixes a couple of bugs on Windows and in the restore command. It also works around an unlikely yet possible situation with rclone which could potentially result in data loss.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.3 (2024-01-14)

The following sections list the changes in restic 0.16.3 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4560: Improve errors for irregular files on Windows
• Fix #4574: Support backup of deduplicated files on Windows again
• Fix #4612: Improve error handling for rclone backend
• Fix #4624: Correct restore progress information if an error occurs
• Fix #4626: Improve reliability of restoring large files

Details

• Bugfix #4560: Improve errors for irregular files on Windows

  Since Go 1.21, most filesystem reparse points on Windows are considered to be irregular files. This caused restic to show an error: invalid node type "" error message for those files.

  This error message has now been improved and includes the relevant file path: error: nodeFromFileInfo path/to/file: unsupported file type "irregular". As irregular files are not required to behave like regular files, it is not possible to provide a generic way to back up those files.

  #4560 #4620 https://forum.restic.net/t/windows-backup-error-invalid-node-type/6875

• Bugfix #4574: Support backup of deduplicated files on Windows again

  With the official release builds of restic 0.16.1 and 0.16.2, it was not possible to back up files that were deduplicated by the corresponding Windows Server feature. This also applied to restic versions built using Go 1.21.0-1.21.4.

  The Go version used to build restic has now been updated to fix this.

  #4574 #4621

• Bugfix #4612: Improve error handling for rclone backend

  Since restic 0.16.0, if rclone encountered an error while listing files, this could in rare circumstances cause restic to assume that there are no files. Although unlikely, this situation could result in data loss if it were to happen right when the prune command is listing existing snapshots.

  Error handling has now been improved to detect and work around this case.

  #4612 #4618

• Bugfix #4624: Correct restore progress information if an error occurs

  If an error occurred while restoring a snapshot, this could cause the restore progress bar to show incorrect information. In addition, if a data file could not be loaded completely, then errors would also be reported for some already restored files.

  Error reporting of the restore command has now been made more accurate.

  #4624 https://forum.restic.net/t/errors-restoring-with-restic-on-windows-server-s3/6943

• Bugfix #4626: Improve reliability of restoring large files

  In some cases restic failed to restore large files that frequently contain the same file chunk. In combination with certain backends, this could result in network connection timeouts that caused incomplete restores.

  Restic now includes special handling for such file chunks to ensure reliable restores.

  #4626 https://forum.restic.net/t/errors-restoring-with-restic-on-windows-server-s3/6943

restic 0.16.2

restic 0.16.2 restores support for ARMv5 which was accidentally not included in the previous one, and also fixes missing documentation on Read the Docs.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.2 (2023-10-29)

The following sections list the changes in restic 0.16.2 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4540: Restore ARMv5 support for ARM binaries
• Fix #4545: Repair documentation build on Read the Docs

Details

• Bugfix #4540: Restore ARMv5 support for ARM binaries

  The official release binaries for restic 0.16.1 were accidentally built to require ARMv7. The build process is now updated to restore support for ARMv5.

  Please note that restic 0.17.0 will drop support for ARMv5 and require at least ARMv6.

  #4540

• Bugfix #4545: Repair documentation build on Read the Docs

  For restic 0.16.1, no documentation was available at https://restic.readthedocs.io/ .

  The documentation build process is now updated to work again.

  #4545

restic 0.16.1

restic 0.16.1 has been released! It contains a few usage and cosmetic improvements, but also one bugfix for a very unlikely but still possible data corruption issue when using the max compression level. Please read the changelog for more details and information on how to verify your repository if you used max compression. Please also note that the auto compression level (which is the default) was never affected.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.1 (2023-10-24)

The following sections list the changes in restic 0.16.1 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4513: Make key list command honor --no-lock
• Fix #4516: Do not try to load password on command line autocomplete
• Fix #4523: Update zstd library to fix possible data corruption at max. compression
• Chg #4532: Update dependencies and require Go 1.19 or newer
• Enh #229: Show progress bar while loading the index
• Enh #4128: Automatically set GOMAXPROCS in resource-constrained containers
• Enh #4480: Allow setting REST password and username via environment variables
• Enh #4511: Include inode numbers in JSON output for find and ls commands
• Enh #4519: Add config option to set SFTP command arguments

Details

• Bugfix #4513: Make key list command honor --no-lock

  The key list command now supports the --no-lock options. This allows determining which keys a repo can be accessed by without the need for having write access (e.g., read-only sftp access, filesystem snapshot).

  #4513 #4514

• Bugfix #4516: Do not try to load password on command line autocomplete

  The command line autocompletion previously tried to load the repository password. This could cause the autocompletion not to work. Now, this step gets skipped.

  #4516 #4526

• Bugfix #4523: Update zstd library to fix possible data corruption at max. compression

  In restic 0.16.0, backups where the compression level was set to max (using --compression max) could in rare and very specific circumstances result in data corruption due to a bug in the library used for compressing data.

  Restic now uses the latest version of the library used to compress data, which includes a fix for this issue. Please note that the auto compression level (which restic uses by default) was never affected, and even if you used max compression, chances of being affected by this issue were very small.

  To check a repository for any corruption, run restic check --read-data. This will download and verify the whole repository and can be used at any time to completely verify the integrity of a repository. If the check command detects anomalies, follow the suggested steps.

  To simplify any needed repository repair and minimize data loss, there is also a new and experimental repair packs command that salvages all valid data from the affected pack files (see restic help repair packs for more information).

  #4523 #4530

• Change #4532: Update dependencies and require Go 1.19 or newer

  We have updated all dependencies. Since some libraries require newer Go standard library features, support for Go 1.18 has been dropped, which means that restic now requires at least Go 1.19 to build.

  #4532 #4533

• Enhancement #229: Show progress bar while loading the index

  Restic did not provide any feedback while loading index files. Now, there is a progress bar that shows the index loading progress.

  #229 #4419

• Enhancement #4128: Automatically set GOMAXPROCS in resource-constrained containers

  When running restic in a Linux container with CPU-usage limits, restic now automatically adjusts GOMAXPROCS. This helps to reduce the memory consumption on hosts with many CPU cores.

  #4128 #4485 #4531

• Enhancement #4480: Allow setting REST password and username via environment variables

  Previously, it was only possible to specify the REST-server username and password in the repository URL, or by using the --repository-file option. This meant it was not possible to use authentication in contexts where the repository URL is stored in publicly accessible way.

  Restic now allows setting the username and password using the RESTIC_REST_USERNAME and RESTIC_REST_PASSWORD variables.

  #4480

• Enhancement #4511: Include inode numbers in JSON output for find and ls commands

  Restic used to omit the inode numbers in the JSON messages emitted for nodes by the ls command as well as for matches by the find command. It now includes those values whenever they are available.

  #4511

• Enhancement #4519: Add config option to set SFTP command arguments

  When using the sftp backend, scenarios where a custom identity file was needed for the SSH connection, required the full command to be specified: -o sftp.command='ssh user@host:port -i /ssh/my_private_key -s sftp'

  Now, the -o sftp.args=... option can be passed to restic to specify custom arguments for the SSH command executed by the SFTP backend. This simplifies the above example to -o sftp.args='-i /ssh/my_private_key'.

  #4241 #4519