prune failed with too many open files

[meise]

Output of restic version

restic 0.4.0 (v0.4.0-1-gf366a63)
compiled with go1.7 on linux/amd64

Expected behavior

restic prune completes without error.

Actual behavior

restic -p ~/.backup/backup.key -r /mnt/backup/home prune                                                                                  
index 0ca66592 has old format
index 33ac50ea has old format
index b828bffc has old format
index 8afcecfb has old format
index 2d91e0cb has old format
counting files in repo
building new index for repo
[8:01] 100.00%  24996 / 24996 packs
repository contains 24996 packs (998111 blobs) with 117.280 GiB bytes
processed 998111 blobs: 734 duplicate blobs, 416.767 MiB duplicate
load all snapshots
find data that is still in use for 10 snapshots
[15:15] 100.00%  10 / 10 snapshots
found 985408 of 998111 data blobs still in use, removing 12703 blobs
will delete 2497 packs and rewrite 1162 packs, this frees 12.021 GiB
open /mnt/backup/home/tmp/temp-078536371: too many open files
TempFile
restic/backend/local.copyToTempfile
  /tmp/restic-build-147414068/src/restic/backend/local/local.go:108
restic/backend/local.(*Local).Save
  /tmp/restic-build-147414068/src/restic/backend/local/local.go:135
restic/repository.(*Repository).savePacker
  /tmp/restic-build-147414068/src/restic/repository/packer_manager.go:122
restic/repository.(*Repository).SaveAndEncrypt
  /tmp/restic-build-147414068/src/restic/repository/repository.go:202
restic/repository.(*Repository).SaveBlob
  /tmp/restic-build-147414068/src/restic/repository/repository.go:556
restic/repository.Repack
  /tmp/restic-build-147414068/src/restic/repository/repack.go:102
main.runPrune
  /tmp/restic-build-147414068/src/cmds/restic/cmd_prune.go:216
main.glob..func11
  /tmp/restic-build-147414068/src/cmds/restic/cmd_prune.go:26
github.com/spf13/cobra.(*Command).execute
  /tmp/restic-build-147414068/src/github.com/spf13/cobra/command.go:599
github.com/spf13/cobra.(*Command).ExecuteC
  /tmp/restic-build-147414068/src/github.com/spf13/cobra/command.go:689
github.com/spf13/cobra.(*Command).Execute
  /tmp/restic-build-147414068/src/github.com/spf13/cobra/command.go:648
main.main
  /tmp/restic-build-147414068/src/cmds/restic/main.go:37
runtime.main
  /usr/lib/go/src/runtime/proc.go:183
runtime.goexit
  /usr/lib/go/src/runtime/asm_amd64.s:2086

ulimit is configured to default 1024:

ulimit -Sn
1024

Steps to reproduce the behavior

I'm not sure how to reproduce this without having my data set.

[zcalusic]

Just got hit with this yesterday, thought it was my repo that is too big, now it looks more like a bug. Right now running with elevated ulimit -n to 65536 files. Will take about 16h before I know if that helped.

restic 0.4.0 (v0.4.0-7-g22f3e21)
compiled with go1.8rc3 on linux/amd64

counting files in repo
building new index for repo
[4:06] 100.00%  12799 / 12799 packs
repository contains 12799 packs (2624340 blobs) with 57.063 GiB bytes
processed 2624340 blobs: 0 duplicate blobs, 0B duplicate
load all snapshots
find data that is still in use for 22 snapshots
[15:36:51] 100.00%  22 / 22 snapshots
found 2108689 of 2624340 data blobs still in use, removing 515651 blobs
will delete 520 packs and rewrite 1262 packs, this frees 5.142 GiB
open /repo/tmp/temp-165795239: too many open files
TempFile
restic/backend/local.copyToTempfile
	/tmp/restic-build-282137169/src/restic/backend/local/local.go:108
restic/backend/local.(*Local).Save
	/tmp/restic-build-282137169/src/restic/backend/local/local.go:135
restic/repository.(*Repository).savePacker
	/tmp/restic-build-282137169/src/restic/repository/packer_manager.go:122
restic/repository.(*Repository).SaveAndEncrypt
	/tmp/restic-build-282137169/src/restic/repository/repository.go:202
restic/repository.(*Repository).SaveBlob
	/tmp/restic-build-282137169/src/restic/repository/repository.go:556
restic/repository.Repack
	/tmp/restic-build-282137169/src/restic/repository/repack.go:102
main.runPrune
	/tmp/restic-build-282137169/src/cmds/restic/cmd_prune.go:216
main.glob..func11
	/tmp/restic-build-282137169/src/cmds/restic/cmd_prune.go:26
github.com/spf13/cobra.(*Command).execute
	/tmp/restic-build-282137169/src/github.com/spf13/cobra/command.go:599
github.com/spf13/cobra.(*Command).ExecuteC
	/tmp/restic-build-282137169/src/github.com/spf13/cobra/command.go:689
github.com/spf13/cobra.(*Command).Execute
	/tmp/restic-build-282137169/src/github.com/spf13/cobra/command.go:648
main.main
	/tmp/restic-build-282137169/src/cmds/restic/main.go:37

[fd0]

Oh wow, that's a bad bug, thanks for reporting it!

[fd0]

Heh, found the bug. =)

[fd0]

Could one of you please test the fix in #778?

[meise]

Works for me now:

restic 0.4.0 (v0.4.0-10-g4ca134a)       
compiled with go1.7 on linux/amd64

index 0ca66592 has old format                                             
index 33ac50ea has old format
index b828bffc has old format
index 8afcecfb has old format
index 2d91e0cb has old format
counting files in repo
building new index for repo
[8:07] 100.00%  25947 / 25947 packs
repository contains 25947 packs (1186932 blobs) with 121.466 GiB bytes
processed 1186932 blobs: 182704 duplicate blobs, 4.487 GiB duplicate
load all snapshots
find data that is still in use for 11 snapshots
[26:15] 100.00%  11 / 11 snapshots
found 992259 of 1186932 data blobs still in use, removing 194673 blobs
will delete 2497 packs and rewrite 2090 packs, this frees 12.021 GiB
creating new index
[6:54] 100.00%  22404 / 22404 packs
saved new index as c1fdabb8
done

Thank you @fd0 for your great work!