Segmentation fault in btrfs.static causes AssertionError

[krichter722]

Running sudo dduper --device /dev/nvme0n1p5 --dir / --recurse fails due to

Dedupe completed for /core:/app/docker/kafka1/data/__confluent.support.metrics-0/00000000000000000000.log
Summary
blk_size : 4KB  chunksize : 128KB
/core has 28 chunks
/app/docker/kafka1/data/__confluent.support.metrics-0/00000000000000000000.log has 1 chunks
Matched chunks: 0
Unmatched chunks: 1
Total size(KB) deduped: 0
Traceback (most recent call last):
  File "/usr/sbin/dduper", line 594, in <module>
    main(results)
  File "/usr/sbin/dduper", line 465, in main
    dedupe_dir(results.dir_path, results.dry_run, results.recurse)
  File "/usr/sbin/dduper", line 456, in dedupe_dir
    dedupe_files(file_list, dry_run)
  File "/usr/sbin/dduper", line 410, in dedupe_files
    ret = do_dedupe(src_file, dst_file, dry_run)
  File "/usr/sbin/dduper", line 225, in do_dedupe
    assert len(out2) != 0
AssertionError

Adding --verbose provides no additional information. dmesg contains

[ 1361.934038] btrfs.static[9668]: segfault at ffffffffb7bd9228 ip 00000000005228d4 sp 00007fff7e417848 error 5 in btrfs.static[401000+189000]
[ 1361.934043] Code: 0e 88 0f c3 c5 fa 6f 06 c5 fa 6f 4c 16 f0 c5 fa 7f 07 c5 fa 7f 4c 17 f0 c3 48 8b 4c 16 f8 48 8b 36 48 89 4c 17 f8 48 89 37 c3 <8b> 4c 16 fc 8b 36 89 4c 17 fc 89 37 c3 0f b7 4c 16 fe 0f b7 36 66

experienced with v0.04-9-g78155b6 on Ubuntu 20.04 with Linux 5.8.0-43-generic

[Lakshmipathi]

Does your mount point has subvolumes?

[krichter722]

@Lakshmipathi Yes, it does. Any more information you want me to provide?

[Lakshmipathi]

Really sorry about the delay. I believe you are running https://github.com/Lakshmipathi/dduper#known-issues dduper works only with top-level default subvolume.

[cactushydrocodone]

Ive removed all subvolumes, as i dont needed them anyway. but still get this error.

[Lakshmipathi]

@RaymondSchnyder Can you share exact error message ? Share details like how big the directory is? does it large no.of small file or small no.of large file? dduper installed in host machine or being executed via docker? I'd like to explore this issue further

[cactushydrocodone]

Can you share exact error message ?

Matched chunks: 0
Unmatched chunks: 62
Total size(KB) deduped: 0
btrfs_lookup_csums search failed.
 Error: btrfs_lookup_csumextent buffer leak: start 631282401280 len 16384
extent buffer leak: start 630993584128 len 16384
extent buffer leak: start 631009378304 len 16384
Traceback (most recent call last):
  File "/usr/bin/dduper", line 594, in <module>
    main(results)
  File "/usr/bin/dduper", line 465, in main
    dedupe_dir(results.dir_path, results.dry_run, results.recurse)
  File "/usr/bin/dduper", line 456, in dedupe_dir
    dedupe_files(file_list, dry_run)
  File "/usr/bin/dduper", line 410, in dedupe_files
    ret = do_dedupe(src_file, dst_file, dry_run)
  File "/usr/bin/dduper", line 225, in do_dedupe
    assert len(out2) != 0
AssertionError

Share details like how big the directory is?

900GB

does it large no.of small file or small no.of large file?

It contains my steam games and there are games with a large amount of small files and games with very large files

dduper installed in host machine or being executed via docker?

On the host machine

How i executed dduper:
dduper -p /dev/sdb2 -d /mnt/games -r -V