Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data corruption when compaction runs on volume server with full underlying disk #1440

Closed
eth0h4ckr opened this issue Aug 31, 2020 · 0 comments

Comments

@eth0h4ckr
Copy link

Describe the bug
When running vacuum/compaction on a volume that has an underlying full disk, data gets corrupted/deleted.

System Setup

  • List the command line to start "weed master", "weed volume", "weed filer", "weed s3", "weed mount".
    weed master -defaultReplication 010 -ip 127.0.0.1 -port 9333 -mdir /var/lib/seaweedfs/master -volumeSizeLimitMB 64000 -volumePreallocate
    weed filer --master 127.0.0.1:9333 -collection test_collection
    weed volume -max=1 -dataCenter DC1 -rack rack1 -dir=/data/d1/seaweedfs -mserver 127.0.0.1:9333 -port 5200

  • OS version
    Ubuntu 16.04.7 LTS

  • output of weed version
    version 8000GB 1.91 2cdad77 linux amd64

  • if using filer, show the content of filer.toml
    filer using default config

Expected behavior
Expected behavior would be to mark the volume or volumes that sit on that particular disk as read only until data is able to be moved off or the volumes moved to a larger disk that are associated to that volume server

Additional context
We noticed the issue in a larger test environment after adjusted the volume sizes. Once we ran compaction, most of the data was missing or corrupted. In this particular environment I have set the disk size to 70GB and the volume size to 64000MB with max of 1 volume. Once the disk is full, compaction is ran and most data is gone/corrupted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant