Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VDO is not compressing some (easy compressible) files. #20

Closed
civic9 opened this issue May 12, 2019 · 1 comment
Closed

VDO is not compressing some (easy compressible) files. #20

civic9 opened this issue May 12, 2019 · 1 comment

Comments

@civic9
Copy link

civic9 commented May 12, 2019

I have strange problem. VDO don't want to compress some of my CSV files
(which are easy compressible, I think). Command line lz4 tool doesn't have any problem with them.

# du /1/test2.csv 
102400  /1/test2.csv

# lz4 -1 /1/test2.csv /1/test2.csv.lz4
Compressed 104857600 bytes into 42259380 bytes ==> 40.30%                      

# vdo status -n vdobackup|grep -i Compression
    Compression: enabled

# vdostats --verbose vdobackup|egrep 'used|compress'
  data blocks used                    : 13056414
  overhead blocks used                : 1292638
  logical blocks used                 : 17262451
  1K-blocks used                      : 57396208
  used percent                        : 27
  compressed fragments written        : 773855
  compressed blocks written           : 305048
  compressed fragments in packer      : 0
  KVDO module bytes used              : 1016029328
  KVDO module peak bytes used         : 1016031648
  KVDO module bios used               : 74572

# cp /1/test2.csv /mnt/backup/1/; sync

# vdostats --verbose vdobackup|egrep 'used|compress'
  data blocks used                    : 13082012
  overhead blocks used                : 1292670
  logical blocks used                 : 17288052
  1K-blocks used                      : 57498728
  used percent                        : 27
  compressed fragments written        : 773855
  compressed blocks written           : 305048
  compressed fragments in packer      : 0
  KVDO module bytes used              : 1016029328
  KVDO module peak bytes used         : 1016031648
  KVDO module bios used               : 74572

As you can see:
"data blocks used" increased by: 25598
"compressed fragments/blocks" are the same.

No other activity at the same time. It is not the only one file with this problem.
I have a few GB of files with similar content and none of them is compressed on the vdo volume.
Compression on other type of files works fine.

My VDO is created on top of standard disk partition. No LVM, no encryption.
kvdo from rhel/centos:

# uname -a
Linux .... 3.10.0-957.10.1.el7.x86_64 #1 SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
# rpm -qa|grep -i vdo
vdo-6.1.1.125-3.el7.x86_64
kmod-kvdo-6.1.1.125-5.el7.x86_64

Test file attached to the issue. test2.csv.gz

Any ideas?

@corwin
Copy link
Contributor

corwin commented Jul 16, 2019

This was discussed on the vdo-devel list: https://www.redhat.com/archives/vdo-devel/2019-May/msg00000.html

@corwin corwin closed this as completed Jul 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants