L2/Gzip/demos compression and decompression validation failed

[f1593570]

I use my own test data, and data size is 512MB to test compression and decompression functions in L2/Gzip/demos but validation failed. the size of data.gz.orig is obviously smaller than original size. Is there any limit on the size of the test data ? (Implement Platform is u200)

[vt-lib-support]

@f1593570 Input size of 512MB must work, which flow is failing ? Xilinx compress & decompress or Xilinx Compress vs Std decompress ? Also what is the compression ratio ?

[f1593570]

The flow is Xilinx compress & decompress, here is my executed command

./build_dir.hw.xilinx_u200_xdma_201830_2/xil_gzip -sx build_dir.hw.xilinx_u200_xdma_201830_2/compress_decompress.xclbin -v part00

Test result shows below :

Found Platform
Platform Name: Xilinx
Using Device: xilinx_u200_xdma_201830_2
INFO: Reading build_dir.hw.xilinx_u200_xdma_201830_2/compress_decompress.xclbin
Loading: 'build_dir.hw.xilinx_u200_xdma_201830_2/compress_decompress.xclbin'
--------------------------------------------------------------
                     Xilinx GZip Compress
--------------------------------------------------------------

KT(MBps)                :608.04
CR                      :1.01
File Size(MB)           :536.871
File Name               :part00

--------------------------------------------------------------
                     Xilinx GZip DeCompress
--------------------------------------------------------------

KT(MBps)                :16.77
File Size(MB)           :536.871
File Name               :part00.gz

part00 part00.gz.orig differ: byte 18874369, line 67009
Validation Failedpart00.gz.orig
XRT build version: 2.7.0
Build hash: 1697a17d5621ee4bf208142fd30a1224866137a9
Build date: 2020-06-02 15:23:47
Git branch: master
PID: 3577
UID: 0
[Tue Jun  9 10:41:05 2020 GMT]
HOST: Chromium
EXE: /STORAGE_4TB/ssc_workspace/Vitis_Libraries06/Vitis_Libraries/data_compression/L2/demos/gzip/build_dir.hw.xilinx_u200_xdma_201830_2/xil_gzip
[XRT] WARNING: Incomplete CU profile trace detected. Timeline trace will have approximate CU End
[XRT] WARNING: Trace FIFO is full because of too many events. Timeline trace could be incomplete. Please use 'coarse' option for data transfer trace or turn off Stall profiling

And detail of file size :

-rw-r--r--. 1 root root 536870912 Jun  9 18:40 part00
-rw-r--r--. 1 root root 531007413 Jun  9 18:40 part00.gz
-rw-r--r--. 1 root root  18877387 Jun  9 18:41 part00.gz.orig

[vt-lib-support]

@f1593570 Thanks for sharing more details, can you please try to decompress using standard GZip ,
Command to use: "gzip -dc part00.gz > out ", check if out and part00 are matching or not.

[f1593570]

out and part00 are not matching neither.

-rw-r--r--. 1 root root  18874368 Jun  9 18:54 out
-rw-r--r--. 1 root root 536870912 Jun  9 18:40 part00
-rw-r--r--. 1 root root 531007413 Jun  9 18:40 part00.gz
-rw-r--r--. 1 root root  18877387 Jun  9 18:41 part00.gz.orig

There is a message after executing gzip -dc part00.gz > out
gzip: part00.gz: invalid compressed data--format violated

[vt-lib-support]

@f1593570 thanks for providing update, similarly can u also try "gzip part00" and try to decompress using xilinx decompress "xil_gzip -sx xclbin -d part00.gz" and check if it works ?

[f1593570]

It's work. part00.gz.raw is matching with part00.
So maybe the problem is on compression of HW.

[vt-lib-support]

@f1593570 great thanks for confirmation. Bug in HW compression might be causing this. Please share part00 for debugging.

[f1593570]

Here is the link of data part00.
Thanks for your help.

[f1593570]

@vt-lib-support , Is there any update ? :)

[vt-lib-support]

@f1593570 we have reproduced the issue at our end. This is a bug we will provide a fix soon.

[vt-lib-support]

@f1593570 This issue is fixed and current master is updated with the fix.
Closing this issue.

[f1593570]

@vt-lib-support I try the code with the fix but it still has problem. I can't decompress 1gb file compressed by fpga-gzip correctly. Could it be that I upgraded Vitis to 2020.1 causing this problem?