QNX系统中，多个dlt.gz文件导出来之后，出现几个文件无法解压的问题

[yfan0000004]

我们对无法解压的文件点击查看文件内部，其中的.dlt文件大小明显小于正常可以解压文件内部.dlt文件的大小，请帮忙看下

[tangzhiqiang3]

QNX: 700 , 最新的dlt-daemon patch:

gzip -d xxx_08_20240424-144242.dlt.gz
gzip: xxx_08_20240424-144242.dlt.gz: invalid compressed data--format violated

[minminlittleshrimp]

Hello @yfan0000004
Please post your issue in English, that the very least requirement for contributing to OSS.
Next time I will close any issue tab like this, thank you.

[minminlittleshrimp]

QNX: 700 , 最新的dlt-daemon patch:

gzip -d xxx_08_20240424-144242.dlt.gz gzip: xxx_08_20240424-144242.dlt.gz: invalid compressed data--format violated

Why trying to compress the already-compressed file here?
Please go for: gunzip -d

[yfan0000004]

In the QNX system, after exporting multiple dlt.gz files, there are several issues where the files cannot be decompressed.
We clicked on the file that cannot be decompressed to view its contents. The size of the ". dlt" file is significantly smaller than the normal size of the ". dlt" file inside the decompressed file. Please help us take a look

[yfan0000004]

The packaged. gz files are all 10MB, but the content size of the file that cannot be decompressed is much smaller than the normal file content size that can be decompressed. It is speculated that DLT performed gz packaging operation before the log file was fully written, resulting in packaging errors and inability to decompress

[tangzhiqiang3]

gunzip -d CSC1_06_20240424-143301.dlt.gz
gzip: CSC1_06_20240424-143301.dlt.gz: invalid compressed data--format violated

Decompression CSC1_06_20240424-143301.dlt.gz still error, decompression CSC1_05_20240424-142833.dlt.gz is ok.

[minminlittleshrimp]

Checking, seem it a real issue, I also cannot unzip the not-fully-up-to-size gz file

Dr.Mint@:~/work/github/dlt-daemon/build/test_gzip (master) 
$ ll
total 16
drwxrwxr-x 2 lum3hc lum3hc 4096 Apr 25 13:07 ./
drwxrwxr-x 9 lum3hc lum3hc 4096 Apr 25 13:02 ../
-rw-rw-r-- 1 lum3hc lum3hc  134 Apr 25 13:06 dlt_logstorage.conf
-rwxrwxrwx 1 lum3hc lum3hc 1134 Apr 25 13:07 Test_gzip_1_20240425-130700.dlt.gz*
Dr.Mint@:~/work/github/dlt-daemon/build/test_gzip (master) 
$ gunzip -d Test_gzip_1_20240425-130700.dlt.gz 

gzip: Test_gzip_1_20240425-130700.dlt.gz: unexpected end of file
Dr.Mint@:~/work/github/dlt-daemon/build/test_gzip (master) 
$ cat dlt_logstorage.conf 
[FILTER1]
LogAppName=LOG
ContextName=TEST
LogLevel=DLT_LOG_INFO
File=Test_gzip
NOFiles=1
EcuID=ECU1
FileSize=10000
GzipCompression=ON

[yfan0000004]

Is the failure to decompress this file due to the file not being fully written, resulting in the failure to execute gzclose(). Our problem is that files that have been fully written and packaged cannot be decompressed, not files that have not been written yet cannot be decompressed. Of course, your problem should also be solved

[tangzhiqiang3]

[FILTER1]
LogAppName=xxx
ContextName=.*
LogLevel=DLT_LOG_INFO
File=CSC1
FileSize=1485760
NOFiles=10
GzipCompression=ON

dlt-logstorage-ctrl -c 0 -p /var/log/ : export logs after the log is taken off disk
dlt-logstorage-ctrl -c 1 -p /var/log/ : recovery logs

But the problem still arises ?
DLT has added gz compression function, and occasionally cannot decompress gz files. Is this a confirmed bug ?

[minminlittleshrimp]

I confirm, it's a bug, dlt internal log showing something wrong with flushing data sync on_msg (sync directly) inside dlt logstorage function.

[ 3181.899657]~DLT~ 6116~WARNING  ~Wrote less data than specified
[ 3181.899661]~DLT~ 6116~WARNING  ~Wrote less data than specified
[ 3181.899665]~DLT~ 6116~WARNING  ~Wrote less data than specified
[ 3181.899669]~DLT~ 6116~ERROR    ~dlt_logstorage_sync_on_msg: failed to gzflush log file

Will check and answer.
Thank you

[yfan0000004]

I confirm, it's a bug, dlt internal log showing something wrong with flushing data sync on_msg (sync directly) inside dlt logstorage function.

[ 3181.899657]~DLT~ 6116~WARNING  ~Wrote less data than specified
[ 3181.899661]~DLT~ 6116~WARNING  ~Wrote less data than specified
[ 3181.899665]~DLT~ 6116~WARNING  ~Wrote less data than specified
[ 3181.899669]~DLT~ 6116~ERROR    ~dlt_logstorage_sync_on_msg: failed to gzflush log file

Will check and answer. Thank you
Hello, how can this problem be quickly reproduced? We did not see this log

[minminlittleshrimp]

There is a fix in the past for many issues in gzip, I might link it to this issue, since I observed this issue at that time, but not fixed yet 😀
Check at: 543087b

[yfan0000004]

OK,do you still plan to solve this problem?

[minminlittleshrimp]

Sure, let me check and provide a fix asap.

[yfan0000004]

Hello, you can try replacing the zlib library with a higher version and then verifying it again.

[tangzhiqiang3]

The currently discovered issue with the inability to decompress the gz compressed logs of DLT is that the inability to decompress occurs only when a gz file is not fully written and is powered off and restarted;