Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark: TurboPFor: IcApp 16 bits Integer Compression - Nanopore Signal Data #50

Open
powturbo opened this issue Nov 13, 2019 · 0 comments

Comments

@powturbo
Copy link
Owner

powturbo commented Nov 13, 2019

Testing TurboPFor, I've made some experiments with nanopore signal data.
File: all 16-bits signal data extracted from multi_fast5_zip.fast5

         ./icapp sig.u16 -Elzturbo,39 -Fs -e83,77,11,39,51,30,76
file: max bits histogram:
04: 0.000% 
06: 0.000% 
07: 0.002% 
08:###### 6.0% 
09:######################################################################################### 89% 
10:##### 5.5% 
11: 0.000% 
12: 0.000% 
16: 0.001% 

file: delta max bits histogram:
00:## 2.4%
01:## 2.4%
02:##### 4.7%
03:######### 9.2%
04:################# 17%
05:########################### 27%
06:######################## 24%
07:######### 9.0%
08:#### 3.7%
09:# 0.8%
10: 0.012%
11: 0.002%
12: 0.000%
13: 0.000%
14: 0.000%

Filesize: 3.097.862 bytes    CPU: Skylake i7-6700  3.4GHz

  E MB/s     size     ratio   D MB/s function (integer size=16 bits) 
   94.48    1275037  41.16%  1740.37 Lztp4z Nibble    Transpose+zigzag+turboanx,64
   81.99    1279609  41.31%  1512.63 Lztpz Byte       Transpose+zzag+turboanx,64
    1.81    1283751  41.44%  2201.98 Lztp4z Nibble    Transpose+zigzag+lzturbo,39 
    1.13    1289797  41.64%  1912.47 Lztpz Byte       Transpose+zzag+lzturbo,39    	
    3.06    1292571  41.72%  1627.03 Lztpz Byte       Transpose+zzag+zstd,22    	
  120.73    1293228  41.75%  2010.29 lzv8zenc         TurboByte+zzag+turboanx    	
    7.32    1296883  41.86%  1946.86 lzv8zenc         TurboByte+zzag+lzturbo,39    	
    7.48    1310074  42.29%  1690.97 lzv8zenc         TurboByte+zzag+zstd,22 	
    6.44    1333780  43.05%  1103.62 vbz              vbz_compression 
  614.78    1432663  46.25%  4419.20 p4nzenc128v16    TurboPForV   zigzag  	
  547.91    1523046  49.16%   752.46 lzv8zenc         TurboByte+zzag+fse    	
    1.55    1577260  50.91%  5779.59 LztpzByte        Transpose+zzag+lzturbo,19    	
   11.68    1577173  50.91%  1789.64 LztpzByte        Transpose+zzag+lz4,12    	
   10.80    1583766  51.12%  6387.34 lzv8zenc         TurboByte+zzag+lzturbo,19    	
   21.06    1587076  51.23%  4863.21 lzv8zenc         TurboByte+zzag+lz4,12    	
  367.44    1632853  52.71%   573.68 Lztpz Byte        Transpose+zzag+fse    	
 6749.15    1676704  54.12%  8775.81 v8nzenc128v16    TByte+TPackV zigzag  	
 6992.92    1676746  54.13%  8927.56 bitnzpack128v16  TurboPackV   zigzag  	
   68.73    1705144  55.04%   959.98 lzv8enc          TurboByte+turboanx         	
@powturbo powturbo changed the title TurboPFor: IcApp 16 bits Integer Compression Benchmark Benchmark: TurboPFor: IcApp 16 bits Integer Compression Mar 23, 2023
@powturbo powturbo changed the title Benchmark: TurboPFor: IcApp 16 bits Integer Compression Benchmark: TurboPFor: IcApp 16 bits Integer Compression - Nanopore Signal Data Jun 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant