From 5f5aa32411ba6d44296e0106f9f06f5c6940c367 Mon Sep 17 00:00:00 2001 From: Yohann Uguen Date: Mon, 13 Feb 2023 02:48:05 -0800 Subject: [PATCH 1/2] reduce simulation problem size Signed-off-by: Yohann Uguen --- .../ReferenceDesigns/decompress/README.md | 2 +- .../decompress/data/gzip/small.gz | Bin 0 -> 418 bytes .../data/snappy/alice29_small.ref.txt | 42 +++++++++++++ .../data/snappy/alice29_small.txt.sz | Bin 0 -> 1256 bytes .../ReferenceDesigns/decompress/src/main.cpp | 55 ++++++++++++------ 5 files changed, 80 insertions(+), 19 deletions(-) create mode 100644 DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/gzip/small.gz create mode 100644 DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.ref.txt create mode 100644 DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.txt.sz diff --git a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/README.md b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/README.md index 6a6365ad39..371de334cc 100755 --- a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/README.md +++ b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/README.md @@ -195,7 +195,7 @@ Snappy is compression format that aims for high throughput compression and decom Unlike many compression encodings, like DEFLATE, Snappy encoding is byte oriented. The Snappy format does not use entropy encoding, such as Huffman or range encoding. -Snappy encoding is like LZ77 encoding, which replaces portions of the byte stream with {length, distance} pairs. For more information on the Snappy format, see the [Snappy](https://en.wikipedia.org/wiki/Snappy_(compression)) Wikipedia article and the [Google Snappy](https://github.com/google/snappy) GitHub repository. +Snappy encoding is like LZ77 encoding, which replaces portions of the byte stream with {length, distance} pairs. For more information on the Snappy format, see the [Snappy](https://en.wikipedia.org/wiki/Snappy_(compression)) Wikipedia article and the [Google Snappy](https://github.com/google/snappy) GitHub repository. The files being decompressed in this tutorial have been compressed using [python-snappy](https://github.com/andrix/python-snappy) which is a python library for the snappy compression library from Google. The basic Snappy format is a *preamble* followed by the *compressed data stream*. diff --git a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/gzip/small.gz b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/gzip/small.gz new file mode 100644 index 0000000000000000000000000000000000000000..57af1cdddeb8a9181dc8f8fae24d65b1b0f2a16c GIT binary patch literal 418 zcmV;T0bTwdiwFoi4eDb619NR*Y-}!ccys`5lHF?KFc5{W?WZ`9nFEu*zotY!e$A~e;Ld({$wV0wLM=1KU&wJV8PCk_p=4-5tz&R>ls`S-M`OD>w~Tv&X*+*-4PpopR6>+InwA zK@B>~29l*Vd(f6dLwUmiBNXzzV>`oloxWzj9$!L(6&vN34vUii1wJF&=$$w85i4}6zd!l@ M1E@y5r=J1<0HILNssI20 literal 0 HcmV?d00001 diff --git a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.ref.txt b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.ref.txt new file mode 100644 index 0000000000..a300afb061 --- /dev/null +++ b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.ref.txt @@ -0,0 +1,42 @@ + + + + + ALICE'S ADVENTURES IN WONDERLAND + + Lewis Carroll + + THE MILLENNIUM FULCRUM EDITION 2.9 + + + + + CHAPTER I + + Down the Rabbit-Hole + + + Alice was beginning to get very tired of sitting by her sister +on the bank, and of having nothing to do: once or twice she had +peeped into the book her sister was reading, but it had no +pictures or conversations in it, `and what is the use of a book,' +thought Alice `without pictures or conversation?' + + So she was considering in her own mind (as well as she could, +for the hot day made her feel very sleepy and stupid), whether +the pleasure of making a daisy-chain would be worth the trouble +of getting up and picking the daisies, when suddenly a White +Rabbit with pink eyes ran close by her. + + There was nothing so VERY remarkable in that; nor did Alice +think it so VERY much out of the way to hear the Rabbit say to +itself, `Oh dear! Oh dear! I shall be late!' (when she thought +it over afterwards, it occurred to her that she ought to have +wondered at this, but at the time it all seemed quite natural); +but when the Rabbit actually TOOK A WATCH OUT OF ITS WAISTCOAT- +POCKET, and looked at it, and then hurried on, Alice started to +her feet, for it flashed across her mind that she had never +before seen a rabbit with either a waistcoat-pocket, or a watch to +take out of it, and burning with curiosity, she ran across the +field after it, and fortunately was just in time to see it pop +down a large rabbit-hole under the hedge. \ No newline at end of file diff --git a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.txt.sz b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.txt.sz new file mode 100644 index 0000000000000000000000000000000000000000..8a21b237a90eb00122b2453a54b6c8d96cb8bba1 GIT binary patch literal 1256 zcmWlZ&x;&I6vw+lH_Q&xuqz{3VFRxr%g&maCGi)@CSaPGUbmCU%sShfO#+70bibLd zOi$NrRrNL_YEDs)f`X#CdJ;r05>NyU>cv3(2XYF6AV?GsUeuRU|EhZRzTfZX_0Kii ztu7p~V3S#I_}#D_tnGr|xg7MOy+N=GVGs8Ey-qOb`n?W&Y4dsO7(3a;QfO#%rDT>} zIAOgTZ3l2E>~@1*FWkEXTYKI1fUZF&jKY2op4&L%x*u8Y&5_jrXm9&FQ80k;LSnhB z+mU4sCPf(V;ZT^?w#<-iu&ZDAnTQceu3?B{k>?^GgOM;s1Ba;QV1&X1)JWTjb8y-od%4$P;LgGbLsjaP6FgSQc5*bgLt;;vH#70|}qQlqcwH)<%fg z`s;O_k*@h_bWWRMCX)3glqr()ihTtLvkbW|l=Gl6PnTEgXEa1|6zxtj1{h9Lay1~+HP zd<6;Vc#jbF+t$SUYw3lRX%VL&3*-H0Q_@#}IFZB!SI(mQiKU&RJ8o4NjoFBj>8Fs; zClBD6Y7~M_IoU}nGj7mZ189^f`?Eq=_>~H!#M(2E6fp1+Ay#sg=q5!BGL8#HFfP2s z-n6+{eH8An503LgQd`PALDpEN_)RKw_=DrtkvqLC05m?i7qJ$=4G2bqFw!EBB{ zE#54&0sox}ndg0LYh3-fK9e)In($wiIc28C=-!K3Dg8j8AlqrIpG-6+W8C;3E4Pcx literal 0 HcmV?d00001 diff --git a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/src/main.cpp b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/src/main.cpp index f4dca99219..37ac4b136f 100644 --- a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/src/main.cpp +++ b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/src/main.cpp @@ -184,13 +184,26 @@ void PrintTestResults(std::string test_name, bool passed) { #if defined(GZIP) bool RunGzipTest(sycl::queue& q, GzipDecompressorT decompressor, const std::string test_dir) { + + +#ifdef FPGA_SIMULATOR + // the name of the file for the simulator is fixed + std::string small_filename = test_dir + "/small.gz"; + + std::cout << ">>>>> Small File Test <<<<<" << std::endl; + bool small_test_pass = decompressor.DecompressFile( + q, small_filename, "", 1, false, false); + PrintTestResults("Small File Test", small_test_pass); + std::cout << std::endl; + + return small_test_pass; +#else // the name of the files for the default test are fixed std::string uncompressed_filename = test_dir + "/uncompressed.gz"; std::string static_compress_filename = test_dir + "/static_compressed.gz"; std::string dynamic_compress_filename = test_dir + "/dynamic_compressed.gz"; std::string tp_test_filename = test_dir + "/tp_test.gz"; -#ifndef FPGA_SIMULATOR std::cout << ">>>>> Uncompressed File Test <<<<<" << std::endl; bool uncompressed_test_pass = decompressor.DecompressFile( q, uncompressed_filename, "", 1, false, false); @@ -202,12 +215,6 @@ bool RunGzipTest(sycl::queue& q, GzipDecompressorT decompressor, q, static_compress_filename, "", 1, false, false); PrintTestResults("Statically Compressed File Test", static_test_pass); std::cout << std::endl; -#else - std::cout << "Only running the Dynamically Compressed File Test when using " - "the simulator flow to reduce execution time." << std::endl; - bool uncompressed_test_pass = true; - bool static_test_pass = true; -#endif std::cout << ">>>>> Dynamically Compressed File Test <<<<<" << std::endl; bool dynamic_test_pass = decompressor.DecompressFile( @@ -215,32 +222,47 @@ bool RunGzipTest(sycl::queue& q, GzipDecompressorT decompressor, PrintTestResults("Dynamically Compressed File Test", dynamic_test_pass); std::cout << std::endl; - -#ifndef FPGA_SIMULATOR std::cout << ">>>>> Throughput Test <<<<<" << std::endl; constexpr int kTPTestRuns = 5; bool tp_test_pass = decompressor.DecompressFile(q, tp_test_filename, "", kTPTestRuns, true, false); PrintTestResults("Throughput Test", tp_test_pass); std::cout << std::endl; -#else - bool tp_test_pass = true; -#endif return uncompressed_test_pass && static_test_pass && dynamic_test_pass && - tp_test_pass; + tp_test_pass; +#endif + } #endif #if defined(SNAPPY) bool RunSnappyTest(sycl::queue& q, SnappyDecompressorT decompressor, const std::string test_dir) { + + +#ifdef FPGA_SIMULATOR + std::cout << ">>>>> Small Alice In Wonderland Test <<<<<" << std::endl; + std::string alice_in_file = test_dir + "/alice29_small.txt.sz"; + auto in_bytes = ReadInputFile(alice_in_file); + auto result = decompressor.DecompressBytes(q, in_bytes, 1, false); + + std::string alice_ref_file = test_dir + "/alice29_small.ref.txt"; + auto ref_bytes = ReadInputFile(alice_ref_file); + bool alice_test_pass = + (result != std::nullopt) && (result.value() == ref_bytes); + + PrintTestResults("Small Alice In Wonderland Test", alice_test_pass); + std::cout << std::endl; + return alice_test_pass; + +#else std::cout << ">>>>> Alice In Wonderland Test <<<<<" << std::endl; - std::string alice_in_file = test_dir + "/alice29.txt.sz"; + std::string alice_in_file = test_dir + "/alice29_small.txt.sz"; auto in_bytes = ReadInputFile(alice_in_file); auto result = decompressor.DecompressBytes(q, in_bytes, 1, false); - std::string alice_ref_file = test_dir + "/alice29.ref.txt"; + std::string alice_ref_file = test_dir + "/alice29_small.ref.txt"; auto ref_bytes = ReadInputFile(alice_ref_file); bool alice_test_pass = (result != std::nullopt) && (result.value() == ref_bytes); @@ -248,7 +270,6 @@ bool RunSnappyTest(sycl::queue& q, SnappyDecompressorT decompressor, PrintTestResults("Alice In Wonderland Test", alice_test_pass); std::cout << std::endl; -#ifndef FPGA_SIMULATOR std::cout << ">>>>> Only Literal Strings Test <<<<<" << std::endl; auto test1_bytes = GenerateSnappyCompressedData(333, 3, 0, 0, 3); auto test1_ret = decompressor.DecompressBytes(q, test1_bytes, 1, false); @@ -285,8 +306,6 @@ bool RunSnappyTest(sycl::queue& q, SnappyDecompressorT decompressor, return alice_test_pass && test1_pass && test2_pass && test3_pass && test_tp_pass; -#else - return alice_test_pass; #endif } From 882f31fc51553c2dab9dfb991f4ec68b68477734 Mon Sep 17 00:00:00 2001 From: Yohann Uguen Date: Mon, 13 Feb 2023 04:25:49 -0800 Subject: [PATCH 2/2] reduce even more Signed-off-by: Yohann Uguen --- .../decompress/data/gzip/small.gz | Bin 418 -> 166 bytes .../data/snappy/alice29_small.ref.txt | 21 +----------------- .../data/snappy/alice29_small.txt.sz | Bin 1256 -> 409 bytes 3 files changed, 1 insertion(+), 20 deletions(-) diff --git a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/gzip/small.gz b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/gzip/small.gz index 57af1cdddeb8a9181dc8f8fae24d65b1b0f2a16c..ca57c751938db4920e25571e44f88c1682b7d07b 100644 GIT binary patch literal 166 zcmV;X09pSZiwFq*D(YhZ17vw_VQpz+Ut@1=aB^jHb7f=zEzP|S!XOxh;kjJJNnA=p zL%W#{I_Qmj1r!D-{lK4i^;*;MBu~z(I|y{REHSey1wAZPg4uUi5;8A$rnosR4%Xd5 za!m}WB{j>YZRs>ioH1hpNg;H%9Y!+#=kO#Blxo4t2jjtReFSze2K{w66Oe5lM}^OE U7zTxG-t`~*0$Mj}U!4E|07{iaE&u=k literal 418 zcmV;T0bTwdiwFoi4eDb619NR*Y-}!ccys`5lHF?KFc5{W?WZ`9nFEu*zotY!e$A~e;Ld({$wV0wLM=1KU&wJV8PCk_p=4-5tz&R>ls`S-M`OD>w~Tv&X*+*-4PpopR6>+InwA zK@B>~29l*Vd(f6dLwUmiBNXzzV>`oloxWzj9$!L(6&vN34vUii1wJF&=$$w85i4}6zd!l@ M1E@y5r=J1<0HILNssI20 diff --git a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.ref.txt b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.ref.txt index a300afb061..cdb4303a19 100644 --- a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.ref.txt +++ b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.ref.txt @@ -20,23 +20,4 @@ on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,' -thought Alice `without pictures or conversation?' - - So she was considering in her own mind (as well as she could, -for the hot day made her feel very sleepy and stupid), whether -the pleasure of making a daisy-chain would be worth the trouble -of getting up and picking the daisies, when suddenly a White -Rabbit with pink eyes ran close by her. - - There was nothing so VERY remarkable in that; nor did Alice -think it so VERY much out of the way to hear the Rabbit say to -itself, `Oh dear! Oh dear! I shall be late!' (when she thought -it over afterwards, it occurred to her that she ought to have -wondered at this, but at the time it all seemed quite natural); -but when the Rabbit actually TOOK A WATCH OUT OF ITS WAISTCOAT- -POCKET, and looked at it, and then hurried on, Alice started to -her feet, for it flashed across her mind that she had never -before seen a rabbit with either a waistcoat-pocket, or a watch to -take out of it, and burning with curiosity, she ran across the -field after it, and fortunately was just in time to see it pop -down a large rabbit-hole under the hedge. \ No newline at end of file +thought Alice `without pictures or conversation?' \ No newline at end of file diff --git a/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.txt.sz b/DirectProgramming/C++SYCL_FPGA/ReferenceDesigns/decompress/data/snappy/alice29_small.txt.sz index 8a21b237a90eb00122b2453a54b6c8d96cb8bba1..4374d4cec71da029554f2cb7756651d31648a0b2 100644 GIT binary patch delta 26 hcmaFCIg^=bJeA6{ChQb3PeSYlBC zxBhdG>$(t06wcxxUh8QB!5dg`f&NZ=hFJ99?%D#eYPPaVmg-80egaF0B;DXI0ii08 z+uHaKf(4(i*y;jzBb03hx#X&ZC0hVY5tiDJ^#ARU(e*Tb;^|rZs736&S64m;mCF4W z*+=Y3pwRkRINXqmFbc1Y#w~~kH=$DH9I!zPhL?~R+9FIFcE5G5zk0Tv zxy8o$YfrswHC!KEdjkwExS8{5fuPc>Jx&*))CPo0@_`qH>994A|2p~n=AzCe=$hgm zJd*4tG(Z|8a^rWRPg~YMvloSmEEcl_-kL}d?kNU%I?TpkC3irlxPpQc#?JusmXiG$ zNQ5*94kWQ*1^NgL@EM_5awBX4ia_UiZ48R_ciB50r>#fhJ@(;Qe&q5lbt#bbHaG!E zWyjz9Q5S*K0@*9N=8(#ER7o%|MBK#A_hxZ-;XcHIPjvyO6?- zG#g%p$?F+RuE03kcNgRRY&c1?K@{#yhP$II+4`khXgx>4R-Pht$CdQBBi)jU|K^p6 z^8qYzL;gbtk3a4@NhpEz?xssKTNT_&_EI-D+FD>R`lChL-iG7?N<2zG?@n>14YG%w z(iLmo-0@JQ_bbA()R<$FaE}}xb;c5(p3}+ d{__TRwV4Bq?qvh1ohP*;Zya@W{fh%U_a7^Z9WDR>