Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.12.2/1.10.2 Forge: Beyond/Skyfactory3/SevTech Ages crashes: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check #146

Closed
iceiix opened this issue May 12, 2019 · 8 comments
Labels
bug Something isn't working

Comments

@iceiix
Copy link
Owner

iceiix commented May 12, 2019

SevTech_Ages_Server_3.1.1 from https://minecraft.curseforge.com/projects/sevtech-ages/files/2686905#additional-files

cargo run --release -- -s localhost:25565 -n

Client crashes on handling FML|HS packet discriminator 2:

thread 'main' panicked at 'Err: IOError(Custom { kind: UnexpectedEof, error: StringError("failed to fill whole buffer") })', src/server/mod.rs:449:33

Forge 1.12.2 with ironchest mod worked in #144

ModList:

'\x02\xf8\x01\tminecraft\x061.12.2\x03mcp\x049.42\x03FML\t8.0.99.99\x05forge\x0c14.23.5.2815\tivtoolkit\n1.3.3-1.12\rmicdoodlecore\x00\x0esmoothfontcore\rmc1.12.2-1.16\rbnbgamingcore\x060.10.0\x0bfoamfixcore\x057.7.4\nmovillages\x051.5.4\rcharcoalblock\x031.1\x0fadvancedmortars\r1.12.2-1.6.22\x0ccrafttweaker\x064.1.14\x05mtlib\x053.0.5\nmodtweaker\x064.0.16\x03jei\n4.15.0.268\x0cabyssalcraft\x071.9.5.2\x06chisel\x11MC1.12.2-0.2.1.35\x06mantle\r1.12-1.3.3.42\x0etwilightforest\x073.8.689\ntconstruct\x111.12.2-2.12.0.115\racintegration\x051.7.0\tfastbench\x051.6.1\x11actuallyadditions\x0b1.12.2-r146\x07baubles\x051.5.2\x0factuallybaubles\x031.1\tanimalium\x050.3.8\x0cantiqueatlas\x054.5.0\tapplecore\x053.2.0\x04base\x063.12.0\x0econtenttweaker\x0c1.12.2-4.9.1\x14immersiveengineering\x070.12-89\x08geolosys\x052.1.4\x0bcyclicmagic\x071.17.16\x0credstoneflux\x052.1.0\x08mekanism\x101.12.2-9.6.0.351\x06natura\x0f1.12.2-4.3.2.66\x08traverse\x051.6.0\x0ebetterwithmods\x101.12-2.3.20-1027\tappleskin\x051.0.9\x13appliedenergistics2\x0crv6-stable-6\x0earmoreablemobs\x051.1.2\raroma1997core\x072.0.0.2\x0baromabackup\x072.1.1.4\x13aromabackuprecovery\x072.1.1.4\x0castikorcarts\x0e1.12.2-0.1.2.6\rastralsorcery\x071.10.11\x05quark\x08r1.5-146\nautoreglib\x061.3-26\x05bdlib\t1.14.3.12\x0cbedrockbgone\x055.0.8\x10betterwithaddons\x0b0.47-hotfix\x13betterbuilderswands\x060.11.1\rbetterwithlib\n${version}\x0bbibliocraft\x052.4.5\x05waila\x061.8.26\x0emodularrouters\x0c1.12.2-3.2.1\x08guideapi\r1.12-2.1.8-63\nbloodmagic\x101.12.2-2.4.0-102\x0cbnbgaminglib\x062.17.5\x0bbonsaitrees\x051.1.2\tbookshelf\x072.3.574\rbuildcraftlib\x077.99.22\x0ebuildcraftcore\x077.99.22\x12buildcraftbuilders\x077.99.22\x11buildcraftfactory\x077.99.22\x12buildcraftrobotics\x077.99.22\x13buildcrafttransport\x077.99.22\x11buildcraftsilicon\x077.99.22\x0fbuildinggadgets\x052.6.6\x07caliper\x061.1.41\x03car\x061.2.12\ngamestages\x072.0.112\x07carryon\x061.12.1\x0ccd4017be_lib\x056.3.3\x08ceramics\n1.12-1.3.6\tchameleon\n1.12-4.1.3\x08chargers\x071.2.0.4\x0echiselsandbits\x0514.30\x06clumps\x053.1.2\x0ecodechickenlib\t3.2.2.353\x0bcyclopscore\x051.0.5\x12commoncapabilities\x052.0.4\x0erefinedstorage\x061.6.12\x10compactmachines3\x063.0.15\x06conarm\x051.2.3\x14cookingforblockheads\x066.4.69\x0fcrafttweakerjei\x052.0.2\x08cucumber\x051.1.3\tdarkutils\x071.8.223\rdeath_compass\x050.0.3\x12despawningspawners\x031.1\x12dimensionalcontrol\x062.10.3\tdimstages\x062.0.23\x08dungpipe\x031.0\nelevatorid\x051.3.8\temberroot\x051.3.9\x0cenderstorage\t2.4.5.135\x0eenderutilities\x060.7.12\x0bvalkyrielib\x0f1.12.2-2.0.19.1\x11environmentaltech\x0f1.12.2-2.0.19.1\x10extendedcrafting\x051.5.2\x10galacticraftcore\t4.0.1.181\x13galacticraftplanets\t4.0.1.181\rmjrlegendslib\x0c1.12.2-1.1.8\x0cextraplanets\x0c1.12.2-0.5.0\x14farmingforblockheads\x063.1.26\x07farseek\x052.3.1\x0bfastfurnace\x051.2.1\x07fat_cat\x050.0.5\x11ferdinandsflowers\x081.10.4.1\x06findme\x051.1.0\x07foamfix\r0.10.3-1.12.2\x08forgelin\x051.8.2\x11forgemultipartcbe\x082.6.1.81\rmicroblockcbe\x082.6.1.81\x15minecraftmultipartcbe\x082.6.1.81\x13galacticrafttweaker\x0c1.12.2-1.0.3\radvgenerators\t0.9.20.12\ngobblecore\r1.12-0.1.6.35\x07harvest\r1.12-1.2.7-20\nhorsepower\x052.6.2\nhuntingdim\x061.0.36\x06igwmod\x081.4.4-15\x0bmcjtylib_ng\x053.1.1\x08immcraft\x051.5.1\x12immersivepetroleum\x051.1.9\rimmersivetech\x061.3.10\x11improvedbackpacks\x0e1.12.2-1.4.0.0\tincontrol\x053.9.4\x06indlog\x051.3.3\x0cteslacorelib\x061.0.15\x13industrialforegoing\r1.12.2-1.12.2\x0finfoaccessories\x061.0.11\x12integrateddynamics\x051.0.7\x18integrateddynamicscompat\x051.0.0\x0finventorytweaks\x141.64+dev.146.2180b27\rironbackpacks\x0f1.12.2-3.0.8-12\tironchest\x111.12.2-7.0.59.842\x0cironjetpacks\x051.1.0\nitemstages\x062.0.46\njourneymap\x0c1.12.2-5.5.4\x04jarm\x051.1.2\x04jaff\x0c1.7_for_1.12\x04jeid\x081.0.2-26\tkleeslabs\x065.4.11\x03lex\x051.0.8\tlttweaker\x061.1.14\x0emagma_monsters\x050.3.0\nmekatweaks\x071.0.0-3\tmercurius\x051.0.6\x07mobends\x040.24\x12mob_grinding_utils\x060.3.13\tmobstages\x062.0.13\x10modularmachinery\x051.9.5\x08morpheus\x0e1.12.2-3.5.106\x07mputils\x051.5.6\x07mpbasic\x051.4.7\x10multiblockstages\x051.1.1\x0fmundaneredstone\x051.1.4\x13mysticalagriculture\x051.7.3\x14mysticalagradditions\x051.3.1\x0fmystagradcompat\x031.2\x0enaturescompass\x051.5.1\x03nex\x052.0.8\x0enoworldgen5you\x051.0.6\tnutrition\x053.5.0\tsamsocean\x051.0.1\roreexcavation\x071.4.137\roeintegration\x052.3.4\torestages\x062.0.37\noverloaded\x060.0.53\x0cpickletweaks\x052.1.1\x07placebo\x051.5.1\x0cplayerbosses\x031.0\x0bplayerskins\x051.0.4\x0epneumaticcraft\x101.12.2-0.8.4-303\rpoweradapters\x051.0.9\x08prestige\x061.1.30\x0cprimalchests\x051.0.3\x06rustic\x051.1.0\x0fthebetweenlands\x053.4.6\x06primal\x070.6.104\x11progressiontweaks\r1.12.2-0.3.40\x0bprospectors\x051.0.2\nreborncore\n3.13.5.421\x0equantumstorage\x054.6.8\x0equickleafdecay\x051.2.4\x0brangedpumps\x030.5\trealdrops\x061.2.14\rrebornstorage\x051.0.0\x0crecipestages\x051.1.1\nreccomplex\x051.4.7\x14refinedstorageaddons\x050.4.3\x07rftools\x047.61\x0erftoolscontrol\x051.9.3\nroadrunner\x051.0.1\x07sampler\x041.80\tscannable\x081.6.3.24\tsevtweaks\x080.2.3-20\x0esev_tweaks_npc\x050.0.4\rsimpleautorun\n1.12.1-1.2\x10simplegenerators\x0f1.12.2-2.0.19.1\x0estoragenetwork\x051.5.7\x0csimplyarrows\x051.0.4\x0espartanshields\x051.4.1\x10spatialservermod\x031.3\x0bstevescarts\n2.4.29.132\x03stg\x0c1.12.2-1.2.3\x0estoragedrawers\n1.12-5.3.5\x07streams\x050.4.4\x05sasit\x061.1.14\x11supersoundmuffler\x081.0.2.10\ttallgates\x051.0.0\x07beneath\x051.5.0\x0ethirstybottles\x051.1.4\x0btcomplement\n${version}\x0ctinkerstages\x062.0.17\x12tinkertoolleveling\x181.12.2-1.1.0.DEV.b23e769\x0ftogetherforever\x051.0.2\ttombstone\x053.3.2\x07totemic\r1.12.2-0.11.5\rtothebatpoles\x0e1.12.2-1.1.0.1\rtranslocators\x082.5.1.77\x07triumph\x063.13.0\x0ftrumpetskeleton\x0c1.12-1.0.2.1\ntumbleweed\n1.11-0.4.5\x06uppers\x050.0.6\x12universalmodifiers\x0f1.12.2-1.0.16.1\x02vc\x065.9.13\x03vtt\x050.6.4\x07waddles\x050.6.0\x0bwailastages\x062.0.23\twanionlib\n1.12.2-2.2\x13watercontrolextreme\x051.0.0\rwaterstrainer\x053.2.0\x05wawla\x072.5.270\x0eweirdinggadget\x031.0\twildcrops\x051.0.1\x0ewitherskelefix\x052.6.3\x06wopper\x071.12-r5\rworldstripper\x0c1.6.0-1.12.2\x04xnet\x051.7.6\x04ynot\x050.2.3\x05yoyos\x0f1.12.2-1.2.2.20\tzenstages\x080.4.1-19\x0bprimal_tech\x050.3.3\x17teslacorelib_registries\x061.0.15'

The server also logs an error during decompression:

[07:25:54] [Netty Server IO #8/INFO] [FML]: Client protocol version 2
[07:25:54] [Netty Server IO #8/ERROR] [FML]: NetworkDispatcher exception
io.netty.handler.codec.DecoderException: java.util.zip.DataFormatException: incorrect header check
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442) ~[minecraft_server.1.12.2.jar:?]
...
Caused by: java.util.zip.DataFormatException: incorrect header check
at java.util.zip.Inflater.inflateBytes(Native Method) ~[?:1.8.0_131]

@iceiix iceiix added the bug Something isn't working label May 12, 2019
@iceiix iceiix changed the title 1.12.2 Forge: SevTech: Ages crashes: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check 1.12.2 Forge: SevTech Ages crashes parsing ModList: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check May 12, 2019
@iceiix iceiix changed the title 1.12.2 Forge: SevTech Ages crashes parsing ModList: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check 1.12.2/1.10.2 Forge: SevTech Ages/Skyfactory3 crashes parsing ModList: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check May 12, 2019
@iceiix
Copy link
Owner Author

iceiix commented May 12, 2019

FTBBeyondServer_1.11.0 ModList packet '\x02\xb1\x01\x03mcp\x049.19\x03FML\t8.0.99.99\x05Forge\x0c12.18.3.2422\x18appliedenergistics2-core\x0crv4-stable-1\rAroma1997Core\n${version}\x0eNotEnoughItems\t2.1.3.220\x12OpenComputers|Core\x071.6.2.7\n\x03000\x0bfoamfixcore\x057.7.4\x0eccl-entityhook\x031.0\x08cofhcore\x064.1.11\x11actuallyadditions\x0b1.10.2-r105\x03IC2\r2.6.252-ex110\x11advanced_machines\x0660.0.4\x15advanced_solar_panels\x054.0.4\x13appliedenergistics2\x0crv4-stable-1\x05bdlib\t1.12.4.24\x08ae2stuff\x070.6.0.9\x0binfinitylib\r1.10.2-0.11.0\tagricraft\x102.0.0-0.10.0-a19\x07Baubles\x061.3.11\x07Botania\x08r1.9-341\x0bAkashicTome\x051.0-5\x03JEI\n3.14.7.419\tappleskin\x051.0.7\x0baquaculture\x051.4.1\x11architecturecraft\x051.7.3\x13Aroma1997CoreHelper\x071.1.1.2\x13Aroma1997sDimension\x031.0\tMorphtool\x061.0-12\x03Psi\x07r1.0-42\x05Quark\x07r1.1-70\nAutoRegLib\x051.0-2\x07bedbugs\t@VERSION@\x13betterbuilderswands\x060.11.1\x0bBiblioCraft\x052.2.5\x08guideapi\t@VERSION@\nBloodMagic\x101.10.2-2.1.11-80\x0cbloodarsenal\x0f1.10.2-2.1.0-22\tbookshelf\t1.4.4.347\x0eCodeChickenLib\t2.5.9.283\x0cbrandonscore\x062.1.11\x06mantle\x101.10.2-1.1.5.205\ntconstruct\x0f1.10.2-2.6.5.10\x08ceramics\r1.10.2-1.1.6b\tChameleon\x0c1.10.2-2.3.0\x10ChestTransporter\x062.5.18\x06chisel\x12MC1.10.2-0.0.13.30\x0bmcmultipart\x051.4.0\x0echiselsandbits\x0512.12\x08shadowmc\x053.6.1\tclipboard\t@VERSION@\x0fCodeChickenCore\t2.4.1.102\rcompactsolars\x111.10.2-5.0.12.336\x14cookingforblockheads\x064.2.45\x0fcrafttweakerjei\x051.0.1\x0cMineTweaker3\x063.0.26\tdarkutils\x0e1.1.8.finalrc2\x0bPTRModelLib\x051.0.0\x05props\x052.4.2\x0bextrautils2\x031.0\x08forestry\n5.2.17.385\x08eleccore\x071.7.431\x0cEnderStorage\t2.2.1.106\rOpenComputers\x071.6.2.7\rdeepresonance\x051.4.8\x11draconicevolution\x052.1.5\x0eStorageDrawers\r1.10.2-3.7.10\nbitdrawers\x040.36\nelevatorid\x051.3.0\x06embers\x050.104\tendercore\x141.10.2-0.4.1.66-beta\x07EnderIO\x0e1.10.2-3.1.183\x11engineersworkshop\x0c1.3.6-1.10.2\x0bvalkyrielib\r1.10.2-0.10.6\x11environmentaltech\x0e1.10.2-0.10.6b\x07etlunar\x0e1.10.2-0.10.6a\x08zerocore\x0e1.10.2-0.1.0.6\x0bbigreactors\x0f1.10.2-0.4.5.30\x11flatcoloredblocks\x0bmc1.10-v4.4\tsonarcore\x053.2.8\x0cfluxnetworks\x051.2.5\x07foamfix\t@VERSION@\x08forgelin\x051.5.1\x11forgemultipartcbe\x082.0.0.38\rmicroblockcbe\x082.0.0.38\x15minecraftmultipartcbe\x082.0.0.38\x04ftbl\x050.0.0\x04ftbu\x050.0.0\x0ffunkylocomotion\x031.0\x07Funnels\x051.1.2\tgendustry\x081.6.5.31\ngravestone\x061.5.13\ngravisuite\x053.0.1\nhammercore\x071.9.4.3\thelpfixer\x031.0\x0bhopperducts\x071.4.8.1\x0btheoneprobe\x061.4.14\x14immersiveengineering\x070.10-61\rLunatriusCore\x081.1.2.38\rInGameInfoXML\x082.8.1.89\x08intwheel\x051.2.6\x0finventorytweaks\x0f1.61-58-a1fd884\rironbackpacks\r1.10.2-2.2.31\tironchest\x111.10.2-7.0.15.804\x07jeibees\x070.9.0.5\njourneymap\x0c1.10.2-5.4.7\tkleeslabs\x053.3.5\x05boots\x031.0\x0bmalisiscore\x0c1.10.2-4.4.0\x0cmalisisdoors\x0c1.10.2-5.2.0\tmercurius\x051.0.6\x17modularforcefieldsystem\x053.0.1\ttestdummy\x041.11\x12mob_grinding_utils\x080.1.8.22\x05mtlib\t@VERSION@\nmodtweaker\x062.0.13\trailcraft\x0610.1.2\nreborncore\n2.13.6.142\ntechreborn\n2.1.17.233\x08morebees\x0e1.10.2-1.4.1.1\x08Morpheus\r1.10.2-3.1.13\x07mputils\x051.2.9\x07mpbasic\x051.1.3\x0cMrTJPCoreMod\x082.0.1.20\x0eshetiphiancore\x053.3.9\x0cmultistorage\x071.2.18b\x06natura\x0f1.10.2-4.1.0.77\x0fnetherportalfix\x031.0\nopenglider\t@VERSION@\x0cp455w0rdslib\x061.0.13\x0charvestcraft\x071.10.2j\x0fprojectred-core\x084.8.4.49\x16projectred-integration\x084.8.4.49\x17projectred-transmission\x084.8.4.49\x17projectred-illumination\x084.8.4.49\x14projectred-expansion\x084.8.4.49\x19projectred-transportation\x084.8.4.49\x16projectred-exploration\x084.8.4.49\x0equantumstorage\x053.3.4\x0brangedpumps\x030.3\x16reborncore-mcmultipart\n2.13.6.142\x11thermalfoundation\x052.1.5\x10thermalexpansion\x055.1.9\x0fredstonearsenal\x052.1.3\x11refinedrelocation\t@VERSION@\x0erefinedstorage\x061.2.26\nxreliquary\x101.10.2-1.3.3.664\x07rftools\x046.13\x0erftoolscontrol\x051.6.8\nrftoolsdim\x045.04\x05roots\x050.208\x07Signals\x031.0\x0fsimplevoidworld\x071.1.0.4\x0esimpleretrogen\x074.0.1+8\x0esimplyjetpacks\x082.1.3.55\x0bSleepingBag\x051.4.0\x04snad\x111.10.2-1.6.09.20a\x0fsolarfluxreborn\x052.17r\x0bstevescarts\x082.2.0.86\x13storagedrawersextra\n1.10-1.1.0\nrscircuits\x051.0.4\x12supercraftingframe\x081.2.1.88\x0fthermaldynamics\x062.0.10\x12tinkertoolleveling\x181.10.2-1.0.1.DEV.f5def58\ttopaddons\r1.10.2-0.15.0\x0btorchmaster\x081.4.1.34\x0cTranslocator\x082.1.4.55\x10usefulnullifiers\x051.3.5\twanionlib\n1.10.2-1.3\x03wcg\x051.0.0\x03wct\x060.1.32\x04Woot\x0c1.10.2-1.4.1\x05wrcbe\x052.0.0\x04xnet\x051.4.0\x05yabba\x0c1.10.2-0.5.1\x07unidict\x0b1.10.2-2.9b'

@iceiix
Copy link
Owner Author

iceiix commented May 12, 2019

FTBPresentsSkyfactory3Server_3.0.15 ModList packet '\x02\xb7\x01\x03mcp\x049.19\x03FML\t8.0.99.99\x05Forge\x0c12.18.3.2477\x0eNotEnoughItems\t2.1.3.220\x12OpenComputers|Core\x071.6.2.7\x0bfoamfixcore\x057.7.4\x0eccl-entityhook\x031.0\rfenceoverhaul\x051.2.1\x11actuallyadditions\x0b1.10.2-r105\x0eCodeChickenLib\t2.5.9.283\x0fCodeChickenCore\t2.4.1.102\x0cEnderStorage\t2.2.1.106\x03JEI\n3.14.7.420\rOpenComputers\x071.6.2.7\x11actuallycomputers\x051.0.0\x07Baubles\x061.3.11\x0bextrautils2\x031.0\x10flyringbaublemod\x140.2.4_1.10.2-d71eb9e\tAppleCore\x052.2.2\rapple_shields\x050.1.7\tappleskin\x051.0.8\x0cautopackager\x061.6.4b\x03Psi\x07r1.0-42\nAutoRegLib\x051.0-2\nbaconators\x053.0.6\x0fcrafttweakerjei\x051.0.1\x0cMineTweaker3\x063.0.26\x04base\x0c1.10.2-1.4.0\x05bdlib\t1.12.4.24\x07bedbugs\t@VERSION@\x13betterbuilderswands\x060.11.1\x0cbiometweaker\x072.3.239\x0eBlockDispenser\x031.0\x08guideapi\t@VERSION@\nBloodMagic\x101.10.2-2.1.11-80\tbookshelf\t1.4.4.347\x07Botania\x08r1.9-341\x0cbrandonscore\x062.1.12\x06mantle\x101.10.2-1.1.5.205\ntconstruct\x0f1.10.2-2.6.5.10\x08ceramics\r1.10.2-1.1.6b\tChameleon\x0c1.10.2-2.3.0\x0bchancecubes\x101.10.2-3.0.1.207\x0bmcmultipart\x051.4.0\nCharsetLib\x050.3.5\x0cCharsetPipes\x050.3.5\rCharsetWrench\x050.3.5\x10ChestTransporter\x062.5.18\rChickenChunks\x082.2.0.52\x08chickens\x054.3.2\x06chisel\x12MC1.10.2-0.0.14.33\x0echiselsandbits\x0512.15\x08shadowmc\x053.6.1\tclipboard\t@VERSION@\x06clumps\x051.0.5\x0bcyclopscore\x060.10.9\x0ecolossalchests\x051.6.0\x12commoncapabilities\x051.3.3\x14cookingforblockheads\x064.2.45\x05Waila\x111.8.17-B31_1.10.2\tdarkutils\x0e1.1.8.finalrc2\x0bPTRModelLib\x051.0.0\x05props\x052.4.2\x0fdimensionalcake\x050.0.1\x11draconicevolution\x052.1.6\x0cDragonMounts\nr46-1.10.2\x0eStorageDrawers\r1.10.2-3.7.10\nbitdrawers\x040.36\nelevatorid\x051.3.0\tendercore\x141.10.2-0.4.1.66-beta\x07EnderIO\x0e1.10.2-3.1.193\x10energyconverters\x081.0.0.26\x11engineersworkshop\x0c1.3.6-1.10.2\x0bvalkyrielib\r1.10.2-0.10.6\x11environmentaltech\x0e1.10.2-0.10.6b\x07etlunar\x0e1.10.2-0.10.6a\x11exnihiloadscensio\x050.1.5\x0cexcompressum\x062.0.97\x14ExtraBitManipulation\x0c1.10.2-2.5.1\x08zerocore\x0e1.10.2-0.1.0.6\x0bbigreactors\x0f1.10.2-0.4.5.30\x0bfairylights\x052.1.2\x14farmingforblockheads\x061.1.16\x11flatcoloredblocks\x0bmc1.10-v4.4\tsonarcore\x053.3.0\x0cfluxnetworks\x051.2.8\x07foamfix\t@VERSION@\x08forgelin\x051.5.1\x04ftbl\x050.0.0\x04ftbu\x050.0.0\x07Funnels\x051.1.2\rGardenOfGlass\x08sqrt(-1)\radvgenerators\t0.9.20.23\ngravestone\x061.5.13\x04grue\x051.3.5\x04gyth\x072.0.0.2\nhammercore\x071.9.4.3\x08hatchery\x060.3.21\nheadcrumbs\x052.0.2\x0bhopperducts\x071.4.8.1\x08hydrogel\x030.9\x14immersiveengineering\x070.10-61\rLunatriusCore\x081.1.2.38\rInGameInfoXML\x082.8.1.89\x05mtlib\t@VERSION@\x10initialinventory\x051.0.1\x12integrateddynamics\x060.7.12\x11integratedtunnels\x051.2.5\x0finventorysorter\t0.11.0+47\rironbackpacks\r1.10.2-2.2.31\tironchest\x111.10.2-7.0.15.804\x0fyunomakegoodmap\x0f1.10.2-6.1.0.26\x19itsbecauseuhasnogoodspawn\x051.0.5\x0eItsRainingFood\x051.0.2\njourneymap\x0e1.10.2-5.5.0b4\tkleeslabs\x053.3.5\x08llibrary\x051.7.7\x10literalascension\x0e1.10.2-1.0.2.2\x05boots\x031.0\x08lootbags\x052.3.9\tmercurius\x051.0.6\x08minemenu\x051.4.5\x08minicoal\x051.1.0\x06natura\x0f1.10.2-4.1.0.80\x0emissing_pieces\x054.3.0\x0crandomthings\x073.7.7.1\x05roots\x050.208\tmoarsigns\x074.2.3.9\x12mob_grinding_utils\x080.1.8.22\nmodtweaker\x062.0.13\x0cmorechickens\x051.1.6\x05shear\x051.1.2\nmoreshears\x051.0.5\x08Morpheus\r1.10.2-3.1.13\x07mputils\x051.2.9\x07mpbasic\x051.1.3\x13mysticalagriculture\x051.5.8\x04nice\x050.1.0\x0enotenoughwands\x051.5.5\roreexcavation\x071.2.102\x0bpackingtape\x050.6.0\x0charvestcraft\r1.9.4-1.10.2g\x0eshetiphiancore\x063.3.11\tplatforms\x061.2.10\nreborncore\n2.13.6.142\x0equantumstorage\x053.3.6\x0brangedpumps\x030.3\x16reborncore-mcmultipart\n2.13.6.142\x0erefinedstorage\x061.2.26\rrebornstorage\x051.0.0\rredstonepaste\x051.7.4\x08reliquia\x050.0.7\x07rftools\x046.13\x0erftoolscontrol\x051.6.8\nrftoolsdim\x045.05\x0cwoodenshears\x1f@MAJOR@.@minor@.@revis@.@build@\x07scanner\x051.4.1\x0cshearmadness\x061.10.2\x12SimpleAchievements\x10MC1.9.4-1.2.2-28\rsimplebarrels\x051.26e\nsimplecorn\x052.2.0\x10simplegenerators\x0e1.10.2-0.10.6a\x0cSimpleLabels\x050.0.1\x0esimplyjetpacks\x082.1.3.55\x04snad\x111.10.2-1.6.09.20a\x0fsolarfluxreborn\x052.17r\x0bstevescarts\x082.2.0.86\nrscircuits\x051.0.4\x07beneath\x051.2.0\x12tinkertoolleveling\x181.10.2-1.0.1.DEV.f5def58\x02tp\x051.1.0\x0btorchmaster\x081.4.1.34\x0cTranslocator\x082.1.4.55\x06ts2k16\x051.0.3\x0ctwitchcrumbs\x052.0.4\x07voidcup\x031.0\x07waddles\x050.5.4\twanionlib\n1.10.2-1.3\x04icse\x071.1.0.0\x05wawla\t2.3.2.215\x0cwithercrumbs\t@version@\x07xprings\x051.3.0\x07unidict\x0b1.10.2-2.9b'

@iceiix iceiix changed the title 1.12.2/1.10.2 Forge: SevTech Ages/Skyfactory3 crashes parsing ModList: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check 1.12.2/1.10.2 Forge: SevTech Ages/Skyfactory3/Beyond crashes parsing ModList: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check May 12, 2019
@iceiix iceiix changed the title 1.12.2/1.10.2 Forge: SevTech Ages/Skyfactory3/Beyond crashes parsing ModList: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check 1.12.2/1.10.2 Forge: Beyond/Skyfactory3/SevTech Ages crashes: error: StringError("failed to fill whole buffer") / java.util.zip.DataFormatException: incorrect header check May 12, 2019
@iceiix
Copy link
Owner Author

iceiix commented May 12, 2019

The ModList packet right before crashing is actually sent from the client:

Sending plugin message: channel=FML|HS, data=[2, 183

if I abort before sending this message, no server error log. Likewise if I abort right after, then the server does log the "java.util.zip.DataFormatException: incorrect header check" error when inflating. So the client is confusing the server. SevTech: 5140 bytes, Beyond: 3823 bytes, Skyfactory: 3809 bytes. Is this beyond some threshold where the packet is supposed to be compressed? No mention of compression on https://wiki.vg/Minecraft_Forge_Handshake but it is only updated for 1.7-1.9, maybe there was a Forge or vanilla change in 1.10+?

https://github.com/MinecraftForge/MinecraftForge/commits/1.10.x

has some changes related to multipart packets (FML|MP), but those are only server to client, not client to server.

https://wiki.vg/Protocol#Packet_format - with and without compression
https://wiki.vg/Protocol#Set_Compression - compression threshold

Receiving from server:

packet = Some(SetInitialCompression(SetInitialCompression { threshold: 256 }))

This is handled and set in Conn compression.threshold, used in write_packet(), so it should be compressed. Uses flate2::read::{ZlibDecoder, ZlibEncoder}

@iceiix
Copy link
Owner Author

iceiix commented May 12, 2019

Using this crate: https://crates.io/crates/flate2 https://docs.rs/flate2/1.0.7/flate2/read/index.html
Could it actually be a bug in flate2-rs? I'm using features = ["rust_backend"], which has this caveat:

There is also an experimental rust backend that uses the miniz_oxide crate. This avoids the need to build C code, but hasn't gone through as much testing as the other backends.

Compressed bytes (2625) '\x81\x1e\x05@[o\xdcHvF"\xb3ojI\x96=;\xb3y\xcbC\xde\x02~\xa8Sd\xdf\x80 \x90\xad\x913ZX\xd6\xc42\x9c\xbc\x19\xc5\xe2aw\xa9\x8bU\x9cSE]\x92\xdd\x1f\xb5\xffp1\x9f|\xba\xfb\xfc\xd7\xdf\x1e\xfe\xf9\xef\xfft\xd2\xdb\xe1\xcd\x0e\xb4;\xf9t\xf7y\xbe\x85\xc2n\x87\xdd\xae\xf8\x14e\xcfK\xd2\xa0-*\xe8Z\xeb?\x9ba\xf0\x8e[\x0e,{\x97\xb2\xb3I\x976\n/\xe5\xa9.S6\x8d\xe7\x92\xce>H\xec\r\xedv\x9b\xeb(\xbc\xf8\xb7\xff\x7fbI.\x86\xbf\x9d\x7f\x89\xf9&\xc4q\x7f\xb8\xcd\xdc\xa7\xb9\x06\xa1\x82\xd6\xea\xdd\xfd\xc0\xe1:\xf6\xc3\x98Y\xd2_\xaf\xa3\xf0\x94\xb0\x86\xc6f\xf1\x1f\xd7\xf1\xd3o\xff\xfa\xe1\xe1\xee?O\x94R\xa7]4}\xe7^l\x14.6\xd8\xa0>\xb7\xd6\x97\x1c\xb2\xcb\xaf\x87\x18\x8f\'\x045\xb3\xb1;\xd8(<\xa9A \xba46\x8f\xc6\xfbW\xd3\xb6.\xbb\x18\xd2)\x81\x14t)\xa4V\'\xb7\xd7\xfaLc\r\xbd\xd2%\xbf\x10\xa9K\xd3>\x99`\xb9\xfd\xd1\x1b{p\x81\xd3d\xad\xa0P\xff\xc9\xb4O&Xn\x7f\xa4\xe8\x8d\xfc\x18L`\x9f\x8a\x1a\n\xf5{3\x0c\xdeq\xcb\x81e\xefRv6\xe9\xa5<\xd5e\xca\xa6\xf1\\R\xd1\xb4\xde5s\x02i\xd4\xd0\xf5\xcc\xb0Ny\xec\xba\xa9\xc2\x1a\n\xbbS\x17:\x17\\~\xf5\xae9#\x90\x82.\x15\x88\xa0\xe6f/\xce\x8a\xe9\xf2[\r\x05U*\x90\x82*\r\xed\xa6\x1f\xcd\xd8xN\x13B\x05\xa2\xe9\xc7\x98Mpf&\x84]Y\xd5t\xfa\xe1h\xd2\xc1\xd9o\xb1\xe7\x82\xa0\xca\xd5\xc9_nn\x17\x15\xa8\xc6\x065\xed\xe6f\x18<\xa7\xa3\x0b\x05Aasj\xfe\x18\x8d\x1d}\x1e\x85\x0bB\r\xba4b\x0f.\xb3\xcd\xa3\xb0\x15\xd3\xe5\x82\xb0A\xf5\xfe\x83\xc4\xde\xd0n\xb7\xb9\x8e\xc2\xbf\xb1\x1fX\xa6\x04\x02A\xbf\xff \xb17\xb4\xdbm\xd2\xaf\xae\xe7\x90\\\x0c\'\x045\xbf\x8b2\x1cr\x8c~BP%\xe9\x93\xdf\x93\x9b\nA\x95\xb5.\xfe{4r\x9c\n\x81\xca\x8dZ|\x18s\xfc\xca\xfb\xcf\xae)\x08\xaa\xd4\xd3\x86\xdbf\xdc\xa7\xf9\xd5\xf7\x9b\xaf\x0f\xb7\xf7_\xae\xde7\x9c3K3:\xdf\xb2\xa4g\x13\xda4Q \x02\x9d~t\x8dw\xf1ZL\x97\x0b\r\x8d\xd5l?\xba\x96\xcd\xe0\xe6W\xdfo\xbe>\xdc\xde\x7f\xb9Z|\xf41\xb6wf\xef\xec[\x02)\xe8R\x83@Tn\xd5\xb2\xf11\xb6F\x12\x07\xe3/\x08\xa4\xa0K\r\x82*\xb5\x9e71\x1e\xd3\x81}7\'\xd4\xa8Q\xd5\x9b\xf3\xeb\xd8\xf2\xf5\xc1\xd9#\x87\xcf\xae\x99k\xac\xb0\x83\xdeV\xcbFLhcH6\nO4\x08D\x93\xde\x84\xec\xf9-\x81\x14tI \xac\xa0\xd5j\x91m\x0c)\xcbh\xf3\x05\x81\x14t\xa9\xb1\xc6\n\xa4f\x96\xc5\xf4\xce\xa63\x02)\xe8\x92@X7\xf3\xeb\x83\xe9\xd9s\x0cK\x02)\xe8R\xa3\x82z{}\xe0\x94\xbf\x89\ti\x88\x92Y&\x1a+\xd0vb\x0f.\xb1\x7fwwM \x05]*(P\x85J\x9d\xf6\xb6\x1f}v\x83\x91\\\x10j\xa8s{p\x89}2\xa1m\\N\x05i\x90\x9e\xa5\x83i\xe3so\x8b\nk\xd0\xdcz74\xd1H;\xbf\xfa~\xf3\xf5\xe1\xf6\xfe\xcb\xd5\xc5ul\xf9\xfa\xe0\xec\x91\xc3u\x14\x9ek\xd4 \x90\xd2g6\xf6\x83\xb19Eo$]\x12HA\x97+(\x90FU\xad\x7f\xb21\x1e]\xd8wQ\x1a\x1f\xed\xf1\xc0\xa6M\x93\x1a\x1a\xf5\xea\xc2\x8a\xe9r~fsdydW\x10\x14hy\xe7\x02\x7f{fsd\xa9&\x15\x14\xf4z\xde\x1a9\x8e\xd9\xf9tN l\xd1\xb9`\xbcX}\xfa\xfb\xb7\xafw\xb1e\xff\xd95\x05AA\x15\x83\xc4!\x15\x1a5\xf4)\xbfd1cv>\xe9\x13\x82\x9auQ8ey]\xac\xa0A\x1bT\xdb\xd5\x8c=[\x1b\x85\xa7\x84\r\xea\x8a\x967\xa1ey\xc8Q\xcc\x9e\xe7\x1a\x1a\x04R\xeb\xb3\xfb\x81\xc3u\xec\x871\xb3\xa4)a\r\x8d\xcdY\xcb<\x08\xa7\x18L\xb0\\\x10jl/[16\x06g\xf9)\xfa1\xbb\x18\n\r\xc2\xea\xfc!G1{\xfeU\xcc3K:#\x90\x82.+l@j\xd1\xb8\xdc\x8ayfIo\x14\xaa\xf5\x82=?\x99\x1c\xc5\xb5\x05\xa1\x82\x9ap\xdf\xb0\xa4B\x81T=\xe7\xd0\xb2\xd8(\xfc\x13\x81\x14t\xa9P\x83\xb0^\x97\rg3\xbd\t-\xcb\xed\xfd9\x81\x14tY\x81@\xdb\xea\x92\xc3\xde\x05fI\xcfQ\x8e\xe9\x10\x87%\xa1\xc2\xba$\x90\x82>}2\xfe\xf8*\x8e\xbdk\xce\x08\xa4\xa0K\x05RX_rxr\x12C\xcf!\x1b\x9f\xd9\x1e\xce\t\xa4\xa0K\x05RX7S\xce~\x0cF\xce\t\xa4\xa0K\x05RX\x9b\xd9\xff\xb1D\x1b\x85\xcf\t\xa4\xa0K\x05\x82\xc2\xfa\xb4q{acs\x94tA \x05]*\xd4X\xa1R\x97\x9d7\xd9F\x1f\x85\xdb\xc6G{L\xa7\xbd%\x90*\x9fj\xd4\xf3\x14\x83\x11\x1b\x85\x8b\n\x1a\xdbe\xe7\xc7\x97\xc0\xf99\xca1\x15\x04\x8d\xd5\xb4\x8b\xa6\xef\xdc\xcb\xfc\xea\xfb\xcd\xd7\x87\xdb\xfb/W\xb3.\xca\x9e\xbd\x0b\x05a\x05\xba\xec\xa2\xec\xb9\x1f}v\x83\x91l\x1b\x9ei((T\xdb\xb3\xdeY\x89\x8d\x8f\xf6h\x1b\x9ei((T\xdb?\xf5.\xb0\x15\xd3\xe5~\xf4\xd9\rF\xb2mx\xa6\xa1\xa0Pm\xdft\xb9\xf1\x85\x82\x82z\xd3\xe5f,\x14\x14\xd4E7\x86\xe3\xab\x8f6\xf61\xbb\x18N\x08j\xfai\x0c\x81}*\x08\x04=\xdfsh\xc7\x94\xe5uFXc\x85\x8a\x16{1O\x9cr\x0c\n7\xcc\xe9\x9c@\n\xba$\xd4 \xd0\xec.\xcap\xe01\x9d\x11HA\x97\x15\x08TM\xfba\xcc\xce\xa7\x82\xa0\xb1\x9b\xf6Cc\x92\xb3\x05\x81P-\xef\xe4\xdb_~\xbf\x8e\xc2w\xb1\x9di(\x10\xb4:O\x07\xcen88\x13l\x14.*T\xd8-\xfb\xd1g\x97r\x14\xb3\xe7)A\x83\xb6\xcd$\x98<\x8a\xb9 \x90\x82.k\x10\x146\x9b\x8b\xc0\xf9\xc02D\xc9\xc6w\xee\xe5\x84\xa0\x16q\xe0\xb0\xf7\xaee\x99_}\xbf\xf9\xfap{\xff\xe5j9\xd4\xab\xd5\xb3\x926y\xd7L\x08\nT-\x0fF\x9e8e+\xa6\xcbS\x02)\xe8\xc7\x8bA\xe2#\xdb,\xdc\x966\n\xcfjlQ\xa3\xde\xfd\xb2\xcd\xc2m\x99\xc5\x844D\xc9&\xbb\x18f5\xb6\xa8Q\xef~\x1e$>\xb2\xcd\xc2m\xc9/\x83\x8fb\xb2\x8baVc\x8b\x1a\xf5\xee\xfc\x8f\xd1\x84<\xf6)G1{.*T\xa8O\xc5\x84=\xb7\xc3\xd8\x0f\xe9D\xa1\xfaY\xb8\x89\x12l\x14.{\xdb\x8f>\xbb\xc1H^hP\x855\xa8\xd6\x97\xf9\xc0\xd2\x1b\xdf\xc51\xb4&\xbb\x18\n\r\xc2\xeam>\xb0\xf4\xc6\xf3\xcb`Br1\x14+\x10v\x17\xc2m\xca1\xb0\x91\xc4\xc1\xf8B\x83P]\nw.p+\xec\xa35\xd9\xc50\xbf\xfa~\xf3\xf5\xe1\xf6\xfe\xcb\xd5\xb9p\xe7\x02\xb7)G1{\x9e\x104\xf4z\xf1"\xec\xdd\x1f\xa3\x91\xd7\xb7\x04R\xd0%\xa1B\x85\xf5\xba\x9eJ\x97c\xf4\xe9\xcd\x1aT\x9dK\x97c\xf4\xc9\xc6\x90%\xfa\x82\xb0\xc6v!]\x8e\xd1\xa7\xd6\xf5oVPu!1\xe6T(h\xb5\x9d>\xb8}0>\x9d\x10\xd4Er\xfd\xe0\xf9)\xba\xf69\x8ao\xa7\x04\x82B}\x9e\\?x\x16\xce\x12\xf7\x1c\xa65\x14\xe8\xdf\xb7\xe7\xc9\xf5\x83\x7f}\xe4<\x18{L3\rB\x85\xd5\xea\xf4\xc13\x0f.\xec?\x9a}A\xa8\xa1\xde\xa4`\xdaK\x02)\xe8\x92\xb0\x86\xdaA+s\x91\xa27\xd2\xf9\xf1E\xb8\x89\x12\n\r\xda\xc8i\xca\xfc\xc4\xc9\x1a\xc9i\xa6\xa1\xa1\xb0]\xbfO9\x8a\xd9s+\xe6\x99%\xf1K\x16\xb3 \x90*\t\x04\xb5\x90d\x9d\xd8\xd1\xe5T\x10\x14\xeawi\x1cX\xac\x98.\xbb\xb0\xef\xc4\xf4<#h\x10\xb6\xdb\x8b|`\xe9\x8do_\x83\xe9\x9dM\x13\r\x05R\xef\xb2\x0bG\x96\x1c\xa3\xf7\xfc\xc4\xde\x85\xfd\x9f\t\xa4\xa0K\x82\x02\xe1\xd7\x9b\xef\xe8V-w\xab\xed<\xc7\xc1\xb4m\x0c\xe9\x8c@\n\xbaT\xa0\x15\xd4i\x8eb\x0f\xbdI\x99eF\xa8A\xa8\xea\xe571!\xf9hM\x8e2\xd3 \xd4X\xad\xde\x8e\x89\xbb\xd1\x87\xd1{\xd79\x96T\x10*\xac\xe6\xcf&\xb8\x18\xbck\x16\x04R\xd0%\xa1:y\xb6\xfb\x82\xa0\xa0N\x9em\x9e(\x10*\xfd\xe6\x7fb\xccK\x02)\xe8\x92P\x83\x8ag\xb1\r\x17\x1a\n\xea\xcdK\xe0\\\x10j\xa8\xe2\xd54\x8dY\x12HA\x97\n+\xd0t\x0c\xaeu6\x9f\x12HA\x97\x1a\xbb\xe6\x1f'

Trying to decompress with Python:

import zlib
zlib.decompress(file("/tmp/c").read())
# zlib.error: Error -3 while decompressing data: incorrect header check

and even with flate2's decompression, right after compression:

--- a/src/protocol/mod.rs
+++ b/src/protocol/mod.rs
@@ -1016,6 +1016,7 @@ impl Conn {
             0
         };
         if self.compression_threshold >= 0 && buf.len() as i32 > self.compression_threshold {
+            println!("Compressing since threshold={} > len={}", self.compression_threshold, buf.len());
             if self.compression_write.is_none() {
                 self.compression_write = Some(ZlibEncoder::new(io::Cursor::new(Vec::new()), Compression::default()));
             }
@@ -1026,6 +1027,15 @@ impl Conn {
             let write = self.compression_write.as_mut().unwrap();
             write.reset(io::Cursor::new(buf));
             write.read_to_end(&mut new)?;
+            println!("Compressed bytes={:?}", new);
+            println!("About to decompress");
+            {
+                let mut reader = ZlibDecoder::new(&new[..]);
+                let mut check = Vec::new();
+                reader.read_to_end(&mut check).unwrap();
+                println!("Decompressed bytes={:?}", check);
+            }
+
             buf = new;
         }

it also crashes: About to decompress
thread 'main' panicked at 'called Result::unwrap() on an Err value: Custom { kind: InvalidInput, error: StringError("corrupt deflate stream") }', src/libcore/result.rs:997:5

Using the default miniz-sys backend:

--- a/Cargo.toml
+++ b/Cargo.toml
@@ -25,7 +25,7 @@ glutin = { git = "https://github.com/iceiix/glutin", rev = "de52fba20678f3931fba
 byteorder = "1.2.7"
 serde = "1.0.91"
 serde_json = "1.0.39"
-flate2 = { version = "1.0.7", features = ["rust_backend"], default-features = false }
+flate2 = { version = "1.0.7" }
 zip = { version = "0.5.2", features = ["deflate"], default-features = false }
 image = "0.21.1"
 rand = "0.6.5"

same result. And the zlib backend, which uses libz-sys:

flate2 = { version = "1.0", features = ["zlib"], default-features = false }

also the same, StringError("corrupt deflate stream"). hmm...

@iceiix
Copy link
Owner Author

iceiix commented May 12, 2019

Testing with flate2 on its own, able to compress and decompress correctly...but not when using cursors:

use std::io::{self, Read};
use flate2::{Compression, read::{ZlibDecoder, ZlibEncoder}};

const USE_CURSOR: bool = false;

fn test(input: &[u8]) {
    let mut compressed = Vec::new();

    if USE_CURSOR {
        let mut encoder = ZlibEncoder::new(io::Cursor::new(Vec::new()), Compression::default());
        encoder.reset(io::Cursor::new(input.to_vec()));
        encoder.read_to_end(&mut compressed).unwrap();
    } else {
        let mut encoder = ZlibEncoder::new(input, Compression::default());
        encoder.read_to_end(&mut compressed).unwrap();
    }
    println!("compressed = {:?}", compressed);

    let compressed = compressed.as_slice();
    let mut decoder = ZlibDecoder::new(compressed);
    let mut decompressed = Vec::new();
    decoder.read_to_end(&mut decompressed).unwrap();

    println!("decompressed = {:?}", decompressed);
    let decompressed = decompressed.as_slice();

    assert_eq!(input, decompressed);
}

fn main() {
    test(vec![1, 2, 3].as_slice());
}

The encoder is initialized once, lazily, with an io::Cursor to an empty buffer. The encoder is then reset to a cursor of the buffer to compress. This causes different results, a corrupt deflate stream, versus initializing the encoder with the input buffer itself. Does compression need a running stream like encryption, or can the encoder be reconstructed on each use? Why does initializing with an empty buffer and resetting produce different results than initializing with the data? It is possible this was always broken, testing on a few server configurations, this code path was not hit.

@iceiix
Copy link
Owner Author

iceiix commented May 12, 2019

Changing to initialize with the input, each time, able to connect to FTB Beyond:

Screen Shot 2019-05-12 at 10 43 00 AM

but are there other consequences? How many times is encryption used, need to confirm additional state is not needed for multiple packets. If not, could remove the compression_write (and compression_read?) member variables entirely. Is compression_read broken too? Uses same io::Cursor technique. Is it possible to stick with cursors but fix the decompression/compression?

@iceiix
Copy link
Owner Author

iceiix commented May 12, 2019

Compressing the &[u8]:

compressed = [120, 1, 99, 100, 98, 6, 0, 0, 13, 0, 7]
decompressed = [1, 2, 3]

Compressing the cursor... produces correct results if initializing with the data, but not if initializing with empty then resetting - seems to be missing header/trailer?

compressed = [99, 100, 98, 6, 0]

New reduced test case:

use std::io::{self, Read};
use flate2::{Compression, read::{ZlibDecoder, ZlibEncoder}};

const USE_RESET: bool = true;

fn test(input: &[u8]) {
    let mut compressed = Vec::new();

    if USE_RESET {
        let mut encoder = ZlibEncoder::new(io::Cursor::new(Vec::new()), Compression::default());
        let mut empty = Vec::new();
        encoder.read_to_end(&mut empty).unwrap();
        println!("read empty = {:?}", empty);
        println!("reset = {:?}", encoder.reset(io::Cursor::new(input.to_vec())));
        encoder.read_to_end(&mut compressed).unwrap();
    } else {
        let mut encoder = ZlibEncoder::new(io::Cursor::new(input.to_vec()), Compression::default());
        encoder.read_to_end(&mut compressed).unwrap();
    }
    println!("compressed = {:?}", compressed);

    let compressed = compressed.as_slice();
    let mut decoder = ZlibDecoder::new(compressed);
    let mut decompressed = Vec::new();
    decoder.read_to_end(&mut decompressed).unwrap();

    println!("decompressed = {:?}", decompressed);
    let decompressed = decompressed.as_slice();

    assert_eq!(input, decompressed);
}

fn main() {
    test(vec![1, 2, 3].as_slice());
}

https://docs.rs/flate2/1.0.7/flate2/read/struct.ZlibEncoder.html

Resets the state of this encoder entirely, swapping out the input stream for another.

This function will reset the internal state of this encoder and replace the input stream with the one provided, returning the previous input stream. Future data read from this encoder will be the compressed version of r's data.

Note that there may be currently buffered data when this function is called, and in that case the buffered data is discarded.

Reading to the end of the encoder after creating it, but before resetting, shows it has data:

read empty = [120, 1, 3, 0, 0, 0, 0, 1]
reset = Cursor { inner: [], pos: 0 }
compressed = [99, 100, 98, 6, 0]

This is the missing header we seek. But it wouldn't be available if the encoder is lazily initialized. What was the point of the compression_write member variable anyways, it was present very early in the project, among the first commits: 6f6d3c96 Main part of the protocol complete, with a curious comment:

    compression_read: Option<ZlibDecoder<io::Cursor<Vec<u8>>>>, // Pending reset support
    compression_write: Option<ZlibEncoder<io::Cursor<Vec<u8>>>>,

The "// Pending reset support" comment was removed in 8a2c247 Minor changes, maybe reset never worked as expected here, since the headers are missing... although, I did update flate2 myself in #4 fd2c51ea

@iceiix
Copy link
Owner Author

iceiix commented May 12, 2019

FTBPresentsSkyfactory3Server_3.0.15 (1.10.2, top 1 on FTB):

Screen Shot 2019-05-12 at 11 56 15 AM

FTBBeyondServer_1.11.0 (1.10.2, top 3 on FTB):

Screen Shot 2019-05-12 at 10 43 00 AM

SevTech_Ages_Server_3.1.1 (1.12.2, top 3 on Curseforge)
#148 1.12.2 Forge: SevTech Ages thebetweenlands crashes StringError("stream did not contain valid UTF-8")

@iceiix iceiix closed this as completed in b98ba3a May 12, 2019
iceiix added a commit that referenced this issue Feb 1, 2020
Previously, the zlib compressor was initialized once, lazily, then reused for each packet by resetting the stream. This didn't properly emit the zlib headers, so packet compression was broken and caused "java.util.zip.DataFormatException: incorrect header check" on the server. Since packets are only compressed above a threshold, this problem manifested itself only on larger servers, such as 1.10.2 FTB Beyond and Skyfactory 3, and 1.12.2 SevTech: Ages, which require sending a large ModList above the compression threshold enabled by the server. Change to instead recreate the zlib compressor, each time it is used as to capture the full header. For symmetry, the decompressor is also recreated each time.

* Removes self.compression_write and self.compression_read instance variables

* Remove self.compression_write, initialize with cursor

* Log compressing/decompressing packets in network debug mode
iceiix added a commit that referenced this issue Feb 1, 2020
Previously, the zlib compressor was initialized once, lazily, then reused for each packet by resetting the stream. This didn't properly emit the zlib headers, so packet compression was broken and caused "java.util.zip.DataFormatException: incorrect header check" on the server. Since packets are only compressed above a threshold, this problem manifested itself only on larger servers, such as 1.10.2 FTB Beyond and Skyfactory 3, and 1.12.2 SevTech: Ages, which require sending a large ModList above the compression threshold enabled by the server. Change to instead recreate the zlib compressor, each time it is used as to capture the full header. For symmetry, the decompressor is also recreated each time.

* Removes self.compression_write and self.compression_read instance variables

* Remove self.compression_write, initialize with cursor

* Log compressing/decompressing packets in network debug mode
iceiix added a commit that referenced this issue Feb 1, 2020
Previously, the zlib compressor was initialized once, lazily, then reused for each packet by resetting the stream. This didn't properly emit the zlib headers, so packet compression was broken and caused "java.util.zip.DataFormatException: incorrect header check" on the server. Since packets are only compressed above a threshold, this problem manifested itself only on larger servers, such as 1.10.2 FTB Beyond and Skyfactory 3, and 1.12.2 SevTech: Ages, which require sending a large ModList above the compression threshold enabled by the server. Change to instead recreate the zlib compressor, each time it is used as to capture the full header. For symmetry, the decompressor is also recreated each time.

* Removes self.compression_write and self.compression_read instance variables

* Remove self.compression_write, initialize with cursor

* Log compressing/decompressing packets in network debug mode
iceiix added a commit that referenced this issue Feb 2, 2020
Previously, the zlib compressor was initialized once, lazily, then reused for each packet by resetting the stream. This didn't properly emit the zlib headers, so packet compression was broken and caused "java.util.zip.DataFormatException: incorrect header check" on the server. Since packets are only compressed above a threshold, this problem manifested itself only on larger servers, such as 1.10.2 FTB Beyond and Skyfactory 3, and 1.12.2 SevTech: Ages, which require sending a large ModList above the compression threshold enabled by the server. Change to instead recreate the zlib compressor, each time it is used as to capture the full header. For symmetry, the decompressor is also recreated each time.

* Removes self.compression_write and self.compression_read instance variables

* Remove self.compression_write, initialize with cursor

* Log compressing/decompressing packets in network debug mode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant