Support both 1.8 and 1.9 chunk format

[rom1504]

http://wiki.vg/SMP_Map_Format#Format

http://wiki.vg/Pre-release_protocol#Chunk_Section

PrismarineJS/node-minecraft-protocol#231 (comment)

This will help mineflayer and flying-squid update to cross-version (1.8 + 1.9)

[rom1504]

this will be useful https://gist.github.com/Gjum/0375b643ec13a42ab3c0#file-chunk_-5_5-dump

[rom1504]

see https://github.com/SpockBotMC/SpockBot/pull/218/files

[Gjum]

C implementation https://github.com/JavaProphet/MinecraftClone/blob/245fca26f1911ff87c04ac774079acb60546e78a/MinecraftClone/src/ingame.c#L201

[rom1504]

https://github.com/Steveice10/MCProtocolLib/blob/4ed72deb75f2acb0a81d641717b7b8074730f701/src/main/java/org/spacehq/mc/protocol/data/game/chunk/BlockStorage.java

[rom1504]

@mhsjlw has started working on that at https://github.com/mhsjlw/prismarine-chunk

[louwers]

Did the chunk format change in 1.10?

[ghost]

Nope, not yet 😄. I doubt the chunk format will change again for a very, very long time as this is probably the most optimized chunk format we've seen (hence how complicated it is, especially in JavaScript).

[louwers]

Here is another implementation in C: https://github.com/broese/mcbuild/blob/master/mcp_packet.c#L625

@mhsjlw This kind of stuff looks quite odd in Javascript, but why is it more difficult? Hard to get good performance?

[ghost]

No, JavaScript doesn't natively support longs which the 1.9 chunk relies on, we've tried ways around it that seem to be successful, but we're not anywhere close.

EDIT: For it to be a proper prismarine-chunk implementation we need to be able to read AND write and load AND dump

[louwers]

What are still currently problems with your implementation?

[rom1504]

http://lunarco.de/minecraft/chunks/

[rom1504]

or https://download.rom1504.fr/minecraft/chunks/

[rom1504]

WIP there https://gist.github.com/rom1504/2d6451dbea299099b861ac6acef6ef7d

[reverofevil]

@rom1504 I don't understand the issue with longs at all. It's getting split into variable length bit fields anyway, there's no difference in the underlying datatype except that count always gets multiplied by 4. To circumvent the issues with computing the size, it's handy to create a structure with two u32 fields.

[Gjum]

The problem lies in accessing the data on these longs. Each entry is offset by some number of bits and can be split across two longs in a nontrivial way: http://wiki.vg/SMP_Map_Format#Example

[reverofevil]

@Gjum And what is the problem?

http://ideone.com/sf0cAR

[rom1504]

@polkovnikov-ph the problem is nobody finished a full implementation for prismarine-chunk 1.9

• mhsjw has started one at https://github.com/mhsjlw/prismarine-chunk but it's not quite working
• so I started a new one recently from scratch, current code is at https://gist.github.com/rom1504/2d6451dbea299099b861ac6acef6ef7d
  but I don't have a lot of time to work on this these days so it's not done yet

I you feel like implementing it, go ahead !

[rom1504]

last progress on this https://github.com/roblabla/prismarine-chunk/commits/master

This issue is 8 months old, might be time to solve it... (blocking mineflayer and flying-squid for minecraft >=1.9)

[GeorgesOatesLarsen]

RE: The WIP you did. I had a go at completing it:

https://gist.github.com/Flynnn/035c17f67158154220db7971e34314b2

Seems to work! It converts the new 1.9 raw chunk data back to the format being read by the 1.8 version of prismarine-chunk. Not the most ideal solution, but it appears to work. Histogram reports 76% air, 14% stone, 6% water, 1% bedrock, and less than 1% of dirt, lava, gravel, gold ore, iron ore, coal ore, and redstone ore.

I'm gonna go ahead and integrate this into a fork of prismarine-chunk.

EDIT:
I should note that I did not check the block-order -- I assumed them to be the same. There's a possibility that my code is loading chunks upside down, backwards, or something similar. I'm not really sure how to test this effectively, though. My cursory reading on the protocols seemed to indicate, though, that the orders are the same, so I doubt this is a problem

[ghost]

👍 Good luck! Keep us updated

[GeorgesOatesLarsen]

AFAICT this should be good: https://github.com/Flynnn/prismarine-chunk But I'd like to wait until someone is able to really test it before doing a pull request.

Notably, a new argument is required for Chunk.Load -- no way around this, except, perhaps, having the argument packed in with the data. Perhaps that would be preferable?

[reverofevil]

@Flynnn Great job! I'll try it right now.

[rom1504]

finally done thanks to @Flynnn

[GeorgesOatesLarsen]

Most excellent! Thanks for all of the help, @rom1504, @mhsjlw