1.16.4 (754) / 1.16.3 (753) / 1.16.2 (751) protocol

[iceiix]

https://minecraft.gamepedia.com/Java_Edition_1.16.2

1.16.4 (754): https://wiki.vg/Protocol — diff 1.16.4 from 1.16.3: https://wiki.vg/index.php?title=Protocol&type=revision&diff=16144&oldid=16091 (no significant changes?)
1.16.3 (753): https://wiki.vg/index.php?title=Protocol&oldid=16091 (version number change only)
1.16.2 (751): https://wiki.vg/index.php?title=Pre-release_protocol&oldid=16001
1.16.1 (736): https://wiki.vg/index.php?title=Pre-release_protocol&oldid=15895
diff 1.16.2 from 1.16.1: https://wiki.vg/index.php?title=Pre-release_protocol&type=revision&diff=16001&oldid=15895

[iceiix]

Lots of packet changes, not yet usable. https://wiki.vg/Pre-release_protocol#Chunk_Data has a new format:

New format, 5 bits per block, containing the following references to blocks in a palette (not shown): 122344566480743131516914101202

0020863148418841 0000000000100000100001100011000101001000010000011000100001000001
01018A7260F68C87 0000000100000001100010100111001001100000111101101000110010000111

[iceiix]

1.16.3 (753) is now out, but according to https://wiki.vg/index.php?title=Pre-release_protocol&action=history, "1.16.3, no changes other than PVN" = protocol version number is the only change. Add both, using same class.

[Titaniumtown]

Is this going to be worked on further? Would like to test out stevenarella on my 1.16.3 server.

[iceiix]

1.16.4 (754) is out now, falling behind. References:

https://wiki.vg/Protocol#Chunk_Data
https://github.com/PrismarineJS/mineflayer/pull/1349/files
https://github.com/PrismarineJS/prismarine-chunk/pull/91/files

22:25 let's say you have bitPerValue = 5, you can fit 12 x 5 bits values, and you'll have 4 bit remaining
22:25 in the old version, you would store the first 4 bit of the value in the same long, and the remaining 1 bit in the next
22:25 in 1.16, you store the 5 bits in the next, and you leave 4 bits of padding

diff -ur 1.15/ChunkColumn.js 1.16/ChunkColumn.js
--- 1.15/ChunkColumn.js	2020-11-22 10:25:27.000000000 -0800
+++ 1.16/ChunkColumn.js	2020-11-22 10:25:27.000000000 -0800
@@ -2,8 +2,7 @@
 const ChunkSection = require('./ChunkSection')
 const constants = require('../common/constants')
 const varInt = require('../common/varInt')
-
-const BitArray = require('../common/BitArray')
+const BitArray = require('../common/BitArrayNoSpan')
 
 // wrap with func to provide version specific Block
 module.exports = (Block, mcData) => {
diff -ur 1.15/ChunkSection.js 1.16/ChunkSection.js
--- 1.15/ChunkSection.js	2020-11-22 10:25:27.000000000 -0800
+++ 1.16/ChunkSection.js	2020-11-22 10:25:27.000000000 -0800
@@ -1,4 +1,4 @@
-const BitArray = require('../common/BitArray')
+const BitArray = require('../common/BitArrayNoSpan')
 const neededBits = require('../common/neededBits')
 const constants = require('../common/constants')
 const varInt = require('../common/varInt')

only change is BitArray to BitArrayNoSpan https://github.com/PrismarineJS/prismarine-chunk/blob/master/src/pc/common/BitArrayNoSpan.js

[iceiix]

https://wiki.vg/Pre-release_protocol#Chunk_Data

This line in src/world/mod.rs load_chunk19_or_115:

                let m = bit::Map::from_raw(bits, bit_size as usize);

mapped in https://github.com/iceiix/stevenarella/blob/master/protocol/src/types/bit/map.rs - this is what allows the spanning. Would need a modified bit map type to not span longs (compare https://github.com/PrismarineJS/prismarine-chunk/blob/master/src/pc/common/BitArrayNoSpan.js vs https://github.com/PrismarineJS/prismarine-chunk/blob/master/src/pc/common/BitArray.js)

[iceiix]

Prototyping tweaking the algorithm based on the example data at https://wiki.vg/Protocol#Chunk_Data. Porting the old bit slicing code:

# https://wiki.vg/Protocol#Chunk_Data
bits=[0x0020863148418841, 0x01018A7260F68C87]

class S(object):
    def __init__(self):
        self.bit_size = 5
        self.bits = bits

def get(i):
    self = S()
    i = i * self.bit_size; 
    pos = i / 64;
    mask = (1 << self.bit_size) - 1;
    ii = i % 64; 
    pos2 = (i + self.bit_size - 1) / 64;
    if pos2 == pos:
        return ((self.bits[pos] >> ii) & mask)
    else:
        used = 64 - ii;
        return (((self.bits[pos] >> ii) | (self.bits[pos2] << used)) & mask)

def seps(e, size):
    e = list(e)
    for i in range(len(e))[::-size][1:]:
        e.insert(i+1," ")
    result = "".join(e)
    return result

print seps("{0:064b}".format(bits[0]), 5)
print seps("{0:064b}".format(bits[1]), 5)

for i in range(0,25):
    print i, "=", "{0:05b}".format(get(i)), "=", get(i)

It should be: 122344566480743131516914101202, but after i=12 crosses the long, causing the difference in this behavior.

Here's how another project https://github.com/adepierre/Botcraft/blob/6b25c8b90f3a129265a370723a556f2944248dcf/botcraft/src/Game/World/Chunk.cpp handled the difference:

#if PROTOCOL_VERSION > 712
                        // From protocol version 713, the compacted array format has been adjusted so that
                        //individual entries no longer span across multiple longs
                        if (64 - (bit_offset % 64) < bits_per_block)
                        {
                            bit_offset += 64 - (bit_offset % 64);
                        }
                        int start_long_index = bit_offset / 64;
                        int end_long_index = start_long_index;
                        int start_offset = bit_offset % 64;
                        bit_offset += bits_per_block;
#else
                        int block_index = (((block_y * SECTION_HEIGHT) + block_z) * CHUNK_WIDTH) + block_x;
                        int start_long_index = (block_index * bits_per_block) / 64;
                        int start_offset = (block_index * bits_per_block) % 64;
                        int end_long_index = ((block_index + 1) * bits_per_block - 1) / 64;
#endif

but they are calculating a running offset, bit_offset, instead of indexing for bi in 0..4096 { ... let id = m.get(bi)

[iceiix]

Tweaking the prototype, adding compensation for padding: 4 bits for every 64, gets the correct results:

# https://wiki.vg/Protocol#Chunk_Data
bits=[0x0020863148418841, 0x01018A7260F68C87]

class S(object):
    def __init__(self):
        self.bit_size = 5
        self.bits = bits

def get_old(i):
    self = S()
    i = i * self.bit_size; 
    pos = i / 64;
    mask = (1 << self.bit_size) - 1;
    ii = i % 64; 
    pos2 = (i + self.bit_size - 1) / 64;
    if pos2 == pos:
        return ((self.bits[pos] >> ii) & mask)
    else:
        used = 64 - ii;
        return (((self.bits[pos] >> ii) | (self.bits[pos2] << used)) & mask)

def get(i):
    self = S()
    padding = i / (64 / self.bit_size) * (64 % self.bit_size);  # ADDED
    i = i * self.bit_size + padding;                            # ADDED
    pos = i / 64;
    mask = (1 << self.bit_size) - 1;
    ii = i % 64; 
    pos2 = (i + self.bit_size - 1) / 64;
    if pos2 == pos:
        return ((self.bits[pos] >> ii) & mask)
    else:
        used = 64 - ii;
        return (((self.bits[pos] >> ii) | (self.bits[pos2] << used)) & mask)

def seps(e, size):
    e = list(e)
    for i in range(len(e))[::-size][1:]:
        e.insert(i+1," ")
    result = "".join(e)
    return result

print seps("{0:064b}".format(bits[0]), 5)
print seps("{0:064b}".format(bits[1]), 5)

for i in range(0,24):
    if i==12:print
    print i, "=", "{0:05b}".format(get(i)), "=", get(i)

output:

0000 00000 01000 00100 00110 00110 00101 00100 00100 00011 00010 00010 00001
0000 00010 00000 01100 01010 01110 01001 10000 01111 01101 00011 00100 00111
0 = 00001 = 1
1 = 00010 = 2
2 = 00010 = 2
3 = 00011 = 3
4 = 00100 = 4
5 = 00100 = 4
6 = 00101 = 5
7 = 00110 = 6
8 = 00110 = 6
9 = 00100 = 4
10 = 01000 = 8
11 = 00000 = 0

12 = 00111 = 7
13 = 00100 = 4
14 = 00011 = 3
15 = 01101 = 13
16 = 01111 = 15
17 = 10000 = 16
18 = 01001 = 9
19 = 01110 = 14
20 = 01010 = 10
21 = 01100 = 12
22 = 00000 = 0
23 = 00010 = 2

now to implement and test

[iceiix]

Added support for the new chunk format, logging in no longer crashes, but the world does not render (there may be other missing changes or bugs).

[iceiix]

Finally now works well enough to connect and view the world (tested with a 1.16.4, but essentially same protocol for 1.16.3 and 1.16.2):

[iceiix]

Tested with 1.16.4, 16.16.3, 1.16.2, able to connect
@Titaniumtown you can now try this with your 1.16.3 (though there may still be bugs or glitches, let me know)