Add metadata variation to items.json and blocks.json

[rom1504]

We need to be able to know things (like the metadata id) about variation of blocks, like http://minecraft.gamepedia.com/Polished_Granite is a variation of http://minecraft.gamepedia.com/Granite

blocks.json currently looks like that :

"1": {
    "id": 1,
    "displayName": "Stone",
    "name": "stone",
    "hardness": 1.5,
    "stackSize": 64,
    "diggable": true,
    "boundingBox": "block",
    "material": "rock",
    "harvestTools": {
      "270": true,
      "274": true,
      "257": true,
      "278": true,
      "285": true
    }
  }

I propose a schema like that instead :

"1": [{
    "id": 1,
    "displayName": "Stone",
    "name": "stone",
    "hardness": 1.5,
    "stackSize": 64,
    "diggable": true,
    "boundingBox": "block",
    "material": "rock",
    "harvestTools": {
      "270": true,
      "274": true,
      "257": true,
      "278": true,
      "285": true
    }
  },
{
    "id": 1,
    "displayName": "Polished Stone",
    "name": "polishedStone",
    "hardness": 1.5,
    "stackSize": 64,
    "diggable": true,
    "boundingBox": "block",
    "material": "rock",
    "harvestTools": {
      "270": true,
      "274": true,
      "257": true,
      "278": true,
      "285": true
    }
  },

]

or similar : maybe some properties never change between two blocks only different by metadata ?

[Gjum]

When de-duplicating properties shared by blocks with different metadata, reading the file probably becomes more difficult. Also when generating these files, all duplicated changes will be updated anyway.
+1 for duplication.

[thejoshwolfe]

It seems like we should not index items by item id, as counter intuitive as that sounds. It sounds like perhaps the name (e.g. "polishedStone") is the true unique identifier. We will certainly need a way to get from an item id and possibly some metadata to an enum object, but since the mapping is complicated, that should probably be a function, not an {} object.

[rom1504]

Recipe.find (https://github.com/andrewrk/mineflayer/blob/master/lib/recipe.js#L19) deals with that same problem by looping over the array. I guess that's a bit inefficient.
An other way to solve that issue would be something like that :

"1": { "0":{
    "id": 1,
    "displayName": "Stone",
    "metadata":0,
    "name": "stone",
    "hardness": 1.5,
    "stackSize": 64,
    "diggable": true,
    "boundingBox": "block",
    "material": "rock",
    "harvestTools": {
      "270": true,
      "274": true,
      "257": true,
      "278": true,
      "285": true
    }
  },
"1":{
    "id": 1,
    "displayName": "Polished Stone",
    "metadata":1,
    "name": "polishedStone",
    "hardness": 1.5,
    "stackSize": 64,
    "diggable": true,
    "boundingBox": "block",
    "material": "rock",
    "harvestTools": {
      "270": true,
      "274": true,
      "257": true,
      "278": true,
      "285": true
    }
  },

}

I don't think indexing by the name is a good solution because we often needs to find a block knowing only its id and metadata (see https://github.com/andrewrk/mineflayer/blob/master/lib/item.js#L15 , https://github.com/andrewrk/mineflayer/blob/master/lib/item.js#L50 and https://github.com/andrewrk/mineflayer/blob/master/lib/plugins/inventory.js#L392 ) : some packets only send the id and metadata (https://github.com/PrismarineJS/node-minecraft-protocol/blob/master/src/protocol.js#L1120)

also see http://wiki.vg/Slot_Data (item damage == metadata)

[rom1504]

Assuming everything but the name, displayName and metadata value are different we could also do that :

{
  "1": {
    "id": 1,
    "hardness": 1.5,
    "stackSize": 64,
    "diggable": true,
    "boundingBox": "block",
    "material": "rock",
    "harvestTools": {
      "270": true,
      "274": true,
      "257": true,
      "278": true,
      "285": true
    },
    "variations": {
      "0": {
        "displayName": "Stone",
        "metadata": 0,
        "name": "stone"
      },
      "1": {
        "displayName": "Polished Stone",
        "metadata": 1,
        "name": "polishedStone"
      }
    }
  }
}

we already have some code to transform the block enum in there https://github.com/andrewrk/mineflayer/blob/master/lib/block.js so it might be fine to deduplicate like that ?

[thejoshwolfe]

we often needs to find a block knowing only its id and metadata

Understood. That means we need some way to get from id and metadata to block enum, but that way doesn't need to be an {} object. It can be a function.

var blocks = {
  "stone": {
    "id": 1,
    "metadata": 0,
    "displayName": "Stone",
    "name": "stone",
    "hardness": 1.5,
    "stackSize": 64,
    "diggable": true,
    "boundingBox": "block",
    "material": "rock",
    "harvestTools": {
      "270": true,
      "274": true,
      "257": true,
      "278": true,
      "285": true
    }
  },
  "granite": {
    "id": 1,
    "metadata": 1,
    "displayName": "Granite",
    "name": "granite",
    "hardness": 1.5,
    "stackSize": 64,
    "diggable": true,
    "boundingBox": "block",
    "material": "rock",
    "harvestTools": {
      "270": true,
      "274": true,
      "257": true,
      "278": true,
      "285": true
    }
  },
};

var blocksById = {};
(function() {
  for (var name in blocks) {
    var block = blocks[name];
    var blockArray = blocksById[block.id];
    if (blockArray == null) {
      blockArray = blocksById[block.id] = [];
    }
    blockArray.push(bock);
  } 
})();

function getBlock(id, metadata) {
  var array = blocksById[id];
  if (array == null) return null;
  if (array.length === 1) return array[0];
  for (var i = 0; i < array.length; i++) {
    if (array[i].metadata === metadata) return array[i];
  }
  return null;
}

module.exports = {
  blocks: blocks,
  getBlock: getBlock,
};

Also, is "polished stone" a real thing? I can't find it on the minecraft wiki.

We'll probably need a metadataMask property to support saplings.

[rom1504]

I meant Polished Granite (http://minecraft.gamepedia.com/Granite#Block_data)

Yeah I guess we could build that index at run time. It's not the most efficient thing, but there are few blocks anyway so it might not be a problem.

If we do that, we need to be consistent though : I think recipes.json should then be indexed by that name too.

Oh and : we need to be really sure that this name always exists and is unique.

We could also have no indexing and store the blocks in a simple array, and do both indexing at runtime.

[Gjum]

we should not index items by item id
the name (e.g. "polishedStone") is the true unique identifier

Recent hints from Grum confirm that (already planned for Minecraft 1.9) the block/item system will not use any consistent IDs at all. Now might be the best time to prepare for that change and use a string ID indexed format.

[thejoshwolfe]

we really need both mappings at the moment. like @rom1504 said, it doesn't really matter if the source is indexed by name or even just a big array that's not indexed at all; we'll need to build at least a few redundant indexes at runtime.

[rom1504]

So there's this http://minecraft.gamepedia.com/Granite#Block_data and http://minecraft.gamepedia.com/Block_states#Stone

I'm pretty sure (I'll know for sure when I'll do the wiki extraction) that variants share everything but displayName, metadata and blockState. (the name is the same)

So I think doing "deduplication" like in #7 (comment) makes sense.

I have no idea what we should be indexing by (id + metadata ? name + block state ? just one of these ?) so doing no indexing and letting the user of the file do it might be the best way.

I think things will be clearer after starting #8

[rom1504]

Related information :

• http://minecraft.gamepedia.com/Category:Data_value_pages
• http://minecraft.gamepedia.com/Category:Block_state_pages
• Block state data #18

[rom1504]

So there's two kind of metadata :

• functional metadata (see http://minecraft.gamepedia.com/Fence_Gate/DV )
• metadata indicating a block variation(see http://minecraft.gamepedia.com/Stone/DV )

Functional metadata is useful for physics and is a bit complicated to store.

Variation metadata is useful for recipes and can be stored with a format like this :

"1": {
  "propThatAllOfIdHave": "val",
  "variations": [
  {
    "metadata": 12,
    "propOnlyThisVariantHas": 123
  }
  ]
}

In order to get only variation metadata, keep the one having a DV column and not a Bit column.

[rom1504]

So blocks variation will have :

• a metadata value
• a blockStateName (if that can be easily extracted from the block state pages)
• a displayName
• other properties ?

the displayName of the variation override the displayName of the block.

[rom1504]

It seems items don't have any metadata variations (no S in http://minecraft.gamepedia.com/Data_values/Item_IDs unlike in http://minecraft.gamepedia.com/Data_values/Block_IDs)

Actually some items have variations : look for the B in http://minecraft.gamepedia.com/Data_values/Item_IDs

[Gjum]

items.js would really benefit from an optional "variants" entry, as most items just have none.
For blocks on the other hand, no idea if this format would be compact/fast/readable. We might just try and see what works, what doesn't, and how to improve it.

[rom1504]

I think an optional variations field for both blocks.json and items.json can work, but it'll be clearer when the data is there (I'm working on that)

[rom1504]

http://minecraft.gamepedia.com/Template:Bst is a bit complicated, starting with juste metadata and displayName only.

[rom1504]

Most of items and blocks variations are now there.

Missing :

• dyes
• flowers

[rom1504]

Most blocks, items and recipes using variations are there.

What's missing :

• some recipes
  • carpet
  • wool
  • banner
• functional block variations (using bits)
• block state / block state name

[rom1504]

Ok remaining issues are now in #24 , #23 and #18 .
Normal variations are there, almost all recipes with variations are there.